Three Truths About Product Analytics

Some things you should know about using data to build better products

A wise man once said, “When you change the way you look at things, the things you look at change.” There’s no better way to describe how your product team can transform what it does when it starts using product analytics to track user behavior.

At Heap, we’ve worked with teams at every level of digital sophistication. We’ve helped teams that use analytics to drive every decision and teams who have yet to discover data. Along the way, we’ve realized a few things that can help ground your team’s approach to product analytics, no matter what level you’re at.

Truth #1: You don’t just need data — you need the right data

In the world of product analytics, there’s a wide range of tools available. Some offer snazzy UIs. Some have amazing analytics capabilities. But the insights that come from even the most powerful-seeming tools are not always accurate, and not always actionable. This is because analytics tools can only analyze what’s fed to them. And if you’re feeding them incomplete data, you’ll get incomplete answers.

It’s like building a house on a foundation with major holes in it. Or buying a Ferrari and feeding it cheap gas. Ultimately, your tool will only be as good as the data you’re able to give it.

How will you know in advance what events will be most important? You can’t. Are the indicators you’re looking at actually the most important things, or are they simply the most important things in the dataset you managed to collect? Unless you’re collecting all the events from your site, you can’t.

For example: you notice an increase in a particular conversion that you’re tracking. That’s good! But do you know why? Could be due to a discount offer…or a new campaign bringing more traffic…or it might even be a false positive caused by an error in the way sessions are being tracked. That’s bad! If you can’t discern exactly what’s going on, you’re gambling with partial information.

Unless your analytics platform can generate a reliable set of indicators to point you toward true north, you can iterate, tweak and ship like mad, but won’t know whether you’re actually making progress. And if you’re not measuring 100% of the metrics around an event, the critical factor to affect change might be a specific behavior that’s not being tracked. But the flip side of this is good news: the better your data set, the more your answers are already there.

Truth #2: Tools that aren’t useful … won’t get used

While lots of platforms bill themselves as a solution, what you end up with is often more like a do-it-yourself kit. When product analytics is extra work for someone (perhaps you!) it becomes their least-favorite chore. Here’s why:

Tracking code manually is a hassle.

Getting the full promised power from many product analytics systems requires begging or borrowing engineering resources in order to install and maintain tracking code, and then relying on ongoing engineering support to keep your data fresh. Most businesses don’t have this human power to spare.

In lots of tools, inquiries take months to answer

For product analytics to be easy, the tools need to be running in real time, all the time. Ideally, anyone in your company should be able to dip their finger into a fresh-running stream of relevant data anytime, in as many points along the stream as they wish. But many tools force you to rigidly define events beforehand, and formulate a specific inquiry plan well ahead of time. This can take weeks to do, on top of the months it can take to collect data.

“Precision tracking” isn’t so precise

Lots of analytics providers tout “precision” tracking. What they’re really talking about is manually-coded tracking, which is restricted and biased by nature. It may be easy to organize, but it only gets this way because “precision tracking” relies on keeping the dataset small, meaning that you can keep your data governed if you only track a few things! When you only have three books on your shelf, it’s pretty easy to keep them organized. But your reading options are…somewhat limited.

You need more than just data, you need smart governance

If your data lake becomes a murky swamp, it doesn’t matter how large it is. Organization is key, so having governance capabilities in the platform keeps things simple and straightforward. Teams should have an easy way to identify, name, and group events. And every user should be free to mine for insights without affecting the shared dataset. When the data pool is clean, everybody in your company can stay aligned, and work efficiently, without limits.

Truth #3: Insights lead to outcomes … but better insights lead to better outcomes

Lots of tools talk about insights. But let’s stop for a minute and think about what that means.

It’s only an insight when you didn’t already know it.

The key thing to remember is that an insight must change your understanding. “The sky is blue” is not an insight. But understanding that the sky only appears blue because blue light scatters more than other colors by traveling in shorter, smaller waves — now that’s an insight. (Well, for many of us.)

Similarly, noticing that a percentage of customers are dropping off at checkout is not an insight. But doing path analysis around the dropoff point and discovering that people who drop off tend to visit your shipping FAQ page first is an insight — one you can take multiple courses of action to remedy. Are users balking at the price…are our shipping costs made clear…is the shipping calculator malfunctioning? That’s the kind of insight that makes product analytics valuable.

Tools are only future-proof if they’re pulling in the right data.

Thanks to our evolving relationship with the digital world, digital experiences are quickly becoming the primary relationship between customers and brands. More than ever, companies need to invest in data foundations that offer complete data, retroactive capabilities, and proactive insights. You never know what you’ll want to look at, or look back at. That’s why you should track everything you can think of. Reliable, quantifiable insights are what help you improve your product, increase engagement, and reduce churn.

Creativity flourishes when curiosity is paired with a scientific method.

Someone we know said recently, “I want data to be so easy that it overpowers my gut reaction.” Data can only do that if your tools are easy to use, and above all, trustworthy.

How we solve these problems

Since we know that the truths above are genuine, we designed our solution around meeting them, with smart governance that doesn’t restrict you to keeping your dataset small in order to keep it manageable. We invented autocapture to get you the right data. We made it future-proof by capturing everything. Then we keep things governed so you can get the insights that matter.

When governance is built into your existing processes, all the key goals — consistency in naming and definition, fresh events, eliminating duplicates, annotation — are simple and straightforward, without no limit to the number of events you can create.

In conclusion

When it comes to making product analytics easier, the best tool is the one that you like to use. Over and over. Every day. In ever-expanding ways. We think Heap alone offers a living, scalable, reliable dataset that remains governed as it grows with you. Nobody else can truthfully say their data is both easy AND trustworthy.

Please contact us with any questions, for a demo, or to get a free trial.