Answering Burning Questions: What data is needed?
Hey there,
If you’re new: Thank you for subscribing to the databeats newsletter where I publish guides for growth practitioners to drive data-powered growth.
Last week I introduced the idea of Burning Questions.
Today’s post is the first in a series that explores the process of getting answers to burning questions.
Let’s get into it.
What data is needed?
Figuring out what data is needed to answer burning questions is an inherently collaborative process – Growth needs to involve Data or Engineering to figure out what data is needed either to arrive at a satisfactory answer or to run a data-powered experiment.
However, a keen understanding of the product, the underlying data model powering the app, and the org’s data infrastructure is empowering for a person hungry for growth. It enables them to have better conversations with their data and engineering counterparts and move fast without compromising on data quality.
Now, let’s use the same burning question we used earlier (in the previous post) to illustrate the process of figuring out what data is needed to get to a satisfactory answer:
“We’re acquiring a ton of users every day but very few end up hitting the activation milestone; what’s preventing the rest from performing the actions leading to activation?”
This also happens to be one of the burning questions I had during my time at Integromat. It was frustrating that we were acquiring close to a thousand users every day but less than 5% of those users were becoming activated and we didn’t have the data to figure out why.
I must add that the path to the activation milestone wasn’t straightforward due to the nature of the product. It had a relatively steep learning curve, particularly for users who were used to a product like Zapier which was a lot easier to use albeit a lot less powerful. Zapier offered a linear, step-by-step process to build workflows (they still do) whereas Integromat’s users would land on a blank canvas with endless possibilities.
This approach was rather novel at the time, but as a result, only a small number of users we acquired were able to create and activate (or turn on) their first workflow and hit the coveted activation milestone. On Integromat, workflows were called scenarios, a term I’ll be using quite often going forward.
The task at hand was to identify where in the onboarding journey were users getting stuck and dropping off, and then figure out ways to fix that to ultimately increase the activation rate (one of the core metrics for SaaS businesses).
Account-level Data
SaaS companies measure activation at the account or organization level rather than the user level, and depending on how one defines an activated account, it can take more than one user for an account to hit the activation milestone. I’ll go much deeper into account-level data in a future post.
Here’s the formula to calculate the activation rate:
Activation rate = number of activated accounts / number of confirmed accounts
A confirmed account is one that has completed the account creation process and the account details have been verified to ensure that the account belongs to a legitimate entity – a person or an organization.
In Integromat’s case, even though activation was measured at the account level and multiple users could work together inside an account, creating and running a scenario (or workflow) didn’t necessitate collaboration – an account owner could alone build a scenario and their account would be considered activated.
At the time, our goal was to get every new user to create their first scenario on Integromat. This included new users who joined an existing account that had already hit the activation milestone. However, and this is something we figured out later, a significant chunk of account owners did not actively use the product themselves but had someone else build scenarios on their behalf. This insight helped us optimize our onboarding and cancellation flows, eventually leading to higher activation and retention rates (we would proactively reach out to account owners who had canceled a paid plan because the primary user who built and maintained the scenarios was either a past employee or a freelancer who was no longer available).