Why you want a dedicated analyst earlier than you might think
- The realities of today’s data driven product management make considering adding an analyst sooner than later attractive
- An analyst can free up product managers to focus on other things, do deeper analysis, investigate beyond specific requests, be an objective party, and inform discovery
Beyond the Product Three
A lot of product literature focuses on a trifecta of three individuals – the product manager, the designer, and the engineers1. Typically, this squad of three key functions is set out to build products in a product team.
There is a lot to like in this characterization. It was particularly drawn up to contrast against feature teams. It tries to illustrate the role of three equal partners where a product manager specifically focuses on product vision and prioritizes a roadmap against use cases and pain points. This allows the team to go from project/feature delivery to product delivery against key results. These teams strive with context and goals, not edicts of what to do.
One of the reasons the literature focuses so much on this Product Three is that it is proven yet still not ubiquitous. The product team is the hallmark of the greatest Silicon Valley companies. All the rising or new big tech players like Spotify & Airbnb use them. At the same time, 80-90% of companies developing technology products still a use a more waterfall method, with many feature teams.
With regards to explaining the value of a product team leading the product process, and the role of the product manager, I think the Product Three works well. But I think a few things have changed since this conception of the product team was formed 20-30 years ago.
First, data has become more available
Whether you are in B2B or B2C, the sheer volume of data has multiplied. Furthermore, the size of that data has multiplied. Product managers can no longer download excel files of all their customers and a financial plan, then work from there to do their job. Product managers are digging into petabytes of telemetry, querying hive databases, and running SQL/python that take hours to run. This takes time, including research to keep up with the latest technologies, to do well.
Second, more of product work involves data
These days, every product team is identifying segments, testing, and utilizing machine learning at some layer of their product. If they are not yet, they should be. While segmentation based on design and product knowledge is okay, doing rigorous unsupervised clustering is even better. While implementing features and looking at changes over time is okay, running and measuring statistical significance from properly sampled A/B tests is even better. Finally, while building a design to put the most important information first is great, building a neural network to put the right categories first is even better.
These two facts have meant that data has become a larger and larger component of the strongest product teams. As a result, I think most people want to enhance their product teams faster than they might think.
To be clear, feature and product teams hopefully work with analysts. The distinction here is that the analyst is 1) full-time on the product team, 2) co-located and sits next to the product team, 3) attends all the same product meetings as the rest of the team, 4) is an equal partner in product decision making, 5) is held to the same set of OKRs as the rest of the team – the results of the product. Like all the other members, they still typically report to an analytics lead who is an expert in pushing forward product analytics and also helps coordinate the data platform so the analyst can focus on the product.
I would highlight five specific key values from adding an analyst earlier than later:
1: Free up the product manager for product work
Explicitly formulating consumer segments, pain points, hypotheses, making tradeoffs, and other core product work comes at the expense of analysis. Every hour a product manager is running a clustering algorithm to identify customer segments themselves is an hour less understanding consumers or building product vision. This tradeoff is very expensive. Being a product manager is a full-time job without taking on the job of analysts. This is the same principle why it is important to have designers or engineers on the team. It pays to be able to focus.
When a product manager can open well-made dashboards and workbooks that update to analyze key metrics, as well as new tests, that enables the product manager to focus on analysis and results. This is the same reason the paid acquisition team, for instance, has channel managers who just spend all their team reading and reacting to the data. Those people do not also write the SQL behind the data. The product manager becomes more of the stock trader who can see the product tradeoffs they have made and optimize towards the overall objective over time.
2: Be able to deeper analysis
If you do not live and breath the data, data becomes a tool. Having someone who owns the data side allows them to come to the table ready to do deeper analysis. For example, coming up with a new rigorous product segmentation takes months for even the best analysts or product managers. So, having someone who has the time and ability to do this is very helpful. Analysts can also go the layer deeper on testing, looking at a wider array of secondary and tertiary KPIs. Similarly, they can help in times of crisis, using analytics to identify pain points.
Structuring the data into useful intermediate tables is something most product managers do not even get time to if the data team does not do that for them. This type of foundational work to speed up future queries and make the data more available is the type of project an analyst has time to take on. Product analysts can also work with data platform and data engineering times to architect systems at the latency and frequency a product team needs.
3: Have the analyst investigate beyond requests/ part-time
Typically three models are in place if the analyst is not dedicated to the product team full time. All have reasons they are suboptimal. In the most common scenario, product teams have access to an analyst based on requests. So when they have time and foresight, typically the product manager’s, to document a request, explain why it is important, then they will get on the analyst’s backlog for investigation. This results in analysts just answering the questions posed, never going beyond. In addition, they lack the context from being in the discovery, design, goal, and other meetings the product team conducts.
In another scenario, the analyst is part-time. This may be the best alternative, especially if the team is waiting to hire into a role. Then of course, the analyst must make tradeoffs with product team meetings, so they have less context. In addition, as they do not have an OKR around the product most of the time, instead it is around delivery of analytics, they have less skin in the game. Typically, the analyst’s work will be helpful but not to the level of thinking that a dedicated analyst whose OKR was the product’s would be.
Finally, sometimes the analysts live in a village, essentially an analytics studio aside. As their data and attention turns to the topic of your product, the product team will be inundated with senior executives and others who have seen the data produced. This can be useful to have an objective party. But in the long term it is not a collaborative product relationship. Consistent analysis as a partner leads to faster results.
4: Have an objective party investigating analytically
Because a product manager is typically intimately involved with the creation of a roadmap and plan, they may be biased towards showing the forward progress on that plan. Analysts are often great at forcing the team to question their assumptions and progress. Was the result really statistically significant? Did we move our topline OKR? These are questions analysts are typically in a position to excel at answering.
A dedicated analyst allows someone to come at the problem purely from a data point of view. This is beneficial in several ways – for example, forecasting. While a product management forecast must bake in gains from product features shipped, an analyst is able to more detached. They can build a forecast purely based on seasonality for instance. Or they can come to the table with a realistic idea of what they lift range might be and help characterize success of a launch.
Having a dedicated analyst is also a form of process accountability. It ensures the product team measures every change. It is usually easier to just ship globally. But Data-Driven Product Teams are less likely to do so. They take the extra effort to set up test and control to appease the objective analytical party.
5: Utilize data in discovery and key product activities
A good product analyst is an equal partner to design and product management in figuring out what to build: sitting in user sessions, iterating on discovery, and developing the product vision. The data person can, for instance, quickly size the potential impact of a change based on the history of similar changes they have analyzed. They can also help scope the objective function, primary KPI, and secondary KPIs.
They also can be equally useful in communicating plans and socializing product victories. By being data focused, the analyst can help put the rigor and work behind analytics to show the impact of changes the product team ships. This can help the product team’s morale, ability to marshal resources and teams for its initiatives, and the growth of the team.
When to Expand
As we all expect, product teams grow as companies do. Any of the following expertise can also enhance a product team depending on the context: (product) marketing, technical program management, data science, machine learning, user research, legal, operations. Some of the key, large teams at Google have almost all these roles. For companies still working up to Google scale, the key is to think about customizing your product team’s growth to your industry, sector, and sales process. Companies may have roles specific to their strategy. Facebook, for instance, has known to have growth analysts: analysts who specifically obsessively focus on the growth metrics.
There are a few common rules of thumb I would suggest thinking about when growing a product team. First, don’t grow it beyond the “two pizza rule,” excluding engineers. I like the two pizza rule but key features on Google Search, for instance, just require so much brain power that they need all the dedicated stakeholders. Engineers are equal partners, especially a good lead, but some products just require heavy engineering lift, especially machine learning or high scale one’s.
Second, start considering “one person per three engineers” (again: we are talking about full-time, part-time you will typically have all the roles). So, start with a development team: less than 3 engineers. Then you move to a feature team: 3 engineers and 1 designer. Then, you go to a product team: 4-6 engineers, 1 designer, and 1 product manager. Once the product grows to the importance of 7-9 engineers, you add an analyst if you are B2C, or a product marketer if you are B2B (or B2C but have a sales force). The next role is sometimes splitting design into visual design and interactive design. Other time’s it is another product manager. Other times it is a test automation engineer or QA person. Let your needs dictate and grow from there. These are just rules of thumb and every situation is different.
Take Away
Most product teams should consider adding a dedicated analyst to the team sooner than they thought – certainly sooner than they would have 20-30 years ago, in light of how data driven product teams have become. In particularly data driven B2C companies, they may consider a full-time analyst, in addition to full time product manager and designer, once a product reaches 4 engineers.
Note
1: Example: Marty Cagan’s Inspired (one of my favorite books). The book is a love letter to the Product Three. Data Analyst was given just three paragraphs in the Product Teams section, Chapter 14 – Supporting Roles.
