- Infernal Ramblings
- Posts
- Why It's Hard to Decide Which Executive Should Lead a Startup's Data Team
Why It's Hard to Decide Which Executive Should Lead a Startup's Data Team
Pondering why, despite staying in analytics my whole career, I've worked for what feels like every major executive at a tech company, ranging from Chief Product Officer to CTO to CFO
I’ve spent just about a decade working in startups. I’d say the core function of my role has been the same just about every startup I’ve worked at, ever since I joined NerdWallet as the first member of the Analytics team in 2014. At every tech company I’ve worked at, my job has been to help business leaders use data to understand what their customers are doing, and to make better decisions based on this data. But despite the stability in my job function, and its obvious value to the business, I’ve had a rotating cast of executive leaders: at different times in my career, I’ve reported into operations, finance, technology, and product executives.
It would be unimaginable for a financial analyst or a software engineer to ever report into anyone other than a finance or a technology executive respectively – yet with data scientists and product analysts at startups, it’s anyone’s guess who our executive boss might be. More than that, at every startup I’ve been a part of, the executive my team reports into has changed during my tenure there. If my function’s core duties and responsibilities have been so stable, why the constantly-changing set of titles and accountable executives?
To boil the rest of this complicated answer down to a simple conclusion: because startups.
Startups big enough to need professional analysts and data scientists are still small enough companies for C-level executives to be hands-on. And technology startups are unique in that they have the scale to amass vast quantities of data about where their customers come from and what their customers do with their product – while still being nimble enough that they can shift the entire direction of the company based on what they see in their data.
Because analysts are a profit center in technology companies, startups typically want to build a company-wide central analytics team that can take a broad view of their business – not silo their analysts under individual functions like marketing and sales. But startups are also too small to hire a dedicated data executive like a Chief Data Officer, and need to operate too nimbly to so rigidly separate their data team and its leader from the rest of the business.
So, startups hire a team of analysts and put them under one of their top business leaders – sometimes it’s the COO, sometimes it’s the CFO, and sometimes it’s even the CTO or Chief Product Officer. But which leader is best suited to manage the company’s data is obviously a tough decision – my newsletter today does not even attempt to come up with a full theory of how these decisions get made. Instead, I want to reflect on why the role of analysts and data teams at technology companies, especially startups, is so special – and why finding the right executive to manage them can be so tough.
To be clear, my subject today is not startups too small to need analysts or data teams – typically companies that are Series B or earlier. Every startup I’ve worked at had already found product-market fit by the time I joined, and was busy working on the problems of operating and further scaling an already-sizeable business. The unique combination of large and fast-growing scale with the agility of being a still-young and -small company is why I think I’ve spent so much of my career at mid-size startups – and also why I’ve seen so much change about the way each startup chose to organize its data team.
Who Best to Manage the Analysts?
As I’ve discussed before, basically every tech company I’ve worked at has organized most or all of its analysts in the same way: individual analysts or data scientists reporting into a functional Analytics team, but day-to-day spending most of their time with a cross-functional tech team consisting of engineers, product managers, designers and analyst(s).
Other ways to organize analysts and data teams exist, but tech companies have settled on this common structure because it clearly has its advantages. By reporting to the same manager and sitting in a singular central team, analysts get the benefit of being part of a broader data community. But by embedding individually into different parts of the product and business, analysts also get the exposure to and ownership of the business that makes them more effective value generators and profit centers in their own right.
An alternative where analysts report to a central team but don’t embed in any business unit leads to “ticket taking” behavior, where analysts don’t understand or feel ownership of any part of the business. And having individual analysts solely report to individual business units rather than any central ownership leads to siloing: it becomes very easy for two different analysts to have two different sets of numbers, and no easy way to reconcile the two.
So, any startup big enough to need professional analysts is almost certainly going to have a centralized Analytics team. And that team is going to have to report into one of the executive team.
It is theoretically possible for the analysts to report directly into the office of the CEO; in practice, I’ve only seen this during a transition period when the executive team itself was going through a shake-up. The truth is that while most CEOs care a lot about the output of the Analytics team that eventually reaches their eyes, most CEOs of mid-size businesses are also too removed from the on-the-ground of daily execution to usefully engage with the Analytics team’s work.
The most common org chart by far that I’ve seen in startups has been to have a handful of senior executives reporting directly to the CEO – the core executive team usually consists of these core roles:
Chief People Officer / VP of HR
General Counsel / Head of Legal
CFO / VP of Finance
CTO / VP of Engineering
Chief Product Officer / VP of Product
COO or a combination of VPs that run operational functions (Marketing if there’s no CMO, Sales, Operations)
The COO role doesn’t exist universally – most mid-size startups are still too small and young to need a clear #2 to the CEO, which is what the COO role de facto is. I’ve typically seen the COO role created when the CEO began to feel it was too unwieldy to manage the individual operational functions’ VPs directly, or when the CEO decided to really focus on some other aspect of the business, usually the product.
The main way I’ve seen the COO role deployed at smaller startups (still on the small end of mid-size) has been when some of the other major functions were also layered under the COO: Finance reporting to the COO is a setup I’ve seen more than once. Mid-size startups are usually (though not always) too small to have C-level executives running Marketing or Sales, and so those functions often roll up to a COO if there is one – or the VPs of Marketing and Sales might report to the CEO directly.
So who should our Analysts or Data Scientists report into? We can rule out HR and Legal off the bat. But otherwise, it’s actually all fair game: I’ve otherwise personally lived through reporting to every other executive from the list above – and multiple reorganizations that moved my team from one of these executives to another.
Why are Analysts So Flexible?
To me the first question this raises is why. Why is this discipline critical enough that every startup past a certain size needs a central Analytics team, but not so critical that there’s a standard playbook for who should manage it? Why is it so fungible and passed around different executives like a hot potato?
I think the answer has a lot to do with the technical product that is software. Traditional companies typically don’t worry very much about their analyst teams. Think of a company like Nike or McDonald’s: there’ll usually be some analysts under Finance, some analysts under Marketing, and some analysts under Operations. Putting all the analysts under one central team or reporting prominently to one executive doesn’t make a lot of sense because in many ways analysts at traditional companies selling physical products are more cost centers than they are profit centers. An analyst’s salary is the ancillary cost associated with running a more efficient marketing campaign or digging into the financial reports – not core to how the product is made or sold.
Facebook may be watching you, but so is every software and internet company you’ve ever interacted with — each of them with their own team of analysts telling them what they should do based on what you’ve been doing with their product
Modern technology companies whose business centers on software – whether that software is Facebook or Salesforce – typically need to organize their analysts quite differently. The sheer amount of data that modern software can collect is a critical ingredient in improving and marketing the company’s product. Whether you’re a mature tech company like Facebook or a startup that’s just starting to get mass adoption of your product, you need analysts to dig into what people are clicking on, how often they’re opening the app, and where new users are coming from. Unlike a physical product like a sports shoe or a restaurant meal, tech companies have the data to understand at the most incredibly granular level what the customer is doing in their product.
This makes analysts essential to the product and fundamental business of tech companies in a way that they aren’t at many traditional businesses: analysts are much likelier to be profit centers than cost centers at a tech company. So it follows that many tech companies are going to care more than a traditional business about ensuring analysts are properly held accountable for the results they’re meant to drive.
This means analysts are often going to be organized into a single team that has clear accountability to some important senior executive who is in charge of decisions and resources that are fundamental to the business. Analysts typically won’t be spread across individual analyst silos buried in the org charts of several different departments, nor buried underneath a relatively anonymous executive. Executives at technology companies will care about having the company’s analysts reporting into them.
But the fact that analysts are much more essential to the business of technology firms still doesn’t explain why they can be put under such seemingly diverse and different executives ranging from the CFO to the CTO to the Head of Product. It would be inconceivable for the Design team to report to the CFO or Engineering to report to the COO in the vast majority of tech companies. Why do the data teams get treated so flexibly by comparison?
Here, I find it useful to look at the organization of a mature technology company, such as Pinterest. Pinterest is large enough that virtually every executive has their own analysts: both the Content organization and the Sales organization had analyst teams when I worked at Pinterest. You need truly massive scale for this to be feasible; I never saw anything like this when I worked at NerdWallet in its startup days, even though NerdWallet also had large Content and Sales teams.
Despite the presence of several analyst teams across the company, the two most influential teams of analysts at Pinterest when I was there were the Product Analytics & Data Science (PADS) team, and the Business Operations team. PADS was the largest team of analysts at the company by far, and reported into Engineering. BizOps, reporting into Finance, was one of the smallest analyst teams, but had the most direct and frequent interactions with the executive team, as we owned the company’s weekly metrics review.
My personal theory is that most startups are in the process of evolving towards a structure similar to Pinterest’s. The end state for analysts at technology companies is for the largest analytics team to be a team of product analysts or product-focused data scientists sitting in the technology group, with a smaller but critical, executive-facing team of business analysts, likely sitting in the finance organization. And startups, being too small to support multiple teams of analysts, end up having to choose between which of these organizational setups they prioritize as they first establish the data and analytics functions at their company.
The End State of a Tech Company’s Analytics Team is to Split into Product and Business Analytics
It seems to me a very natural end state for large, mature technology companies to have multiple data teams spread out across the company, with two teams specifically taking primacy: a large product analytics/data science team under a technical executive, and a smaller business analytics team likely under the Finance org. This was obviously the set up at Pinterest, but every startup I’ve worked at has felt like we were building towards this outcome if we ever got to Pinterest scale. One of my previous startup employers, Thumbtack, has already made the transition to this type of organization even though it remains a privately-held firm.
It is intuitive that a technology company will have a large data team in its technology organization. Whether they report to the product executive or an engineering leader, these product analysts have to play an essential role in how the technical teams understand their users and the way they use the product – so it’s logical that in the mature end state, you’d have a team of product analysts sitting somewhere in the technical organization.
And at the same time, it tracks that you’d want a separate team of analysts driving corporate reporting and birds’ eye analyses of the business for senior executives. Unlike a company selling physical products where the business metrics are mostly financial or operational, a technology business’s metrics come from data generated through highly technical processes within their product itself. So you can’t have a team of financial analysts working with this technical data to answer executive questions about the health of the business; you need a dedicated team of relatively more technical analysts capable of being the interface between very vague, high-level executive questions about the business and the nitty gritty raw data generated by the company’s technology.
Since the business analysts are in charge of all the non-financial metrics about the overall state of the business, it makes sense that they should report into the CFO’s organization. Just as only the Finance team is trusted to tell the company how much money it’s making – not the analysts on the Sales or Marketing teams – it makes sense that the business analysts responsible for telling the company how many users the company has should be part of the same broader Finance organization.
In a large technology company, the product analysts have highly specific operational duties related to shipping product, just as marketing analysts have highly specific roles in operating marketing campaigns. The leaders of these functional analytics teams do not want their analysts distracted with the kind of bigger, vague strategic questions that executives often have for analysts. So every technology company that reaches a certain scale starts to see the value in having a dedicated business analytics team, and the most natural home for such a team tends to be in the Finance org.
You’ll note that this mature end state does not have a Chief Data Officer running a unified data team, which would be the case in some large traditional businesses. No company I’ve worked at has had a Chief Data Officer, or shown any sign of being on track to hire one at some point. I think this comes back to my theory of how analysts are profit centers in technology companies. Separating the analysts at the company from the rest of the business is just too hard to do in a technology business, where analyzing data from the product is so clearly a major generator of value for the business.
The chief product or technology officers are just not going to want another C-exec controlling a major potential driver of value for their teams; they will push to have their own product analysts. And because of the value of technical data to a technology business, these product analysts will become the most important team of analysts at the company in their own right.
The main important set of analysts left over will be the business analysts who interface directly with the senior executive team – and there’s no point in having a CDO overseeing a rump team of business analysts. You just don’t need more than a small team of analysts to oversee corporate metrics for even a very large technology business; not when you already have a large team of sharp analysts in your technical org. So it’s a very natural end state for big technology businesses to end up like Pinterest, with the most important/highly-visible analysts at the company being a large product-focused data team in the technical org and a smaller business-focused data team in the financial org.
Thus we have the beginning of an answer to our original question – why do startups seem so flexible about who their data teams report into? It’s because there are two types of executives who ought to manage the bulk of the company’s analytical firepower if the startup ever makes it into a tech giant. Ultimately, the company’s analysts will largely report into either the technology group or the finance group. So most startups will put their very first data team under a technology leader or a finance leader.
The question of how to choose between the technology or the finance leader comes back to whether the startup seeks to emphasize product analytics or business analytics more when it’s first building its early data team. If the emphasis is on product analytics, the company will likely staff its data team and analysts under a product or engineering executive. If the emphasis is on business analytics, the company will put its analysts – who may or may not also be grouped together with the data engineers or analytics engineers that typically comprise the rest of the data team – under whichever executive is leading its finance team. This might be the CFO, or it might be the COO if the CFO role does not exist yet at the company.
With this framing in mind, we’re finally in a position to ask: so how do startups choose whether to emphasize product analytics or business analytics when they’re first starting to build a data team? How should they be making this choice? I don’t pretend to have any sort of final answer to these, but I will share my thoughts in future posts. In the meantime, if this has spurred any reflections or questions, or you’ve thought of something you’d like me to address in a follow-up newsletter, feel free to drop me a line.
Reply