Tuesday, February 26, 2019
Measuring and benchmarking the four vital signs of SaaS – TechCrunch
SaaS metrics should be to a management team what patient vital signs are to an emergency room doctor: a simple set of universally understood numbers that allow a doctor to quickly know how ill a patient is and what needs fixing first.
Heart rate, blood pressure, respiratory rate and temperature are the big four vital signs in the ER. Everyone knows what they are, what they mean, and what good and bad looks like. When a patient is wheeled in, the doctor does not start by asking the EMT, “How exactly are we defining heart rate?” This shared understanding allows for rapid evaluation, then fast, focused action.
Not so much in SaaS, where discussions about definitions are all you hear. There are too many metrics, too many things to measure and too many useful but incompatible ways to measure them. This results in a loss of clarity, comprehension, and — most importantly — comparability across different companies.
At Scale we’ve spent the last 20 years evaluating investments in SaaS and other subscription companies. We have built an internal shared belief of what the Four Vital Signs of SaaS are, and how exactly to measure them. We have opted for simplicity over complexity in selecting these metrics. This has allowed us to benchmark accurately across companies and to know what a realistic version of “good” looks like.
Scale recently launched Scale Studio, an open-to-anyone tool that gives cloud and SaaS companies performance benchmarks based on these vital signs and 20 years of data across more than 300 companies.
The four vital signs of SaaS
The vital signs of SaaS are Revenue Growth, Sales Efficiency, Revenue Churn and Cash Burn. Almost everything that matters about the financial performance of a SaaS business is captured in these four metrics.
Revenue Growth matters because growth is the central purpose of a startup, and thus for an investor the most important driver of if value can be created at all. We have found that at each stage of a company’s development there is a minimum required level of growth below which a startup will struggle to attract venture capital. We’ve analyzed this Mendoza Line for SaaS growth previously on TechCrunch.
Sales Efficiency matters because software at scale is all about distribution, and thus the relationship between dollars invested in sales and marketing and dollars back via revenue is the key determinant of how much value is created per dollar invested. In a perfect world I would call this Distribution Efficiency, because calling it Sales Efficiency tends wrongly to narrow the focus on this metric to just sales, but that ship has sailed.
Revenue Churn matters because as growth slows the impact of churn escalates and provides an upper bound on how big a company can become. More fundamentally, high churn is just the financial evidence of a product that is not delivering value to customers. Products that do not deliver value cannot build value for their investors, which is why my former board colleague at Box, Mamoon Hamid, is right in saying that every company needs a non-financial north star.
And of course, Cash Burn matters because, well, duh — try running a company without it.
Measuring and benchmarking the four vital signs of SaaS
Vital Sign No. 1: Revenue Growth
There are multiple ways to measure revenue and thus Revenue Growth. ARR-based metrics are more forward-looking, but GAAP revenue tends to be calculated more accurately and is thus more comparable across companies. It typically lags ARR by about a quarter, and for simplicity we have not excluded services revenue. The simplest measure of Revenue Growth is the quarterly GAAP revenue run rate compared to the same quarter GAAP revenue one year ago (or a year from now for a forward-growth estimate). We also measure revenue growth using the ARR Growth Rate and a forward-looking measure of ARR growth that we call iCAGR.
We benchmark Revenue Growth by looking at companies at a comparable revenue run rate because revenue growth rates decline fairly predictably as absolute revenue scale increases (as is clear in the chart below). This means that the top quartile Revenue Growth rate of 123 percent at a $20 million revenue run rate would represent bottom quartile growth at a $2 million revenue run rate.
The chart and table below show, for various revenue run rates, which revenue growth rates represent top, median and bottom quartile performance. The Scale Studio data set has 300+ public and private SaaS companies, some of which have become public, many of which have not and most of which have raised at least some venture capital.
Using this table, a team can quickly benchmark their company’s performance. For example, a company that grew last year 80 percent from $11 million to $20 million is growing at just above 50th percentile growth for SaaS companies at that stage, which is 78 percent. We can also generate a separate table showing the same data on a forward-revenue basis, to allow a company to answer the related question: If I am at a $20 million run rate now, and I grow next year at 50 percent, how will I be doing relative to other SaaS companies?
It is also interesting to see that the data here broadly agrees with a separate calculation we did recently around the Mendoza Line for SaaS growth that tracks at or just above the bottom quartile of growth rate. The Mendoza Line was derived by math; this table was generated from real data — it is good to see both estimates roughly agree.
Another way to look at the same data is to think about the revenue trajectory over time, or “how many years to $100 million.” The graph and table below show for a consistent top, median or bottom quartile company (at the cutoff points) how long it takes to grow from $1 million to $100 million. A company that grows consistently, just at the top quartile cutoff growth rate, takes six years to get to a $100 million run rate, a median performer takes eight years and a bottom quartile performer does not yet get there in 10 years. This is a calculation with all sorts of survivorship bias problems, because the slow growth companies tend to get acquired and not make it all the way to $100 million, but the analysis is roughly right.
This data also matches well to various rules of thumb. An example is the T2D3 (triple-triple-double-double-double) rule, which is also shown in the table above. T2D3 matches top quartile performance for the first four years and becomes just a little aspirational in year five. If you fail to double from year four to year five and only grow at 90 percent, we would still be glad to talk to you! (The data also roughly matches the Bessemer State of the Cloud Report, which shows an estimate of times to $100 million for best-in-class companies).
Vital Sign No. 2: Sales Efficiency
Sales Efficiency metrics (and again the reminder to think of this more broadly as Distribution Efficiency!) measure the relationship between dollars in (spent on Sales & Marketing) and dollars out (in the form of new revenue). For a recurring revenue business, by far the most intuitive way to measure this concept is by dividing the Gross or Net New ARR for the quarter by the fully loaded Sales & Marketing spend for the same quarter. The Gross SE metric measures the effectiveness of the company in generating new ARR, and the Net SE metric measures the overall effectiveness of the business in both generating and retaining revenue.
We love the simplicity of this calculation and its direct actionability. It can be explained in 30 seconds at a Sales Kick-Off meeting in a way that no other measure can. “We gave you this money, you gave us this ARR.” We are not a fan of putting lags in this method (comparing this quarter’s Net New ARR with last quarter’s S&M spend). There is some logic to the idea that there is lag between spend and results, but once you start to adjust, you end up with all sorts of special pleading. Keep it simple.
For a vital signs diagnosis, we prefer this metric to the complexity of the LTV/CAC calculation. An LTV to CAC works really well for consumer businesses and for B2B businesses that have fairly consistent deal size and low net churn. A former Scale portfolio company, HubSpot, did a brilliant job of orienting their business around this metric. However, for enterprise businesses with highly variable deal sizes and strong positive net cohort growth over time, the calculation becomes arbitrary. The underlying idea is real, namely that enterprise customers can have lower Gross SE but higher ultimate value as the cohorts grow, but trying to track and explain quarterly fluctuations is hard.
Another complexity we choose to avoid is using Gross Margin instead of Revenue. It is of course more correct to use Gross Margin, but especially at the early stages of a SaaS company, Gross Margin fluctuates based on fixed cost recovery issues that significantly distort the calculation.
The problem with an ARR-based Sales Efficiency metric is it doesn’t allow easy comparison across companies. ARR is not reported by public companies and private company ARR numbers are often suspect. Our workaround was to slightly tweak the calculation for Net SE, replacing the numerator (Net New ARR) with the intra-quarter difference in GAAP revenue multiplied by 4 (annualized). We call this formula using GAAP instead of ARR the Magic Number and it should be equal to Net Sales Efficiency with a one quarter lag (to allow ARR to convert to GAAP).
As a rule of thumb: When you’re talking to your team and want to keep it simple, talk Gross and Net Sales Efficiency; when you want to do benchmarking against other companies, use Magic Number.
The benchmarking results here are very different. Unlike Revenue Growth, which clearly declines as absolute revenue increases, we have found Sales Efficiency to be fairly consistent across the entire SaaS universe. The median Magic Number for our data set is between .8x and .7x, and the range from top quartile to bottom quartile is between .5x and 1.5x. This matches the public company data set where the median is .7x today.
Payback (on a revenue not a gross margin basis) is simply the inverse of this number, which implies that the average SaaS company is earning back in revenue what it spends in sales and marketing in one divided by .7 years — 17 months — with a top/bottom quartile range of eight months to two years.
We have also observed something that does not come in this table, which is that Sales Efficiency tends to be persistent over time for a given company, especially after $10 million. A good go-to-market model at a $10 million run rate tends to still be a good model at $100 million. And bad sales efficiency at $10 million is hard to change later.
The high top-quartile Magic Number for the $1 million revenue run rate represents an anomaly early on, as often founders are doing the selling themselves (and probably not allocating their costs to Sales & Marketing!). Pretty quickly top quartile Magic Number falls to 1.4x and then to 1.0x at scale.
Vital Sign No. 3: Revenue Churn
The simplest way to measure Gross and Net Churn is by taking Churned ARR (Gross) and Churned less Upsell ARR (Net) and dividing it by opening ARR for the period, usually a quarter. In the tables below, we show Gross Churn by quarter and annualized.
We acknowledge that this metric is a horrible oversimplification. For the Sales Efficiency calculation above, the simple method is also, we believe, the best method, but for the churn calculation this simplification comes at a significant cost in terms of being able to diagnosis underlying issues. At high growth rates especially, this measure understates actual churn. However, the vital signs framework calls for simplicity to allow consistent, relevant benchmarking across companies. If this simple benchmarking exercise exposes a churn problem, then a deeper dive using retention analysis and a cohort analysis is an absolutely required next step.
The data above shows Darwinian selection at work. Early on some companies have huge churn but they have to either improve or die. At a $20 million revenue run rate, even bottom quartile companies have annualized Gross Churn hovering around -22 percent.
Vital Sign No. 4: Cash Burn / Operating Income
Internally, we measure cash burn by looking at free cash flow for a quarter (operating cash flow less capex) and compare it to cash on the balance sheet to calculate a cash out date. For confidentiality reasons, we do not ask for cash balances in Scale Studio. A reasonable proxy for cash burn is Operating Income, and the chart and table below show Operating Income as a percent of Revenue at different revenue run rates.
To illustrate what this metric means, at a $20 million revenue run rate the median company in the data set is losing 63 percent of revenue, $12.7 million dollars, or colloquially “burning $1 million a month.” In Scale Studio you can also further benchmark total operating expenses one level down, across each of Gross Margin, Sales/Marketing, R&D and G&A.
The most important point to make about this metric is that in a recurring revenue business, operating income, or “burn,” however calculated is not a measure of efficiency. Instead it is a measure of how aggressively a company is investing. A high operating loss coupled with a high growth rate and high sales efficiency is an aggressive but probably sensible strategy provided of course the company has access to capital. A high burn, low growth company is a disaster in the making.
Exactly how much burn for exactly how much growth will be the subject for another post, but any comment that tries to link burn rate to value creation, without taking growth rate into account, is simply wrong. Many of the most successful SaaS companies were in the bottom quartile on this metric at $100 million in run rate revenue, but were also in the top quartile on revenue growth. Worth highlighting again is the proviso regarding access to capital. If the cash runs, out even the best business dies.
The very best companies are those such as Veeva and Atlassian where a high Sales Efficiency allowed them to simultaneously be top quartile on growth, and top quartile on operating income profitability. It is no accident that companies with these characteristics get premium public valuations.
What are your company’s vital signs?
Getting started with Scale Studio is simple: you enter nine basic data points for each of your trailing eight quarters then generate a benchmark report. The benchmarks use a sample set consisting of companies at the same revenue stage as yours. This allows for much more accurate benchmarking, especially for Revenue Growth and Operating Income which, as we have said, are a direct function of revenue run rate. The benchmarks give you a sense of your performance that is clear, concise, and comparable. Your report might say something like:
“At your current revenue run rate of $5 million, your Y/Y Revenue Growth rate of 150 percent is in the second quartile for companies of your size, your Magic Number of 0.8 is in the second quartile, your Gross Churn of -1 percent is in the top quartile, and your Operating Income of -152 percent of revenue is in the second quartile.”
Vital signs don’t cure patients, doctors do. SaaS vital signs don’t fix companies, management teams do. But realistic benchmarking metrics do what ER vital signs do: pinpoint issues, provide actionable context and allow you to get to work.
Jeremy Kaufmann contributed to this article.
Rotten Tomatoes tackles review-bombing by eliminating pre-release comments
The film-rating aggregation site Rotten Tomatoes has announced it will no longer allow users to comment on or register early anticipation for movies, following a series of coordinated attempts to sabotage the ratings on a few select upcoming films.
The most recent attack was on Captain Marvel, which received a wave of negative early ratings from people complaining about the “social-justice warrior politics” of star Brie Larson. The backlash largely stems from an interview where Larson said she was tired of being interviewed by “overwhelmingly white male” journalists and that she wanted to see more opportunities for diverse voices in the field.
A new blog post from Rotten Tomatoes’ product team announced changes to tackle this issue. “Unfortunately, we have seen an uptick in non-constructive input, sometimes bordering on trolling, which we believe is a disservice to our general readership,” the post reads. “We have decided that turning off this feature for now is the best course of action. Don’t worry though, fans will still get to have their say: Once a movie is released, audiences can leave a user rating and comments as they always have.”
This isn’t the first time Rotten Tomatoes has dealt with organized attacks on a specific film from certain groups. Star Wars: The Last Jedi was bombarded with negative reviews that also cited “SJW ideologies” as a reason. At the time, a Facebook group called “Down With Disney’s Treatment of Franchises and its Fanboys” organized a concerted action to drop the film’s audience score on Rotten Tomatoes.
One review, posted by a user named Johnny S, read at the time:
If you like stunning visuals, superficial character development, a choppy, incoherent, illogical plot, plot holes, SJW concepts such as capitalism is bad, animal abuse is bad, “make love not war”, every bad guy being a white male, and neutering every positive protagonist male in this movie, or you are only a casual fan of Star Wars, you’ll probably like this movie.
Rotten Tomatoes isn’t the only platform that has come up with measures to tackle review-bombing. Steam has also implemented product changes to dissuade users from doing it, and helping other people understand when it might happen.
Captain Marvel is just the latest example of review-bombing on Rotten Tomatoes, but the new product change also comes before Star Wars: Episode IX, for which some were expecting to see similar treatment.
Lady Gaga’s Tattoos | POPSUGAR Celebrity
Years ago, Lady Gaga made a pact with her father that all of the tattoos she got would be inked only on the left side of her body. "He asked that I remain, on one side, slightly normal," she explained to Rolling Stone. "So I only have my tattoos on my left side. I think he sees [my right side] as my Marilyn Monroe side and he sees [my left side] as my Iggy Pop side."
Two dozen tattoos later, you'd think that Gaga might be running out of space (or ideas), but the eccentric pop star never seems to tire of getting new ink. From tributes to her fans to sweet mementos for her family, all of Gaga's 24 tattoos contain special meaning — and while you can't miss some of her pieces, we're sure there are a few tats that you've never even noticed. Read on to see the beautiful meaning behind all of the recent Oscar winner's many tattoos.
Pick Your Side in Operation Grand Heist with a New Season of Content for Black Ops 4
Xbox fans! Call of Duty: Black Ops 4’s Operation Grand Heist begins February 26. Pick the side you fight for in this exciting third free season of content which includes hundreds of new items and updates for all three modes of Black Ops 4, including a Specialist making her Black Ops 4 debut, new playable destinations across the Blackout map, two brand-new Multiplayer maps, a redesigned and restocked Black Market, and much more.
New Specialist: Outrider
Provide overwatch for your heist crew with Outrider, the powerful Specialist making her Black Ops 4 debut in Blackout and Multiplayer. Eliminate your opponents in Multiplayer with explosive precision using Outrider’s devastating longbow, Sparrow. Unlock new outfits and personalization accessories in the Black Market exclusively for the Brazilian Specialist.
Breakout New Blackout Updates
Drop into the roughest, toughest, and wildest new location, Ghost Town, which features two new destinations featuring the classis map Outlaw above ground, and the chilling Zombies map Buried below ground. Additionally, Operation Grand Heist introduces a new Blackout mode called Hot Pursuit, launching on Xbox on March 1. Hot Pursuit is fast-paced and frenetic, and debuts three new vehicles – SUV, muscle car and PBR assault boat, all in special variants themed to Operation Grand Heist – as squads race to supply drops placed around the map containing high-level armor, weapons and grenades to dominate their opponents. Plus, Operation Grand Heist brings new challenges and continued releases to Blackout throughout the Operation.
New Zombies Features
Operation Grand Heist delivers new features to the Zombies mode including a new round-based Gauntlet challenge mode for the Classified map, along with a brand-new perk addition called the Ethereal Razor that substantially increases a player’s melee damage with area of effect and life re-gen for each hit.
New Black Ops Pass Content
In Operation Grand Heist, Black Ops Pass owners get instant access to two exciting and original new Multiplayer maps, Casino and Lockup. Experience the chaos of an exotic sportscar robbery gone wrong in Casino, then get mixed up in a raid to destroy evidence in Lockup. Finally, drop into Blackout with an exclusive character that returns from Dead Ops Arcade – Cosmic Silverback!
Get ready to experience Operation Grand Heist on February 26 and enjoy this exciting new season of content which includes new items and updates for all three modes in Black Ops 4.
Call of Duty: Black Ops 4 is available now on the Microsoft Store for Xbox One. Click here for purchase details.