Beyond HumanBig PictureCatalystsConnected WorldExchangeMarketing MixNew MoneyNew SchoolPeople SciencePulse
Company Name
Job Title

Why the smart money is on AI; The future of sales forecasting

sales forecasting sales forecasting

The average accuracy rate of sales forecasting is lower than a coin flip. Could a data and artificial intelligence led revolution fundamentally change the process forever?

The Princeton University professor, Burton Malkiel, famously claimed in his bestselling book, ‘A Random Walk Down Wall Street’, that:

“A blindfolded monkey throwing darts at a newspaper’s financial pages could select a portfolio of stocks that would do just as well as one carefully selected by experts.”

Subsequent studies – which modelled a random selection of stocks over an extended period – actually showed that the ‘monkeys’ outperformed the experts in many cases, highlighting the sheer unpredictability of the stock market.

What if the same was true for sales forecasting?

Surely, businesses across the globe aren’t making revenue predictions that warrant the analogy of blindfolded monkeys throwing darts.

The sad reality is that, in many cases, they are.

Industry-wide research carried out by The Bridge Group highlighted that forecast accuracy, or lack thereof, was the second biggest challenge for sales leadership (after increasing productivity). Further work from CSO Insights revealed a miserly 46% of all sales forecasting was deemed accurate; in other words, a lower hit rate than a coin flip.

There are few other areas of business that are afforded such a high fail rate, particularly when you consider the impact of such significant inaccuracy. A missed forecast can cause low employee morale, job loss, diminishing shareholder value, and ultimately company failure.

A startling example of inaccurate forecasting occurred in February of this year. When LinkedIn announced revenues for the first quarter of 2016 of $820 million, it missed its estimates of $867 million by $47million. In the absence of any guidance, 30% of the company’s market cap was wiped immediately.

James Cakmak, an analyst at Monness Crespi Hardt & Co, when questioned on the dramatic share price tumble, stated, “In this market, there’s no mercy for a miss.”

How could a company which generates such a high percentage of its revenues through a predictable subscription model report a discrepancy of such magnitude from its estimates?

While there is a general acceptance that forecasting is by its very nature, unpredictable, such a low level of accuracy means current sales forecasting should, finally, be pulled into question.

“46% is disturbing, especially because the VPs of Sales sitting on top of that inaccuracy will probably disappear when the accountability reaper appears,” said Andreessen Horowitz Partner, Mark Cranney, who believes better results come from redoubling efforts on process, rather than an over-reliance on data.

“If you roll up your sleeves and shine a bright light on why the average sales forecast is so inaccurate, you will probably find a lack of maturity and/or scrutiny in several areas. You can start to fix forecasting issues by taking complete inventory of the sales process. Technology is replacing or enabling many company-internal and customer-facing workflows, but I don’t believe technology will ever completely replace the expert intuition and granular processes that sales organizations need in complex enterprise selling environments.”

Cranney represents the point of view that process – and the level of management talent associated with implementing it properly – combined with the intelligence gained from integrating technology is the key to ensuring organizations can consistently produce predictable results.

A theory has emerged that the challenge of reliable sales forecasting has less to do with management skill and more to do with what managers don’t (and can’t) see, because it is external to their operations.

Collective[i] co-founder, Stephen Messer, believes that sales processes like internal sales forecasting give the illusion of being correlated to the quality of sales management, but that they are in reality irrevocably flawed.

“If companies like LinkedIn, a technology company with unprecedented access to the best talent in the world, can’t predict their own revenue, there’s something larger at work here,” says Messer.

That something, in his view, has to do with the fact that forecasting needs to be informed by internal and external patterns of activity, looking both forwards and backwards – something that only machines with access to troves of cross-sector data can accomplish.  

What’s happening within an industry, the economy, or even changes specific to individual customers are likely to have more of an impact than the process a manager uses to evaluate the revenue their teams are recording in CRM. Put another way, no manager would have enough insight into real-time market changes to accurately predict revenue through their current forecasting process. As a basic example, Messer points to the fact that forecasts are generally established on a weekly basis, typically on a Friday.

A flaw in itself.

This rigid weekly cycle lacks agility, and is a far cry from the real-time analytics that are used to great effect in other functions, such as marketing, finance and increasingly, HR. By producing a forecast once a week, it becomes viewed as a static probability when in fact, it’s more of a snapshot that changes on a daily basis depending on the internal activities of team members.

This also means that disruptive, macro-events – such as Brexit in the UK or the recent attempted coup in Turkey, both of which had a huge economic impact and can threaten the likelihood of a deal closing – are left unaccounted for until the following week’s review.

Pablo Dominguez is VP Global Business Operations for NYC cloud advertising platform AppNexus. His company is focussed on data-driven results leading to more robust forecasting processes, and, as such, he feels current methods are in need of an upgrade. “I think we’re at something of a turning point in companies – or at least we should be – where the traditional forecasting process is not really yielding results that are measurable or that you can do anything with.”

“The question is; is there a better way for people with data-driven insights to provide an actual forecast number that is more accurate versus wasting the commercial team’s time? The answer is undoubtedly yes.”

If you believe that combined data sets, automation and artificial intelligence are the only way to accurately forecast revenue, the sheer time cost sunk into the traditional forecasting process is an astounding waste of resources. Given that forecasting can take up to a day to complete per sales professional, that’s 20% of each individual’s working week allocated to a non-revenue generating activity.  In fact, shortcomings in accurate forecasts can often lead to loss of revenue in the form of end of quarter discounting, in last ditch attempts to correct for an inaccurate forecasting process. 

The conversations that occur between sales professionals, their managers, and ultimately sales leaders is all time that could be spent on generating leads and closing deals rather than speculating about the likelihood of a sale.

In Messer’s words, “whether you believe that forecasting is art or science, you can’t ignore the reality that the 20% of time sales managers and operations teams spend compiling a forecast takes away from generating revenue. With a 46% accuracy rate, there’s an obvious upside to using machine generated intelligence to predict outcomes”.

When considering explanations for that 46% hit rate, perhaps the single greatest concern with forecasting is the distance the person making the final call is from the deals themselves.

Take the typical hierarchy of knowledge within a sale, running from the buyer, to the sales representative, to their sales manager, and finally to senior management.

Clearly, the buyer has the most information on the likelihood of a deal closing, which means that each passing of the baton within this information flow is somewhat distorted by a combination of opinion, perception, ego, and lack of insight into what is actually happening.

This (sometimes literal) ‘telephone game’ effect means that by the time a sales leader submits the final forecast for the next week across multiple salespeople, the low percentage of accuracy amongst sales forecasting starts to become more understandable.

Adds Pablo Dominguez, “You’ve got to hold managers accountable for the conversations they have with their teams. There’s a human element that you can never completely fix, but [with the right technology] you can get much closer to the pin, as opposed to being 20 points away. If the data shows that 60% of what you continue to commit gets pushed out to the next quarter every quarter, well, now I know you’re unable to accurately forecast 60% of the time”

The notion of an ego-driven sales person is not a new one. David Mayer and Herbert M. Greenberg writing in Harvard Business Review noted that powerful ego drive was one of the fundamental ‘two basic qualities’ that underpin any successful salesperson. As such, untangling the perception that sales projections are more often than not misguided might feel like a particularly stinging attack on an experienced sales manager is competency or ability. However, the issue at stake is of the quality of data offered to make sales forecasting truly valuable, rather than the acumen of any particular individual.

“Using the right technology, no company should have a LinkedIn-esque surprise. I’m often asked why more companies haven’t adopted this approach. One of the key challenges is getting experienced sales managers at senior levels to accept that a machine or new piece of technology can do a better job than they can in this specific aspect of the sales process,” added Messer.

Messer’s theory is especially appealing when you consider that according to McKinsey, more than 40% of public companies miss earnings targets within three days of reporting. So why do so many companies continue to rely on process and blame management for unpredictable misses? According to Messer, it is for the same reason – the accuracy of the sales forecasting process being tied to the manager themselves. The illusion of control. The bias we have that human decision making is better than decisions guided by data.

According to Messer, it is for the same reason – the accuracy of the sales forecasting process being tied to the manager themselves. The illusion of control. The bias we have that human decision making is better than decisions guided by data.

“Someone who has been doing sales for a number of years, in some cases decades, understandably believes that they are best positioned to judge the likelihood of deals closing. They naturally amplify the value of their experience relative to the extent of the unknown. An analogy we often employ is that of how most people believe they are good drivers, when in fact, the biggest factor in road safety is the quality of the car. For sales, the biggest factor is the quality of sales intelligence, not human judgment. It’s very simple, the better the intelligence, the more effective the manager, not the other way around.”

Assuming Messer is right, sales forecasting is about to undergo a radical transformation where a blend of automation, machine intelligence, and human judgment will replace manager specific sales forecasting processes, long considered an essential investment of management time.