I recently read a blog posts that suggested that big data was going to revolutionize management science. It suggested that historically, businesses have been driven by management fads that were merely hypothesis-based approaches backed up at best by a few case studies. In the future, with access to unbiased data, we will be able to move to an evidenced-based management approach and through hypothesis and management fads to the wind.
While I do understand the author’s point, there are a few challenges with this thinking.
Is data really unbiased?
Let’s start first with the data itself. The notion that there can be unbiased data and the analysis is dubious. The decision on what data to gather, how it is structured and the methods used to analyze it are all based on someone’s hypothesis around what types of findings and insights they anticipate deriving from the data. In relatively few instances can one capture all possible data and run all possible forms of analysis on it to gain every possible insight. That takes too long and is too costly. As businesses continue to push for more rapid responses to the market, some of which are resulting in near real-time actions, it is imperative to hypothesize and prioritize for the sakes of both efficiency and speed.
Should evidence-based management be the goal?
Now let’s address the desirability of evidence-based management. If we are talking about a health care situation, I absolutely want the decisions the doctors make to be evidence-based. Please show me that this treatment has been tried preferable thousands of times on people with the same issue I have and the results have consistently been successful. No trial and error desired here.
However, there is no single most desirable answer that is applicable to all businesses as there is for doctors working to return their patients back to health. In fact, the data gathered by most businesses will already be influenced by past decisions the company has made regarding their product and service offering. The decisions they are looking to make as a result of analyzing the data must also vary by company. A mere repeat of past behavior that drove current success is insufficient insight into what is required to deliver a differentiated offering that is responsive to ever changing market demands.
In other words, there must be some element of strategic direction, based upon hypotheses that are informed by, but not restricted to, past successes to shape a successful future.
Importance of striking a balance
The wider adoption of cloud enabled big data is obviously providing companies with the ability to analyze massive amounts of data more quickly and cost effectively than ever. It is becoming rare that a management team must rely upon a decision made purely based upon a hunch. There is likely some data available to provide some insight into the approach being considered. This should reduce the number of unsuccessful outcomes.
At the same time, data should not be considered a substitute for the wisdom, creativity, and gut feelings that senior leadership brings to the table. Big data should be considered another tool – arguably a potentially very powerful one – that is available to management to drive thoughtful decision-making about the future of their companies that are both sufficiently competitive and responsive in the market.