Data is abundant—so abundant that it is sometimes hard to imagine its scale. In 2020, global data volumes were estimated at 64.2 zetabytes. By 2025, that figure is expected to nearly triple, hitting 180 ZB. If anything, the data volume curve is bending upward as new technologies reach the market. For example, autonomous vehicles may generate 1.4 TB of data every hour.
But enterprises have long struggled to fully unlock the value of data across an organization. Valuable though it may be, data often sits siloed, unstructured, and unused. This has raised a paradox in big data circles: If data is so valuable, why do nearly half of projects never make it out of the pilot stage?
A recent report from 451 Research, titled Profitable Outcomes Linked to Data-Driven Maturity and commissioned by BMC, suggests that things may be changing. Organizations are doubling down on their investments in data management despite an uncertain economic climate, and I think we may be seeing the emergence of a virtuous cycle. Data management strategies yield success, which begets more investment, which leads to more success, and so on.
It’s a transformation that I am excited to see, in part because of the downstream effects. The growth of data may be changing the way leaders make decisions.
THE CHANGING SCOPE OF DECISION-MAKING
Agility And Democratization
Leaders need the right data at the right time. That’s a clear driver behind the growth of self-service data programs. Empowering leaders to seek their own data increases agility while fostering more evidence-based decision-making.
Approximately one-third of respondents reported that their organization has a self-service data program. But another 25% report that they are either in the pilot stages or plan to have one within the next six months. Clearly, these programs are gaining traction because they can support rapid, strategic decision-making. Indeed, the top benefit that respondents to the survey foresaw from self-service programs is more agile decision-making—cited by 61% of respondents.
Resiliency
Few companies, if any, were fully prepared to deal with the turbulence of the global pandemic. And yet, some companies adapted quickly. The ability to leverage digital tools was a key differentiator. According to the International Monetary Fund, in the immediate aftermath of the pandemic, “digitally-enabled firms faced a lower decline in sales by about 4 percent than less digitally constrained companies.”
For executives, the pandemic—in addition to being a global tragedy—was a case study of the need for resiliency planning. When executives evaluate data management programs, they aren’t only asking how it might help increase sales or customer satisfaction, but how the data can be used to make the company more adaptable and resilient, and how it can anticipate changes in market conditions.
CHALLENGES REMAIN
While continued investment in enterprise-wide data management strategies has increased, especially among those with existing data programs, organizations may continue to struggle in this area.
Information, for example, remains siloed. Only 36% of respondents have a data fabric technology or tool that makes information available across platforms and functions. But a larger set of respondents said their organization plans to implement one within the next year.
The democratization of data may also encounter bottlenecks because organizations are likely to have far more data consumers than data suppliers. If data consumers do not have the skills to extract, analyze, or interpret the information they need, they may place pressure on their upstream colleagues for additional support.
Finally, and perhaps most importantly, it is important to note that the successes we are seeing aren’t always spread throughout the organization. Organizations tend to have the most experience with data in areas that focus on profitability and revenue, and less so with areas dealing with operations and processes.
THE HALLMARK OF SUCCESS
Overcoming these challenges requires a new strategy that makes data management more efficient. We find that pilot projects help companies understand what works and what doesn’t. These learnings often provide a basis for continued improvement.
Organizations should focus on automation of data flows into an enterprise data warehouse, to make data as standardized, clean, and usable as possible. Organizations should also focus on tooling that accelerates analyzation and visualization of data. My own company has experienced this shift—for ourselves, and our customers. Each week, more than half of my firm’s employees access our enterprise data warehouse. We’ve saved 40 to 50 days per year in sales operations.
DataOps practices, an emerging capability that borrows workflow practices from the world of DevOps, can help most organizations overcome these challenges. But done right, organizations may be able to maximize the benefits of data throughout the organization.
Ayman Sayed is the CEO of BMC Software.