Skip to content
Fast Company Executive Board

The Fast Company Executive Board is a private, fee-based network of influential leaders, experts, executives, and entrepreneurs who share their insights with our audience.

Agentic AI Workflows: Who Is Accountable?

Enterprises fail to budget for AI-ready data that demands consolidation, governance, security, and indexing.

[Photo: Getty Images]

Profile photo of Greg Statton

As companies rush to adopt agentic AI to speed up business processes and improve efficiency, AI and data governance are driving up costs.

Mistakes are costly. In one case, a global financial services firm created an AI agent to reconcile anomalies in its accounting ledger. Still, a null value caused the tool to enter a recursive reasoning loop, generating more than 15,000 high-cost API calls in under an hour. The total bill: about $50,000 and a locked database, according to Google’s AI risk and resilience report published in September.

Better data governance and controls could have blunted the impact, but companies have not prepared their data for AI. Sixty percent of AI projects that lack AI-ready data will fail, according to Gartner. Poor data management and governance practices also lead to AI inefficiencies and a lack of return on investment, a major issue as token usage is expected to increase 24-fold over the next four years.

The problem for companies is that the cost per terabyte of data remains unknown. The consolidation, governance, security, and indexing required to make enterprise data AI-ready represent a hidden cost that most organizations are not tracking and, in most cases, have not built. Data is not a static asset either: The ongoing costs of storing and continuously updating data are rarely included in ROI calculations.

Companies should be tracking the fully loaded three-year cost per terabyte—the cost of AI data readiness. When they don’t track this, it eats away the majority of value created by using enterprise AI. Agentic data issues such as context bloat, retry loops, and cache misses can lead to a rapid increase in costs—in some cases, up to 30 times the cost per request, even as per-token costs decline. Much of this is due to poor data readiness and a lack of governance.

DATA IS FUNDAMENTAL TO AI PROJECTS

In many ways, the current AI moment resembles early cloud adoption with shadow IT spinning up resources, employees bypassing governance in pursuit of efficiency, and companies experiencing growing pressure to modernize workloads to realize the full cost benefits. Companies adopting modern cloud frameworks are able to create predictable and manageable IT operations.

In addition, well-managed data infrastructure is more defensible. Cyberattackers are already using AI in many aspects of their attack chains. In one instance, an AI-assisted threat actor used markdown playbooks and an automated reconnaissance and credential management framework, known as “Recon.” That helped it gain access to a network and steal more than 23,000 secrets, such as API keys for cloud and AI services, according to a description of the attack published by Google.

Data governance and management could not only detect such unauthorized access, but also ensure that, before an attack, the data is properly provisioned and protected.

The ultimate goal is to be boring in the sense of making AI’s capabilities, costs, and ROI predictable. As Singapore’s minister for foreign affairs, Vivian Balakrishnan, told CNBC in a recent interview, making technology and business environments trustworthy can lead to lower transaction costs.

“Trust is basically a way of lowering transaction costs,” he told CNBC. “Being predictable, being boring, being reliable, being trustworthy has got real value.”

EXPAND EXISTING TOOLS

The industry is spending a lot of time selling two different aspects of enterprise AI to customers: the dirt-to-first-token startup time, and custom harnesses to eke out the most measurable performance from AI. Both efforts rely on good, governable data, and that’s something that nobody has been talking about because there is a perception that it is hard to do. Yet the capabilities to measure compute and storage costs in the cloud, clean and protect data, and manage users’ access to AI services are built into much of the technology most enterprises have already deployed.

Take backup software. It’s one of the few enterprise tools that touch every application across data centers, clouds, and remote sites. It already consolidates, de-duplicates, indexes, and normalizes data. By leveraging this trusted technology on intelligent platforms, businesses gain greater ability to manage their data, track access to data stores, and set governance policies.

In its report on AI project failures, Gartner recommended that companies expand their existing data management practices that have already proven effective. Companies should add the necessary AI-specific data features to support agentic operations, such as vector data stores, chunking, sampling, embedding, and integration with retrieval-augmented generation (RAG). Businesses should also not view data management as a project to complete, but as an ongoing effort. Data always changes, and new AI capabilities may require new types of data structures.

In the cloud era, missteps took days or weeks to compound into significant problems. With AI, they take hours. The organizations that succeed will have the strongest guardrails on the data feeding them.

Greg Statton is VP and CTO of APAC at Cohesity.