Skip to content
Fast Company Executive Board

The Fast Company Executive Board is a private, fee-based network of influential leaders, experts, executives, and entrepreneurs who share their insights with our audience.

Hidden costs are enterprise AI’s next challenge

Change your approach to manage your AI bills.

[Photo: Getty Images]

Profile photo of Kiva Kolstein

President and Chief Revenue Officer · AlphaSense

Actual usage dictates today’s AI budgets. What starts with a handful of a company’s users experimenting with a new tool can quickly evolve into dozens of workflows, hundreds of employees, and thousands of prompts every day. And as AI usage grows, the bill grows with it.

More visibility gives finance, revenue, and technology leaders a much better understanding of AI resource use and fast-growing costs. Visibility also introduces tough questions: Which workflows are creating value? Which are just consuming compute? Where is AI activity misaligned with business impact?

As the conversation shifts to economics, leaders want to understand what they’re getting from AI credits consumed and whether investments produce meaningful business outcomes. While AI should increase business value, if rising token consumption outpaces business impact, you may no longer be able to justify the investment.

Why AI costs scale faster than teams expect

You ask a question, and a well-reasoned answer appears a few seconds later. Even better, an agent has an answer before you prompt it. On the surface, the interaction feels simple.

Underneath, a great deal happens before that answer reaches you. The agent loads instructions, retrieves information, and assembles context. Agents may check multiple sources, rerun tasks, or process the same information more than once.

Each step consumes tokens, or the small units of text an AI model reads and writes. Tokens are what you actually pay for, so the more steps, the higher the bill.

Since January, mentions of “token costs” in corporate documents such as earnings transcripts have nearly tripled, according to our data. Many organizations are learning this the hard way, with nearly 8 in 10 IT leaders reporting they were surprised by charges due to AI models or how much the models were used. Most of an AI’s work happens out of sight, so costs are only reconciled when the invoice arrives.

Most AI platforms require organizations to piece together the underlying information layer themselves. This relies on different data sources, business applications, and access permissions across multiple tools.

As a result, companies must ensure the AI pulls the right information, preserves the right context, and produces answers that can be verified if challenged. In high-stakes environments, every additional connection introduces more complexity, more opportunities for error, and ultimately more cost.

The cost of accuracy

Many organizations also encounter unexpected costs. An AI platform may seem inexpensive when evaluated on subscription price alone. A forecast built around a handful of use cases changes once AI becomes part of daily workflows. As a result, consumption and cost can accelerate in ways that are difficult to predict.

In complex environments, inefficient architecture can lead the model to undertake extra work just to produce a reliable answer, when the real question is “Can I trust and defend what it tells me?”

A workflow that appears inexpensive during a pilot can look very different as teams add more data sources, load more context into prompts, and introduce additional validation steps. A reliable answer requires extra tokens with every additional search, trip between systems, and step needed to refine the answer.

Why quality becomes an economic issue

Enterprise AI’s deeply embedded assumption is that if you connect a model to enough information, intelligence will follow.

In practice, that assumption breaks down more often than many realize. A system can connect to dozens of sources and still miss the most important piece of evidence. It can retrieve information without preserving context, and surface facts without understanding why those facts matter. The answer may look polished, cited, and complete, while still missing the critical signal necessary to make a confident decision.

The challenge isn’t always visible. Weak answers often look strong. They read well and may even include citations. A prompt 20% less efficient can drive costs 200% higher as models spend more tokens compensating for weak retrieval and noisy context. These small inefficiencies add up quickly.

Organizational growth becomes harder to sustain when AI consumes more resources without creating proportionally more business value. That’s when rising token costs stop being a technology issue and start becoming a business liability.

Token efficiency starts with architecture

In less than a year, a technical discussion about tokens has become a boardroom priority, showing up in budget reviews, operating strategies, and conversations about return on investment.

Every architectural decision has downstream consequences. Content quality, retrieval design, context preservation, and how information moves through the system all influence what AI must do before producing an answer. Those decisions shape how accurately chief information officers forecast AI demand and how confidently chief financial officers allocate capital behind it.

Bringing together trusted information and context into a single intelligence layer reduces unnecessary work throughout the system before an answer ever reaches the user.

The result is a system that scales more predictably, creates much better token efficiency, and gives teams answers they can trust without introducing unnecessary complexity and manual reviews as AI usage grows.

Build discipline around AI spend

As AI usage grows exponentially, executives and IT leaders must ensure business value grows alongside it. Enterprise AI’s next phase will demand greater operating discipline from leaders. Managing token costs must become an ongoing review of how teams use AI, where the system introduces unnecessary complexity, and whether the architecture helps teams reach trusted answers efficiently.

As chief information officers optimize architectures, chief financial officers will have greater confidence that AI investments translate into measurable business value.

Organizations building that discipline early will better handle AI spending and enable systems where efficiency improves as usage grows. This creates an advantage that scales effectively over time, rather than creating unpredictable, runaway costs that compound.

Kiva Kolstein is president and chief revenue officer at AlphaSense.