Skip to content
Fast Company Executive Board

The Fast Company Executive Board is a private, fee-based network of influential leaders, experts, executives, and entrepreneurs who share their insights with our audience.

The AI industry fixes the wrong problem

The problem is one layer down, when truth degrades.

[Photo: Getty Images]

Profile photo of Michael Quoc

Founder & CEO · Product.ai

A few months ago, I spent three hours researching a fitness tracker.

It wasn’t a complicated decision. I just couldn’t tell if the information I found about any tracker was true. I ran searches, read reviews, watched comparison videos, and scrolled Reddit threads looking for someone who had actually used the thing.

By the end, I had more data than when I started and less confidence in any of it.

Everyone has done this. The systems we rely on to connect us to the right answer, including search engines, review platforms and recommendation algorithms, were optimized for engagement. For clicks. For time on page.

When attention is the business model, noise wins. Spam outcompetes authenticity, hyperbole outranks accuracy, and the honest answer to a simple question gets buried under content engineered to perform. That was already the problem before AI. Now AI is making it structurally worse.

Relevance isn’t truth

Most AI conversations treat hallucinations as flaws, or bugs that better training will eventually patch. Yet, I would argue that hallucination is the system doing exactly what it was built to do.

Large language models are relevance engines and attempt to predict what a plausible answer looks like, without access to verified information. The most likely next word, phrase, or sentence given a context. This is genuinely powerful but also categorically different from knowing what is true.

A relevance engine asks, “What sounds right?” A truth engine asks, “What is right?”

You cannot solve the second question by getting better at the first. AI does not have a verified layer underneath it.

For entertainment, that’s fine. But it’s a problem for decisions that cost you money, health, or time. AI needs ground-truth information it can actually reason from, not plausible-sounding content it was trained on.

The missing floor

Think about what Wikipedia did for the early internet. While imperfect, it was built on a system that continuously pressure-tested claims against sources. When you searched for something, Wikipedia surfaced near the top because it was one of the most reliably grounded pages on the internet. It gave the rest of the internet a floor to work from.

The AI economy does not yet have that floor.

Right now, AI systems operate within an increasingly synthetic information environment loop. Content generated by AI, optimized for AI, then fed back into AI.

A model trained on plausible-sounding content produces plausible-sounding content that gets scraped and used to train the next model. Truth degrades at every iteration. What is left is the confident-sounding average of the internet, and that is exactly not what you want when you are trying to make a real decision.

I call this the beige singularity, where everything collapses toward the noise-averaged mean, delivered with complete conviction. The stakes grow higher as AI moves from generating text to taking action, including making recommendations, booking services, executing purchases, and coordinating tasks.

An agent that acts on plausible-but-false information wastes your time, costs you money, and sends you to the wrong doctor. It buys the wrong thing. The gap between “sounds right” and “is right” matters enormously when the system is doing things on your behalf.

We’ve been here before

Social media arrived faster than trustworthy infrastructure to make it trustworthy, as companies leading the charge optimized for growth. We all know how that played out, and we still live in that world.

AI’s broader shift is prompting companies to make decisions now about what these systems are built on and whose interests they serve. The decisions will be harder to undo than a content policy update.

Building the alternative requires solving the hard problem: Verifying truth at scale, in real time, in a way that resists the adversarial pressure to game it.

Only better infrastructure will fix this. And it is the part of the AI economy that is not getting nearly enough attention.

Michael Quoc is founder and CEO of Product.ai.