Artificial intelligence (AI) may be more human than we all realize—at least when it comes to making mistakes. As more organizations deploy AI, there has been an accelerating number of headlines about the errors generative AI is making.
These errors are so common that there are now websites dedicated to measuring the error rate for various generative AI engines. An example is Vectara’s analysis, which benchmarks the top 25 large language models. The latest results as of November 2024 showed that the top of the leaderboard had an error rate of 1.3%, ranging up to a rate of 6.6% for the bottom of the rankings. Vectara’s analysis of prior data sets revealed error rates as high as 30%.
That’s not .0013% or .0066% or .030%—which would make them essentially rounding errors. These are shockingly high error rates, particularly given how critical a role AI is projected to play in corporate decision-making.
These errors can have far-reaching consequences for business leaders, especially when relying on AI for critical decision-making processes.
GETTING TO THE HEART OF AI ERRORS
These errors have attracted intense analysis from both computer scientists and human psychologists alike, and the most common term for these errors comes straight out of a psychology textbook: AI hallucination. The hallucination term has gained traction in technology media, but I think the more accurate term for these errors is AI confabulation, as this article in Psychology Today argues.
Calling these errors “AI confabulation” more accurately gets to the heart of the errors revolving around how AI interprets commands, data, and context. AI isn’t seeing hallucinations like they are floating through the video for Lucy in the Sky with Diamonds or watching Laser Floyd at the planetarium.
AI is doing something far more common among humans: mishearing requests, misinterpreting information, filling in gaps with mistaken assumptions, making clumsily logical leaps, and coming to the wrong conclusions. Furthermore, some of the errors seem to be driven by false confidence or even outright fabrication (aka lies). How very human of AI to be guilty of so many kinds of fallibility.
THE REAL-WORLD IMPACT OF AI CONFABULATION
These AI confabulations aren’t simply an academic curiosity, though. They are having a significant real-world impact on organizations. In the past few months alone, articles like this, this, and this have discussed the significant risks AI confabulation poses to the success of AI investments and even to the reputation of companies using the technology.
One thing is clear: Business leaders need to understand the issue and have a strategy for mitigating these risks.
HOW TO MITIGATE AI CONFABULATION
To identify strategies for mitigating these risks, it is instructive to look at an application of AI that utilizes highly technical data because the potential for confabulation increases as the complexity of data and queries increases. AI analysis of location-based data is a great example not only because of the complexity of the data but also because how you make queries is so critical.
Location-based data often exists in structured and unstructured datasets across multiple computing systems in an organization. The data is also highly technical, often with multiple layers of visual, numerical, textual, and geographic information that is highly contextual. This poses significant challenges to AI analysis that increase the risk of confabulation, but there are ways to mitigate that risk.
For business leaders, implementing strategies to address AI errors in such complex data sets can be the difference between reaping the benefits of AI or encountering costly setbacks.
STRATEGIES FOR REDUCING CONFABULATION
1. Use retrieval augmented generation (RAG)
One of the best weapons against AI errors is a big dose of context that gives the LLM domain knowledge that mirrors the expertise of an experienced industry professional.
RAG equips LLM models with domain knowledge they would not otherwise have from the public data they are built on. This helps the LLM better understand the data and thereby produce better results.
2. Have a data strategy to analyze data quality
Unfortunately, data quality can vary greatly, particularly for technical data like location-based information. Not all data is AI-ready, and using data that is lacking in accuracy, timeliness, and richness will undermine the accuracy of AI-driven analysis.
Having a data strategy that assesses the quality of data, enhances its accuracy and richness, and breaks down barriers to timely access is an important way to reduce AI confabulation.
3. Ensure queries are as precise as possible
Another important tool for mitigating confabulation is enhancing the way AI queries are asked for location intelligence use cases. Given how complex location-based data is, the precision of queries can make or break the success of a given task.
To make these queries as precise as possible, I recommend leaders have an engineer serve as a second set of eyes for queries, ensuring that they will translate from natural language into SQL instructions that minimize the chance for confabulation. This is akin to putting your best geospatial/location intelligence professional at your workforce’s side.
4. Use a knowledge graph
A fourth important tool for mitigating confabulation is to measure, teach, and repeat through the use of a knowledge graph. LLMs are built to learn over time, and that is one of our best weapons against confabulation.
Using a knowledge graph, your organization can perform fact verification to identify areas where mistakes are happening and where fine-tuning needs to happen. Simply put, it uses a model to check the model and teach the model—accelerating the learning curve for LLMs in ways that reduce errors. This is akin to providing 24/7 tutoring to your LLM from your best geospatial/location intelligence professional.
This approach is also a blueprint for lowering the frequency and severity of confabulation for other use cases of AI.
AI NEEDS MORE WISDOM
Yes, AI is fallible just like people are. Just like many people, it is very smart but doesn’t know what it doesn’t know. What it needs is more wisdom—wisdom that helps it make smarter decisions and fewer mistakes. And these strategies give us the power to help it become wiser.