Skip to content
Fast Company Executive Board

The Fast Company Executive Board is a private, fee-based network of influential leaders, experts, executives, and entrepreneurs who share their insights with our audience.

6 innovative ways leaders can better measure AI ROI

Here are six unconventional AI ROI metrics that accurately capture the true impact of an organization's AI investments. 

6 innovative ways leaders can better measure AI ROI

[imagineRbc / Adobe Stock]

Profile photo of Ismail Amla

Senior Vice Preside, Kyndryl Consult · Kyndryl

More than a year after the hype around artificial intelligence (AI) kicked into high gear, global business leaders are still waiting to see their investments pay off.  

That’s not to say global C-suites are starting to feel bearish. Most businesses are still investing in both traditional AI—which includes machine learning—and generative AI. Even still, only 42% of leaders currently see positive ROI on their AI investments, according to my company’s recently published Kyndryl Readiness Report.

The sentiment is justified, as a clear discrepancy exists between expectation and reality. To date, no major productivity lift from AI has borne out in economic statistics. Perhaps we’ve been looking for gains in all the wrong places.

With that in mind, I propose some unconventional AI ROI metrics that more accurately capture the true impact of an organization’s AI investments. 

1. TRACKING “RETURN ON DISRUPTION” (ROD)

Start by measuring how AI disrupts and redefines internal workflows or business models. Look for manual processes eliminated or modified, improvement in decision-making, or the speed at which processes can run. The more “disruption,” the higher the value AI brings.

Consider a healthcare example. An October 2024 study in the Journal of the American College of Radiology found that AI in radiology delivers strong financial returns. By comparing the impact of AI-use versus no AI-use in radiology workflows over a five-year period, researchers could measure the impact of AI’s “disruption.” The ROI calculator showed a 451% return, jumping to 791% when counting radiologist time savings. The AI system also caught over 1,200 critical problems that doctors would have missed otherwise.

2. CONSIDER THE “AI AGILITY QUOTIENT” (AIAQ)

Another strategy is to evaluate AI ROI based on readiness for future challenges. This metric could include the number of emergent opportunities that AI helped capture and the frequency of rapid adjustments an enterprise can make via AI insights.

Printer-maker Lexmark developed a machine-vision quality assurance system called Optra Edge Visual AI. The system processes live video feeds from assembly line cameras, performing real-time detection, classification, and segmentation of objects. The system worked across multiple production tasks, including analyzing front covers and model badges, tracking photoconductor drums, and managing pallets. Running on-site computation via the edge on their factory floor, engineers quickly adapted it for different inspection needs. Performance metrics proved its success: 40% faster inspections with 99% accuracy, with the ability to run 24/7 and instantly alert operators about defects through existing manufacturing systems. What started as an internal solution paid for itself in five months, prompting Lexmark to transform it into a product they now sell to other manufacturers.

3. DEPLOY A “TEAM CREATIVITY INDEX” (TCI)

Consider Amazon‘s “Hands off the Wheel” automation initiative which automated routine tasks in retail management—like inventory, pricing, and warehouse stocking decisions—to free up employees for more creative work. The system evolved from one where employees could override automated suggestions to 80% automation of pricing and purchasing decisions, among other tasks, by 2016. While employees initially worried about their changing roles, Amazon reassigned them to product and program manager positions focused on innovation and providing more space for creative work better handled by humans.

4. ADOPT THE “AUTOMATION FATIGUE RATIO” (AFR)

Instead of looking at direct cost savings or efficiency gains, measure how AI reduces fatigue and burnout by automating repetitive tasks. Survey employees on perceived workload reduction and use metrics like fewer burnout-related absences, higher job satisfaction scores, and fewer reported incidents of automation errors to calculate AFR.

For example, a five-week quality improvement study at Stanford Health Care examined how healthcare providers adapted to using AI-generated drafts for patient message responses. While participants didn’t reduce the time spent on messages, they reported that the AI significantly improved clinician well-being metrics. The task load and exhaustion scores dropped substantially. The upshot? AI integration didn’t save time; it reduced perceived workload and burnout among healthcare providers.

5. EMBRACE THE “RISK AVERSION REDUCTION INDEX” (RARI)

AI can enable bolder moves by reducing perceived risk. Think about adopting RARI as a metric, which involves tracking the number of higher-risk, high-reward projects undertaken after AI implementation, and the success rate of these initiatives compared to previous risk profiles.

Look at JPMorgan’s COIN (Contract Intelligence) system which combines AI efficiency with human expertise. The system uses unsupervised learning to analyze credit agreements across 150 different attributes with greater accuracy than human reviewers. What takes hours for a human to review takes seconds for COIN. By automating the review of 12,000 wholesale contracts and reducing loan-servicing mistakes, COIN enabled JPMorgan to handle larger loan volumes more confidently. Rather than replacing workers, the bank redirected lawyers and loan officers to more complex risk analysis tasks, creating a powerful combination of machine precision for routine review and human judgment for sophisticated decision-making. 

6. “CYBORG SYNERGY INDEX” (CSI)

Taking a page from AI expert and Wharton professor Ethan Mollick’s paper about AI Cyborgs, this metric seeks to quantify synergy between human workers and AI. CSI is designed to measure joint outcomes that neither AI nor humans could achieve alone, such as hybrid solutions or surprising problem-solving methods enabled through the human-AI partnership.

A 2024 Harvard study of 194 physics students found those using an AI tutor learned twice as much as peers using traditional learning methods. Available 24/7, the AI tutor provided personalized feedback and self-paced learning. The students could engage with material without judgment, and reported higher engagement and motivation levels. The success was notable as it occurred in an already well-taught course. As a result, the university looks to pilot similar chatbots in other courses.

It’s not surprising that AI ROI is also hard to pin down. New technologies rarely impact productivity or sales figures until years after introduction. Remember, productivity gains from the personal computer, the smartphone, and other technologies are still debated by economists. Using these non-traditional metrics, businesses can design better ways to measure the ROI of AI and gain a deeper understanding of AI’s impact beyond conventional financial indicators. This is critical when dealing with frontier technologies that may radically redefine modes of work.