Skip to content
Fast Company Executive Board

The Fast Company Executive Board is a private, fee-based network of influential leaders, experts, executives, and entrepreneurs who share their insights with our audience.

Why multisensory AI holds great promise

Combining senses to improve understanding will define the next era of technological possibility

[Photo: Getty Images]

Profile photo of Ismail Amla

Senior Vice Preside, Kyndryl Consult · Kyndryl

Today, a shopper struggling to read a cereal box in a grocery aisle may pull out their smartphone, open an accessibility app to capture the text, and wait for it to be read aloud. Soon that same shopper may instead ask their earbuds for help—because the wearable device sees what the shopper sees. No phone, no app, no aiming. The device is fully present in the moment.

Companies are reaching late-stage testing on artificial intelligence-enabled devices that act as both eyes and ears. This is a key moment in the evolution of AI technology: It points toward a future in which AI does not just hear or see but also combines senses the way humans do to generate far more knowledge and understanding than traditional technology could ever offer. 

As consumer technology is rewired, people will move closer to experiencing fully multisensory AI—an interface shift that will be among the most consequential evolutions since the smartphone.

WHY COMBINED SENSES COMPOUND UNDERSTANDING

Human cognition has always been multisensory. Combined senses produce understanding that no single channel can deliver alone. A chef tastes with their nose as much as their tongue, because much of what we call flavor depends on our sense of smell. A mechanic diagnoses an engine by listening for specific sounds, picking up on strange odors, and feeling how the housing vibrates. A nurse evaluates a patient by visually observing posture, using physical touch to measure pulse strength, and listening for any breathing difficulties.

When sensory channels are combined, perception and accuracy compound in ways no single input predicts. Research on multisensory integration has documented this for decades. When the brain receives correlated information from two senses, neurons fire at rates well above what either input would produce on its own. That neural boost translates into greater precision: Sight, touch, and sound together deliver a fuller picture than any one sense could alone.

 

THE PATH OF AI’S SENSORY PROGRESS

In the case of AI, hearing was the first sense to truly get utilized. Voice models surged in adoption after the remarkable introduction of continuous speech capabilities. Not long after, AI harnessed the power of sight. And it is the modality that finally lets machines perceive the world directly rather than through the user’s description of it. Google reports that Lens handles more than 20 billion visual searches a month. Meta’s Ray-Ban smart glasses sold over 7 million units in 2025, more than triple the previous year. Camera-equipped AirPods are the anticipated next entrant, and their ubiquity gives them a significant head start.

Interestingly, multisensory AI also holds huge promise for improving software development. To bridge the visual gap, Cursor, one of the leading AI coding tools, gave its agent the ability to control a browser, take screenshots, and watch how the running application behaves on screen. The agent stopped relying on code and logs alone and started seeing the product the way an engineer sees it. Engineers, in turn, can speak directly to coding tools to add new features. They can also verbally describe challenges with AI agents that can draw on visual data sources to find solutions.

 

HOW DOES YOUR PRODUCT SEE (AND HEAR) THE WORLD?

For leaders, the shift toward multisensory AI necessitates asking which senses their product currently uses and which it is forcing users to compensate for. For example, a medical product that only processes imaging is asking the clinician to integrate everything else by hand. And an AI system that can’t see what a user can see will limit the design potential. When evaluating whether a product has enough multisensory inputs, business leaders should also focus on their effectiveness. They should assess whether sensory signals help users better understand information, make decisions faster, and increase engagement. Ultimately, the goal is to ensure that every sensory input serves a clear purpose, improves the experience, and works as part of a cohesive system.

Companies that recognize this early will win not by bolting on one new sensory capability, but by treating perception as a stack, building products that combine senses to improve understanding, just like the human brain. Critically, the potential of multisensory AI will rest on whether companies can modernize the infrastructure, data flows, governance, and security needed to let AI perceive across systems in real time.

The move to multisensory AI marks an evolution—from machines that interpret what we describe to machines that perceive the world with us. For consumers, the results will be magical. And for businesses, they will define the next era of technological possibility.

Ismail Amla is senior vice president of Kyndryl Consult.