Skip to content
Fast Company Executive Board

The Fast Company Executive Board is a private, fee-based network of influential leaders, experts, executives, and entrepreneurs who share their insights with our audience.

AI could be the answer to safer gaming environments

AI systems can be powerful tools to promote healthy online gaming behaviors and to build communities that not only provide fun and meaningful experiences, but drive growth and revenue.

Woman gaming

[Images: Anton/Adobe Stock]

Profile photo of Jeff Mills

President · iMerit Technologies

When I think about the gaming space, I think of it through several different lenses: the Business Partner, the Parent, and my favorite, the Gamer.

Looking at the gaming space through these different lenses can definitely change the perspective I have when it comes to safer gaming environments. As a gamer, I want the best gameplay. As a parent, I am thinking about safety and exposure. And as a Business Partner, I am thinking about how to help my customers maximize their consumers’ experiences and ultimately help them drive long-term revenues. That said, for anyone who regularly plays video games—or is the parent of someone who is online regularly—one fact is clear: it can be a jungle out there.  

There are times when the combination of online anonymity and a competitive environment can lead to behavior that is not ideal. Case in point: A recent study by the ADL found that 83% of adult gamers have experienced harassment in online multiplayer games. Moreover, 71% reported severe abuse, including stalking, sustained harassment, and physical threats. For anyone who has spent time online, those numbers shouldn’t surprise. 

Those stats may make some nervous—especially when looking through the parent lens. Any good parent wants to protect their kids from the worst behaviors. However, the fact that bad actors are a part of online gaming isn’t really the issue. Video game studios are aware of this problem and have invested heavily in exciting new tools to promote safety, ensure fair play, and build communities that reinforce those standards and enhance the experience for their players. 

It’s not surprising that creating healthy communities—and solving the complex issues that are inherent in creating such an environment—is a major focus for gaming studios. After all, positive gaming experiences lead to greater player engagement, and subsequently greater revenue. As a business partner, I have spent significant time working with my team in tandem with some of the top gaming companies in the world to leverage AI to impact these efforts.

HOW AI FORGES HEALTHY ONLINE GAMING COMMUNITIES

Game studios have four primary AI-powered tools at their disposal to help create healthier and safer environments for players. Let’s take a quick look at each one and how they are used:

Sentiment Analysis

Using AI models such as Automated Speech Recognition (ASR), game studios can analyze the tone and sentiment of online speech and/or messages. If a player is using aggressive or inappropriate language, making threats, or exhibiting other undesirable behaviors, the AI can flag it and ensure that studios and community managers can take appropriate action. 

Behavior Tracking

Similarly, AI systems can track player behaviors and identify larger patterns of harassment, bullying, or other actions that violate community standards. When the AI flags the pattern, it can refer the player to moderators so that appropriate measures can be taken. 

Chat Filters

Using language models, AI systems can monitor text chat to filter out messages with inappropriate language, threats, or other harmful behaviors. The AI can be trained to recognize specific keywords and phrases and automatically flag or delete messages where that language can be found. 

Player Matchmaking

Player matchmaking systems have been around for nearly two decades. However, the level of sophistication in the AI driving the systems has dramatically improved. As the AI system analyzes player behavior and performance, it can create more balanced player matchups—which results in a better user experience and greater engagement. 

THE ROLE OF HUMANS IN CREATING AI-DRIVEN SAFE COMMUNITIES

These AI systems are powerful tools to help game studios create healthy communities and positive experiences. However, one of the least understood aspects of these new AI systems is the critical role that humans play in training and teaching the AI models. For the AI system to recognize the patterns of behavior in an effective way, it must be trained by humans to understand the context it needs to do its job. 

This process first requires vast amounts of data that must be collected and organized in a manner that allows the AI system to learn. These datasets are tied to the tools mentioned above, including things like chat logs, speech recordings, and gameplay trends and behaviors.

From that point, human data annotators are used to manually classify the data—typically labeling it as positive, negative, or neutral. The AI model learns as it consumes the labeled data and builds an understanding of what to do when certain events occur. As the system becomes more sophisticated, the need for human experts with an understanding of the underlying technologies and the nuances of human communication becomes even greater. These experts are a vital part of helping the system learn how to deal with subtleties such as sarcasm, humor, and even different accents and dialects. 

For the process to be successful, game studios need AI partners that offer the right combination of technology tools, human talent, and annotation techniques. There are specialized providers of tools and services that provide this service, can scale it appropriately, and provide the expertise to ensure the model functions as intended. 

The bottom line is that video games are big business. It’s estimated the video game industry will generate nearly $400 billion in revenue this year. With that much revenue at stake, game studios must invest to create healthy and safe gaming communities that deliver the experience that will keep players around. AI systems can be powerful tools to promote healthy online gaming behaviors and to build communities that not only provide fun and meaningful experiences, but drive growth and revenue.

So, next time you see something that gets your attention while a game is being played, know that many of us are looking at this space from multiple lenses and the gaming companies themselves are investing heavily to ensure that each perspective ends up with the best of experiences.


Jeff Mills is the Chief Revenue Officer at iMerit.