Society is grappling with the question of how to protect children and adults from abuse and harm online while maintaining democratic values of free speech and user privacy. Governments are implementing online safety laws, like Australia, the U.K., the Philippines, and others, having struck this balance after public consultation with experts and survivors. While the U.S., Canada, and the EU are still debating, Australia is poised to strengthen its law through its Online Safety Act Review.
France, for its part, is using judicial powers to hold accountable tech leaders alleged to have violated domestic laws in the operation of their platforms, with the arrest and indictment of Telegram CEO Pavel Durov for alleged “crimes related to illicit transactions, child sexual abuse, fraud and the refusal to communicate information to authorities.”
With an estimated 300 million children impacted by online child sexual exploitation, laws are clearly needed to address this avalanche of abuse. For instance, even though US law requires companies to report suspected child sexual abuse material (CSAM) on their platforms after learning of it, many companies do not proactively detect CSAM and therefore fail to report it. Case in point: Transparency submissions to Australia’s eSafety Commissioner indicate that companies do nothing to detect livestreamed child sexual abuse on their platforms, apps, and devices, even when it manifests as child sex trafficking.
Pause for a moment to let that sink in. We’re talking about turning a blind eye to children being repeatedly sexually abused live on their platforms, apps, and devices by their users for their users to watch in real time, in what Europol calls “a persistent threat” that “stands out as the main form of commercial sexual exploitation of children and as a major source” of new CSAM. In the Philippines, this impacts nearly half a million children, with victims in Colombia, the U.S., U.K., Romania, and elsewhere.
What can be done to address this? Tech sector actions to address financial sextortion and even malware give governments clues on how to strike a balance between child protection and user privacy because, after all, companies are acutely aware of the need to strike this balance for commercial purposes.
For instance, Apple touts its “Communication Safety” tool as user-privacy-preserving, explaining that “Communication Safety uses on-device machine learning to analyze photo and video attachments and determine if a photo or video appears to contain nudity. Because the photos and videos are analyzed on your child’s device, Apple doesn’t receive an indication that nudity was detected and doesn’t get access to the photos or videos as a result.”
Meta-owned Instagram likewise reports that “Nudity protection uses on-device machine learning to analyze whether an image sent in a DM on Instagram contains nudity. Because the images are analyzed on the device itself, nudity protection will work in end-to-end encrypted chats, where Meta won’t have access to these images.”
Similarly, “WhatsApp automatically performs checks to determine if a link is suspicious. To protect your privacy, these checks take place entirely on your device. Remember, because of end-to-end encryption, WhatsApp can’t see the content of your messages.”
Clearly, companies can address child exploitation, including CSAM, while preserving user privacy—the principles and tools just need to be applied across the board while working to prevent harm in the first place.
In another promising sign, Meta, Google, and others joined Project Lantern, a Technology Coalition initiative using signal sharing to improve child exploitation detection. In August, Lantern expanded to the financial sector, adding Western Union and Block. Financial-sector participation is game-changing because money transfer and other financial services are routinely misused to commit live video child sexual abuse and financial sextortion globally.
Financial regulators, like Australia’s AUSTRAC and Canada’s FINTRAC, have shared child sexual exploitation indicators with financial entities to enhance detection, reporting, and disruption. Project Lantern’s cross-sector signal sharing can leverage evidence-based tech and financial sector knowledge to disrupt child exploitation online and CSAM. To maximize Lantern’s impact, companies should ensure that their reports are “actionable” and make it into law enforcement hands—through the CyberTipline, suspicious activity reports, and in response to lawful data requests. Companies should also use it to combat the trafficking of children to produce CSAM, including via live video.
Still, regulation is needed to catalyze industry-wide change globally across harm types through standards established by government leaders charged to balance child protection with other fundamental rights.
Here are two key recommendations for what governments and corporations can do right now:
1. Maximize the first-ever Global Ministerial Conference On Violence Against Children.
This November’s Global Ministerial Conference in Bogotá is a tremendous opportunity for governments to follow survivor leader recommendations by making and funding serious pledges to end violence against children, including sexual violence online. Because hands-on sexual violence against children often has an online component, commitments should include establishing online safety legal frameworks that require companies to build their platforms, apps, devices, and operating systems safe by design to prevent child sexual abuse. Governments should equally pledge to provide access to trauma-informed justice and recovery services for survivors by enforcing laws that protect children from violence.
2. Elevate online prevention in the global conversation.
While most enacted or debated laws focus on detection and reporting, it is time to explore a more preventative approach that protects the privacy of CSAM survivors and prevents ongoing child sexual abuse. Multi-sector stakeholders can consider what survivors from the Philippines and globally are demanding: blocking and preventing access to, sharing, rendering, uploading, storage, and production of CSAM at every conceivable opportunity—or what I call a “CSAM wipeout.”
Even global hotel chains, like Marriot International, are doing their part by blocking access to known CSAM websites, in an innovative partnership with Internet Watch Foundation and Cisco. It is time for a zero-tolerance approach to CSAM, with safety tech and big tech firms leading the way to develop and explain innovative tools that secure protection and privacy.
Ultimately, when companies elevate online safety side-by-side with profit and user privacy, and when governments put resources behind their global pledges, we will move closer to a world where every child can grow up safe—at home, at school and online: a world where child protection—not CSAM—is at our fingertips.