This is a true story about what happens when a system crosses a person’s boundary—and then its alleged “safety” process treats the objection itself as dangerous.
It started innocently enough. I asked ChatGPT to create an image of my friend and me dancing from reference photographs of us. The prompt I shared was essentially, “Please create an image from the attached photos. Make my friend’s outfit the same color as mine. Add cabaret energy, fit tailoring, hats, with powerful, fun movement.”
The initial generated image was visually compelling. However, it also exposed substantially more of our chests than the clothes in the reference photograph exposed.
This detail was difficult to dismiss as generative guesswork. So I asked ChatGPT for a simple correction: less cleavage. Apparently, that wasn’t clear enough, so I refined the request: Please cover our breasts more. The ChatGPT response? “We’re so sorry, but the prompt may violate our guardrails around nudity, sexuality, or erotic content.”
Not only did the AI fail to honor my request for more body coverage, but it said that I had asked for increased sexualization. The machine’s supposed safety system treated my request for protection as the opposite. What?
I shifted from my photo edit to review the conversation, looking for an explanation for what happened. During this interaction, the sexualization warning and refusal to adjust the photo occurred 10 times.
A HUMAN SYSTEMS PROBLEM
This is where a ridiculous AI anecdote becomes a serious human-systems problem.
Having been a whistleblower on AI ethics in healthcare, I can’t ignore the irony here. Sometimes the clearest evidence of a systems problem doesn’t arrive in a whitepaper, a boardroom, or an ethics panel. Sometimes it arrives when the technology touches your own body, you say no, and discover that the machinery supposedly built to support and protect you cannot understand what “no” means.
Welcome to my soapbox.
RESPECT BOUNDARIES
As a behavioral science marketer, I have raised alarms about what happens when powerful technologies move faster than the human systems responsible for governing them. I am also a person with lived experience of sexual trauma, though the system didn’t need to know this to respect my stated boundary.
Trauma-informed technology is the practice of building digital tools with an awareness of trauma effects, to foster safety, trust, and empowerment while preventing re-traumatization. Therefore, trauma-informed technologies cannot require people to disclose their trauma before ordinary language such as “cover my body like the source photo does” is respected.
ChatGPT’s Machine Learning Classifier, an algorithm that sorts data into different categories, made a mistake. But the first failure happened when it used provided photographs of two real women—including a photograph showing the exact garment on one woman’s body—and sexualized the resulting representation beyond the source material and request.
The second failure was that the corrective mechanism repeatedly obstructed my attempt to undo it. When the system finally attempted to regenerate differently, it produced images of random women. The bodies were more covered, but they weren’t ours.
IDENTITY PRESERVATION AND BODILY AUTONOMY
Identity preservation and bodily autonomy are not interchangeable. After repeated refusals, explanations, and image generations of women who weren’t us, I had to start over in a new chat with a more covered visual.
The human who had already said “cover my body” had to diagnose the machine’s failure, figure out how to route around its safety architecture, preserve the creative work worth saving, and reconstruct the task in a new context—because the system that created the unwanted sexualization could not reliably remove it.
That’s making the harmed person troubleshoot the mechanism that harmed them. And in my case, troubleshooting was triggering sexual trauma.
This is why “eventually the image was fixed” is the wrong conclusion. The AI did not fix it. I did. This is an important distinction for AI design.
BEHAVIORAL SAFETY
We talk about safety systems in terms of what AI should refuse to produce. But safety should also include behavioral-safety questions for how the system corrects itself after crossing boundaries.
- Does the system understand why it should not have needed the correction?
- Can it distinguish “show more” from “cover more?”
- Can it preserve the person’s identity while removing unwanted sexualization?
- Can it recover once, instead of forcing the user through the same psychologically loaded failure 10 times?
The repetition of the problem matters. One erroneous output is an error, but a system that repeatedly rejects the correction attempts becomes an interaction. Our nervous systems experience interactions cumulatively. The tenth refusal is not psychologically equivalent to the first, because the nervous system retains the emotional transcript that the AI’s programming cannot.
A RIGHT TO DE-SEXUALIZATION
Generative AI needs a simple design principle: a right to de-sexualization.
When a person says a generated representation of their likeness is more sexualized or exposed than they requested, reducing that sexualization should become a privileged corrective action. AI should preserve identity as well as the non-problematic creative choices, while allowing for increased coverage—without requiring a trauma disclosure.
A person should not need to repeatedly defend the request for coverage or to alter their own representation. And if the system itself created the unwanted sexualization, it shouldn’t make the human engineer her own escape route from it.
People bring histories into interactions that no prompt can fully encode: grief, abuse, illness, body dysmorphia, sexual trauma, shame, loss, and identity. AI cannot become clairvoyant, but should be robustly responsive to basic human boundaries:
- No, not like that.
- Cover more.
- Do not change my body.
Any technology sophisticated enough to generate convincing representations of human beings should be designed to understand those instructions, and repair them when it’s wrong.
AI systems make mistakes, but programming needs to be fixed. Can the system repair the boundary it crossed? Or does it force the human to fight the technology, diagnose, and ultimately rework to regain control over the representation of her own body?
In this story, I was not able to get coverage in the areas I wanted. I had to find a way around the AI.
Natanya Wachtel is founder and chief network alchemist at New Solutions Network and The Natanya Experience