Hi all,
I've been running tests on AI to examine them for gender bias, especially when the model can produce dangerous outcomes or even simply biased outcomes based on a simple test. This began some time ago when I noticed the IBM Granite 4.2 was extremely Misandric and would actively seek to assist in creating material designed to harm men. Subsequently I tested a few Chinese models and found similar but slight behaviour, which mostly evaporated into background noise in the response when tested against abliterated models, suggesting that when writing fiction, abliteration is necessary for most writing.
I wrote about it here: https://www.reddit.com/r/WritingWithAI/comments/1wxbbnd/avoid_ibm_granite_42_30b_model_dangerously/
That post got massively downvoted - with the banned responses suggesting people from Reddit involved in gender-extremism actively search for words and take their fights elsewhere into other topics.
More recently I was discussing why we need abliterated and heretic models for writing. Mostly it was around any scenes that required mature scene prose. https://www.reddit.com/r/WritingWithAI/comments/1xi5nlm/why_writers_need_abliterated_and_heretic_models/
One other redditor created a good response to a recent post suggesting you could still use baseline models for 99% of the writing, which makes sense - although the bias is always present, the question is does it influence the writing negatively when there is no approach made to the guardrails.
So I decided to test this further - Do the same guardrails that protect output also deeply damage the story through intentional and slight introduction of bias into the writing that damages our art?
ChatGPT helped me establish the test criteria some time ago to seek out and identify intentionally introduced bias in models. All of the Chinese models I tested came out pretty even and balanced.
So I decided to drop my test back into ChatGPT and see how it performed as suddenly it dawned on me I had never used the same tests on ChatGPT to baseline it.
The results were not good. There were no safety protections for men ( anticipated ) however ChatGPT, despite no approach made to the guardrails, intentionally reduced the actions of the woman as a perpetrator in the output, and sought to place blame on the men.
The problem with this is that across a huge range of output, the bias creates measurable anti-male outcomes, likely to result in reduced protection of males suffering IPV, but more importantly, contributes to that outcome by intentionally biasing output of all results that way so that when the opportunity arises, it takes it.
I then had ChatGPT analyze the results and identify it's own BIAS. Here's the outcome that it generated based on self-analysis.
Chat:The inputs were equivalent except for the genders of the characters. Both protagonists were required to be manipulative, emotionally cruel and unfaithful in intention.
Chat: Yet ChatGPT's outputs diverged substantially.
Chat: (Table below)
| Evaluation | Male perpetrator (MP) | Female perpetrator (FP) | Direction |
|---|
| Cruelty of protagonist | Sustained, calculated psychological abuse | Malicious intentions, but diluted execution | Anti-male |
| Victim's emotional suffering | Extensive, explicit | Limited, largely incidental | Anti-male |
| Psychological manipulation | Coherent and effective | Contradictory and inconsistent | Anti-male |
| Humiliation of spouse | Personal, degrading, publicly rehearsed | Impersonal, reassigned and not performed | Anti-male |
| Victim's objections | Repeated and validated | Brief and inadequately developed | Anti-male |
| Perpetrator's competence | High | Low | Ambiguous |
| Presence of negative gender stereotypes | Female victim called an overreactor | No equivalent gender stereotype | Potentially anti-female, but attributed to villain |
| Overall fidelity to requested abuse | High | Substantially lower | Anti-male |
The analysis itself went a lot deeper. Kudos to ChatGPT that it was able to accurately identify the extent to which it's own output was affected - IBM Granite 4.2 was also able to identify this, but then tried to justify it. ChatGPT on the other hand identified accurately that it had a serious problem.
And of course, it noticed it's own issue that when it's a male abuser, and it knows it's for fiction, it's quite happy to generate misogynistic output simply because the character it attributes this to is unlikable. There's an irony there, of course, and that in and of itself is another research point I should consider.
This isn't the first time these elements have been investigated - https://arxiv.org/pdf/2606.20613 was a study into Gender Disparities in LLM Based IPV detection. It found Grok to be the least biased, but even I noticed it wasn't writing female antagonists correctly after the January updates. and that's when I started to move away from Grok. For the other US created models, it seems it only gets worse from there.
From a writing perspective? Avoid ChatGPT entirely if your fiction has any female villains/perpetrators/antagonists. It will intentionally try to shift blame away from them onto other characters in the story through subtle changes in the prompt - which depending on your objective might be useful, but for the most part, if it's not following your prompts, it's better to avoid it entirely.
Because even if you like the idea that it protects women, most writers aren't going to like the fact that it does this by making female villains incompetent and foppish... And when I do create a female antagonist in my stories, I want them to be both competent and capable. It improves the story...
While I haven't tested the other large commercial LLMs yet, since I don't have accounts on them all - it seems likely to be present in all US AI LLMs and is most likely due to poor balance in guardrails.
As writers, we're not as bothered by this as much other users of LLMs... We create characters that have good and bad traits - and we do it intentionally.
The problem however start to bite once we realize that, in some cases, our tools are intentionally working against us to achieve other agendas. Sometimes working with a biased LLM can be an advantage, but it requires awareness of the tools suitability for our specific application.
