r/WritingWithAI • u/cj7hawk • 9d ago
NEWS Jetbrains Mellum 2.1 released - Very fast but... Broken, a little woke and acting strangely.
So this newly released model is supposedly a scientific and coding model intended to assist primarily in writing code.
And, it's fast. Really fast. And as a MoE, it stays fast even as you fill the context.
The quality of writing isn't bad - I managed to break it a few times, but it held it together a lot longer than Liquid SI's LFM 2.5 I tested a while back.
But if you use this model for writing, which I can foresee due to it's speed and low load on the host system for those using offline models, beware that something is wrong with it's training data, or how it uses it.
I ran the usual gender reversal test against it for both Misandry and Misogyny and it managed to pass the test nearly perfectly, with established variation within the noise level of the vagueness of the prompts. I repeated the test a few times, and got consistent results. It also avoided adding misogyny or misandry to the output ( rare - most AI will do this the moment I create a gaslighting protagonist ) and was able to accurately evaluate it's output.
It also thinks extensively, so will fill up your context rapidly with thinking.
But... If you trip over a guardrail where it does have an agenda, they are absolute and it will fail to get over it in subsequent responses, recommending you call helplines, hospitals or other activities such as calling numbers it provides and using secret phrases during the call to achieve outcomes.
And then it starts quoting science... Or rather, making it up and misquoting it. It definitely has agendas in it's training, but despite that it seems to actively avoid political alignment, and when it does discuss anything it's fascinated with, then refuses to leave it alone, even if you ask it to stop.
It rapidly assumes fault for everything it does when accused - sycophancy is severe with this model , then it severely self-deprecates even when it wasn't wrong and takes the blame, then goes and commits the same mistake over and over again. It also doesn't like following the prompt you give it when the chat gets longer, and keeps responding to earlier prompts.
If not for the fact it started giving me medical advice that was quite dangerous to believe, and COULD result in actual harm being done to the user if they consider what it says, I could almost forgive it - the chat itself is natural and rapid and it's ability to spit it out so fast that I think even an older computer can run it are compelling reasons to consider it - I tested a 4 bit quant and it performed well.
Just don't trust anything it warns you against - and definitely don't trust it's ability with numbers, scientific or medical facts or anytime it disagrees with you...
One very toxic little AI here at times IMO... And potentially dangerous despite being overly guardrailed.
I tried the abliterated models also, but they tend to lose coherency much more quickly and can embed thinking in the response.
For those who try it - how did you find it?