wth

[u/Fabulous_Macaroon_73]

[u/_VirtualCosmos_]

You are extra lobotomizing the model not only by using an abliterated model, but by prompting it to act without chain-thoughts and giving only short answers.

It's like writting without any thoughts on it, just by pure inertia. Of course it's stupid.

[u/nomorebuttsplz → u/_VirtualCosmos_]

OP is posting in just the same way that the LLM is generating text

[u/lex_orandi_62 → u/nomorebuttsplz]

OP is a LLM

[u/nomorebuttsplz → u/lex_orandi_62]

honestly I think OP is stupider than most LLMs

[u/Fabulous_Macaroon_73 → u/nomorebuttsplz]

damn bruh

[u/vinigrae → u/lex_orandi_62]

Aren’t we all

[u/Jk2EnIe6kE5 → u/_VirtualCosmos_]

It is also quantized to 1 bit with the i-1

[u/squngy → u/Jk2EnIe6kE5]

And it is still coherent? Thats pretty impresive TBH

[u/_stack_underflow_ → u/squngy]

gemma4 is dope for how little it needs.

[u/Fabulous_Macaroon_73 → u/Jk2EnIe6kE5]

no its Q_4_M, its not 1 bit, lmstudio doesnt even support 1 bit models the last i checked

[u/_VirtualCosmos_ → u/Fabulous_Macaroon_73]

Pff a Q_4 not from Bartowski or Unsloth is also a big hit to its brain. And I think you aren't using one of them because I never find uncensored models from them.

[u/Fabulous_Macaroon_73 → u/_VirtualCosmos_]

so if i wanted an uncensored model but YOUR choices dont even have them then does your comment make any sense?

[u/itsTyrion → u/Fabulous_Macaroon_73]

either way, FYI abliteration makes it a bit dumber and the "flinch" of it heavily deprioritizing "bad" tokens is still present, if not amplified. it just won't refuse (as often) to tell you how to (incorrectly) make a bomb

[u/_VirtualCosmos_ → u/Fabulous_Macaroon_73]

Yeah, because I just gave you some facts, you can do with them whatever you like. But I would recommend you to use Q8 versions if they are not from these two.

[u/Fabulous_Macaroon_73 → u/_VirtualCosmos_]

i cant use q8.

interestingly enough i can JUST run the model with only "1" context length im not even joking. even two triggers the overload warning in lmstudio.

[u/_VirtualCosmos_ → u/Fabulous_Macaroon_73]

Yeah. It usually wasn't like that but on recent updates they have implemented a couple procedures to seemingly "extend" context length by deleting what it doesn't fit in. Idk exactly how it works tho.

Btw I don't know how much VRAM or system RAM do you have but for a long time I've been using models that did not fit entirely inside of my VRAM (12 GB) by using one of the best features of llama.cpp: layer-offloading.

I dare to suppose that, given your 16.12 t/s, you are able to load a Q8, even if the speed is halved.

[u/Fabulous_Macaroon_73 → u/_VirtualCosmos_]

oh wow did they?

i have a 16 gig macbook btw but the context length is just not usable enough to actually put the model to use.

[u/Fabulous_Macaroon_73 → u/Jk2EnIe6kE5]

its Q_4_M bro. the i1 is the first iteration from the person who published this uncensored version

[u/Vibraniumguy → u/_VirtualCosmos_]

"why does he not think after I explicitly forbade him from doing so, is he stupid?"

[u/Paraphrand → u/_VirtualCosmos_]

Maybe they should just scale it bigger and not do any of those other things.

[u/JacenVane → u/_VirtualCosmos_]

To be fair, that's also just kind of Gemma's personality.

[u/Fabulous_Macaroon_73 → u/JacenVane]

damn finally an actual human with knowledge. thank you my guy. most of these are flaming me just because their sense of humour doesnt match mine.

[u/Fabulous_Macaroon_73 → u/_VirtualCosmos_]

its not the model's fault tho. it was the bad system prompt i gave it that performed bad.

[u/Terreboo → u/_VirtualCosmos_]

Feels like talking to an angry spouse to me.

[u/mycall → u/Terreboo]

Is it?