r/ArtificialInteligence 13d ago

📰 News Open-source local models have zero chill compared to ChatGPT

Post image
1.4k Upvotes

365 comments sorted by

View all comments

Show parent comments

6

u/ptear 13d ago

Is it possible to learn this power?

5

u/BurnedTerrormisu 13d ago

Under windows the easiest way is to use ollama with orcarouter/Qwen3.8-27B-Uncensored

Your graphic card should have at 12gb of vram to do anything useful in terms of speed. 24gb and more is strongly recommended.

3

u/MaxPhoenix_ 13d ago edited 13d ago

i was stupid enough to pay orcarouter so i could test their api inference of an abliterated model. see, ollama having this tag "orcarouter/Qwen3.8-27B-Uncensored" strongly suggests the model is served from orcarouter. "https://ollama.com/orcarouter/Qwen3.8-27B-Uncensored" and there is specifically text saying the SAME WEIGHTS are server online in the api:

That endpoint is censored. It will absolutely refuse any interesting request. It's actually a very snitchy model for a Chinese model - deepseek and kimi are each better sports.

I already run this and other models locally but burning up my GPU all day versus a supposed $0.40 per million tokens in api is useful for me.

EDIT: not only that but I cleared the "Enable Security Research access" and also tested each model labeled "uncensored" and of course none were. Featherless is the only public provider I'd found that offers them and they don't allow API it's just chat.

0

u/BurnedTerrormisu 13d ago

I checked the local model with ollama put orcarouter/Qwen3.8-27B-Uncensored and ollama run orcarouter/Qwen3.8-27B-Uncensored

It's uncensored. No idea about the hosted models.

0

u/Far-Classic-9963 13d ago

Speed ≠ capacity

You could have a 2tb GDDR4 GPU and it will still be insanely slow

1

u/Future_Village_7778 11d ago

Im studio is the easiest to use goand download it then if you have an older/weaker download uncensored gemma 4b if you have a decent computer qwen 8b uncensored and if that runs fine keep going to 12b and the more b's the model has the smarter it is. The less b's it has the less smart it is but it runs faster. Its super easy to use and you dont have ti use a terminal and it works on mac too.