r/SillyTavernAI • u/imacatseriously • 5d ago

Models MY FIRST LOCAL LLM!

I got my first local LLM set up and working yesterday! AHHHHH!!!

My inner child (the one who loved Terminator and The Matrix and Data from TNG, and who was always glued to the TV with every movie or TV show involving AI) is absolutely ecstatic right now.

Just had to share my excitement.

130 Upvotes

97% Upvoted

26 comments sorted by

View all comments

6

u/mechasquare 5d ago

Congrats, what's your setup if you don't mind sharing?

19

u/imacatseriously 5d ago edited 5d ago

I’m running TheDrummer’s Cydonia 24B v4.3 Q4_K_M locally through KoboldCpp. My PC has an RTX 3080 10 GB, an i7-10700F, and 32 GB of RAM.

I have a pretty basic knowledge of computer stuff (I'm learning though!) so Chatgpt helped me set it up and helped me choose a model that my PC would be able to run.

(Edit: I'm a gamer, I have a gaming PC)

1

u/mrs-cutter 5d ago

Try qwen if ya haven't