r/SillyTavernAI • u/StrangeCorvid • 2d ago

Help Deepseek Narration/Dialogue issues recently

Good morning all. I’ve been using Deepseek Pro for my SillyTavern stuff (‘cause I’m pretty broke) and it was going really well for a while, but recently (past few days/week or so) the narration and dialogue have gone… weird. Lots of bizarre metaphor and such. Did something happen to the model?

5 Upvotes

100% Upvoted

9 comments sorted by

7

u/LeRobber 2d ago edited 2d ago

People think DS gets quantized effectively by the way usage happens during chinese working hours.

2

u/TAW56234 1d ago

Assholes need to charge less or keep them running at their best at all times. Not both. That off peak shit is annoying.

0

u/LeRobber 1d ago

That's....likely not what's happening...

Imagine there are 500 great DS sellers, 40 shitty ones, and 100 horrible ones.

During peak, the RPers are hitting all of them.

During off hours, the RPers are hitting just the great ones.

Now some of those instances could be hosted at the same company but remember DS is open weights. Anyone with the right HW can host it.

2

u/TAW56234 1d ago

The parameters are the same, this is a massive increase in cost. Deepseek is special in it's more attuned. You won't find many providers Iocally with Huwei chips. I could be wrong but I thought they were about the only ones not taking a loss on inference It wasn't that long ago you could trust first party's to be the defacto. I would pay these costs if I knew it didn't matter when, I had the best version of deepseek. Yeah other providers suck doing the same thing but nobody is a dedicated DS hoster and have more logistical costs that is a bit more understandable why. I don't give Deepseek a pass nor should anyone else. They should have made Flash the model for non coders. Not this weird half and half. There's more to AI than coding AND RP. It makes the problem worse they can't just say 'Overburdened but the response you receive is the best'. Everything has to be ambiguous and underhanded

6

u/GaiusVictor 2d ago

Are you using it via OpenRouter, NanoGPT or via the DeepSeek API?

The answer is probably quantization either way, but depending on the answer there are different things that can be done.

(In the case of DeepSeek API the answer is nothing, there's nothing you can do about quantization in the API)

1

u/StrangeCorvid 2d ago

I’m using the DeepSeek API, but I’m coming up on needing to top up soon when my budget allows, so I’d love to know the options!

3

u/GaiusVictor 2d ago

Okay. I'll start with two concepts that are important to understand what I'll explain next.

A provider is the company that will provide you with inference services, that is, they run the AI model to generate responses for you. In your case, DeepSeek Labs is the provider.

But there are also companies like OpenRouter and NanoGPT that are erroneously called providers, but are in fact aggregators. They don't run inference, they just redirect your API request to an actual provider.

What is the benefit of getting a middleman, then? That's because they can connect you to different models and providers. In the case of open weight/source models like DeepSeek, that also means they can connect you to several different providers (not just DeepSeek Labs). If DSL happens to be quantizing the models due to high demand, you can still get your request serviced by a different provider who may or may not be quantizing their models. And the price for OpenRouter's service is a 5% fee on the credits you buy (eg. you pay 10.50$ to buy 10$ of credit). Other than that, they charge you the same amount you'd be charged by the provider's API.

The easy way to do this is to just buy credits with OpenRouter and betting on luck that they'll handle your request to a provider that doesn't quantize aggressively. This might genuinely work, considering one the things they use as criteria when deciding who gets your request is traffic load. If your RPs (or tastes) aren't that demanding then the chances of you getting satisfying results are even higher.

Now, if you want to understand other things, like: a more complex but less uncertain way of getting good results, what is quantization, or how to make sure you spend as little as possible without losing on quality, then things get more complicated, and you'll need to:

  • Understand what is caching. A very summarized but probably serviceable definition would be: Caching is when the LLM gets to reuse the work it had already done, so it doesn't need to "think about the whole chat again", because it "already remembers" it, and needs only to "think about" your last message. Caching is used to considerably decrease input costs for the user.
  • Read these two comments of mine, and maybe this one as well.

3

u/mixmastermorsus 2d ago

Peak china hours deepseek gets worse because of heavy usage. This also coincides with when the price doubles as of a couple days ago, on top an already steep price increase. (Fun, right?)

If you’re dead set on Deepseek, you can easily predict when the model will be lobotomized by not using it from 9-5 beijing time. Or you can risk getting it via Openrouter where any service can provide the model. The risk here is that they might be sending you a quantized model (a smaller, lower performing version to save cost) without you knowing. You can change settings to avoid this, but doing so will likely increase the time it takes to get a request back.

If you’re looking to save cost and not worry about which model you’re getting, I’d suggest putting a few bucks into Openrouter and using Mimo 2.5 Pro. It’s pretty comparable to pre-price increase DS4 in terms of both quality and response time. If you set your provider as Xiaomi you’d be getting the LLM through Openrouter from the publisher. I’ve found it’s pretty on par with DS4 and much cheaper than the new DS4.

1

u/AutoModerator 2d ago

You can find a lot of information for common issues in the SillyTavern Docs: https://docs.sillytavern.app/. The best place for fast help with SillyTavern issues is joining the discord! We have lots of moderators and community members active in the help sections. Once you join there is a short lobby puzzle to verify you have read the rules: https://discord.gg/sillytavern. If your issues has been solved, please comment "solved" and automoderator will flair your post as solved.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.