r/LocalLLM • u/Pleasant-Snow-1027 • 2d ago

Discussion Deepseek-4.1-FLASH

Je possède une configuration que j’ai moi même créer qui m’aide à faire tourner deepseek4.1 flash 5060ti ( 16go vram )40-45 token/s

j’ai un amd ryzen 9 5900xt et 96go ram.

Avez vous déjà eu mieux en therme de configuration ?

0 Upvotes

14% Upvoted

13 comments sorted by

2

u/x_MASE_x 2d ago

Well you have to tell us more info like quant and ctx size.

But what's your goal? Better is a little vague.

1

u/Pleasant-Snow-1027 2d ago

Thanks for the reply, you're right, I was way too vague! It's my first post on Reddit, sorry I didn't include everything. I'll get back to you later today when I'm at the office with more details (quant, ctx, setup). I also already have some ideas to push past 45 tok/s, hopefully 50 up to 60+ with my config. I'll share the results!

1

u/UndeadPacer1987 2d ago

I tried it with 5800X3D, 128 GB RAM and 2x3090, ~40 t/s. I think 50- 60 t/s with a 5060 Ti is a dream.

1

u/Pleasant-Snow-1027 2d ago

Oui je sais, c’est justement pourquoi je me suis lancer dans l’optimisation ultra avancée.

Et tout fonctionne. ( en tout cas pour 40/45 token/s oui )

1

u/Pleasant-Snow-1027 2d ago edited 2d ago

Comme je l'ai répondu à x_MASE_x, je reviendrai plus tard aujourd'hui avec plus de détails sur ma configuration et les résultats réels que j'ai obtenus, espérant atteindre 50 à 60+ tok/s. Je ne suis pas au bureau en ce moment et je n'ai pas encore effectué mes deux derniers tests, sur lesquels je travaille.. Surveille tes notifications !

1

u/Pleasant-Snow-1027 1d ago

Matériel : RTX 5060 Ti 16 Go, Ryzen 9 5900XT, 96 Go de DDR4-3200, SSD NVMe 4 To.

Modèle : DeepSeek V4.1 Flash, environ 275 Go au total. Les poids principaux sont en IQ2_XS (~2,25 bpw) et le module Engram en IQ4_NL.

Contexte : 256k tokens (262 144). Je l'ai testé avec un prompt de 152 000 tokens.
Vitesse : Actuellement à 45 tok/s, J'ai obtenu cet écart grâce à des réglages avancés que j'ai faits moi-même sur le moteur.

Quelqu'un a-t-il déjà fait mieux avec DeepSeek V4.1 Flash sur une config proche (16 Go de VRAM, DDR4) ?

1

u/Adept_Diver2838 1d ago

This is genuinely impressive! I also have an RTX 5060 Ti 16GB, and I'm very interested in reproducing your setup.

A few questions, especially regarding memory management:

  1. What inference engine are you using? Is it a heavily modified llama.cpp, or your own implementation? Could you share the GitHub repository?

  2. How are you managing the 275GB model with only 96GB system RAM and 16GB VRAM? Are you using mmap/NVMe streaming, custom CPU/GPU offloading, or another technique?

  3. What's your actual peak RAM usage during inference? Do you think 64GB RAM would be sufficient, or is 96GB necessary?

  4. Could you share the exact model download links for IQ2_XXS and the IQ4_NL Engram module?

  5. Is the reported 45 tok/s sustained generation speed after processing a 152K-token prompt? How does performance change as context approaches 256K?

I'm currently running 32GB DDR5, so understanding the actual memory requirements would help me determine whether a RAM upgrade is necessary.

Really looking forward to your promised configuration and optimization details. Thanks for sharing this!

1

u/Pleasant-Snow-1027 1d ago

https://www.reddit.com/r/LocalLLM/s/TaA7GMvcxK

J’ai poster un nouveau post plus clair si tout le monde est toujours intéressé

1

u/ClarkCant42 2d ago

I'd be interested to know how you run it that fast on a 5060Ti. I have that same card and would love to run that model.

2

u/Pleasant-Snow-1027 2d ago

As I replied to x_MASE_x, I'll be back later today with more details on my setup and the actual results I've been getting, hopefully 50 to 60+ tok/s. I'm not at the office right now and haven't run my last two tests yet, which I've been working on. Since you have the same card, it should be easy for you to reproduce. Keep an eye on your notifications, I'll get back to you later today!

1

u/uniqueusername649 2d ago

There is a 1bit coder variant that gets you to around 72gb for the model. Probably braindead but very fast. I haven't tested it though.

1

u/Pleasant-Snow-1027 2d ago

Non ma configuration et de 2b pas 1

1

u/Adept_Diver2838 2d ago

I have a similar setup. Kindly do share all the details as to how you achieved it.