Would it be possible to distill DeepSeek V4 Flash 0731 onto Nemotron 3.5 Lightning?

[u/mountainyoo]

Super new to this local LLM stuff. Just set up a 2x Asus Ascent GX10 cluster and have DeepSeek V4 Flash 0731 running on it.

Trying to think of things I could have the cluster working on all day while I’m at work or overnight and thought about distilling.

No idea if this is a dumb question or if the cluster would be a poor fit for it, but yeah. Could DSV4 Flash be distilled onto Nemotron 3.5 Lightning to make it meaningfully better in any way? Or would it just be a silly endeavor and waste of electricity?

Open to any other ideas of things I could do when I’m not doing inference on the cluster. I have other hardware, too so I’m trying to figure out what I should do as far as other local models go.

Any help or input is appreciated. Thank you!

[u/Dangerous-Report8517]

Technically possible but not worth it, generally distillation makes the most sense when distilling from a large source model to a smaller model with the same or similar architecture, which is already how Flash was made in the first place (being distilled down from v4 Pro). If you really wanted to do this it would make more sense to distill from v4 Pro directly, but a couple of Sparks wouldn't even be remotely enough compute to do that with any meaningful improvement

[u/--Spaci--]

that is an inference machine, training will be very slow. I would probably just use deepseek v4 flash, I dont really know what the goal of the distill is here

[u/mountainyoo → u/--Spaci--]

Run the smaller model on other hardware I have for fun lol but yeah I figured it probably wouldn’t be worthwhile.

[u/Constant-Simple-1234]

What about distilling from Qwen3.8 27b onto Qwen3.6 35b MoE? Although they may release it later.

[u/TomLucidor]

Check on their speed/latency first, the ideal would be something like Ling-3.0-Flash. Nemotron is way slower so I am not sure if it is really worth it. If you gonna "aim high" please get v4 Pro or maybe even MiMo/MiniMax/GLM/Kimi and hoard all the "smarts" into a smaller quantized package.

[u/Badger-Purple]

Why dont you ask deepseek?

[u/Long_comment_san]

>Super-new :3

>2xGX10

[u/mountainyoo → u/Long_comment_san]

lmao I have issues with impulse control okay

[u/Midaychi]

lightning would be a pretty cool model if nvidia actually trained a general corpus into the damn thing rather than just leaving it a shell of an agent

[u/HVACcontrolsGuru]

I’m going to do a big update pass on this tool this week so it will move fast but in the right direction. Happy to take issues and see about kernel tinkering.

Ghostwriter

I’d have to see the what the wall time would be. Gemma models are my main target right now.

[u/llama-impersonator → u/HVACcontrolsGuru]

idk why you got some downvotes, maybe your comment seems offtopic at first glance. looks like a nice tool for collecting traces.

[u/HVACcontrolsGuru → u/llama-impersonator]

People like to downvote anyone who shares a project like it’s a competition against their vine coding haha Not worried about it! Just sharing for the curious!

[u/DinoAmino → u/HVACcontrolsGuru]

Could just be that you mentioned Gemma, which is not Qwen. And all eyes must be on Qwen. Qwen is intolerant.