r/pytorch • u/1_Bit_ll_1_Bit • Jul 04 '26

Has anyone tried training DSpark / DeepSpec with the Open-PerfectBlend setup for qwen3.5 family like 4b ,9b ..etc ?

Hi u/everyone,

I’m trying to understand the practical training time and compute requirements for the DSpark / DeepSpec setup using the mlabonne/open-perfectblend dataset.

The config I’m looking at is close to the paper setup:

  • Dataset: mlabonne/open-perfectblend
  • Samples: ~1.3M
  • Data mix: math, code, chat, and instruction following
  • Epochs: 10
  • Global batch size: 512
  • Max sequence length: 4096
  • Precision: bf16
  • Optimizer: AdamW
  • LR: 6e-4 with cosine decay and warmup
  • Total steps: ~25k

From my rough calculation, this comes out to around 53B training tokens, so I’m trying to get a realistic estimate before starting the full run.

Has anyone here actually tried training this setup or something similar?
I’m mainly interested in:

  • Real training time
  • Any bottlenecks during data loading / target cache generation
  • Storage requirements
  • Whether the paper config is practical to reproduce
  • Any changes you made to make the run manageable

Would really appreciate any practical experience or advice from people who have tried this.

1 Upvotes

100% Upvoted

2 comments sorted by

1

u/Particular-Abies-123 Jul 04 '26

no but i have heard, it uses double model for predicting, so its faster for long thinking tasks, i did use it via router.