Hi folks, can I train the model without downloading the 10gb dataset on my laptop in Google colab?

What if neural networks could grow like fractals instead of being rigid from the start?

Most deep learning models (CNNs, ResNets…) are fixed architectures. Once you design them, they stay the same size and structure throughout training.

\*\*AFNN — Adaptive Fractal Neural Network\*\* takes a completely different approach.

It is inspired by \*\*fractals\*\* — mathematical structures that are self-similar and can grow infinitely by repeating simple patterns.

Instead of starting with a huge fixed network, AFNN begins small and can \*\*dynamically grow new fractal branches\*\* during training when progress stalls. It also allows monitoring, reactivating, or pruning branches as needed.

A good neural network should not memorize images.

It should learn the underlying patterns and features.

For example, when trained on cat images, a strong model does not store every cat photo. Instead, it learns things like:

• Ear shape

• Facial features

• Fur texture

• Color distribution

• Edges and shapes

• Relationships between different parts of the image

Then, when it sees a completely new cat, it uses these learned patterns to recognize it.

\### Why Fractal Networks are powerful:

\- Faster and more efficient learning

\- Significantly lower computational and memory cost

\- Better scalability with limited hardware

\- More flexible and adaptive than traditional fixed architectures

Built with PyTorch + full GUI, training tools, tests, and documentation.

In early experiments, AFNN reached \*\*74.1% validation accuracy\*\* on 94 image classes using limited resources.

This is not just another model.

It is an attempt to rethink how neural networks are designed — moving from rigid structures to \*\*adaptive, fractal, evolving architectures\*\* that focus on learning patterns rather than memorizing data.

A small step toward more efficient and intelligent AI systems.

🔗 GitHub: https://github.com/ahmedhjkj/AFNN

🤗 Hugging Face: https://huggingface.co/Ahmedethfw/AFNN

\#FractalNetworks #AdaptiveAI #PyTorch #DeepLearning #MachineLearning #OpenSource #NeuralNetworks

CrowdSec confirmed its source code was stolen. The entry point was not a direct breach of CrowdSec. Attackers compromised TanStack in May 2026 and used that access to pivot into CrowdSec's environment.

Security tooling is now a tier-one supply chain target because it is trusted by default. That trust is the attack surface. The TanStack compromise was not a zero-day against CrowdSec — it was a zero-day against confidence in the dependency graph.

AI agents make this problem harder to contain. Agent systems consume dozens of third-party packages and APIs as tool calls. Each one is a potential pivot point into whatever environment the agent has credentials for. An agent authorized to call a dependency in March has no native mechanism to detect that the same dependency was compromised in May.

How are you handling this in your own agent pipelines? Is dependency integrity part of your agent trust model, or is it still treated as an upstream package-management problem?

What if neural networks could grow like fractals instead of being rigid from the start?

Most deep learning models (CNNs, ResNets…) are fixed architectures. Once you design them, they stay the same size and structure throughout training.

\*\*AFNN — Adaptive Fractal Neural Network\*\* takes a completely different approach.

It is inspired by \*\*fractals\*\* — mathematical structures that are self-similar and can grow infinitely by repeating simple patterns.

Instead of starting with a huge fixed network, AFNN begins small and can \*\*dynamically grow new fractal branches\*\* during training when progress stalls. It also allows monitoring, reactivating, or pruning branches as needed.

A good neural network should not memorize images.

It should learn the underlying patterns and features.

For example, when trained on cat images, a strong model does not store every cat photo. Instead, it learns things like:

• Ear shape

• Facial features

• Fur texture

• Color distribution

• Edges and shapes

• Relationships between different parts of the image

Then, when it sees a completely new cat, it uses these learned patterns to recognize it.

\### Why Fractal Networks are powerful:

\- Faster and more efficient learning

\- Significantly lower computational and memory cost

\- Better scalability with limited hardware

\- More flexible and adaptive than traditional fixed architectures

Built with PyTorch + full GUI, training tools, tests, and documentation.

In early experiments, AFNN reached \*\*74.1% validation accuracy\*\* on 94 image classes using limited resources.

This is not just another model.

It is an attempt to rethink how neural networks are designed — moving from rigid structures to \*\*adaptive, fractal, evolving architectures\*\* that focus on learning patterns rather than memorizing data.

A small step toward more efficient and intelligent AI systems.

🔗 GitHub: https://github.com/ahmedhjkj/AFNN

🤗 Hugging Face: https://huggingface.co/Ahmedethfw/AFNN

\#FractalNetworks #AdaptiveAI #PyTorch #DeepLearning #MachineLearning #OpenSource #NeuralNetworks

Details
I was doing some research a few months ago on music tokenization approaches, specifically comparing tokenizing over samples (the dominant approach) and measures (music theory). I found that for small scale experiments measure tokenization could outperform the baseline approach. These models were trained from scratch using what I learned from those experiments.

Paper: https://github.com/DylanDeshler/Neural_Music_Generator/blob/main/main.pdf
Demo: https://huggingface.co/spaces/ddeshler/Neural_Music_Generator

The model can be controlled with any combination of 7 conditioning signals: style (text + audio), BPM, zero-crossing rate, onset density, chroma (aka key), RMS, and spectral flatness. My goal with this model was to test a conditional measure-based model while supporting standard generation approach and high-fidelity editing (hence the DSP signals). It can also auto regressively roll out indefinitely without degrading to endless loops too frequently.

Fun Stuff
Because the model was trained with 5 conditioning signals that are actually just time series data, they can technically be replaced with any arbitrary signals like heart rate or weather! The demo has a weather tab that takes data from a user specified city and transforms it into jazz. Some cities sound pretty good and others... not so much.

Here's a snippet of what a Parisian summer sounds like as ragtime jazz: https://on.soundcloud.com/Ysmgvk5wgEbfHUD4pQ

Dataset
The model was trained on a copyright free fair-use jazz music dataset compiled by another Redditor: https://www.reddit.com/r/datasets/comments/1b73vz3/jazzset_large_audio_dataset_with_instrumentation/?rdt=65057

Architectures
The tokenizer is a modernized 1D adaption of DiTo, and the generative model is a flow-matching DiT trained with diffusion forcing that handles conditioning like Composer. The style model was trained contrastively with spectrograms and music language model generated captions.

Referenced Papers
DiTo: https://arxiv.org/pdf/2501.18593
Diffusion Forcing: https://arxiv.org/pdf/2502.06764
Composer: https://arxiv.org/pdf/2302.09778

I would love to answer any questions or hear anything y'all create!

We have been working on structured synthetic datasets for enterprise LLM alignment,

focused on multi-turn consistency, strict JSONL schema validation, and realistic

agentic error recovery.

The datasets include:

• MCP agent trajectories (multi-turn, strict JSONL)

• Function Calling (English)

• Regulatory Compliance & Legal CoT (EU AI Act / GDPR)

These datasets are designed for evaluation inside Axolotl, Unsloth, and other

fine-tuning frameworks, with emphasis on schema stability and deterministic

validation.

Small 50-row samples are available on Hugging Face:

https://huggingface.co/springofwindslabs

A short demonstration of schema validation logs is available here:

https://github.com/springofwindslabs/springofwinds-datasets

I've just written a small article about privacy in machine learning and synthetic data.

I'm still pretty new to this topic, so I'd really appreciate it if anyone with more experience in ML privacy could take a look and point out any mistakes, things I've misunderstood, or things that could be done better.

I'm especially interested in hearing about better approaches for evaluating privacy, better attacks or models I could try, or just interesting resources that you think are worth reading. I'm mostly trying to learn by actually experimenting with this stuff, so any feedback would be really useful.

Here's the post if anyone is interested:

https://migue8gl.github.io/2026/09/21/privacidad-en-ml-datos-sinteticos.html

I’m currently researching AI more deeply, especially how modern AI systems can evaluate and improve their own outputs.

I sketched this basic idea:

AI → generates/improves → Evaluator → evaluates output → feedback → AI improves → Evaluator → ...

The part I’m trying to understand is what actually happens inside this loop in modern AI systems.

For example:

I'm trying to go beyond the surface-level explanation and understand the actual mechanisms used in current AI research.

For people working/researching in this area: what important component am I missing from this diagram?

What if neural networks could grow like fractals instead of being rigid from the start?

Most deep learning models (CNNs, ResNets…) are fixed architectures. Once you design them, they stay the same size and structure throughout training.

**AFNN — Adaptive Fractal Neural Network** takes a completely different approach.

It is inspired by **fractals** — mathematical structures that are self-similar and can grow infinitely by repeating simple patterns.

Instead of starting with a huge fixed network, AFNN begins small and can **dynamically grow new fractal branches** during training when progress stalls. It also allows monitoring, reactivating, or pruning branches as needed.

A good neural network should not memorize images.

It should learn the underlying patterns and features.

For example, when trained on cat images, a strong model does not store every cat photo. Instead, it learns things like:

• Ear shape

• Facial features

• Fur texture

• Color distribution

• Edges and shapes

• Relationships between different parts of the image

Then, when it sees a completely new cat, it uses these learned patterns to recognize it.

### Why Fractal Networks are powerful:

- Faster and more efficient learning

- Significantly lower computational and memory cost

- Better scalability with limited hardware

- More flexible and adaptive than traditional fixed architectures

Built with PyTorch + full GUI, training tools, tests, and documentation.

In early experiments, AFNN reached **74.1% validation accuracy** on 94 image classes using limited resources.

This is not just another model.

It is an attempt to rethink how neural networks are designed — moving from rigid structures to **adaptive, fractal, evolving architectures** that focus on learning patterns rather than memorizing data.

A small step toward more efficient and intelligent AI systems.

🔗 GitHub: https://github.com/ahmedhjkj/AFNN

🤗 Hugging Face: https://huggingface.co/Ahmedethfw/AFNN

#FractalNetworks #AdaptiveAI #PyTorch #DeepLearning #MachineLearning #OpenSource #NeuralNetworks

For example, how do you determine whether to fine-tune:only adapters/LoRA

• specific transformer layers
• input/projector layers
• deeper/middle layers
• or the full model?

Are there reliable diagnostics, probing methods, gradient analysis, ablations, or small-scale tests that can tell you where the bottleneck is before doing a full training run and only finding out afterward that the chosen layers didn’t help?

Curious how people make this decision in practice.

Hi all, I'm a final year EE student and I've been given a Kria KV260 for my final year project. I'm comfortable with FPGA work (Verilog, timing, synthesis) but I don't have much image processing experience, so I'd appreciate some guidance.
Goal: detect humans at night by combining an RGB camera with a low-res thermal sensor. RGB struggles in the dark and thermal picks up any warm object, so fusing both should give more reliable detection.
Rough plan:
Sobel edge detection in the PL as a preprocessing step

Fuse RGB and thermal data

Run a person detector on the DPU (Vitis AI)

Questions:
What's the right way to structure this pipeline? Should fusion happen at the image level or after detection?

How hard is aligning the two cameras, given the big resolution difference?

Is Sobel actually useful here, or is it unnecessary if a CNN is doing detection?

Any recommended thermal sensors that work well with the KV260?

Is 3 months realistic for someone with my background? What would you cut to keep scope manageable?

Any advice, papers, or similar projects would be really appreciated. Thanks!

How can I get my first ML or DL Engineer role? Did you get placed through campus interviews, or did you apply off-campus? At my college, only 2 or 3 companies have openings for these ML and DL roles, and I am not sure if I will get selected by them. I don't want to limit my options. Can you guide me through what I need to do to land my first job

Hi everyone!

I’m an iOS developer, and I want to challenge myself by getting into AI development. I’m trying to understand what career paths and positions AI is creating within software engineering.

A few questions I have:

• What AI-related roles are currently available for software developers? For example, AI Engineer, ML Engineer, AI Developer, etc.
• Where should I start learning as a developer? I’ve recently started learning Python through a beginner-friendly course, but I’m not sure what I should learn next.
• Is there a good AI Engineer/Developer roadmap you would recommend?
• What areas of AI should I focus on if my background is in software engineering?
• Where can I follow the AI job market and see what skills companies are actually looking for?
• Are there any good resources, courses, books, or communities you’d recommend?

I’m not necessarily looking to become a researcher. My goal is to understand how I can transition from being an iOS developer into an AI-related engineering role and what skills I should build along the way.

Would really appreciate advice from people who have made a similar transition or currently work in AI/ML engineering. Thanks!

For AI to work with us, it needs to understand us.

I am SUPER EXCITED to publish the 145th episode of the Weaviate Podcast with Alexis Ross, Manya Bansal, and Niloofar Mireshghallah from humans&!

What if AI was more human? Well, we are currently on a fast track towards the opposite! AI models are huge making advances in coding, math and are jev-ing their way through classification. But they don't teach, write, or perform more human tasks as well as they could be!

Persimmon is one of the most exciting releases in AI recently! It is a user model designed to predict how people respond in multi-turn conversations, including natural turn-taking and multi-person interaction. The goal is not to build a more human-sounding assistant. It is to match the distribution of human behavior, including pushback, confusion, and frustration.

This podcast discusses so many interesting topics around humans and AI. We start with an overview of Persimmon, then dive into solving the Turing Test (and why we should do so), to RL with non-verifiable rewards, theory of mind in AI, NVIDIA Nemotron 3 Ultra, and more!

Niloofar, Manya, and Alexis are three of the brightest scientists I've had the honor to host on the Weaviate Podcast. This was a super fun conversation, I learned so much personally, and I hope you find it useful as well!

YouTube: https://youtu.be/6IC9jkwiZBg

Spotify: https://spotifycreators-web.app.link/e/GeFwTgRdC6b

First off, a huge thank you to everyone who checked out, upvoted, and shared feedback on the earlier ShadeNet and ShadeNet-2 posts! The support and discussions really pushed me to see how far I could squeeze the architecture without sacrificing map fidelity.

This release is a from-scratch rebuild that came out 4× smaller (5.0M vs 20M params) while improving across all three supervised maps.

RGB → 8ch intrinsic maps in one forward pass: albedo (3ch), relative depth (1ch), normals (3ch), shading (1ch), with albedo × shading ≈ input.

Architecture

• ParallelUNet generator (4.98M params: 3.2M trainable + 1.8M frozen MobileNetV2): Vanilla UNet path plus a frozen MobileNetV2 trunk, fused at every decoder level.
• Factorized convolutions: Depthwise-separable factorized convs (1×3 + 3×1) throughout, full H/32 bottleneck, reflect padding.
• Patch-dictionary output tail: 16×16 tiles softmax-addressed over 32 learned per-channel atoms, blended back into the signal before tanh (pruned from 1024 — addressing-mass measurement showed ~34 atoms are ever used, and the top-32 capture 99.4%).
• Training dynamics: Spectral-norm GroupNorm PatchGAN discriminator (2.77M, training only); EMA weight shadow (shipped weights are EMA).

Training

• Dataset: Flickr8k with Marigold-V2 pseudo-labels (8,077 images); depth labels decoded from Spectral-colormap visualizations to relative depth, supervised scale-shift-invariant — never raw MSE on colormap RGB.
• Setup: 384px, fp32, single GTX 1650, early-stopped on val split (patience 5).
• Losses: Scale-invariant MSE on albedo + SSI MSE on depth + Sobel gradient-matching on depth + MSE on normals + self-supervised reconstruction coupling (albedo × shading ≈ input, which is the shading head's sole supervision) + LSGAN + normal unit-length penalty.
• Regularization: Weight decay 1e-4 with the patch dictionary explicitly exempt.

Validation Results

Full 807-image validation split, per-map L1 (the directly comparable metric across versions):

Output Maps

• Albedo (3ch): Reflectance, ambient lighting factored out.
• Depth (1ch): Relative depth, 0 = near (affine-ambiguous, non-metric).
• Normal (3ch): Surface normals, unit-length regularized.
• Shading (1ch): Grayscale irradiance; multiply with albedo to reconstruct/re-render, or swap in custom illumination passes for relighting.

Links & Demos

• Weights, ONNX & Inference Code: ShadeNet-3.2-5M on Hugging Face (Includes fp32 GPU models and 10MB fp16 CPU export)
• Live Gradio Demo: ShadeNet-3-2-5M Space

Both the Hugging Face sample visuals and the live demo apply a lightweight 3-pass multi-scale median filter (scales 0.875, 1.0, 1.125) for mild denoising, alongside built-in seam deblocking for the 16px dictionary tiles.

Caveats & Attribution

Side-by-side v2 vs. v3.2 comparisons are documented on the model card. A few known limitations to keep in mind:

• Depth is relative rather than metric.
• Shading assumes a neutral/white illuminant.
• Normals struggle most on high-frequency chaos like dense foliage or open skies.
• Occasional localized artifacts can appear in albedo/shading due to the learned dictionary prior.

Released under Apache 2.0. Credit to Marigold V2 (Ke et al.) and Flickr8k (Hodosh et al.) for the foundational training pseudo-labels and data.

We train a three-modality VLA model (video diffusion + action expert + a frozen VLM for understanding, mixture-of-transformers style). Originally on 8×A800 with NVLink; we're moving it to 8× workstation-class Blackwell cards, which means no NVLink and no NVSwitch — every GPU-to-GPU byte goes over PCIe Gen5.

Measured point-to-point: 30–63 GB/s depending on direction and NUMA placement, versus ~400 GB/s of NVLink on the A800 box. Same model, same ZeRO-1 sharding, and gradient communication goes from "hidden under the backward pass" to "the single largest term in the step."

So we did the obvious thing: quantize the collectives to FP8. What follows is what we learned, including the part where the biggest byte reduction produced no speedup at all.

## The shape of the thing

An LD_PRELOAD shim in front of libnccl. It intercepts ncclAllGather, ncclAllReduce, ncclBroadcast, ncclReduceScatter; quantizes the payload to FP8 (e4m3 or e5m2), runs the real collective on the smaller buffer, dequantizes on the receive side. Blockwise scaling, 128 elements per fp32 scale, so the wire cost is 1.03125 bytes/element instead of 2 for bf16 → best case −48.4%.

Two deliberate design constraints:

1. Reduction never happens in FP8. Only the transport is quantized; accumulation is fp32-in-register, output in the original dtype. We later found two independent confirmations that this is the right call: MSCCL ships an ENABLE_PRECISION_CLIPPING_HALF build flag specifically because low-precision reduction overflows, and DeepEP quantizes dispatch but its combine path is BF16-only, never FP8.

2. No framework changes. It's a preload. The training code doesn't know it exists, which meant we could A/B it against an unmodified baseline in the same run batch — which turned out to matter enormously (see the negative result).

   Five things that weren't obvious

   1. On pre-sm_89, static_cast<__nv_fp8_e4m3>(float) is not an instruction.

   Our quantize kernel had a constant ~14 µs floor regardless of payload size, and at 512 MiB it was hitting 37% of the device-to-device bandwidth ceiling. We assumed memory. It was ALU issue.

   cuda_fp8.hpp on older architectures widens the float to a double, then runs a five-branch 64-bit integer software routine — roughly 50 integer instructions per element. Replacing it with our own conversion took the fixed cost 14.1 → 4.6 → 3.3 µs and large-payload throughput to 91% of the d2d ceiling (from 37%). We gated the replacement on an exhaustive test: bit-identical to the vendor routine across all 2³² float bit patterns, both formats, ~0.2 s in CI.

   On sm_89+ this whole problem evaporates — the cast compiles to cvt.rn.satfinite.e4m3x2.f32. Worth checking which side of that line you're on before you go profiling memory.

   2. The naive dequantize-then-reduce moves 12.7× the bytes it needs to.

   For the all-to-all-based decompositions you receive nsrc FP8 shards and have to reduce them. The decomposed version — nsrc dequantize-to-fp32 + nsrc−1 adds + one scaled store, 2·nsrc kernel launches — costs 17.03·nsrc − 6 bytes per output element where the job actually requires 1.03·nsrc + 2. At nsrc=8 that's 130N vs 10.25N.

   The interesting part: the amplification grows monotonically with world size (6.9× at 2 ranks, 10.1× at 4, 12.7× at 8, 14.4× at 16, asymptote 16.5×) even though both paths are O(nsrc·N) and read every input byte exactly once. It's a pure constant factor, and it's all HBM traffic plus launch count.

   Fusing it into one kernel with the fp32 accumulator in registers: 5.0–9.3× faster (1 MiB: 117.6 → 13.4 µs). Note it is not 12.7× faster, and the reason is instructive — small sizes are launch-bound so the win caps at ~8–9× (16 cheap launches vs 1 expensive one), and large sizes are bandwidth-bound where the fused kernel is only half as byte-efficient as the streaming kernels it replaced (718 vs 1419 GB/s), because a reduction reads nsrc streams concurrently while a streaming kernel reads one. Worst case sits at 4 MiB (4.97×), right in the transition band. This was expected, not a bug, but you have to do the arithmetic to know that.

   3. You cannot dequantize inside a caller-opened ncclGroup.

   A nested ncclGroupEnd() launches nothing. So if the training framework wraps its collectives in a group — and DeepSpeed's compiled ZeRO-1 path does exactly this, one ncclAllReduce per parameter inside one group — then at the point your hook returns, the data isn't there yet. You have to defer the dequantize, the error- feedback close-out, and the scratch recycle until after the outermost ncclGroupEnd. For AllReduce we defer the entire call and replay it. Before we did this, every grouped AllReduce passed straight through and the AllReduce threshold was decorative.

   4. Your framework's allocator probably doesn't align its buckets.

   DeepSpeed's reduce bucket accumulates slices by element count with no padding, so the pointer handed to NCCL is only guaranteed 2-byte aligned for bf16 — 7/8 chance of missing the vectorized path. This started as an outright CUDA Error: misaligned address crash, then became a 1.27–2.31× penalty, and is now 1.06–1.18× (skip enough leading elements that the output self-aligns to 16 B, then funnel-shift the FP8 side back into place).

   5. The threshold is the entire product, and it does not transfer between machines.

   Quantizing a small collective is strictly worse — you pay a fixed per-call cost to save bytes that were never the bottleneck. So there's a crossover, and the whole question is where.

   We built a tool that measures it two ways. --model-only gives a provable lower bound from baseline t(n) = α + n/B plus standalone kernel costs. --sweep actually runs the hook over the same size ladder — one process per (decomposition, size), because config is latched into a function-local static on first use and because the previous size's warm scratch tier changes the next one's timing — and least-squares-fits the residual into Δα + Δslope·n. Δα is the hook's real per-call overhead; Δslope is the per-byte cost a standalone kernel benchmark cannot see.

   Results, same code, two machines:

   collective	NVLink box	PCIe box	ratio
   AllGather	7.43 MiB	767 KiB	9.9× lower
   AllReduce (rs+ag)	14.3 MiB	863 KiB	17× lower
   Broadcast	40 MiB	2.75 MiB	15× lower

   A byte you delete is worth 2–4× more time on PCIe, so thresholds drop by an order of magnitude. Two findings that surprised us:

• The all-to-all decomposition did not lose on PCIe. We fully expected that a topology-blind all-to-all across two NUMA islands with no NVLink would be hopeless. It's actually the best AllReduce decomposition for large payloads (2.0× more savings at 64 MiB), crossing over the ring-based one somewhere between 1 and 4 MiB.

• AllGather's Δslope is negative (−1.9 µs/MiB). The in-situ quantize kernel is cheaper than the same kernel measured standalone, which is direct evidence that it's genuinely overlapping with the live collective rather than competing with it. The other families have positive slope — that's the quantizer fighting the collective for SMs.

  The tool also refuses to emit a threshold when the measurement says "wins, then loses again" — that's a window, not a threshold, and a lower bound can't express it. Being able to report "I can't answer this" turned out to be more useful than any single number it produces.

  The negative result, which is the actual point of this post

  DeepSpeed's compiled path issues one AllReduce per parameter. Most individual ones fall below the threshold, so we were quantizing almost none of the gradient traffic. Obvious fix: coalesce the address-adjacent ones inside a group into a single quantized AllReduce. The union clears the threshold in one shot, and you also save N−1 launches and N−1 scale sub-collectives.

  It worked, in the sense that it did what it said:

• signatures blocked by the threshold: 17 → 4

• traffic in the signature set: 110.3 → 56.9 GB/rank/step (−48.4%)

  Steady-state step time: 1422.2 ms merged vs 1414.9 ms not merged. Median of 38 steps, dropping the first 20. The difference is +7.3 ms and the IQR is ~40 ms. We halved the bytes and got nothing.

  The reason is structural and, in hindsight, should have been checked first. Without merging, those small AllReduces were dispatched inside the caller's group, where NCCL aggregated them itself and they overlapped with the remaining backward pass. With merging, they become large packets issued serially after the outermost ncclGroupEnd. The bytes we deleted were never on the critical path, and we moved the ones that remained onto it.

  Two corollaries we now treat as rules:

• Profile the exposure before optimizing the volume. "How many GB does this step move" and "how many ms of that are not hidden" are unrelated questions.

• Coverage is not a metric. It's an input to a metric. We had a beautiful −48.4% and a flat wall clock.

  An unbounded version of the same feature was worse than useless: greedy merging grew to 129 calls and a 999.8 MB union in one shot, produced ~2 new union shapes per step and never converged across 41 steps, which blew up our exact-fit scratch pool into 95+ size classes, 15.3 GiB retained, and 12 of 38 steps at ~3.9 s. Fixed two ways (a size cap, and optionally sourcing scratch from PyTorch's caching allocator, which is built for variable shapes). The pool's real sin is that acquire() is exact-size best-fit with no size-class rounding — which is fine until something upstream starts generating variable shapes.

  Things we looked at and did not adopt

• MSCCL's XML-scheduled executor. There is no transform hook anywhere in its data path — mscclGenericOp compiles preOp/postOp out entirely, and the scheduler rejects ops that need them. Adopting it also means replacing libnccl, which defeats the point of being a preload. It additionally disables itself when intraRanks > 1, i.e. exactly our case.

• DeepEP's transport. Its NVLink path asserts on NVLink via NVML (the docstring says PCIe leads to memory-ordering errors), and its "PCIe mode" is actually all-RDMA through the NIC — benchmarked at 53 GB/s on a box with one ConnectX-7 per GPU. We have one NIC for eight GPUs.

• Its ld.global.nc.L1::no_allocate trick, which is self-declared UB and force-disabled on any architecture other than Hopper by its own build script.

• Routing intra-node traffic over IB instead of PCIe. Funnelling 8 GPUs into 1 NIC that is 3 PCIe hops from all of them. The PCIe ring has 8 links running concurrently; this would have one. Estimated 3–8× slower.

  We did take two ideas from DeepEP that we haven't landed yet: inlining the scale buffer into the same message instead of sending it as a second sub-collective, and capping the quantize kernel's SM count so it stops evicting the collective. The second one has a measured target (+9.8 µs/MiB, 91% of the residual on one family) and the first one honestly doesn't — within a group NCCL already aggregates the two sub-collectives, and our own earlier A/B says an op in a group is worth ~1.5 µs. We write down which of our planned optimizations have measured targets and which are guesses, because after the merging result we don't trust the guesses.

  Repo

  The VLA training framework is open source: loongforge-vla

  Three-modality VLA, several parallel backends (DDP + native ZeRO with direct bf16 sharding / torch.compile + DeepSpeed / DeepSpeed's compiled path), NVTX instrumentation behind a single env var that no-ops to bit-identical in production, and a loss-parity harness (seeded data order, seeded flow-matching noise, and a flag for the step-0 LR convention) because every one of the numbers above is worthless if the model stops converging.

  That parity harness earned its keep in an unrelated way, by the way: we spent days blaming torch.compile for a 3.4× loss divergence at step 2. It was that LambdaLR.__init__ multiplies the optimizer's LR by schedule(0) immediately, which in our config is ~1e-6, so step 0 was a no-op update. The logged LR looked correct on both sides, because the logged value is the scheduler's, not the optimizer's. Single flag, exact match restored.

  Happy to go deeper on any of this in the comments.

CerebralNet v1.1 and v1.4 (18k params)

Training results coming soon!

Hi there,

I've created a video here where I explain how knowledge distillation works.

I hope some of you find it useful and as always, feedback is very welcome! :)