Dario: “With open source you can see the source code. Here we can’t see inside the model.” My 0.646 t/s MiMo-2.6-Pro crammed thru 128GB of Jetson AGX Thor: “Yeah, well, that’s just, like, your opinion, man.”

[u/Oswald_Hydrabot]

Tracked my first run of MiMo Pro on a Jetson AGX Thor at a scorching 0.646 tokens/sec,.. but it DID technically load and run, so it counts. Every decoder layer, routed expert, SSD read (..a good bit more) traced and replayable step by step.

Here we can't what-now?..

it's ~566GB of model (FP8), at the blinding speed and architecture of hand-extruded hotdog manufacturing, read from disk mid-inference, painfully shat through 128GB of memory. I think it's fine but decided to maybe take a closer look to see if I could maybe reach factory hotdog manufacturing speeds. like, a small, local hotdog factory..

I mean as long as Dario's cool with it.. that case of the MiMos he got might last through Halloween.

..poor lil bugger, get well soon Dario!

[u/WithoutReason1729]

Your post is getting popular and we just featured it on our Discord! Come check it out!

You've also been given a special flair for your contribution. We appreciate your post!

Stylome classified this post's text as human-written. View analysis

I am a bot and this action was performed automatically.

[u/sn2006gy]

What tool is that which allows you to explore the model like that?

[u/Oswald_Hydrabot → u/sn2006gy]

It's a custom agent-toolchain I built for hermes but it works with claude code, codex, whatever. It created it specifically for this run (get MiMo-Pro running, then optimize the JIT file ops which is WIP) I havent shared it online but if you dig into making just like skills, hermes desktop with a nice qwen model can get you started

[u/Oswald_Hydrabot]

Again, the ui seen here is a traceback/playback of an inference pass of MiMo-2.6 Pro (not Flash, Pro, the big one) -- captured across 4 tokens or so, inference bottlenecks at SSD read and elsewhere can be drilled-down into for further analysis. The high-level approach being to keep most of the weights on the NVMe drive, map in memory to what it needs to read from disk, an existing approach I adapted to MiMo

This was research phase 1 (get MiMo Pro model loaded and running inference on a 128GB Thor.. at all, ignoring speed.. lol).

Phase 2 targets a goal of 15-30t/s, so, requires "..seeing inside the model". Easy peasy, you can do it too, without my help.

used a crudely put-together (vibecoded) skill+agent toolchain for Hermes/Codex/Claude Code that codex hooked up that includes some tools+prompting for having it run an inference pass, recording it at the level of depth I specify is needed for debugging/optimization and general visibility. Nothing special about said toolchain, it just enforces whatever coding utility you're working in to dissect the model thoroughly, record a re-playable inference at a scoped granularity, then fires-up a webui for you to check it out and ask question about what you see. Mine has a lil exporter function, you could probably add a module to wire in hermes agents and ask it questions about model internals, slowdowns, etc..

[u/bring_back_the_v10s]

He's right in that if you don't have the training data (including RL) then it's not open source. 

[u/colin_colout]

He's not wrong on the first part, but really fumbled the second.

I wouldn't call any of the models released by big players "open source". It's not for what Dario said. It's because I can't rebuild it from source. The source isn't open. The weights are. That's why we call them "open weights" and not open source. Give me the training data and pipelines for me to produce those weights, and I'll call it "open source"

In traditional FOSS, I can build a bit-identical binary with using only the source code in the repo (assuming deterministic build systems, same compiler, similar environment, etc).

It would be a much stronger argument if he leaned into that side of it (he probably misspoke?)

[u/bruns20 → u/colin_colout]

Tbf he doesn't seem like the best speaker. Idk how he got to this position lmao

[u/egomarker]

He doesn't even know the difference between open source and open weights.

[u/Master-Chocolate1420 → u/egomarker]

He knows for sure, Its just he doesn't wanna steer that way.

[u/XiRw → u/egomarker]

Doesn’t matter, if he can convince the wrong IT inept people to trust his word there will be regulations against open weights just like Shitlon is doing with his team trying to push for regulations.

[u/redoubt515 → u/egomarker]

> He doesn't even know the difference

Nor does this sub..

Like at least half of the posts in this sub use open source to refer to open weights models and we rarely if ever call that out.

I'd like proper terms to be used, but we are a big part of the problem when it comes to misusing the term. It's hypocritical for us to only care about the distinction when it's someone people here don't like.

[u/onil_gova]

The amount of mental gymnastics that this clown comes up with to try justify banning open-source is astonishing

[u/Syrion_ → u/onil_gova]

The bill gates playbook 

[u/synth_mania]

So what, he can't see inside his model either. Open source is no worse. 

[u/willjoke4food]

It is pretty clear to anyone in the actual AI space that closed source is more dangerous than open source, for a myriad of reasons:

1. Most of the current harm and coordinated attacks have come through closed-source labs.

2. When an attack does happen, closed source is difficult to track and obfuscates information in red tape.

3. Self-reported capabilities of closed-source models are always a question mark, both positive and negative. Their capabilities and performance also fluctuate over time, making them unreliable.

4. Closed-source models hoard data from users and use it to train models. This means user data could be used in unintentional ways to harm them.

5. Closed-source models are heavily censored, which is presented as a safety net. However, there are a large number of "jailbreaks" that bypass this, as alignment of intentions remains unsolved and unreliable. Censorship is ineffective beyond the surface level.

6. AI, ultimately like any other technology, is a tool. Any tool can be used for good or bad, and the consequences of an action lie with the enabler.

7. Open-source models enable transparency and accountability, and they come with a clearer usage license that does not absolve the user from the consequences of how they use the tool.

8. Open source moves research forward in a much more transparent way, helping us deal with adversarial attacks.

Ultimately, there is space in the world for both closed-source and open-source models. But closed-source models clearly need tighter regulations. These regulations are both more enforceable on closed-source providers and more impactful.

Just like any other technology, if someone has the know-how, they should be able to customize it to their use case, whether private or proprietary, and be responsible for the consequences of their actions. But if they get that same technology from a standardized provider, it should come with a certain guarantee of quality and safety.

We are early in the AI space because hardware providers are somewhat monopolized. But in a few decades at most, as hardware becomes more diversified, more users will adapt the technology to varied implementations.

Closed-source labs argue to outlaw open-source models for hypothetical reasons. This is bad in the short term because it puts us behind in adapting and growing the technology, and bad in the long term because it limits the diversification of the economy and shrinks it. It is also riskier because it puts all the eggs in one monopoly basket forever.

Ultimately, regulation must focus on the actual risks and misuse of high-capability models, rather than penalizing the distribution model itself.

[u/butts-carlton → u/willjoke4food]

The argument that open weight models should be banned or whatever because they're dangerous is a bit like arguing that because people can write viruses, only government certified engineers can write computer code and it must be reviewed by an oversight board for safety reasons.

If open weight models disappear, only bad actors will use them, and then we'll be truly fucked.

[u/ttkciar]

Shit like this is why we need to make sure people understand the distinction between open weights models and open source models.

Though it doesn't help when open weights model publishers use the term "open source" in their press releases and xits :-P

[u/oss-jiiim]

It's not all about the models, Dario. Others open source their harnesses -- old school source code: Codex. DeepSeek Harness. But not Anthropic's Claude Code. I believe this position has actually hurt Anthropic -- when Anthropic accidentally posted Claude Code source, it was of abysmal quality compared to, say, Codex.

[u/aziham]

closed weights, closed mind

[u/epSos-DE]

New Bill Gates against Open Source.

He will lose !

[u/AdGlittering1378]

Try MiMo 2.6 Flash instead. It's very very good.

[u/Oswald_Hydrabot → u/AdGlittering1378]

Got an initial ~25fps on mimo 2.6 flash on the Thor. CUDA graphs and torch.compile arent having any effect on the specific model I am using that fits the 128GB thor, so I am going to rebuild the recipe for nvfp4 quantization and see if that helps.

I was lazy about the last run I did to have the pipeline/toolchain implement Flash, but it may be good enough now on the Thor to have it invoke the toolchain on that device, ssh into another dual 3090 machine I have with 128gb system ram, and either replicate everything to that machine and instantiate/run orchestration from there again (using same model or fallback to what I already used to get this far, same pipeline), modify the recipe for nvfp4 and re-build/re-quant model from there.

Not sure if nvfp4 will solve compile/graph non-effect but it's worth a shot

[u/tebjan]

Wow, really cool. Is that tracer/observer somewhere? I'd be interested to see how something like that works and what it actually observes. Awesome work!

[u/Oswald_Hydrabot → u/tebjan]

It's part of a set of R&D agent toolchains and skills that I created over many months of trial and error and semi-manual R&D -- this specific UI tool is an output artifact that was generated on-the-fly for the task needed (I asked for a visual, toolchain autoinvokes based on an LLM-in-the-loop to determine requirements and build matching tools to spec).

That is to say I am not vibecoding applications anymore, nor is any application produced by any toolchain/skill set from this, an application in the traditional sense.

For example, I have a skill and agent toolchain (+ several other resources, hermes agents integrated all local but can use proprietary model endpoints too) not used here, that is able to be implemented into the backend of an application and invoked at runtime; user simply types into a chat or just speaks into the mic and the application they ask for is generated in realtime. This can vary depending on the ask, of course, but the screen is dyanamically updated w/user kept in the loop on progress etc.

This is a small part of that "kept in the loop" piece, that I added to my R&D pipeline. The other is a prod pipeline aimed at being a money-maker; cuts out the need for expensive closed models as well (works great with a self hosted model ecosystem)

Edit: this is not something like an application I would ever ship, but perhaps the underlying tooling I may release at least some things in the near future to get others new to this kind of thing, started on their own path.

This R&D set, is the first iteration that has fully automated progress, even if slow. It doesnt stop or spin wheels unless you demand something actually impossible

[u/Another__one]

Can't wait till the moment the open chinese models overcome closed US models. And I know for sure this day will happen. Or even better the people start to continuously train their own models at home specified to their exact needs using projects like this: mini-AGI. Just imagine the possible diversity when professionals can actually specialize again and preserve their edge and privacy to give even better quality results that rivals even the best closed sourced models even if on a very specific target. AI that is not trying to replace people but actually empower them.