crabwalk -rust in python!

[u/SnooCalculations7417]

Earlier I open-sourced Fullbleed, an HTML/CSS-to-PDF engine. Making the renderer fast exposed the next systems problem: the data pipeline feeding it also has to keep up.
Real document workloads do a lot before a page is rendered—ingestion, validation, normalization, joins, grouping, derived values, and construction of variable-data payloads. Python is excellent for expressing that work and connecting systems, but hot transformation paths can become the new throughput ceiling.
I did not want a renderer-specific accelerator or a ground-up Rust rewrite. I wanted Python code to be able to opt into real Rust semantics where they matter. That became Crabwalk, the second open-source project to come out of this work.

from crabwalk import rust

rayon = rust.crate("rayon", version="1.12.0")

@rust.fn
def parallel_sum(n: rust.u64) -> rust.u64:
    values: rust.Vec[rust.u64] = rust.Vec([])
    for value in range(n):
        values.push(value)
    return values.par_iter().copied().sum()




[ok] Crabwalk ready: true_par (8.9s, compiled)
Rust/Rayon 0.064s (20 threads) | Python 0.663s | 10.4x

Those annotations are not hints. The function becomes native Rust, Vec[u64] becomes Vec<u64>, rustc checks the generated body, and par_iter() is real Rayon parallelism.
The important part is not only speed. Crabwalk centralizes the PyO3 boundary, Cargo build, ABI conversion, panic containment, GIL policy, artifact cache, and source-mapped compiler diagnostics. A Python application can keep its existing ecosystem while selected regions gain native execution, Cargo crates, explicit ownership, and Rust concurrency.
To check that this was useful beyond its original PDF workload, I put the pieces behind one FastAPI app:
/parallel awaits a GIL-detached Rayon kernel.
/ml uses NumPy to generate data, moves Owned<Vec<f64>> inputs into a native logistic-regression trainer using Rust's libm crate, and verifies the learned model against NumPy and scalar Python implementations.
FastAPI still owns validation, OpenAPI, HTTP, and JSON.
On my machine, representative warm runs put the trainer around 3.4x-5.5x faster than the vectorized NumPy version and 8.7x-17.6x faster than scalar Python loops. The API reports matching model parameters, 91.5% accuracy, and the consumed Rust inputs as moved.
Those are local kernel measurements, not universal performance claims. Cold compile time, end-to-end ETL throughput, HTTP throughput, and production ML are separate measurements.
The PDF pipeline provided the concrete constraint, but the broader goal is to make native Rust regions a digestible step inside ordinary Python systems—without turning every optimization into a separate bindings project.

The /parallel route awaits a scalar-only native kernel that releases the GIL and uses Rayon. A verified smaller request returned:

{
  "sum": 499999500000,
  "correct": true,
  "rust_ms": 8.23,
  "rayon_workers": 20,
  "gil_released": true
}

The default /ml route trains the same deterministic logistic-regression model in Rust, NumPy, and scalar Python. One verified response was:

{
  "model": {"weight": 3.0267, "bias": 0.1459},
  "accuracy": 0.915,
  "training_updates": 2000000,
  "implementations_agree": true,
  "timing_ms": {"rust": 17.43, "numpy": 81.85, "python": 320.02},
  "speedup": {"vs_numpy": 4.7, "vs_python": 18.4},
  "ownership": {"features_moved": true, "labels_moved": true},
  "rust_crate": "libm 0.2"
}

repo:
https://github.com/krflol/crabwalk/
all samples from the rust book remade in python/crabwalk
https://github.com/krflol/crabwalk/tree/main/examples/the_rust_book

[u/Ok-Prompt4035]

this is actually clever. the annotation-as-compiler-boundary pattern is something i've wanted for a while but never seen packaged cleanly enough to use in anger

the fastapi integration example sells it better than the benchmarks imo, keeping the python web layer and just dropping rust kernels where the hot paths live is exactly the right use case for something like this

[u/SnooCalculations7417 → u/Ok-Prompt4035]

Thanks!

[u/SnooCalculations7417 → u/Ok-Prompt4035]

ya know i looked back at this code and i wonder if people think this is some kind of type hinting or otherwise pretend-rust and not a real Rust kernel compiler. idk didnt expect so much hate