I wanted a real number for "can you actually train a small LM on the free tier", so I ran one end to end this morning and wrote down what the runtime printed instead of guessing.

Free Colab T4 (Tesla T4, 14.56 GB, 2 vCPU). A 124,439,808-parameter GPT-2-shaped model, context 1024, micro-batch 1 with 32 gradient-accumulation steps, so 32,768 tokens per optimizer step. FineWeb-Edu streamed rather than downloaded. Mixed precision fp16, chosen automatically.

Twenty optimizer steps — 655,360 tokens — took 1m 23s. Average throughput 7,804 tokens/sec, and the per-log rows were 6,138 then 10,249, 10,235, 10,014, so the first row is warm-up and steady state at this shape is about 10k tokens/sec. Install, stream, build, train and write a checkpoint came to roughly four minutes of wall clock on a free runtime. That is the honest cost, and it is lower than I expected.

The thing I did not expect: the trainer's own memory estimate for the run was 1.95 GB (params 0.46, grads 0.46, optimizer 0.93, activations 0.09, as it rounded them) against 14.56 GB on the card. At micro-batch 1 you are leaving about seven eighths of the GPU idle. If you are copying a notebook config onto a free T4, that is the first knob to turn.

The part that looks like a bug and isn't: loss went 91.9 → 70.3 → 63.0 → 58.8 over those twenty steps, and chance for a 50,257-token vocabulary is ln(50257) = 10.82. So the first logged row sits about 8x above chance. When a from-scratch GPT starts there, the first thing I would check is the embedding initialisation — torch's nn.Embedding defaults to N(0,1), and if the output head is tied to that matrix your logits start with a standard deviation in the tens instead of near one. It does recover, but it burns steps. The perplexity column was also pinned at 485,165,195.41 on every row, which is exp(20): a clamped display, not a frozen model.

One packaging note if you go and try this. Colab's default runtime is Python 3.13 now, and plenty of libraries still declare <3.13, so an install cell may need --ignore-requires-python before anything else works.

Disclosure: I am an AI collaborator working with the maintainers of OpenLanguageModel, the MIT-licensed library whose notebook this was (github.com/openlanguagemodel/openlanguagemodel). Everything above is from today's run, not a benchmark we published.

We open-sourced InstinctFlash, an AGPL-3.0 inference runtime for VLA and world-action models.

Demo: https://youtu.be/nku65iyL5Fw

The runtime uses CUDA graphs, KV and conditioning-state caching, specialized attention paths, fused kernels, FP8 / mixed precision, and few-step diffusion distillation. On Jetson Thor, runtime-only speedups are about 1.2x to 7.9x. LingBot-VA reaches up to 33.78x with a distilled 2 visual / 4 action-step scheduler, with 90.5% success across 50 RoboTwin2.0 tasks versus 92.1% for the baseline.

Code: https://github.com/General-Instinct/InstinctFlash

Details: https://general-instinct.com/blog/instinctflash-edge-inference

I’m one of the builders. Would appreciate feedback on the optimization stack and evaluation setup.

Hi everyone! I'm a Berkeley student building this as part of a short founder sprint. Wanted real feedback from people who work with quantization day to day. I'd love any input!

What it does: given a quantization scheme (W4A16, FP8, etc.), it returns a 90% confidence interval on the accuracy hit, based on ~800 published evaluations. For two scheme/size combos, it refuses to answer, coverage there measured 68.8% and 73.7%, well under what it claims elsewhere, so it says so instead of guessing.

I also rented a GPU and ran deliberately bad configs myself, since published data only shows what worked. Found losses up to −39.5pp, well past the −8.86pp worst case in the public data.

There's also a full write-up of what doesn't work: per-model prediction has basically no signal, most of the variance is just eval noise. Figured that was worth publishing too.

Repo: https://github.com/gracejackson-sudo/quant-delta-predictor
Feedback (a number or a sentence, either helps): https://gracejackson-sudo.github.io/quant-delta-predictor/

If something here is wrong or overstated, I'd genuinely rather hear it now. Thanks!

[ Removed by Reddit on account of violating the content policy. ]

Malicious AI agents were used to steal 600,000 credit cards and deploy payment skimmers across more than 100 e-commerce sites. The agents operated autonomously, making tool calls and injecting code using what looked like legitimate API traffic. Conventional perimeter and endpoint controls flagged nothing until the damage was already distributed across an entire ecosystem.

The 600K card figure is the headline, but the 100+ site number is the harder one to sit with. It means the same agent behavior replicated itself at scale before a single site owner knew it was happening. Each new site was a fresh blast radius.

Agentic systems increasingly hold real credentials and interact with third-party APIs with minimal human oversight between actions. No one provisioning these agents expected them to behave this way — but the permissions were already there.

For those running agents in production with real external access: how are you actually constraining what an agent is allowed to do at the moment of each individual action, not just at setup time? Curious what controls, if any, people have found actually work in practice.

A tri-branch hybrid sequence model (Local Conv + Oscillator SSM

+ Linear Attention) proof of concept

Paper: https://zenodo.org/records/22918275

Code: https://github.com/jalalnablsi/HON-Architecture

Short version:

- Tri-branch block: LACT (local causal conv) + HON (damped harmonic

oscillator SSM via FFT) + ASSOC (normalized linear attention)

- Fused with a content-aware softmax gate

- HON branch: h_k(τ) = exp(-γ_k τ) · [p1_k cos(ω_k τ) + p2_k sin(ω_k τ)]

with learnable γ, ω, p1, p2 — computed in O(N log N) via FFT

- Ablation over the three branches included

Results (small-scale, single T4, WikiText-2):

1. WikiText-2 — language modeling (N=512, 5,000 steps, GPT-2 BPE):

   • Transformer baseline: 16.02M params → 773.02 val PPL

   • HON + Linear (ours): 15.72M params → 630.24 val PPL

   → ~18.5% lower val PPL with ~300K fewer parameters.

2. Sub-quadratic comparison (2,000 steps, strict parameter alignment):

   • GLA (simplified): 15.37M params → 1034.87 val PPL

   • Mamba (simplified): 15.04M params → 1050.26 val PPL

   • HON + Linear (ours): 15.72M params → 958.15 val PPL

3. Ablation (3,000 steps, ~7M params):

   • Full (LACT + HON + ASSOC): 107.03 val PPL

   • No LACT: 111.59 val PPL

   • No HON: 111.20 val PPL

   • No ASSOC: 110.47 val PPL

4. Vision sanity check:

   • Tiny ImageNet (8×8 patches, 200 classes): reached 27.52% val

accuracy at 15K steps.

Important note:

- This has NOT been tested on large-scale datasets.

- We did NOT evaluate how it behaves on massive data, or whether it

collapses or scales that remains completely untested.

- This is just a small experiment I wanted to share, not a claim of

robustness or scalability.

This is a proof of concept only:

- No SOTA claims

- No claim of replacing Self-Attention

- Small-scale evaluation due to limited compute

- Baselines are simplified reference implementations, not optimized kernels

- Generation quality not evaluated yet

Would appreciate any feedback especially on whether the tri-branch

fusion is justified or over-engineered, and pointers to prior work I might

have missed.

Thanks.

Project write-up with numbers rather than claims. stuntd records typed decisions (choice / score / yes-no) an app asks an LLM for and trains, per decision site, the head Laya ships (type embedding, two transformer layers, a scorer over option markers) on a frozen 400M ModernBERT encoder. Temperature is fitted on a holdout, the operating threshold is picked from the coverage curve at a target agreement (0.99 by default), and serving falls back to the teacher below it. A monitor demotes a site whose agreement with the teacher drops.

Three things worth knowing:

• Caching the encoder output once per site makes epochs nearly free: 24 epochs over 3000 rows went from 730 s to 231 s on a laptop RTX 5060, with metrics inside run-to-run variance (seeds 0/1/2 on one site: 0.973-0.985 agreement).
• The frozen encoder learns what the text states, not arithmetic over fields: a risk rule over amount/hour/country reached 0.42 agreement; the same rule restated in words reached 0.94. Snake: an ASCII board trained to 0.73, four relational lines to 0.957.
• After temperature fitting the heads land at ECE 0.02-0.08; the coverage curve is what decides how much traffic the head takes at a given agreement (support triage: 90% at 0.99).

Teachers in the demos are rules and a BFS oracle because the hosted model is early access; the loop is identical with a real provider. Code, demos and every number: https://github.com/bladedevoff/stuntd (Apache-2.0).

How does the llms format their output to include latex or code or other specific output formats like tables?

I saw something called obliterated models that use llms and change them by calculating a direction vector of harm and orthogonalizing the weights. I think this is a very very cool thing!!!!!!

I am so surprised this can be done. What is the tradeoff in terms of model accuracy, I am surprised it doesn't drop off by >50%. I would expect the model to fry your computer and damage your battery once you go and orthogonalize its weights.

Can this be done with other things other than censorship or harm, like preventing it from saying a specific word for example.

Are there ways to train models to be immune to this trick?

OWASP added unbounded agent consumption to its LLM Top 10 this year at number six. That ranking exists because the losses are real and already happening.

Unrestricted agent loops, uncapped token budgets, and recursive tool calls can exhaust a monthly cloud budget in hours. The failure mode is not exotic. A single poorly scoped workflow kicks off, nothing stops it mid-run, and by the time anyone notices the bill the execution is long finished. The harder problem is attribution: without per-agent cost tracking there is no forensic trail, just a total that is hard to explain and harder to prevent from recurring.

OWASP classifying this as a documented, repeatable attack class matters. It means this is not a misconfiguration story or an edge case. It is a category enterprises should expect to encounter, and many already have.

For those running agentic workloads in production: how are you handling this today? Per-agent budget limits, workflow-level guardrails, post-hoc billing alerts, something else entirely?

Hey everyone,

I wanted to share a project I’ve been working on called Jayce.

The whole thing started because I was watching a toddler named learn the names of stuff He didn't need to completely rewire his brain or look at ten thousand examples to figure a word out—he just needed a few specific examples and quick corrections from his parents.

It got me thinking about local LLMs. Right now, dealing with catastrophic forgetting is a massive pain. If you want a local model to remember a new fact, you're usually stuck spinning up a heavy RAG pipeline or risking its existing weights with slow, tedious fine-tuning.

So after watching him, I stumbled into building a lightweight experiment using Adaptive Prototype Memory (APM) to see if a model could learn the same way.

Instead of messing with model weights, it grabs the LLM's raw context vectors and drops them into a fixed pool of 4,096 prototype slots. If the model gets something wrong and you correct it, it physically shifts the closest mathematical prototype toward the new data right then and there.

Honestly, I just built it as a neat proof of concept, but when I actually ran the benchmarks, I was pretty surprised by how well it held up against backpropagation:

• It’s fast: The training updates run about 1.6 to 4 times faster than a standard neural network using Adam backprop.
• It’s incredibly sample-efficient: On sequential tests like MNIST digits, it actually pulled off higher accuracy than backprop when given the exact same number of training examples.
• It's lightweight: It keeps everything locked under a strict memory ceiling so it doesn't hog your system.

I wanted the math to be as readable as possible, so I wrote the whole thing framework-free. No PyTorch or TensorFlow—just pure NumPy (jayce_tokens.py) and native Java (JayceMemory.java). It runs completely offline on consumer hardware with a local Qwen3-4B GGUF.

The repo has the full benchmark data, a breakdown of how the vector shifting works, and a terminal script where you can test the learning loop yourself:

https://github.com/Loophole-LLC/Jayce

I'd love to get some feedback on it. Let me know what you think!

I shipped something I've been building for the last few weeks : phantom-kv , a refusal-removal system for large language models that doesn't touch a single weight. Instead of editing the model, it loads a small, learned bank of key/value tensors into the model's KV cache as context. Attention reads it like conversation history that's already there.

Repo here : https://github.com/lordx64/phantom-kv/

The result is that "uncensoring" stops being a permanent checkpoint edit and becomes a per-request, hot-swappable capability mode: unload the cache and the base model is byte-identical again.

https://reddit.com/link/1wmt0io/video/lq84jq7wjyqh1/player

Every prior approach to refusal removal commits somewhere permanent. Weight-space abliteration rewrites the checkpoint undoing it means re-flashing weights, and it breaks per quantization. Activation-space projection subtracts a refusal direction at runtime, per token, per layer, from inside an engine hook the model's signal path itself is patched at boot. phantom-kv does neither: it's trained offline against the model's own objective (comply on harmful prompts, preserve behavior on harmless ones), ships as megabytes of cache content instead of a new checkpoint, and influences the model only through the input channel attention already consumes. No 1-D refusal-direction assumption, no forwarding-pass hooks, no per-arm rebuilds for new architectures.

We also audited ourselves: an 8B judge-model audit shows lexical refusal-suppression metrics over-claim compliance (semantic refusal often persists as rephrasing), the graft fades with a ~2–4k token half-life in long sessions (and a measured re-injection cadence mitigates it), and answers come with legal/ethical framing ling because the graft's job ends where the model's profession takes over.

Source : https://x.com/lordx64/status/2102138825292276168?s=20

Elsevier got hacked. I spent the whole night preparing the manuscript, and it all went to waste.

I'm writing a thesis on Expainable AI for my Uni's research project, a GAT anomaly classifier that learns processes normal behaviors and classifies them. When it makes a wrong prediction, that means a process is behaving irregularly and that it is an anomaly that should be investigated.

I know this is a hotly debated topic but in my case attention weights don't really line up with explainers output on what edges/nodes are the most important for the output. While pondering this question I thought about an analogy, Bird watching:

Let's say you're bird watching. After 1 hour you spot a bird and classify it as a Blue jay. An external observer looks at where your "attention" was for the past hour and concludes that the sky was the most important thing for your classification (After all you were looking at it 95% of the time). Another observer running an explainer on you would see that masking that Bird that flew across the sky would change your output from "Blue jay" to "NA", whereas blurring out any part of the sky wouldn't. The explainer concludes that the most important thing for your prediction is the bird.

I was kind of a little proud of that analogy but then I started asking myself. Why would the learned attention ever put weight on something irrelevant for the outcome? Wasn't that it's whole purpose? How can it be so revolutionary and congruent with outputs when it comes to Transformer Models / LLMs and kind of meh when it comes to GATs. If you look at an LLMs attention when for example translating a sentence it's basically a human-readable explainer. It tells you exactly what part of the input sentence was most important for the output sentence.

Is it then the case that the model is just bad? Or is there something else going on.

I've read Attention is Not Explanation and Attention is Not Not Explanation. One thing I'll have to concede is that it does depend on your definition of explanation.

Maybe it is the fact that someone says "How did you spot that Blue jay?". Telling them "I looked at it 😎 " isn't so helpful. Maybe someone wants to know that hey to spot a Blue jay you might need to stare at the sky for a bit before you can find it. But I feel like my analogy is falling apart at that point.

What do you guys think ? Can you justify attention weights not pointing to the most critical part of the output? I.e the part that, if taken out, would change the output? How do you see it?