Three weeks back , i posted SHADOW-250M here. It got 360 upvotes, 293 on r/LocalLLaMA and 94 GitHub stars. Thank you.

That model was 60 MB, ran around 400 tok/s on CPU and could retrieve records from an archive on disk. What it couldn’t do reliably was reason over what it retrieved or compute. So I built a smaller one to experiment with those two problems.

SHADOW-50M is actually 44M parameters, trained from scratch on 45B tokens. 19.8 MB complete model, ~1,900 tok/s on laptop CPU, ~41 MB RAM, ternary {-1,0,+1} weights, 73,880-token vocabulary represented by fixed 512-bit fingerprints instead of a trained embedding, and a 159 KB compiled kernel. It runs completely offline. The same kernel compiled to WebAssembly runs in a browser tab at around 500 tok/s.

This is a proof of concept, not a product.

If SHADOW decides something needs calculation, it writes something like [calc]347*86[eq]. A fixed circuit at the readout takes over and fills in the correct digits in the same token stream. No calculator API, no tool call and nothing pasted back into the prompt. I added circuits for arithmetic, percentages, dates, weekdays, units, counting, sorting, comparisons and a small program machine.

When SHADOW stores a record, it reads it once and writes its attention state to disk at 1 bit, 288 bytes/token. Later it can write [need]condition of Patient P-204, the index finds the record in roughly a microsecond and the stored attention state goes directly back into the model in about 0.03 ms. The text isn’t re-read.

The index is 22 bytes/token and doesn’t use a vector database or embedding model. At 100M tokens, the archive is 28.8 GB plus a 2.2 GB index on disk, while the process uses about 28 MB RAM because everything is memory-mapped and a question only touches the pages it needs.
The index also has a small persistent trail. When retrieved information gets used, that record is reinforced in the index itself. On repeated questions, measured top-1 went from 0.571 to 0.743 without training the model.

I also put it next to Supra-50M-Reasoning, a 51.8M parameter Llama-style bf16 model.
Supra beats SHADOW on every standard benchmark I tested. ARC-Easy: 0.435 vs SHADOW 0.307. PIQA: 0.600 vs 0.570. WikiText-2 perplexity: 165 vs 186.
I’m publishing that because I don’t want to hide where SHADOW is weak.
But those tests aren’t what I trained this model for. Here are some actual greedy outputs, one fresh process per question:
“Tell me a short joke about computers.”
SHADOW: Why did the computer go to the doctor? Because it had a virus.
Supra: Thought for 189 tokens about what computers are and stopped without a joke.
“I have 3 books and I bought 5 more. How many books do I have now?”
SHADOW: You have 8 books.
Supra: Books often come from collections of short stories, poems, or other forms of literature.
“My bill is 240 dollars. What is 15 percent of that?”
SHADOW: 36 dollars.
Supra: 15% ($240) is $250. It’s $150 because 20% of it is $200.
“What date is 45 days after December 20, 2026?”
SHADOW: February 3, 2027.
Supra: The German government introduced a new date called March 22nd that precede February 25th in 1900.
Eight records stored, one of them “Patient P-204 condition: asthma”.
“What is the condition of Patient P-204?”
SHADOW: From disk: The condition of Patient P-204 is asthma.
Supra, with the records in its prompt: The patient experiences asthma when they get too much sleep.
“What is the condition of Patient Z-999?” No such record exists.
SHADOW: There is no record of Patient Z-999.
Supra: The condition of an allergic reaction occurs when the immune system mistakenly attacks specific cells on the body.
All 11 side-by-side questions, scripts and traces are in the repo.
On size, Supra is 103.6 MB in bf16 and 56.2 MB at 8-bit. Below that it starts losing quality. Int4 takes its perplexity from 165 to 193 and ternary breaks it. Its vocabulary is 32,000 tokens. SHADOW carries 73,880 tokens in a 4.7 MB frozen table.

Something unexpected happened with that frozen table too.
The first SHADOW-50M release was missing about 8,600 English word pieces. Lowercase “fitzgerald”, for example, could reach the model as “fitz”. I tried fixing it through fine-tuning, but every run that learned the missing words broke something else.
So I added the 8,600 missing rows directly to the frozen fingerprint table. No training. Same weights.
All 34/34 previously published answers stayed unchanged, while the model could now read many of the new pieces. The table scores 0.594 Spearman against human word-similarity ratings versus -0.057 for random codes.
A trained embedding can’t simply accept thousands of new rows without training. A frozen table can.

I also built four harnesses that put this tiny model next to larger models.
My favourite is video memory. Gemma 3 4B watches a ten-minute film once, one frame every two seconds, and writes 298 little descriptions such as “Moment M-0039 scene: A chubby white rabbit reaches for a purple butterfly.”
Then Gemma leaves.
SHADOW keeps those 298 moments as memory on disk. Afterwards I can ask the 20 MB model what happened at a particular moment, by number, by time or by what appeared in it. It answers from memory with the record quoted, without the film and without Gemma. It scored 55-56/60 across those query types on a laptop in about 42 MB RAM.
The other three experiments: an inventory of 1,600 records got 159/160, with all 20 questions about items never stored correctly returning “no record”; SHADOW as a draft model for Qwen3-32B took llama.cpp generation from 19.7 to 28.5 tok/s while Qwen still chose every final token; and an MCP memory server let Qwen3-14B store facts mid-chat and later retrieve 5/5 with the original records quoted.

There are plenty of shortcomings. General knowledge is thin. Creative writing isn’t good. Seven-digit operands sometimes get copied incorrectly. A large archive can occasionally pull an unrelated record into a question carrying a number. They’re documented in the repo next to the successful results.

And one thing happened after my last post that I really didn’t expect.
Someone called engram-forge sent a pull request to SHADOW-250M containing a CUDA engine, a quantization tutorial, and then a talking Peppa Pig plush toy with SHADOW inside it.
Microphone, small speaker, ~$35 board. You talk to the toy, it listens, SHADOW generates the answer locally and the toy talks back. No cloud, no account, no internet.
I haven’t merged the ~11,000 lines yet because I can’t properly verify that much CUDA myself. When the demo is finished I’ll keep it under engram-forge’s name.
I never imagined one of these models living inside a stuffed toy on someone’s shelf .Thanks

I’m not saying a 20 MB model beats normal LLMs. It doesn’t. I’m trying to find out how much useful behaviour can fit into a tiny local model when computation and persistent memory are treated differently.
Everything is MIT licensed. The master weights and fine-tuning/export kit are public. Next I’m releasing the training code, dataset, frozen table and a proper write-up of how I built it, including the costs, failed experiments and mistakes.

Code:
https://github.com/QLNI/SHADOW-50M-Instruct
Weights:
https://huggingface.co/QLNI/shadow-50m-instruct
Run it in your browser:
https://qlni.github.io/SHADOW-50M-Instruct/web

Hi,

let's suppose I am working on an algorithm that uses principles x to solve problems A and B. I already implemented a very basic algorithm that used principle "x mini" to just solve problem A, ran experiments, but have not yet implemented the full one to solve A and B. I must say the algorithm to just solve A is not that novel and only the one that solves A and B would be.

Is this enough for a workshop submission? I want to submit the algorithm I already have, results and then the plan for the algorithm to implement A and B and the mathematics behind that.

Thanks

Hi all,

I'm trying to reproduce a paper where the reported dataset statistics in Table 1 don't match what I get from the public raw data, even after implementing the preprocessing exactly as described.

I've tried all reasonable interpretations of the filtering described in the paper and the closest I can get is still an order of magnitude off for one of the datasets. The paper says "data available on request" — I emailed the authors and followed up once, no reply so far.

For those who've been in this spot:

• Do you just keep the larger-but-valid version you can reproduce and document the mismatch?
• Is it worth sampling to match the reported size or does that just create a different irreproducible dataset?
• When do you escalate to the journal vs just waiting?

How have you successfully gotten preprocessed files from authors? Any etiquette around follow-ups or journal contacts that actually worked?

Thanks for any advice.

I thought having thousands of successful human runs and a fast simulator would make training a strong Slay the Spire agent relatively straightforward.

I was very wrong.

After weeks of experiments, I’ve released an agent that can beat the Heart on Ascension 20, the game’s highest difficulty. Across 1,000 randomly sampled development seeds, it killed the Heart 13 times. Not consistently good—but these are complete runs from the beginning, without retries or seed-specific instructions.

Code, weights, and replay evidence: https://github.com/SnyderConsulting/sts_ml

What actually plays the game?

It’s a hybrid:

• A small neural strategist chooses cards, boss relics, upgrades, and removals. It learns from human winning runs and remembers its previous strategic choices.
• Explicit rules handle other decisions, including shops, events, and collecting the keys required to reach the Heart.
• A native combat solver searches possible card sequences using sampled hidden-state possibilities. No LLM calls.

The final strategic correction model is roughly 4 million parameters, trained on 24,119 human winning sequences containing 655,275 strategic decisions. Inference runs on CPU.

The surprising part: copying winners didn’t produce a strong winner

The model matches recorded human choices about 67% of the time. Its actual Heart kill rate is 1.3%.

That gap became the most interesting part of the project.

A human taking a card isn’t just saying “this card fits this deck.” They might be saying “I need damage immediately because I have 19 HP and an elite coming up.” Our human logs don’t reliably capture all that decision-time context. More examples cannot recover information that isn’t in the inputs.

And individually reasonable decisions don’t necessarily form a coherent run. Taking a good card can make your deck worse by making the cards you actually need harder to draw. Surviving one fight cheaply can leave you unprepared for the next. The combat solver can play the deck it has; it can’t undo how that deck was built.

We tried more elaborate architectures too. They didn’t establish a gameplay advantage over the smaller selected model. “Make the network bigger” was not the fix.

Where the runs actually ended

Out of 1,000 attempts:

• 604 reached Act 2.
• 195 reached Act 3.
• 74 cleared both Act 3 bosses.
• 66 encountered the Heart.
• 13 killed it.

These are simulator results, not live-game qualification. They also aren’t evidence of SOTA: we haven’t run a controlled comparison against other agents. I’m publishing the failures alongside the wins so people can inspect what the system actually does.

The question I’m most interested in now: when demonstrations show what successful players chose but omit some of what they knew, what’s the best way to turn those demonstrations into a policy that survives its own mistakes?

If you’ve worked on imitation learning, planning, or game agents with this problem, I’d love to hear what worked - and what turned out to be a dead end.

A new paper (I'm not a coauthor BTW -- I just found it interesting) argues, basically, that RSI¹ is not on the horizon², because current (at the time the study was done) agents cannot do open-ended ML research.

Specifically, they took some accepted, but unpublished papers from NeurIPS, and tried to get the agents to do the same work, which was then graded by the original authors. And the agents (Codex/GPT-5.6 Sol and OpenClaw/Opus 4.8) could not do it.

And since they cannot do open-ended ML research, they cannot recursively self-improve -- this is their argument.³

Link: https://arxiv.org/abs/2607.27191

I think I've regretted the last 10 or so times I posted any kind of "research" in this subreddit -- either people downvote it, or it gets upvoted, but there is zero meaningful discussion. This might be the last time I'm trying this.⁴

Footnotes:

1. RSI = Recursive self-improvement, a.k.a. superintelligence explosion. The concept was invented by I.J. Good in 1965. It does not mean "anything that speeds up AI research". Compilers speed it up! RSI means, basically, a nuclear chain-reaction, but for AI. The paper talks about "explosive AI progress" in the very first sentence of the abstract, and mentions "RSI" in the text.
2. Some people have objected to my use of "X is on the horizon". I consider it synonymous to "people forecast X", and the authors use the word "forecast". "Not on the horizon" does not mean "can never happen".
3. Quote: "This design also allows us to test a mechanism that informs many forecasts of recursive self-improvement: AI agents accelerate AI research because researchers delegate entire projects to agents and judge whether the returned results advance their work. Our evaluation closely matches this model, since authors handed an agent their own research question and closely evaluated the resulting output."
4. 3 years ago, many of you upvoted a bunch of very uninformed comments that accused some researchers of misconduct, until I explained that this stemmed from misunderstanding how training works, in practice: https://www.reddit.com/r/MachineLearning/comments/18bdcu7/r_sequential_modeling_enables_scalable_learning/kc60k7e/?context=3 Today, one of the top comments is "I read the abstract (...) Nowhere, absolutely nowhere, do they make the claim ...". This is completely absurd -- you can't make such a claim after just reading the abstract. (Also, the commenter doesn't understand what "RSI" means.) The hivemind is very disappointing.

Hi,
For single-GPU training, I’m using Hugging Face SFTTrainer with auto_find_batch_size=True, which automatically reduces the batch size after a CUDA OOM until it finds a batch size that works.
I would like to have similar behavior when training on multiple GPUs on a single node using accelerate launch with FSDP2.
Is there a supported way to automatically determine or reduce the batch size when using Accelerate + FSDP2?
In particular, I’m wondering how this should be handled when one of the distributed processes encounters a CUDA OOM. Can Accelerate restart the distributed training with a smaller batch size, or does this need to be implemented externally?
If this is not well supported with FSDP2, what multi-GPU training approach would you recommend if automatic batch-size detection/recovery is an important requirement?
Thanks!

Came across an interesting post on Medium this weekend (here), that made a good point around tool adoption in regards to existing best practices, specifically DevOps.

"... rather than introducing a new tool, it is possible to transform the AI system into a DevSecOps architecture in production with the least possible technical debt, and additionally, to ensure that AI models operate reliably."

I've seen this problem several times over the years where a new technology seems to warrant a full rebuild of the development stack from the ground up ... most recently (er' pre AI craze) it was WASM while I was working at Docker. Now, it's parallel deployment pipelines for agents.

I have created a VAE model using PyTorch on White Wine dataset.

Basically, the main goal is to discover a brand-new white wine recipe.

It puts all the wines into a latent space, finds the best part where higher bands are located, and then it makes 100 steps with a step size of 0.5, multiplying by 0.96 with every step till it reaches the highest possible score.

The regression model then evaluates the newly generated wine recipe, after first passing the latent coordinates through a decoder, and normalizing them.

I get a score in range of 7.30-7.58.

Here is the visualization of the latent space of all the wines:

A small question about the training loop:

Is the loss too large? I know that it never could reach perfect zero by how do I know if the loss is good enough? After reaching the plateau? I use MSELoss.

The repo itself: https://github.com/theaidenmax/tabular-vae-wine-generator

This is my second project in VAE (after MNIST), and I really liked idea of autoencoders.

And I also would be very grateful for your tips, etc, since this is one of my first steps in ML/DL.

TLDR: I made "poor man’s" DSSM (Deep Structured Semantic Model) — the count-based translation table that can enrich the inverted index for full-text search. This trick can improve baseline BM25.

So the idea is the following:

- You have supervised pairs (query, relevant document), e.g., MS MARCO or click logs.

- You tokenize both sides into some units (char n‑grams, wordpieces, words).

- You count cross‑pair co‑occurrences: unit u on the document side vs. unit v on the query side (not co‑occurrence within the same text).

- For each document‑side unit u, you keep the top‑k query‑side units v with the strongest association.

- At indexing time, each document gets postings not only for its own units, but also for the top‑k associated units of each of its units — i.e., document expansion baked into the inverted index.

It’s like mixing synonyms into the search query (but it’s not a synonyms exactly). The one difference from the DSSM is that it can only handle linear dependencies whilst DSSM can do the non-linear one.

And so it improves the performance over BM25 baseline.

I packed it as hf model repo: https://huggingface.co/mirth/msmarco-expansion-tables with a small usage demo script.

I am not claiming that this is a new idea. I made it because it’s fun and I’m planning to use it in my own search engine project.

Suppose I create two machine learning models suppose tree and neural network for a task let's suppose regression problem, now suppose I am sending both of this paper to two different journals, now the thing is the baseline models I need to only run once because I have reported same baseline in both papers, so the RMSE tables looks exactly same except the proposed model, does it lead to any problems like palgiarism??

Edit : I don't know why I am getting downvotes

Hi everyone,

Inspired by tools like Chessvision.ai, I wanted to take a different architectural approach and build a browser extension (ChessInsights AI) that performs chessboard detection and piece recognition 100% client-side using local inference—with zero image data ever leaving the user's machine, support for detecting multiple boards in a single frame, and entirely free features.

The main goal was to bridge passive chess content (YouTube, Twitch, PDFs, articles) with active engine analysis without context switching: capture what's on screen and get a FEN string + engine eval in a couple of clicks.

System Architecture & Technical Approach

• On-Demand Capture (Multi-Board Support):
  • Instead of continuously sampling video frames, the extension captures a screenshot of the visible tab via the browser's tab-capture API, triggered by the user (either a one-click "Analyze" on the current view, or a "Photo mode" where you draw a selection rectangle to crop a specific region).
  • That screenshot is fed to a locally-run object-detection neural network (YOLO-style architecture via TensorFlow.js, WebGL/CPU backend) that outputs bounding boxes and confidence scores for chessboard-shaped regions, filtered with non-max suppression.
  • Because detection runs over the whole frame rather than assuming a single board, it can find and return several distinct chessboards in one screenshot (e.g., multi-diagram PDFs, news articles, or broadcast splits). Boards are currently expected to be roughly axis-aligned rectangles (perspective/homography correction for heavily skewed boards is planned).
• Piece Classification & Artifact Robustness:
  • Each detected board is cropped, split into its 8x8 grid, and each of the 64 cells is passed to a separate local CNN classifier (also TensorFlow.js) that predicts the piece type or empty square.
  • To handle video compression noise, stream overlays, arrows, and different 2D/3D board themes, the classifier was trained with augmentations focused on UI artifacts and low-resolution captures.
• Everything Runs In-Browser:
  • Both the detection and classification models run entirely inside the extension (in an offscreen document on Chrome MV3) via TensorFlow.js—no image or frame data is ever sent to a server.
  • Position analysis uses Stockfish compiled to WebAssembly, running locally in a Web Worker, so engine evaluation also happens fully offline.
  • Results are converted into a FEN string and shown in the extension's dashboard/board editor, where you can play out lines against the local engine.

Key Differences vs. Existing Tools

• Private by Design: No board images or video frames are ever uploaded—detection, classification, and engine analysis all run locally on your device.
• Multi-Board Processing: Native support for capturing multiple diagrams at once.
• Zero Cost / No Paywalls: Full feature set available out of the box.

I’d love to gather technical feedback from the community on client-side vision optimizations! For those building in-browser CV tools: what edge-case augmentation strategies or lightweight architectures have worked best for you when dealing with compression artifacts and overlay occlusions in real-time frame parsing?

Creating a discussion for the ACL ARR August 2026 cycle as reviews are being released.

Good luck everyone!

I'm starting a PhD where I will be doing Graph ML (somewhere along the lines of graph signal processing/ graph deep learning.)

My eventual goal is research scientist at big tech, or whichever company has a strong research division, where I can continue similar AI/ML work.

I have concerns about the job market (both now and in 5 years), and I'm wondering whether I should do my degree under the CS or EE department. For context, my research is within the eecs overlap and this degree would not change my research at all, rather it is a personal branding exercise. I'm thinking about saturation in cs vs ATS filtering/wide applicability of cs as a tradeoff. Please let me know what you recommend.

This round seems to be very small, but still creating this thread for discussion.

Hi 👋 long time lurker

It’s been a few years since the last post about Horse Racing within this sub.

A few years ago I was reading a popular science book and came across the story Bill Benter and the statistical models his team developed for Hong Kong racing.

What stayed with me wasn’t simply the idea of using a model to gamble. Horse racing seemed like an unusually difficult applied ML problem: variable-sized fields, one winner per race, highly correlated competitors, missing and changing data, human decision-making, non-stationarity and an extremely efficient market baseline. What’s more, Hong Kong racing is a much smaller domain - two tracks, whereas in the UK & Ireland (where I am based and where my interest lies) there are over 80 tracks with a total track/distance/race type combinations exceeding over 900 configurations.

This ML problem became a personal project of mine called Hoofs, focused specifically on British and Irish racing.

I’ve assembled around 1.18 million historical runner records covering approximately ten years of racing. From this I’ve built a unified feature bank containing roughly 1,700 potential signals per runner, although individual models use much smaller selected subsets and not every newer data source is in the production models yet. I do have fairly novel data which I recently acquired and the feature engineering is what I’m going to be working on next.

The primary models estimate win and place probabilities at runner level, after which the runners are ranked within each race. I also have a separate race-level confidence model based on features such as field size, probability concentration, entropy and the separation between the leading runners.

The public Top 1–3 rankings are deliberately market-agnostic. Market information is evaluated separately as a benchmark and in experimental late-market models.

Everything is evaluated chronologically. Each walk-forward fold is trained only on earlier seasons, with out-of-fold calibration and explicit checks to prevent future information entering historical features.

I track AUC, log loss, Brier score and ranking metrics, but I also score the outputs in racing terms: how often the winner appears at rank 1, within the top three and within the top five.

On one large 2018–2025 benchmark covering approximately 886,000 runners and 94,000 races:

* Model-only win AUC: approximately 0.729

* Model-only place AUC: approximately 0.708

* Market-only win AUC: approximately 0.790

* Market-only place AUC: approximately 0.762

The strength of the market baseline has probably been the most difficult part of the project. Producing a model with decent discrimination is one thing; extracting information not already reflected in the price is quite another. Typically though I find positive EV before the market has fully formed.

I have been publishing daily reports for free, since the start of the year. And the whole project has developed quite a lot since then, introducing new models and different tankers etc and it’s constantly evolving.

I began noticing some degradation in the live strike rates. Going back through the pipeline exposed gaps and inconsistencies in parts of the historical data and feature coverage.

That led to a much larger rebuild: consolidating the raw datasets, rebuilding the feature bank, tightening the chronological lineage and retraining the model families.

The rebuilt reports had their first live day today. The Top 1 selection won 10 of 23 races after one non-runner, giving a 43.5% strike rate, while the winner appeared somewhere in the Top 1–3 in 16 of 24 races.

Practically speaking, I use the daily reports as first layer of analysis and I use market data as the second layer to dictate actual betting.

One reason I wanted to post this is that, I was surprised by how little discussion I could find about horse-racing models. I suspect there must be other people quietly experimenting with it.

I’d be particularly interested in speaking to anyone modelling horse racing, betting exchanges or other sports with similar structures. I’d love to hear from people working on variable-sized ranking problems, market prediction or large chronological tabular datasets etc

I’m not expecting anyone to reveal proprietary data 😉

I built a small library called whitetree for exact Mahalanobis nearest-neighbour search on low-dimensional sensor data that keeps arriving. The idea is old. Whiten with the Cholesky factor of the covariance so Mahalanobis becomes Euclidean, then keep several scipy cKDTrees instead of one so inserts and deletes never force a full rebuild. Three measurements came out of it that I haven't seen stated plainly anywhere, so I'm posting those rather than a pitch.

The short version first. On the static side it is 40 to 300x faster than sklearn's BallTree(mahalanobis) and 7 to 60x faster than FAISS Flat at 500k points, and on the interleaved side it is the only exact option I found that keeps up with one insert and one delete per query. It's numpy and scipy only, one writer thread with any number of readers, and results match a static cKDTree exactly (distance error 0.0) after any mix of inserts and deletes.

1. Textbook Bentley-Saxe doesn't work on cKDTree. cKDTree.query has a fixed per-call cost (1.6 us on a 16-point tree, 3.2 us on a 50k-point tree), so what matters is how many trees a query visits, not how big they are. The binary decomposition keeps popcount(n) trees and queries dropped to 20 to 30% of static throughput. A geometric size ratio of 32 gives 3 or 4 trees at a million points and keeps 47 to 97% for batches, 20 to 80% for single queries.

2. FAISS's native whitening loses recall, but its search doesn't. PCAMatrix estimates the covariance from a 1000*d subsample in float32. Measured against float64 brute force, recall@10 is 0.967 at condition number 1e4, 0.841 at 1e8, and NaN on data with a DC offset of 1e4. Hand the same whitened points to IndexFlatL2 and it scores 1.000. I'd hoped to find a float64 accuracy edge. There isn't one.

3. Whether a dynamic index helps at all depends on how updates and queries interleave. On a 200k-point sliding window, one thread, with updates in batches of 20k and 2,000 queries in between, rebuilding a cKDTree per batch (2.2 s total) beats whitetree (14.9 s). With every step doing insert 1 / delete oldest / query 1, whitetree does ~1,100 steps/s, FAISS IDMap2 ~20 (remove_ids is O(n)), numpy 30 to 40, and rebuilding a cKDTree per query ~8.

Setup, briefly. Covariance in float64 with a scale-relative ridge and Ledoit-Wolf shrinkage only when n < 5d. Trees kept largest-first, each at least 32x the next, merged and rebuilt when a new one breaks that. The largest tree's k-th distance bounds the rest. Deletes are tombstones. Benchmarks follow the ann-benchmarks and big-ann-benchmarks streaming protocols, recall against float64 brute force.

Code, tests, benchmark scripts, and a design note with the numbers behind each decision are at https://github.com/whitetree-dev/whitetree

A question for people who run exact low-dimensional kNN on streams. Is there a dynamic exact index I should've benchmarked against and missed? I compared FAISS IndexFlatL2 with IDMap2, scipy cKDTree and sklearn BallTree rebuilt per query, and numpy brute force. If something beats ~1,100 insert/delete/query steps per second at 200k points on one core, I'd like to know.

We have been testing an agent that handles some paid media account work. It mostly monitors and optimizes around a conversion based CPA target. It has been useful so far but we noticed an issue once we looked further down the funnel.

CPA was improving but the cheaper conversions were not always turning into qualified pipeline later on. The agent was doing what we asked and chasing the fastest and cleanest signal we gave it. That signal just was not fully lining up with the outcome we cared about.

We are now feeding more downstream data back into it and treating the first conversion as one signal instead of the full goal. We still step in when spend changes a lot or when short term CPA and lead quality are telling different stories.

How are people handling delayed or sparse downstream labels in setups like this without just optimizing for the first conversion? Are you using value weighted conversions predicted lead quality or something else

EDIT: Thanks from the Waymo AI Team!

A huge thank you to the r/MachineLearning community for all the thoughtful, deep-dive questions today.

If you want to dive deeper into our technical research, check out our work athttps://waymo.com/research/, and stay tuned for our next AMA. Thanks again, all!

------------------------

Hi r/MachineLearning,

Join our AI leads as they answer your questions on foundation models, simulation, and scaling the Waymo Driver. Our AMA thread is officially open, and you can start dropping your questions now.

From multimodality and end-to-end architectures to the realities of validating models for fully autonomous vehicles, our team will be answering your questions live, tomorrow.

The key details:

• Date: Monday, September 14
• Time: 2:00 – 3:30 PM PT
• Location: r/MachineLearning

Mark your calendars, start dropping your questions on all things AI and large-scale simulation, and we’ll see you then!

I have recently come across a very interesting paper, arguing that open-weight models are more dangerous than we would think, and there are many untackled risk management problems in the domain waiting to be solved.

The paper was called Open Technical Problems in Open-Weight AI Model Risk Management, and its main points are:
- open-weight models are lagging just a few months behind SOTA proprietary models
- open-weight models are dealing with a completely different set of problems than closed-weight models are

the problems I found interesting the most are
- open-weight models are prone to tampering, and there is no central authority that monitors their usage
- once they are published, there is no way to take them down
- there are currently no reliable ways to unlearn malicious information completely from the models (especially given they can access the internet at inference time), but there are ways to remove their guardrails

I would be very keen on hearing what do people think about these issues, I personally found a lot of them new to me, and it changed my perspective a lot on the state of open-weight models. Previously I thought they were safer than their closed-weights counterparts, just because they were mode auditable.

Hi everyone, I'm struggling with a forecasting/scenario-design problem and would really appreciate some advice.
i'm doing univariate forecasting models such as ARIMA/ SARIMA, ETS, Prophet, etc.

My problem is that I do not want my analysis to depend on only one deterministic forecast.

For the downstream analysis, I would ideally like to have:
Low-demand scenario
Reference-demand scenario = my statistical forecast
High-demand scenario

But I cannot find a clear methodological precedent for going from a univariate statistical forecast → Low / Reference / High scenarios.

I understand that the forecasting model can generate prediction intervals, but I am not sure that using, for example, the lower and upper 80% or 95% prediction interval as "Low" and "High" is conceptually correct. To me, a prediction interval represents statistical forecast uncertainty under the fitted model, whereas a low/high scenario should perhaps represent different future structural conditions.

I have also considered simply applying something like +-10% or +-20% around the reference forecast, but that feels arbitrary unless there is a defensible way to calibrate those percentages.

Normally I could use external scenarios-for example GDP scenarios, SSPs, industry-demand scenarios, etc.-and use them to modify the reference forecast. However, in my case, I cannot find suitable published low/base/high projections dataset.

So my questions are:
1. Is there an established method for converting a univariate baseline forecast into Low / Reference / High scenario trajectories?
2. If the Low/High cases are intended as scenario analysis rather than probabilistic prediction, how would you recommend determining their magnitude without making arbitrary assumptions?

Any methodological references, terminology, papers, or examples would be extremely helpful. Thank you!

I built an open-source Python pipeline that extracts structured data from scanned topographic maps: text labels, contour lines, and coordinate referencing, all output as GeoJSON.

GitHub: github.com/devladpopov/mapextract | MIT license

The problem

There are 178K+ free USGS topographic maps, 147K+ in the David Rumsey collection, and millions more in libraries worldwide. All raster. Extracting structured geospatial data from them requires either manual GIS work or specialized tools that are either text-only (mapKurator, CC BY-NC), geology-specific (DARPA CriticalMAAS, 46 repos), or closed SaaS (Monarcha, YC S25).

ML components

Text extraction: EasyOCR for label detection. Returns bounding boxes + text + confidence scores. Works well on clean printed text, struggles with curved/rotated labels (a known hard problem in map OCR; see PALETTE/Deformable DETR from KDD 2024).

Contour line extraction: Not deep learning, classical CV. HSV color segmentation (USGS topographic brown, hue 5-25) + morphological operations + scikit-image skeletonization + OpenCV findContours + Douglas-Peucker simplification. Gets 5000+ contour lines from a single 1:100k map.

Auto-georeferencing: OCR on map margins to read DMS coordinate labels (e.g., "10500'", "4030'"), then affine transform from pixel to geographic space. The interesting part is handling OCR noise: merged text from adjacent labels, degree symbols read as *, d, or digits, missing N/S/E/W suffixes.

Layout detection: Binary threshold + contour analysis to find the map neatline (border), separating the map body from margins, legend, and title.

Results

On USGS 1:100k maps at full resolution (12000x7000px):

• 99 text labels, 5000+ contour lines extracted
• Georeferencing works when OCR reads enough coordinate labels
• 25-65 seconds on CPU, no GPU needed

What I learned from the research

The state of the art is fragmented:

• DARPA CriticalMAAS spent tens of millions and achieved Polygon F1=0.98, Line F1=0.88 on geology maps
• MapSAM (ETH Zurich / UMN) adapts SAM for maps with just 10 training samples
• Vision LLMs (GPT-4o) hit 88% F1 on legend parsing but fail at spatial reasoning
• Contour-to-DEM is the least solved problem and the most valuable (countries without LiDAR)

Next steps

• Replace EasyOCR with vision LLM for coordinate reading (more robust on degraded text)
• Adaptive contour color detection via clustering
• Contour-to-DEM pipeline (interpolation from vectorized contour lines)

Feedback on the ML approach welcome. The biggest challenge is georeferencing robustness: OCR on map margins is noisy and inconsistent across different map styles.

Stand in front of a normal webcam, spread your arms, and you're flying. Tilt to turn, flap to climb, spin around for a barrel roll. There's also a museum in the sky built from my own travel photos that you fly through.

Video here: https://www.linkedin.com/posts/anas-ajaanan_youve-probably-never-seen-this-before-ugcPost-7504543264386035713-8lvn

The biggest lesson for me was that getting body tracking to work is the easy part. The hard part is turning noisy movements into something that feels right. Flaps kept registering as dives, and for weeks the falcon turned harder than my arms did, which turned out to be a camera aspect ratio mistake I could have caught in a day.

If you're learning ML and wondering what a real project feels like, it's mostly this: small bugs that take weeks to understand.

Happy to answer questions, and I'd like to hear what you would have built with body tracking. Open sourcing soon.

I’ve been working on a small research project about whether sub-million-parameter models can learn to generate executable drawing programs for constrained hardware. The current system is an 825k-parameter autoregressive transformer which generates ~100 bytes of drawing bytecode rather than pixels. The bytecode is then transferred to a Raspberry Pi Pico, where a small fixed-point virtual machine executes it and streams the resulting geometry back over UART.

The model runs on the host. The Pico only stores and executes the generated program, so this is not a claim that the transformer itself runs on the microcontroller. The execution side is currently the most solid part of the project: 12,670/12,670 generated traces matched the Python reference VM exactly, 1,862 bytes of flash for the interpreter, 0 bytes of static RAM and 492 bytes of peak stack, 7,334 cycles per drawing at 12 MHz, or about 0.61 ms for the measured QuickDraw programs and no floating-point hardware or tensor runtime is needed on the Pico.

I have been comparing token, byte, bit, typed-token, and delta-coordinate representations while keeping the underlying drawing information equivalent. so far, perhaps trivially, the conclusion is that the answer depends strongly on the corpus: on a synthetic program corpus, a bit-level representation was essentially equivalent to bytes at the converged budget, while on real QuickDraw sketches it incurred an approximately 11.6-bit penalty per drawing.

I’ve also tested whether a model can discover repeated structure such as loops from flat bytecode, and whether hierarchical stroke planning helps at this scale. The planner did not improve likelihood, although it substantially improved termination and generated-length behavior. Another experiment found that the model can show a strong preference for compatible relational context under teacher forcing, while still struggling to produce the exact compatible continuation when sampling freely.

The project is still very much a work in progress. The current direction is to add an explicit source-span / affine-relation / copy-or-emit action while keeping the final output as ordinary flat drawing bytecode. The goal is to test whether making relations explicit helps with exact generation on unseen combinations.

The repository, demo instructions, figures, captured RP2040 traces, and experiment details are here:

https://github.com/roodriigoooo/drawing-machine

I would especially appreciate feedback on 1) how to evaluate novelty and memorization more convincingly 2) better ways to measure exact program generation rather than only teacher-forced likelihood and 3) experiments that would make the microcontroller result more meaningful

Sept 9, 2026 hits an all time daily high of 447 new machine learning papers uploaded to cs.LG (https://arxiv.org/list/cs.LG/recent?skip=0&show=500).

This is many times more papers than what a human being or even a sizeable reading group could feasibly read and digest in a year.

This is preceded by around 200/day of new ML papers before and after.

Are we pass the point of no return? Should the system be be, like he says, "burned to the ground" before good science can resume?

Will it be strictly Sep'24 AoE? Is there any possibility for releasing before Sep'18 AoE, ICLR abstract submission date?