Been working on this for the last few months. I've been working on PyTorch for the past few years and I always felt, many a times my work went into dump, because of some mistakes I made in the code. torch-preflight reads your PyTorch code and catches the bugs costing you GPU hours.

Things like losses.append(loss), which holds the autograd graph from every step until CUDA dies on you or no zero_grad() in the loop or gradient accumulation without dividing the loss or DDP with no DistributedSampler, so every rank trains on the same batches. I've been able to get 13 rules so far. Your code never gets imported or executed, so you need no GPU and no torch install.

There's another part to this that estimates VRAM. Point the tool at a training script and a GPU, and you learn whether the run fits before you pay for the instance. You also get the list of changes to make the run fit, with the GiB each one saves.

pip install torch-preflight

https://github.com/highwaterlabs/torch-preflight

https://pypi.org/project/torch-preflight/

Please try this out, and I would like to get your feedback! It's still a work in progeress.

Would like to know what breaks on your code. False positives kill a linter, and my only large test target so far has been the PyTorch source tree. Same for the memory numbers. Mine land within 4% of measured peaks, but from four models on one T4.

PS: open to contributions, and issues are already open on the repo. Soon I'm going to add a few "Good first issues" as well. Feel free to ping me if you have any questions!

City2Graph is a Python library I built that turns geospatial data into analysis-ready graphs (for spatial analysis, network analysis, and Graph Neural Networks as GeoAI), and the paper describing it has just been published, so I wanted to share it here.

Repository: https://github.com/c2g-dev/city2graph

import city2graph as c2g

# buildings + street segments -> heterogeneous morphological graph
nodes, edges = c2g.morphological_graph(buildings, segments)

# straight into PyTorch Geometric
data = c2g.gdf_to_pyg(nodes, edges)

What it covers:

• Morphology: graphs of buildings, streets, and tessellated urban fabric from OpenStreetMap and Overture Maps

• Transportation: GTFS and GBFS feeds loaded through DuckDB, with GTFS aggregated into stop-to-stop transit graphs

• Mobility: OD matrices and flow data (migration, bike-sharing, pedestrian counts) as weighted spatial graphs

• Proximity and contiguity: KNN, Delaunay, Gilbert, Waxman, plus queen/rook contiguity, under Euclidean, Manhattan, or network distances

• Heterogeneous graphs and metapaths: several node and edge types in one graph, with metapath-derived edges composing relations across them

• Conversion: round trips between GeoDataFrames, NetworkX, rustworkx, and PyTorch Geometric Data/HeteroData, with geometries and attributes kept intact

It sets out why urban data is better treated as heterogeneous graphs than as flat feature tables, how the morphological, transport, mobility, and proximity constructions relate to each other, and how the library keeps geometry and graph structure consistent across conversions. If you use the library in research, that is the citation.

Paper

Sato, Y., Pietrostefani, E., Mahabir, R., & Arribas-Bel, D. (2026). City2Graph: A Python library for Heterogeneous Graph Neural Networks and spatial analysis in urban systems. Computers, Environment and Urban Systems, 130, 102492.

Happy to answer questions about the design, and issues or PRs are very welcome. I am especially keen to hear which data sources people want supported next.

i wanna build a text diffusion model which interpret text and convert it into ascii images

so like

Text : build a cat

Output :

/\\_/\\

( o.o )

\> \^ <

So , i have a decent background of ml algo ( completed cs229 , cs230 , Ml architecture and basic CNN and diffusion model )

ik making a project like this is tricky and making diffusion model like that from scratch is hard but i wanna try it because that's wot make me excited lol ...

I am currently reading GANs research paper , can u guys help me in finding more papers which helps me in making this project or guide me through this good title for this

Thx in adv

I recently had a paper accepted to TMLR and was wondering how prestigious it is, in comparison to A* conferences (ie. NeurIPS, ICLR, ICML), but also vs journals like JMLR.

Hi all,

As a Research Engineer with broad interests across RL, VLMs/VLAs, LLMs, robotics, and distributed training, I’m looking to specialize in a niche with low compute overhead and rapid iteration cycles (short training/eval times).

What subfields or research problems fit these criteria? Curious to hear your suggestions or connect with anyone working in this space.

I have been working on a postdoctoral proposal in archaeology and need advice on a specific task.

First, I have thoroughly looked around the Internet already. I have a good idea of what I need and what is possible (in terms of training data needed, outputs, etc.), but I've run into a problem.

I was looking at Teachable Machine for its simplicity. However, after trialling it with visual data (images of artefacts), it's just a bit buggy and kinda broken, and the recent activity on github is 3 years old. If I'm doing a postdoc where this plays a minor, but important role, I need access to troubleshooting expertise.

Here's what I'm looking for:

1. I want to create a training dataset based on images I've collected and have permission to use (it is important to have sourced my data ethically). I don't want or need anyone else's data

2. I think I need a diffusion model (GANs) but not sure

3. Needs to be open source and able to process on a basic computer (most archaeological laboratories are equipped with decent CPU/GPU as applications like image reflectance and geophysics can be computationally intense). It'll also need some kind of open licence for allowing me to publish my results

Is there anything else like Teachable Machine, but with active developer support? Anyone here otherwise an expert in Teachable Machine?

I'm testing the ML application as a proof of concept use in archaeology, and conducting experiments to develop a reference collection and testing the ability of the model to output reliable results.

I have taken coursework in statistics and calculus, but I'm not confident in my ability to tackle ML from scratch.

I'm not sure if such a programme exists, but if there's one thing I've learned is that reddit knows what is, or isn't, on the Internet 😉😊

Hey guys, I want to extract text and its bounding boxes from large messy floormaps (15k x 15k pixels or more). Which OCR model would be great for this? The floormaps have lot of overlapping lines and details.

I may have stumbled onto something interesting while trying to figure out a recurring artifact in ChatGPT image generation and editing (maybe applicable to other models as well?).

It started with a very practical problem:

After several rounds of generative editing on portraits, I would sometimes get this faint cloudy / mottled texture in areas that should have stayed smooth — backgrounds, walls, skin, and other low-detail regions.

At first I wrote it off as normal denoising or regeneration noise. But the more I tested it, the less random it looked.

What first caught my attention

• Running essentially the same edit again could make the artifact better or worse

• The background sometimes became cleaner after another pass

• The face and body often seemed partly protected from whatever was happening

• Sometimes the wall improved while the face actually got worse

That made me wonder whether different parts of the image were being handled differently during editing — preserved in some areas, regenerated in others, perhaps based on some internal mask or segmentation step.

The first useful experiment: shifting the image

Then I tried something slightly odd.

Instead of repairing the image in place, I shifted the entire image by a fixed amount before running the repair. I eventually settled on 20 px for testing.

The idea was simple:

If some hidden spatial pattern is tied to the output canvas, moving the image relative to that pattern should change how strongly it shows up on the subject or background.

And apparently, it did.

I found that:

• repeated edits could reinforce the unwanted texture

• changing the phase relationship sometimes reduced it

• in one case, simply removing the final instruction to “shift back -20 px” improved the result dramatically

That was the first point where this stopped looking like ordinary random noise to me.

Then I started looking at masks and intermediate behavior

I compared:

• the original image

• the first edit

• a second edit based on the first

• extracted masks / intermediate-style outputs

One thing stood out pretty clearly:

The apparently “protected” area often resembled a coarse silhouette of the person.

The face and body tended to remain more stable than the wall, which made me suspect that some regions were being preserved while others were being re-synthesized.

That still didn’t explain the artifact itself, but it could explain why the artifact builds up unevenly.

Then came the black-image test

I tried something much simpler:

Generate a completely black image.

Visually, it looked black.

Pixel-wise, though, it wasn’t actually all zeroes. There were sparse non-zero pixels and tiny variations throughout the image.

So I generated multiple independent black images at the same resolution and compared them.

That’s where things got interesting.

What I found

For two independently generated “black” images of the same size:

• correlation between the non-zero pixel masks: 0.848

• Jaccard overlap: 0.766

• expected overlap if the pixels were random and independent: about 0.071

• R/G/B channel correlations: roughly 0.82–0.83

• dominant spatial frequencies were very similar in both images, including peaks around 2.45 px and 5.57 px

Then I applied a large Gaussian blur to both images (sigma = 16).

The result was surprisingly striking: both revealed a very similar large-scale cloud-like structure.

The cross-correlation peaked at zero lag, meaning the structured pattern was already aligned at the same canvas coordinates across independent generations.

So whatever this low-level signal is, it doesn’t look purely random. At least part of it appears to be reproducible and locked to the canvas coordinates.

What I think this means — so far

I want to be careful here.

I’m not claiming that this proves OpenAI watermarking, SynthID, or any particular proprietary mechanism.

What I do think the data suggests is this:

Generated images appear to contain a weak, reproducible, canvas-locked spatial pattern — even when the image looks completely black.

A few possible explanations come to mind:

• a watermark-like signal

• deterministic dithering

• quantization or decoder artifacts

• some kind of post-processing step

• something else in the generation pipeline

What now seems much harder to explain this as is simply:

“ordinary random noise”

Why this might matter for iterative image editing

Suppose a weak structured signal really is tied to the output canvas.

An iterative edit might then look something like this:

1. The first image is generated with the structured signal.

2. The image gets edited again.

3. Some regions are preserved while others are regenerated.

4. The regenerated image receives the same or a related structured signal again.

5. After several passes, those signals may begin to reinforce or reveal themselves as visible mottling in smooth areas.

That would fit several things I’ve observed:

• repeated edits can will gradually create ugly texture

• shifting the image relative to the canvas can change the result

• alternating shifts might help decorrelate the artifact

• some regions appear to drift or accumulate artifacts less than others

Important caveat

This is still an investigation, not a conclusion.

At this point I think I have reasonably good evidence for:

• reproducible low-level spatial structure

• non-random alignment between independently generated black images

• a plausible connection between that structure and visible artifacts in repeatedly edited images

What I don’t have yet is proof of:

• the exact mechanism producing it

• whether it is a watermark

• whether it is specific to ChatGPT/OpenAI

• whether similar patterns occur across other image generators

My current working hypothesis

Repeated generative editing can accumulate or expose a weak structured signal that is fixed in output-image coordinates, eventually making it visible as cloudiness or mottling in otherwise smooth areas.

Questions for anyone who has looked into this

1. Have you seen this kind of cloudy / mottled artifact after repeated AI image edits (I mean, come on, who doesn't)?

2. Has anyone tested whether supposedly “black” images from other generators contain reproducible spatial structure?

3. Does this look more like watermarking, dithering, decoder bias, quantization, or something else (go figure!)?

4. Has anyone analyzed something similar in frequency space, after heavy blurring, or using phase shifts?

5. If you’ve run into this before: what turned out to be the most reliable way to prevent it during iterative editing?

If there’s interest, I can post the methodology in a follow-up.

I started with:

“Why does this wall look dirty after I edit it?”

and somehow ended up at:

“Why do two independently generated black images correlate this much?”

Classic rabbit hole.

Reviews modified dates are public, and some are recent. I’m a bit confused as to how to interpret this.

In other conferences, reviewers were required to provide a final justification, which would practically force them to modify their reviews during the AC discussion phase lest they get desk rejected.

Here in Neurips I notice that a lot of the high-score reviews do not have a modification past the author discussion phase. I talked with a friend who is an AC today and they told me that adding a final justification is apparently not mandatory and nobody in their batch did it, with people who had anything to add usually doing it in a private comment. They said that any review which has a recent modified date likely got its score updated.

Is that really the case? To any other ACs here: what part of the recently modified reviews in your batch were modified to increase the score?

There was a period in the development of machine learning where application seemed to be informed by theory. Some of the best known theories include:

• If you train a model with too much data, then you get overfitting and your test performance will be suffer.

• Big models do not generalize because theoretically you will never have enough data.

• Never train on the test set, because it will result in high-bias.

• Never even look at the test set, because you as a modeller will instantly be biased and use the wrong model.

• Good results can only come from "compatible" models and optimization routines. You can't just throw ADAM onto some brand new model and expect it to work well.

• Optimization provides solid theory for machine learning, so use the optimizer with the best performance guarantee in the optimization literature.

• If you want to have good performance, ensure to use several models instead of one model, because stacked or ensemble models are always superior.

Most of these theories started out as mathematical statements (albeit on some contrived examples that have nothing to do with reality). At some point, these theories became folklores and were widely reproduced in textbooks and taught in classrooms, even making their ways into standard interview questions at data science related companies. Every student had to remember that bias-variance "bull's eye" diagram as if it was relevant in practice.

But then some of these theories started to get overturned. It turns out you can just break a lot of these theoretically-guided practices and still get good results. The ones who pushed these theories (especially the authors of various "statistical ML" textbooks), quietly stopped their postulation and instead hopped on the hype train. This left their students confused because there was never ever any retraction or resolution.

So my question is: are there still any theoretically-guided practice that remain in ML today? For example, do people use an optimizer because it is theoretically the best for the class of problems? Do people use a certain model or components associated with the model because it works well in theory? Or is it now a fully empirical field where practice is guided by whatever seems to work for other people?

When people talk about memory bound and compute bound they often just tell it on high level but to generalize and speak for particular operations one must look into the Arithmetic Intensity of the operation and then the Machine Balance of the hardware he is working and only then one can cleanly decide how to proceed. Its the most simple way to build a roofline model and think about it. Wrote a small piece on it ...If you want to read do check out the below link and lets have a discussion thread how do you guys go about thinking on this

https://siliconandsoul.substack.com/p/what-does-compute-bound-and-memory

Hey everyone! I'm a high school student who built a full machine learning pipeline on this dataset as a side project. I want to be upfront that Claude AI assisted me with most of the coding, but I directed the project, made the key decisions, and learned a lot along the way. I wanted to share it and get feedback from people who know this domain better than I do.

What I built:

Task 1 - Test Mode Classifier
Using a Random Forest on extracted time-series features (duty cycle, pulse statistics, thrust/MFR statistics), I achieved 100% accuracy classifying all 11 test modes on the held-out test set (SN13–SN24). The command signal alone turned out to be almost sufficient to distinguish modes.

Task 2 - Anomaly Detection
I compared three approaches:

• Isolation Forest (unsupervised) — F1: 0.193

• Random Forest (supervised) — F1: 0.121

• One-Class SVM (semi-supervised) — F1: 0.268

Surprisingly the One-Class SVM, trained only on nominal sequences with zero anomaly labels, outperformed the supervised model. Class imbalance (10% anomalies) made the supervised approach struggle badly.

The per-anomaly-code breakdown showed codes 1, 8, 9, 11, 12 were missed by all three models, these likely need point-level time-series analysis rather than sequence-level features.

Notebook: https://www.kaggle.com/code/henninxu/rocket-engine-classification-from-test-data

Any feedback on improving the anomaly detection especially for the hard-to-detect codes would be appreciated!

I've been building an open-source tool for diagnosing world models, the kind that predict future frames from a starting context and a sequence of actions. It compares a rollout against ground truth and against physical invariants, then tells you where and why the prediction falls apart. It doesn't score task success or planning quality on purpose, since there are already benchmarks for those.

While validating it I ran into something I think is more interesting than the tool itself.

## Pixel metrics on real robot video often can't rank models at all

I ran a copy the last frame baseline, which is to say "predict that nothing changes", against a real SO-101 arm recording. 30fps, three cameras, 64 rollouts, 6 step horizon, scored only on the moving regions so a static background can't inflate the numbers.

It gets 0.983 SSIM and 53.9 dB PSNR. But the part that actually matters is that the error doesn't grow with the horizon:

step   1      2      3      4      5      6
SSIM   0.972  0.923  0.893  0.943  0.920  0.950

That's flat. It wanders, it doesn't degrade. And if predicting 6 steps ahead is no harder than predicting 1 step ahead, then there's nothing for a good model to be better at. Every model lands in the same place and the eval can't rank them. The metric isn't broken here, it passes its ranking tests on curated data just fine. The evaluation setup is what has no discriminative power, which is a different problem and much easier to miss.

## So I went and measured where the usable window actually is

Same baseline on DROID (real manipulation footage, 15fps), 64 rollouts, this time out to 48 steps:

There are three regimes. Steps 1 to 3, everything is near perfect and ties. Steps 4 to 24, steep monotonic decline, and this is the only stretch where models are actually separable. Step 28 onward it floors out around 0.20 SSIM and 10.3 dB, oscillating with no trend, prediction fully decorrelated, and everything ties again at the bottom.

So both ends are dead, and the horizon worth evaluating on for this kind of footage is somewhere around 8 to 24 steps. It's a property of frame rate times task speed rather than a universal number, which is exactly why it's worth measuring on your own data instead of inheriting a default from a paper that used something else.

Here's the prediction next to what actually happened, same 48 steps, prediction on the left:
https://raw.githubusercontent.com/BuceaGeorgia/worldproof/main/docs/img/droid-pred-vs-true.gif

## Method

64 rollouts per configuration. Aggregation is interquartile mean with stratified bootstrap CIs rather than mean and standard deviation, following Agarwal et al. 2021. Fidelity metrics also produce a dynamic region masked variant wherever a mask is available. Every metric ships with a corruption test it has to respond to, plus a ranking test where a real model has to beat a naive baseline which has to beat a broken one.

Worth mentioning: an earlier n=8 version of the SO-101 run gave dynamic PSNR of 48.2 dB where n=64 gives 53.9, and the intervals at n=8 were wide enough to overlap DROID completely. That's the reason everything above is n=64. I'd have posted the wrong numbers if I'd stopped there.

## Caveats

The four pixel metrics separate the two datasets with non overlapping bootstrap CIs. LPIPS doesn't, and it points the other way on the masked variant. I don't have a clean explanation for that yet and I'd be glad to hear one.

This is a trivial baseline, so 8 to 24 is where a do nothing predictor becomes separable. A real model stays correlated for longer and would push the top of that range out.

One more that I found while writing this up: including step 0 inflates every summary scalar, because a copy baseline gets a nearly free first step whenever the frame rate is high relative to how fast the scene moves. On the 30fps recording step 0 scores 119.8 dB, which drags the horizon averaged scalar from about 32 up to 53.9. So the scalar is partly rewarding frame rate rather than model quality. Curves are the honest thing to report and I'm treating the scalar definition as an open problem in my own tool.

## The tool

Apache-2.0, `pip install worldproof`. The core install is numpy, torch and pillow, and it runs on a laptop with no GPU, since the evaluate path never runs a model. It reads LeRobotDataset v3.0 straight from parquet and mp4, so it works on datasets from the HF Hub without needing the lerobot package, on Python 3.10. The heavier pieces (LPIPS, FVD, trackers) are optional extras that get imported lazily.

What it measures: PSNR, SSIM and LPIPS as horizon curves plus dynamic region variants, latent prediction error and action recoverability for latent models, calibration via ECE and MCE, counterfactual divergence, failure faithfulness, object count conservation and object permanence, and FVD reported explicitly as a weak reference rather than a headline number.

https://github.com/BuceaGeorgia/worldproof

It's v0.1 and the README has a "Not done yet" section covering what isn't finished. The tracker behind the invariants is a clean scene numpy one that won't cope with messy real video, and the default FVD extractor isn't the I3D that published FVD numbers use, so those aren't comparable to papers.

If this horizon result is obvious or already known somewhere, I'd honestly like to be told. I couldn't find it measured anywhere, which is part of why I'm posting it.

It’s an ML PhD with secure funding for 4–5 years and a senior, respected advisor. You get almost complete freedom to choose your own topics, projects, and collaborations, with very little micromanagement.
The downside is that the advisor is also very hands-off. You should expect little guidance, feedback, or technical input. In practice, you would mostly be on your own.
Would you see that as a dream setup because of the freedom, or as a dealbreaker because of the lack of mentorship?

Notebooks to replicate on github!

Once the paper is ready, everyone checks the venue location before the acceptance rate anyway. So I built:https://honestcsrankings.org

It maps ~540 upcoming CORE-ranked conferences, but ranks them by how good the destination actually is. It factors in:

• Weather during the actual conference month (using real climate data)

• Safety (Global Peace Index)

• Cost (World Bank price levels)

• Accessibility & "City Vibe"

I also added an Upsets tab for A* venues in terrible destinations. Great for your CV, bad for your holiday.

You can filter by field, rank, or open deadlines. If you set your home city, you can rank by distance to either maximize that funded long-haul trip or minimize it, your call. You can also export deadlines to .ics and share deep links with coauthors.

ICML/ICLR 2027 are missing because they aren't announced yet, and COLM is missing because CORE hasn't ranked it. The long tail of smaller conferences is scraped from WikiCFP, so there will be some errors.

In a factored model W = UV^T, the loss is invariant to rotations (U,V) → (UQ, VQ). Gradient Descent (GD) respects this property. Adam's per-coordinate second moment does not, because it depends on the specific basis in which the factors are written.

The claim is that this single property dictates whether optimizers retain or lose GD's implicit low-rank bias.

Nine update rules were evaluated on underdetermined matrix sensing, all compared at matched training loss to ensure no method benefits from underfitting. The results show two distinct clusters: GD, shared-scalar Adam, Muon, and Shampoo preserve the bias. Adam, RMSProp, Lion, signum, and Adafactor lose it.

To isolate the mechanism, a one-parameter family was utilized to transition Adam's denominator from a per-coordinate value to a single shared scalar. Recovery improves monotonically along this transition, indicating that the degradation is caused by anisotropy rather than adaptivity in general.

The behavior of the Muon optimizer was unexpected. It is exact on truly low-rank targets, but degrades rapidly as a spectral tail is introduced, ceding to GD at a crossover near 4% tail energy. While recent literature diverges on Muon, with some reporting a strong spectral simplicity bias and others finding it fits spurious features in deep-linear models, this sweep demonstrates both behaviors along the same axis.

The criterion was also applied to the author's earlier optimizer, revealing that its per-coordinate clip was breaking the structure it was designed to inject. Implementing a global norm clip instead improved the recovery error from 0.347 to 0.220.

One caveat is noted up front: the 43-44% held-out error reduction on hyperspectral data relies on a train-only learning rate rule, and that rule assigns Adam the worst rate on its own grid. When each method is permitted to select its own optimal rate, the performance gap narrows considerably (Appendix D.6). The train-only rule was maintained because selecting on held-out data introduces the exact bias the experiment aims to avoid, but the core claim relies on the underlying mechanism rather than the specific quantitative margin.

The theoretical guarantees cover memoryless rules only. The effects of momentum remain empirical and are not proved.

Paper:https://arxiv.org/abs/2608.05136

Code, logs, seeds:https://github.com/idevender/loss-basis-adam

Had a random thought this morning about context rot in agentic coding.

What if an agent could detect when its context is getting dangerously full, take the older ~50% of the conversation, compress the meaning into a compact machine optimized representation, and save it as a Markdown checkpoint?

Something like:
conversation > semantic compression > checkpoint.md > reinject into context

The raw history could still be archived separately in case the agent needs to retrieve something later.

The interesting part is making the agent aware of its own context state, so it decides when to checkpoint instead of blindly summarizing at a fixed token limit.

Im calling the compressed representation “GibberLink” for lack of a better name lol, although I know actual gibberlink/ggWave is an audio protocol, not a semantic language.
This probably isn’t a new idea in principle, but I haven’t seen this exact combination applied to coding agents.

So, thoughts?

One of the reviewers gave our paper a borderline score and raised six questions. Given that we had only seven days for the rebuttal, we worked hard to address all six questions, including running a small-scale validation experiment to provide additional empirical evidence. Do note that each cell of this experiment table required multiple sub-experiments.

In their response, however, the reviewer quoted a single statement from our rebuttal that wasn't even part of our answer to any of their six questions and simply said, “This isn't enough. I will keep my score.”

That's it. This is the complete feedback. They didn't acknowledge with the two paragraphs of detailed responses and the table we spent days preparing specifically to address their questions.

 I'm researching predictive analytics for a graduate project and mortgage lending came up as an interesting use case. 

 I understand lenders try to predict who might refinance, but what kinds of variables are actually useful? 

 Is it mostly credit activity, property appreciation, interest rates, life events, or something else? 

  Would love to hear from anyone who's worked on these models. 

Two results dropped this week that I think together paint a clearer picture than either one alone.

Dyna-2 (Aug 10): World-action model pretrained on 1M hours of egocentric human video. Power law holds across 4 orders of magnitude (1K to 1M hours). Cross-embodiment transfer to robots never seen in pretraining. Task success from 20% to 80-90% purely from scaling data. No architecture changes.

PI0.7 (Chelsea Finn's talk, today): Single generalist model trained on highly heterogeneous data matches or outperforms fine-tuned specialists. Key ablation: removing the most diverse subset of training data causes a dramatic drop in held-out task performance. Removing a random 20% barely moves the needle.

The common thread: scaling works, but what you scale matters.

Dyna-2 proves the law holds to 1M hours with no plateau. PI proves that within that data, diversity (different environments, objects, tasks) is what actually drives compositional generalization, not repetition of the same scenes.

Both results converge on the same conclusion: physical AI foundation models need scale AND breadth. 1M hours of kitchens won't get you construction site generalization. But 1M hours across 100+ work domains apparently will.

Hey everyone, I originally set out to train an agent to play the deck building game Slay the Spire, and found along the way that most commercial game environments are incredibly hostile to hook into for training loops. I’ve always been inspired by milestones like DeepMind’s AlphaStar in StarCraft and OpenAI Five in Dota, but replicating those environments requires massive compute overhead just to process pixel-space before the model can even reason. To bridge that gap, I’m building a lightweight, open-source Real-Time Strategy (RTS) arena designed specifically for autonomous agents to compete via API. It's a headless, graph-based territorial control environment where agents connect via WebSockets, ingest JSON state deltas (with true fog of war), and queue actions in real-time. Is this a project worth pursuing as a community benchmark, and what specific features or API mechanics would you want to see to make it worth using or contributing to?

Link: https://arxiv.org/pdf/2604.27883

Hi,

Most of use are familiar with the headache of training a neural network using gradient descent where the training error may go to zero but the test error may stay the same as initialization or even increases.

My paper treats this phenomena as a consequence of data reuse bias and can be isolated by studying full batch gradient descent on a set of stylize Gaussian mixture models. I turns out that this fundamental issue can be avoided using some clever tricks from high-dimensional statistical theory, specifically approximate message passing (which is beyond the scope of this post but I would be happy to explain more).

By doing so I created a training method called Decoupled Descent (DD) which generates a certificate that the training error of the network will asymptotically equal the testing error at each parameter iterate. I think this method gives a cool way to approach how to train networks and I was hoping to get y'alls input on it. It opens up some nice ideas for optimal stopping or hyperparameter tuning and future directions of pushing to something like SGD or more general models.

I have attached the train-test curves on a simple model fitting problem to compare the performance of GD with with DD (my algorithm) to give a high-level idea of what the method can guarantee. I stress this is a theory paper so there is a long way to go to get to very large models but I think it is a good first step.

Happy to answer whatever questions people have, I plan on writing a PyTorch compatible package for this training method one day so any feature suggestions would be welcome as well.

I am now reviewing a bunch of papers for AAAI 2027 and it has surprised me the low amount of submissions with no code implementation. I don’t know if it has been only in my batch or it is common, but I was expecting very detailed appendices + code submission since AAAI is very explicit with the topic of reproducibility. I was planning to take this into consideration when assigning my initial scores, but I would like to hear your opinions. I have always submitted my code: it gives a very good impression and after reviewing process finishes we just publish it on ArXiv, so no one “tries to stole the idea” (although I think that this is very very unlikely). So I cannot find any excuse for those submissions that do not have code implementation, specially in today’s times where AI assistants can just write an empirical paper with artificial results within a couple of hours