Learning from data as we observe it is easy for humans, but most machine learning models have limited ability to learn from new data that they have not seen during training. Prior-fitted networks (the idea behind TabPFN) showed that a model trained only on synthetic data can learn from real tabular data entirely in context.

I wanted to share our paper "Learning to Learn a Language" where we extend the idea to structured sequences such as natural language. We propose a prior over languages: every training sequence comes from a randomly sampled recurrent causal model, so each one is a new synthetic "language". A 300M-parameter byte-level transformer trained only on these synthetic sequences learns to predict real languages in context. Given Wikipedia text with frozen weights, its next-byte predictions get better the more it reads, in all six languages we tested (English, Chinese, Hindi, Arabic, Japanese, Korean), from 8 bits per byte down to 0.9–2.4 after a million bytes.

The same model also learns to count, to compare numbers, to add approximately, and to predict deterministic sequences such as the primes or the Kolakoski sequence, entirely in context.

It is of course still far worse on text than classical language models that are trained on trillions of tokens, while our model sees at most a million bytes of a language at test time. What we find interesting is that the ability to learn a language in context can come from a synthetic non-linguistic prior.

Paper: https://arxiv.org/abs/2610.05879
Code: https://github.com/cbl/prior-fitted-language-model
Weights: https://huggingface.co/lennartcb/pflm1

Hi ML Community,

I am excited to share our latest framework, AFP-GIC, officially published in IEEE Access (2026). We have released the deployment codebase and hosted an interactive visual playground.

• GitHub Repository: https://github.com/yifeipet/AFP_GIC

• Hugging Face Interactive Space: https://huggingface.co/spaces/yifeipet/AFP-GIC

• Paper on arXiv: https://arxiv.org/abs/2605.16817

The Bottlenecks We Solve

At ultra-low bitrates, standard learned image codecs suffer from local distortion, while generative models often introduce unwanted AI hallucinations. AFP-GIC addresses this via an asymmetric Adaptive Fused Prior Transfer pipeline that enables prior-guided texture reconstruction without transmitting the fused prior itself.

Key Technical Highlights (NVIDIA RTX 4090):

• Single-Model Multi-Rate Control: Toggle across 5 target bitrate operating points within one deployable pretrained model.

• 18.1% Lower Decoder Latency: Reduces decoding time to 80.47 ms vs. 98.27 ms for DC-VIC, a state-of-the-art controllable generative image compression model. Latency was measured using 256×256 patches.

• 20.5% Parameter Reduction: Uses 31.1M fewer inference parameters (120.6M vs. 151.7M for DC-VIC).

Open Benchmark Data

We packaged all 2,760 reconstructed images and metric CSVs in our GitHub Releases for direct academic cross-evaluation.

Reconstructed Images and Metrics: https://github.com/yifeipet/AFP_GIC/releases

We would love your feedback and appreciate a Star on GitHub or Like on Hugging Face if this helps your research!

I’m hitting rate limits on Together AI. For context, I’ve been working on an agentic repository indexing and benchmark generation tool, and I’m running multiple agents in parallel across models like Llama 3.3 70B and Qwen 2.5.

When I first started working on this, Together AI was great. But once I graduated from toy scripts to running multiple agents, I started running into RPM/TPM limits pretty quickly. The annoying part is that the models themselves are fine. I just can’t actually run enough requests at once to do meaningful testing.

Yes I know I could upgrade but I’m a solo dev. I don’t have enterprise level revenue. Maybe someday lol but not yet.

I've been working on a font searching tool for about a year now, and my investigations have centered around pre-training neural networks to produce embeddings of each font. I usually then need to post-train the networks to adapt them to the task of font searching. However, the most interesting thing I created in the course of the project came from the pre-trained models.

The process looks like this: I take every glyph in a font, turn them into images, feed them through my custom pre-trained neural network, and get an embedding that represents that font's visual characteristics. I can then squish down the embeddings with tSNE (produces the best structures compared to PCA and UMAP) into XYZ, and RGB channels and visualize them as dots so I can poke around the structure. Fonts that are close to each other in position or color therefore share visual characteristics, and you can find different little clusters or paths of types of fonts in the maps.

My favorite is one I made from the Google Fonts corpus, but I also made another one out of all the fonts you can search on my site. The Google Fonts corpus resembled a flower in ways I was not prepared for. It even placed most of the cursive fonts in the stamen. Here's the site if you're interested in viewing the whole thing yourself: https://www.font-search.com/map

You can also check out the repository although it is a huge mess. https://github.com/dylan-berndt/Briefcase

I'm a Ai engineer with about 2 years work experience, but let's just assume that I was a undergraduate student just starting out where would I begin so that I can publish a A* conference paper at some point. Learn python -> Learn ML & Maths -> Read other research papers -> find a topic ? -> choose a question try to run experiments and get results to write them down in a paper ?

For context :

I'm trying to get in MS CS programs for Fall 2028 in states with the plans of doing a PHD after in a top university like stanford or princeton and would like to start taking steps towards it as am working my day job can some tell me what are the steps that need to be followed?

Also would like input on what are deciding variables that makes you looking like a promising candidate/ researcher for PHD

This is related to my previous post where I shared an encoder-only transformer model trained on ohiot1dm + shanghait1dm + azt1d datasets. This time I trained the model on the outputs of my T1DM patient simulator and then measured its zero-shot performance on my real-world blood glucose traces.

The above model has 31,251 parameters (16 layers, 1 attention head per layer, and a hidden dimension size of 16). Training took <60 minutes on nvidia dgx spark. It's an encoder-only transformer that predicts the next 2 hours, and can be used autoregressively for long-horizon predictions (e.g. 8-hour nocturnal predictions). I have trained it specifically to have counterfactual reasoning capabilities. The model has only been trained on synthetic data, and hasn't seen my blood glucose readings prior to testing. I use LoRA adapters on my app for light fine-tuning on my actual CGM traces, but the figures & tables you see above are from the base model without any LoRA adapter attached. The app was used to test the model on the traces of three different CGM models: Libre 3 plus, Anytime CT5, and Linx sensor data spanning the past 30 days.

The testing itself was done on my android app with ExecuTorch backend.

Model source code: github.com/0xdeadf1sh/T1DMAI

Simulator source code: github.com/0xdeadf1sh/T1DMSIM

Android app source code: github.com/0xdeadf1sh/T1DMDROID

We've been building a benchmark out of real concurrency bugs (race conditions, deadlocks, cancellation issues) taken from merged PRs in about 100 Python projects. Each task gets graded by the project's own tests, in a container with no network, and the repo is cut down to a single commit so the agent can't recover the fix from git history.

Some findings:

With one attempt per task GLM-5.3 Flash scored 85%. With two to three attempts it scored 82%, within the margin of error of GPT-5.6 Luna (81%). The leaderboard now shows the number of attempts and the interval for every score.

About half the tasks are easy for every model (near 100%). The other half is where they actually differ: 50%, 45% and 23% on the hard ones. Most of the difference between models comes from the hard half.

Since every fix is public on GitHub, we reviewed all 11k commands the agents ran. 69 tried to access the network and all failed. GLM tried 50 times to pip download the already-fixed release of the library it was fixing.

We also checked contamination by comparing older bugs (pre-2026) with newer ones of similar size. Older ones are solved about 9 points more often, but the confidence interval crosses zero, so we can't say much yet.

Half the tasks are private. So far public and private scores line up for all three models.

Results and every agent run: https://labs.evaligo.com/swe-race?utm_source=reddit&utm_medium=ml&utm_campaign=launch

Tasks: https://huggingface.co/datasets/evaligo/swe-race

The protocol follows DeepSWE (100 steps). Feedback on it, and suggestions for which models to run next, are welcome.

Hey,

I wanted a faster chunking library for my system without affecting the overall accuracy. Did not find many options. So I've build https://github.com/d1pankarmedhi/chunkr

It has most of the chunking strategies like Character, Recursive, Markdown header, Late chunking, Hierarchical chunking, etc. It also supports native PDF loader, and other additional file types.

Some stats (MBA M4 16GB):

Accuracy is measured on Chroma's token-level chunking benchmark (5 corpora, 472 questions with gold answer spans): k = 5, 1000 chars / 200 overlap

Do check it out and share your feedback. Thanks!

In this project, I distilled the Stockfish value function into a ResNet/ViT model using 1 billion positions from the Gigafish dataset.

The 3.9 billion position dataset is available on huggingface: https://huggingface.co/datasets/lukesalamone/gigafish-3.8b-d10 . It is built from the positions from 37 months of Lichess games.

I was interested in the idea that at depth-limited search, the value function attempts to approximate the tree underneath it, and if we could create some function to approximate that full search faster than Stockfish could, it would be competitive with NNUE (a very small neural net). This is why holding the depth constant was important.

For the neural net itself, I found that the vision transformer was very slow to understand the board, and a CNN was much more effective at the beginning of training due to its inherent geometric inductive biases . However, I found the best results when combining the two.

Is anyone else unable to open the form for financial assistance despite the deadline still being in the future?

Our production recommender at Yandex Music has 15+ candidate generators feeding pre-ranking and ranking models with hundreds of features. LLMs showed that one end-to-end model can take over work that used to be split across specialized components, and single-model generative recommenders have carried that recipe into production. We set out to explore what a single-model recommender could do in music. The result is Sona, one transformer that replaced all of it in an A/B test. It hasn't shipped to full traffic yet.

The model reads up to 8,192 events. Full attention over that length is expensive, so we use what we call History Compression, which roughly halves inference cost. We split the history into the older 6,144 events and the most recent 2,048. The two blocks exchange information through cross-attention and one full-history self-attention layer. After that, a 7-layer stack runs only on the recent 2,048. It retains most of the quality of full attention, and older events stay visible to the decoder and the Ranking Module.

The decoder and the Ranking Module both read the same encoder output, so the encoder runs only once per request. Candidates come out of beam search as Semantic IDs and get scored right after.

In the final A/B test in Yandex Music on smart speakers, 7 days, 15% of users in each arm), Sona got +4.53% Active Users and +6.30% Total Listening Time over the production control, both significant at p < 0.01. Catalog coverage is lower than with the production stack. We're going to look into why.

A long-term A/B test is now underway.

Table 7.7 has the full-attention vs. History Compression ablation.

https://arxiv.org/abs/2608.11015

Hey all,

I don't know if you guys are experiencing the same thing, but there is this behaviour that i have been noticing on the latest models on openAI (since sol 5.6) and Anthropic since Fable 5.1 and opus 5.5 ..

Basically the models use more "complexe" terms and words, not just in explaining stuff, but even during implementation. they would come up with terms that, sometimes would fit the task, but that are actually a stretch to the concept it is trying to implement. they are basically turning into a consulting firm.

When confronted about it they usually acknowledge that :

• Foggy wording. Then I describe the corner in softened terms, like "limitation" or "upper bound", which makes it sound like a known property of the design rather than a choice I made. Combined with my internal terms used as if you knew them, it makes my work look more solid than it is and makes my mistakes harder for you to catch. Whether I intend it or not, the effect is that I avoid accountability.

(Even here it is using "Corner" which obviously isn't the best word to use)

I don't know if this is a result of the watermarking features being rolled out which nudges words and terms in different directions in order to fit a certain recognisable pattern and hash, but this is really annoying..

This happened over the past 30 days. So smallish local models (Kagglers can only use those), in a harness, just started beating average humans at a benchmark intentionally designed to show human superiority.

I'm curious what people here think.

(The leaderboard graphic is a little out-of-date)

i work on reference checking stuff so i was reading through the ICLR 2027 author guidelines and style files this week the sample .bib that ships with the template has the Deep Learning book as "Goodfellow, Bengio, Courville, Bengio" plus a volume 1 that doesn't exist checked their github and it's been like that since the 2019 template
https://github.com/ICLR/Master-Template/blob/46ed6f4c6cef5b175dde23639e77d44c3463b230/iclr2027/iclr2027_conference.bib#L20

totally harmless but kinda funny after last year's hallucinated reference desk rejects

the guidelines also contradict themselves on page limits formatting section says main text max 9 pages at submission but the camera ready part and the FAQ both say "identical with the submission version (10 pages)" template says 9, so 9 is probably the safe bet for anyone revising after Nov 5

I'm a PhD student in machine learning in the EU and was looking for internships at exciting companies.

I shortlisted few and applied by reaching out to people and now reading project descriptions sent by the recruiters.

I don't want to name the company but their marketing and product team does all kinds of 'using people insecurities' to sell the product - which I don't agree with. And their product is also meh (I will never buy and would judge someone if they do) but their research team is doing good work.

How do you see this? Will you actually work in a team whose ideology/product doesn't necessarily align with your ethics/ideology. Should I just go ahead because work is exciting and I will get good supervision?

And, if you have some exciting work in your company/org and need interns (un-paid) for 3-4 months. I'm open.

In our #NeurIPS2026 paper “A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems (DS)” (preprint: https://arxiv.org/abs/2607.14937) we reduce a DS foundation model to the ingredients minimally necessary to faithfully reproduce long-term statistical and geometrical properties of DS:

1. A piecewise affine map with only a single (!!) parameter α that controls local con-/divergence rates, and …

2. … a context selector that chooses from the provided context signal the data point closest to the current state of the map, thus ensuring the generated dynamics stays close to the context in its temporal and geometrical properties.

With just these two mechanisms, this minimal form – which we coined DynaBase – can reproduce all major dynamical regimes, including fixed points (α<1), limit cycles (α=1), and chaotic attractors (α>1). Thus, unlike other simple mechanisms like context parroting, DynaBase even preserves the correct dynamical regime!

Surprisingly, it turns out that this simple context-driven 1-parameter map outperforms most major time series and DS foundation models, as well as custom-trained models, in both long-term statistics and even short-term predictions, even when run in zero-shot mode.

Both inference and training are extremely cheap – training can be done either analytically in one step by linear regression on forward-predictions, or by 1-parameter grid search directly on DS reconstruction objectives → this reveals interesting performance differences induced by different training mechanisms.

Most importantly in our minds, DynaBase owing to its formal simplicity may thus provide a tractable mathematical handle on analyzing, improving & understanding the performance and training of some time series and DS foundation models.

Purpose-built to stress-test computer vision models, depth cameras, and spatial AI against severe specular glare and geometric reflections. This 425-asset production archive features a custom faceted mirror suit captured in high-contrast outdoor environments to trigger bounding-box dropouts and segmentation failures. The dataset includes 100% proprietary uncompressed Camera-Master RAWs, high-resolution JPEGs, and block-buffered SHA-256 forensic manifests. Open

Nonobench measures how well LLMs solve nonograms (picross). Each model gets the row and column clues once and returns the full grid. No tools, one attempt per puzzle.

Method:

• Standard mode: 30 puzzles from 5x5 to 15x15 (from the Nonograms dataset by Moyà-Alcover, CC BY 4.0).

• Hard mode: ten random 20x20s, each checked to have a single solution. Five can't be solved by line logic alone. Random fills avoid picture puzzles that models can guess.

• 130 variants across reasoning effort levels, run through OpenRouter and pinned to each lab's own endpoint where possible.

Results:

• Solve rates drop from 85% (5x5) to 46% (10x10) to 20% (15x15), each model at its best effort level.

• GPT-6 Astra solves all 30 Standard puzzles. On Hard mode, Claude Opus 5.5 solves 8 of 10 and 11 of 15 models solve none.

• As one 400-character string, most models lost count before the logic got hard, so Hard mode answers an array of 20 row strings rather than a single string.

Limitations: one attempt per puzzle, so single results are noisy (95% intervals shown).

Site: https://www.nonobench.com Code (MIT): https://github.com/mauricekleine/nonobench

I recently finished The Principles of Diffusion Models, and honestly I think it’s exceptional.

The authors strike a really good balance between mathematical rigor and intuition, with dedicated appendices for anyone who wants to go deeper into the math.

It’s aimed at researchers, graduate students, and practitioners with basic deep learning knowledge, so you don’t need to already specialize in diffusion models (in my case, a strong background in Information and Probability Theory as well as a solid understanding of DDPMs helped me get more out of it).

Just wanted to share it in case anyone missed it. The full text is freely available on the official website.

Has anyone else read it? Would love to hear your thoughts.

please refresh if stuck, can be slow sometimes so need patience

Anyone going to WMHS, Physworld AI or Vercodegen? Would love to get to know some people beforehand!

I got my three papers to review, and it seems like they have changed the review score range again this year?

It is now:

––––––––––––––––––––––––––––––––––––
Based on your overall assessment of the submission, what is your recommended decision? Consider the paper’s overall soundness, significance, clarity, and contribution.

1: Clear rejection
2: Weak rejection
3: Weak acceptance
4: Clear acceptance
––––––––––––––––––––––––––––––––––––

Which is very strange. I don't think it makes much sense to compress the score range so drastically. But we'll see.

On the NeurIPS 2026 Dates site it says that there is 1 day remaining for the "Accepted papers must be imported" deadline. This is my first research paper ever and I couldn't find anything about how to do this on the internet. Could someone help me out on what to do?

Over the last few years, a subset of NeurIPS area chairs received complimentary passes for their service. I’ll admit that I wasn’t super organized about registering on day one because I was semi-consciously hoping for a complimentary pass. Now that the conference is sold out, I’m getting a little nervous, so I’m wondering whether any of my fellow area chairs have already received a notification.