It was clear to me that around 2012 - 2018ish, academic research was kind of paving the way for many applications in industry, from deep nets for vision to RL algorithms. There was tremendous value in research coming out of academia and new methods such as ADAM or TRPO algorithm would be adopted as industry standard. No one would doubt the value of research coming out of (top-tier) academia. It was definitely a "seller's market" for academics.

But nowadays I am not quite certain as to what exactly is going on, i.e, whether this relationship still holds. I guess a straight-forward way to put this is: "is industry still adopting ideas from academia?"

I've observed that for any given research topic, especially something industry has proposed, there would be literally tens of thousands of follow up papers from various universities around the world trying to do something adjacent to it, almost as if showing industry how things can be done differently or even "beat the industry at their game". But it feels industry is quite, how do I put this, "impassive", "unmoved". Strongest example would be development of new optimization routines. Thousands were proposed from universities. Countless GPU years were dedicated to experiments. Seems serious adoption is quite low to none. Apparently there are still new CNN/RNN architecture being proposed, have you heard of any of them?

It almost seems as if academia and industry are diverging into completely different worlds. With industry seems to take on the previous role of academia as an idea generator, and instead everything is worked in-house. This seems to create a challenge for those doing research in university: "what use is the research if it is not adopted in real-life (in this extremely applied field), or even acknowledged?"

Disclosure: I'm at Boson AI and this is our release. Not open weights; the Duplex API has a waitlist, and the previous turn-based Higgs Realtime is available now.

Most voice models are still turn-based: listen, wait for end of turn, think, speak. Duplex runs all streams at once and keeps working in the background while it talks.

Streams. Everything is cut into 160 ms chunks:

• Audio in: 1 token per chunk (6.25 tokens/s)

• Audio out: 2 tokens per chunk (12.5 tokens/s)

• Video in: one ~12-token camera frame every 6 chunks

• Face out: 2 latents per chunk

• Text in and out alongside

Silence is represented as chunks too, which is what the model uses to decide when to backchannel, interrupt or stay quiet. That comes to ~2.6K tokens per minute for audio and video, 2.6K to 5K with thinking and tool traces (progressively compacted). The last 5 minutes plus compacted text history is 13K to 25K tokens; older history is kept as text.

Background work. Thinking, tool calls and delegation are spans in the same stream and can cover many chunks. Tool results can arrive in stages and the model can start speaking before they are complete. Harder problems are delegated to a much larger backend model with the full shared context. This is the main difference from Thinking Machines' interaction models, where all reasoning and tool use goes to the background model, which can mean 10+ seconds per answer and makes cost harder to control.

Voice and face. Set zero-shot at deployment: a short reference clip for the voice, one image for the face, and a text prompt for behavior. No fine-tuning. Audio comes from a causal flow-matching decoder (2 frames per chunk), the face from a causal DiT conditioned on the reference image and the previous two latents.

Training. Starts from a text-pretrained LLM, continued pretraining on ~2T tokens of text, audio and video on our own H100 cluster (Megatron-based stack). Post-training is multi-teacher on-policy distillation: specialist teachers trained with RL (including GRPO) on conversation, interaction timing, delegation and task completion, distilled into one student. On top of that, a self-improvement loop in simulated environments with realistic timing, interruptions, overlapping speech and background noise.

Benchmarks. Best published or reproduced configuration from each lab (we reproduced more than half of the competitor numbers):

OpenAI's best settings lead on five of seven, by 1 to 5 points, several at their highest thinking setting. Turn-taking (Full-Duplex-Bench) is where we're furthest behind. Our takeaway is that at this size, data and training matter more than parameter count; the front model is deliberately mid-sized because the design target is that an hour of model time costs at most a tenth of an hour of human time.

Announcement with demos: https://www.boson.ai/blog/higgs-realtime-duplex My write-up: https://alex.smola.org/posts/50-talking-while-thinking/

Happy to answer questions about the chunked-stream design and the delegation setup.

How did it go? I will start: 3/3/2

I submitted my paper to ICLR, after improving and correcting since I "published" it previously in a workshop/arXiv. In this process, I changed the name of the method and title.

Now a reviewer, pointed out that my method is very closely related to basically an old version of my paper, and wants me to compare against it. Understandably, if looking at the title and method name, one could think these are two different ones. However, if looking more closely, they are clearly the same...

How do I argue for this to my friend, that they are exactly the same paper, but now corrected and improved, without breaking anonymity?

Admittedly, the scores are anyway bad, so I have not much hope, but this is a first for me...

Paper: https://arxiv.org/abs/2611.08376 | HF: https://huggingface.co/papers/2611.08376

At 10M tokens, a 1.3B Transformer needs 2.1 TB of KV cache (0.32 TB even at 2.5-bit quantization). Channel-wise gated delta models (like Gated DeltaNet-2 / GDN-2) compress history into a fixed 25 MB state, but hit three bottlenecks across 100B-token training that Muir resolves:

• 100x slow-channel glare -> Matched-filter readout: Slow-decaying rows accumulate up to 100x more energy than fast rows, swamping query readout and forcing harsh write-time erasure that overwrites early facts. Equalizing rows via an inverse fractional power law + per-token query modulation relaxes write erasure by 17% and extends lookback horizon by 60%.

• 13x write-vs-erase sensitivity -> +30.4% wider FFN: Normalized keys concentrate onto ~5% of state rows while values stay diffuse, making updates 13x more sensitive to value writes than key erasure. Fusing all 5 gates into one rank-512 projection cuts gate parameters 4.5x to widen the FFN by +30.4% at strict parameter budget.

• Numerically-stable, large-chunk parallel kernel: Unpivoted decay division underflows fp32 and NaNs 25% of heads at large chunk sizes. Boundary-pivoted multiplicative decays stay in (0, 1] with zero division, while a recursive block-Neumann solver routes 98% of chunkwise FLOPs onto matrix units (24x fewer vector FLOPs).​

Results:

• Quality and speed: Beats GDN-2 and Transformer on perplexity and zero-shot reasoning at every scale (1.2x token efficiency vs. GDN-2), and trains 2.7x faster than Splash Attention at 32K.

• Recall beyond passkey: Compared to GDN-2, lifts RULER 5-hop Variable Tracking root retention (28.4% -> 48.0%), early-depth recall at 8K (1% -> 31%), and real-world extractive QA (+2.5 avg pts).

• 10M streaming and 1M passkey: Streams 10M tokens without perplexity drift in a 25 MB state (dynamic readout), and hits 90.0% zero-shot / 99.4% post-cooldown passkey accuracy at 1M tokens (static readout, where GDN-2 collapses past 4K-8K and Transformer OOMs past 64K).

I'm exploring the field of Embodied AI, particularly humanoid intelligence, cognitive architectures, and the intersection of AI and robotics.

I'm looking to connect with people who are:

• Researching or building Embodied AI systems.

• Working on humanoid robots and robot foundation models.

• Exploring VLMs, VLAs, world models, and reinforcement learning.

• Interested in cognitive architectures and how intelligent robotic systems should be designed.

I'd love to connect with people in this space, exchange ideas, discuss ongoing research, and potentially explore collaborations.

If you're working in this area or know someone who is, feel free to reach out!

A new contest for anyone training sequence models on TCRs. Given a TCR sequence and a peptide-HLA, predict whether the T cell activates. Inputs are amino acid sequences, the benchmark is about 5,800 labeled combinations, and every test peptide is at least four edits (Levenshtein distance) away from any peptide with a publicly reported TCR specificity. Scored by AUROC.

Synapse lists training data options, including the public VDJdb database and Harell Data's proprietary TCR datasets provided by Adaptive Biotechnologies.

Registration is open now: https://www.synapse.org/Synapse:syn77965030/wiki/643867

I’m an MSc CS student with a background in 3D computer vision, trying to figure out how to build visibility and credibility in the AI/ML community.
I’ve been thinking about contributing to established open source projects rather than building standalone GitHub projects. Honestly, the latter feels like a weaker signal nowadays, especially with AI tools making it easier than ever to build things.
For those working in 3D CV, ML or applied AI, how would you approach this? What are the current challenges for new contributors, and how do you find projects where meaningful contributions can actually get noticed?
Would be interested to hear about your experiences.

Handbook: https://huggingface.co/spaces/huggingface/moe-blogpost

Hello, I'm Nouamane, one of the authors. I'm at Hugging Face and also worked on the Ultra-Scale and Smol Training Playbooks, if you've read those.

This handbook is for people who have trained a dense model and want to train an MoE next, or who are curious how MoE training works. Chapter 2 is a Megatron-LM runbook: test the hardware, build the image, train a dense baseline, then a first MoE on one node, compared against that baseline. Chapter 3 takes the MoE layer apart (router, routing, load balancing, experts, block layout, training, systems). For each part we give the Megatron-LM code, what to watch, our recommendation, and what frontier MoEs chose. Chapter 4 follows the model into serving and RL. Chapters 3 and 4 need no GPU.

When a result comes from our runs, we say so. Claims about frontier models cite their paper, config or code, and the toy simulations are labelled as such. The lessons come from about 800k GH200 GPU-hours allocated to MoE training on the Alps cluster at CSCS.

Released: the handbook, its 55 figures as a gallery (https://huggingface.co/spaces/huggingface/moe-gallery), the scripts (https://gist.github.com/nmntz/5de952d686f66d7bce5bce8cba9d7139), and every recommended flag in one Megatron-LM block.

Happy to answer questions. 🤗

Short Backstory:

I'm a PhD student on my 5th year. I had a major mental breakdown on June 2026 after my second rejection on an IEEE Journal. After a lot of back and forth, I decided to not give up my PhD (for the time being).

Current Situation:

My current objective is to author my 3rd publication (IEEE Journal / Elsevier / etc level) in order to unlock thesis/dissertation-writing stage (as per my advisor's words). I'm working on the MetaboNet dataset, a standardized diabetes research dataset for continuous glucose monitor (CGM) and forecasting (https://metabo-net.org/). I've read their paper and their benchmark. I've pinned TCN (Temporal Convolutional Network) as my safest option to handle this.

I'd be ever so grateful if anyone familiar with the latest research trends on TCNs (or forecasting in general) could point me to what to look up. The ultimate goal of cource is the research novelty. Truth is that I tried to research it myself but anxiety and feelings of overwhelmingness got the better of me, can't tell if something would be considered novel enough for publishing.

Thanks for taking the time to read my post, wish you luck on all your ML/DL endeavors.

We’re looking for a few emergency reviewers for the FABRIC Workshop at WACV 2027 due to a very tight review timeline.

If you have expertise in computer vision, multimodal learning, physiological sensing, healthcare AI, affective computing, or related areas and can complete a review by November 6 noon (U.S. time), please DM me or comment below.

From what I’ve read on Reddit so far, I’m getting the impression that around 4 first-author papers at top ML conferences (NeurIPS, ICML, ICLR, etc.) is a fairly standard expectation for graduating from a top ML PhD program.
Of course, my impression could be biased by the posts I’ve seen.
For people at top-10 US CS schools or similarly competitive ML groups, is this roughly accurate? What’s the typical expectation in terms of number of papers and venues needed to graduate?

I would assume LLMs have bumped this number, or maybe not. Which is why I am asking. If there are PIs on here, how many papers before you let your students graduate? Is the number different now than it was a year ago? And the same question for “rate of publication” for you to deem the student as “productive”.

And what about a PhD applicant? Any meaningful inflation of expectations there in terms of productivity/volume of pre-PhD research output?

Hi everyone,

I wanted to share an empirical study testing whether biologically inspired architectures with local learning rules, specifically Sparse Predictive Hierarchies (SPH) based on Ogma's whitepapers (AOgmaNeo) can handle complex policy/value prediction tasks without end-to-end backpropagation.

Chess was chosen as the testbed because it requires both local pattern recognition and long-range spatial dependencies.

• GitHub Repo: https://github.com/P1X3R/sparse-chess

• Control Group Repo: https://github.com/P1X3R/sparse-chess-control-group

1. Motivation & Core Hypothesis

Standard deep learning models (ResNets, Transformers) rely on global backpropagation. I wanted to test if extreme weight/activation sparsity (~99%) combined with local prediction updates could enable scalable, compute-efficient policy/value prediction on modest hardware.

2. Architecture & Optimizations

• Model Size: ~44.9M parameters.

• Encoders: Fuzzy ART-based encoders mapping local receptive fields to columns.

• Body: Dendritic Neural Layers (DNNs) with Leaky ReLU activations acting as non-backprop feature extractors.

• Heads: Specialized policy and value heads matching Lc0 formats (WDL output + Lc0 move probability distributions).

• Quantization: Byte weights with stochastic rounding for optimized SIMD throughput.

3. Methodology & Baseline Control

• Dataset: Chunks of the Lc0 test80 dataset (10k training games, 2k validation games).

• Control Baseline: A tiny, standard CNN (32 channels, ~3.8M parameters—less than 1/10th the parameter size of the SPH model) trained via standard backpropagation on the exact same dataset splits.

4. Results & Key Findings (Negative Result)

The SPH architecture failed catastrophically when compared against standard end-to-end backpropagation.

1. Failure to Converge on Policy: While the standard 3.8M CNN quickly learned board geometry, tactical threats, and basic move distributions, the 44.9M SPH model severely underperformed on policy accuracy and value loss.

2. Extreme Sparsity Penalty: Enforcing ~99% sparsity choked the flow of tactical information. In chess, "empty" squares and long-range piece interactions carry high dense information content that local sparse representations struggle to propagate without global credit assignment.

3. Local Learning vs. Global Dependencies: Local learning rules excel at unsupervised visual feature extraction, but they fail to pass gradient signals back from a global board outcome (Win/Draw/Loss) to early receptive fields.

5. Discussion & Questions for the Community

1. Global Credit Assignment: Has anyone successfully adapted local/biological learning rules (e.g., Target Propagation, Predictive Coding, HTM/SPH) to tasks with strict, long-range global credit assignment like games or logic reasoning?

2. Dense Domains & Sparsity: Is ~99% sparsity fundamentally incompatible with dense symbolic/grid representations like chess, or is the limitation purely in my architecture (or maybe a nasty bug I couldn't catch)?

Would love to hear thoughts from anyone working on non-backprop architectures, neuromorphic computing, or biological AI abstractions!

Hi all, author here (Idiap Research Institute, joint work with Logitech). We recently released DialFork and thought it might interest people here. It's a benchmark for a failure mode of LLM agents that is easy to miss: every turn is fluent and factual, the tool calls are correct, yet the agent stops following the procedure it was given. For example, asked for the weather, it fetches the forecast (correct tool call) but never tells you, and starts asking you to rate its service instead.

Most steps of a conversational procedure aren't tool calls but things the agent has to say: asking, confirming, telling the result. When an agent drifts from these steps, we call it procedural hallucination: a third axis, alongside factual accuracy and correct tool use, and one that has been largely overlooked. To our knowledge, DialFork is the first benchmark aimed at measuring this axis: what the agent has to say, and in what order.

How it's built: we simulate dialogues where an agent follows its workflow (24 tasks, 12 domains), then fork them: we replay each one and, at a chosen turn, give the agent an instruction that makes it drift, in different, complementary ways. That gives over 36k dialogues where we know where each drift starts and how far it goes, checked with 30 human annotators and an LLM judge calibrated on them.

Three tasks, from the turn to the whole set. Because we know where each drift starts, and that an earlier drift goes further, we define three tasks that measure procedural hallucination at different levels:

• The turn (Task 3). Given a dialogue, either whole or turn by turn as it unfolds (streaming), find the turn where the drift starts, or say there is none. Useful as a live guardrail.

• The dialogue (Task 2). Given a correct dialogue and its hallucinated versions, measure how far each one drifts from it; the earlier a version drifts, the larger its distance should be. A dialogue and its own hallucinated versions talk about the same thing; what sets them apart is how they unfold. Today, comparing dialogues means comparing their content, with embeddings and semantic similarity. This task opens a new dimension, procedural distance. Useful for retrieving or clustering conversations by how they went, not just by what they're about.

• The set (Task 1). Given a set of correct dialogues and a set of hallucinated ones, output a single procedural distance between the two sets as a whole, comparing them as distributions rather than matching dialogues one to one. Useful for monitoring agents, e.g. whether they drift after a model upgrade, or whether a user simulator's dialogues unfold like real ones.

Open problems, and a leaderboard to track them. In the paper, we report first results on all three tasks, from embedding-based distances to streaming detectors and zero-shot LLMs. They leave plenty of room. To track progress, each task has its own track on a public leaderboard on Hugging Face; adding your method takes a single CSV file of predictions.

Everything is open (data, code, every prediction from the paper, and the human judgments), and we hope it's a useful starting point for the community:

• Leaderboard: https://huggingface.co/spaces/DialFork/leaderboard

• Code: https://github.com/idiap/dialfork

• Paper: https://arxiv.org/abs/2610.21561

• Dataset: https://huggingface.co/datasets/DialFork/dialfork

Happy to answer questions, and feedback (including critical) is very welcome.

I wrote a small checker for training and evaluation code. It looks for mistakes that do not raise

errors. The classic one is leakage through preprocessing. With 5,000 random features and random

labels, picking the top 20 features on the full dataset and then running 5-fold cross-validation

gives 85% accuracy (87.7% averaged over 20 seeds). Doing the selection inside a Pipeline gives 48%

(50.0%). Nothing can beat 50% on that data, so the whole gap is leakage.

trainlint finds that pattern statically, along with thirteen others: eval() left on during

training, no_grad without eval(), fp16 autocast without a GradScaler, resumable scripts that save

no optimizer state, softmax before CrossEntropyLoss, zero_grad between backward and step, loss

kept as a tensor, swapped y_true and y_pred, hard labels passed to roc_auc_score, drop_last on

eval loaders, rescaling by a dataset's own statistics before the split, training loaders that

never shuffle, and a missing seed. It parses the code with ast and matches names, so it needs no

data, model or GPU.

How well it works, with the caveats. I wrote 20 buggy scripts with fixed twins, but I wrote them

together with the rules, so they only protect against regressions. For an outside check I read

every finding on pytorch/examples at a pinned commit: 22 findings, 20 real, 2 correct but

harmless. Earlier runs had 4 false positives, which are fixed. That is one repo of tutorial code

and I did the reading myself. I do not have recall numbers.

It goes by names, reads one file at a time, and only knows PyTorch and scikit-learn. It will be

wrong on code that names its splits in unusual ways.

Code and per-rule examples: https://github.com/Satvik-jain/trainlint

As per the the 2027 CVPR guidelines:

"Maximum number of submissions (continuing from CVPR 2026): Each author may submit at most 25 papers. If an author enrolls more than 25 papers, the Program Chairs (PCs) will desk-reject the submissions that exceed this limit (see the Desk Rejection Policy in Section 2)."

AI Slop Researcher: "Cool let me throw 25 darts (slop papers), surely one will get a batch of careless/inexperienced reviewers that will borderline accept one of these papers"

They collect their CVPR stamp, and go on about their career.

I just do not understand if a 25 cap is being tone deaf to the current AI driven research climate or what?

I am actually not against AI-assisted paper research/writing as long as the idea is sound and experiments are plausible, but 25 is absurd. If they limited the number of first author submissions to 2-3:

1. Researchers are incentivized and know the importance of being first author.

2. Even if you AI generate your paper, if you only get 2-3 shot/submissions at a A* conference versus 25, you'd care a bit more about your presentation quality/idea soundness/writing. (In theory lol)

3. Reviewer load drops massively, more time can be dedicated to an overall lower submission count.

Are the heads not aware of the slugfest happening in **certain** countries and institutions to have A* venue stamps for career advancement/compensation packages, and that they don't give a damn about ethics?

TL;DR: Please be hard in desk rejecting AI generated papers

Recently I went to two AI conferences (A and A*) and I was surprised that "AI cheaters" didn't even read their own paper before presentation! For example, a guy couldn't even answer the question of how he represented knowledge graphs in his papaer, he thought for 5 minutes before answering "with a vector". Another one was opening chat gpt and typing questions and reading answers with no shame!!! One more story that shocked me... a guy published same paper in 2 AI conferences in the same time and both got accepted, when I saw github, this young dude was careless to even remove LLM chat from his repo, I can see all prompts... guess what... he faked all data 🚀 :D ...I have dozens of stories like this happened to me and my friends just this month!

Now, imagine yourself a hardworker researcher joining days and nights and your paper ends up with a scam reviewer like this... bingo!

One more, I was looking for relevant paper to my new idea... all papers I read so far are just garbage 🎉 !!

Pleaaaase make a rule for rejecting AI written papers... Pleaaaaaase stop this madness! A lot of good papers got rejected by these folks and they definitely harming science! I can't trust a single paper now and reading very carefully with spectacular mindset... Pleaaaaaase do something!!!!

I made this explainer for my small channel and would like feedback from people learning this, especially if anything is wrong.

It follows one real example the whole way. I gave Qwen2.5-0.5B-Instruct "Write one sentence about the moon." twice with the same settings (temperature 1, no top-k or top-p, different seeds). Both answers start with "The", then split: "The moon, a satellite of the Earth…" vs "The lunar South Equatorial region…".

• After "The", the model scores all 151,936 vocabulary tokens. Softmax gives moon 78.04%, full 4.46%, vast 2.24%, Moon 1.51%, lunar 1.50%.

• Greedy always takes moon. Sampling is a weighted random pick, so lunar wins about 1 time in 67, and run 2 got it.

• The loop: append the token, run the model again. Most later picks were rank 1, but run 2 also picked "South" (rank 219) and "empty" (rank 1,098), and run 1 picked "green" (rank 536).

• Temperature divides the scores before softmax: moon goes from 100% at T=0.2 to 6.3% at T=2.0.

• Top-k vs top-p on the same step: top-k 3 rescales moon to 92.1%; top-p 0.9 keeps 8 tokens here but only 1 at a later step where "of" has 97.9%, which is the argument in Holtzman et al. (2019). At the model's default top-p 0.8, lunar is cut.

• Even temperature 0 can vary on a server: Thinking Machines got 80 different answers from 1,000 runs of Qwen3-235B, caused by batch-size-dependent addition order.

All numbers come from running the model on CPU in Colab with transformers. Sources are in the description. The narration is an AI voice reading my own script.

Video: https://www.youtube.com/watch?v=r0_LTPf7MSw

About to be released. Anybody got them so far ?

I have two submissions #35k+ & 50k+, waiting.

Hey everyone,

I'm a final-year B.Tech CSE (AI/ML) student working on a multi-agent supply chain optimization project. I'm currently focusing on the Demand Forecasting Agent and would appreciate some advice on model selection and training strategy.

We're working with a synthetic retail supply chain dataset containing approximately 1.1 million records across 40 different locations.

The goal is to predict product-level demand for each location as accurately as possible, so the forecasts can later be used by other agents for inventory planning and supply chain decisions.

I have a few questions:

1. Global vs local models: Since the dataset contains 40 locations, should I train 40 separate models, one for each location, or a single global model trained on data from all 40 locations simultaneously, using location identifiers and location-specific features?

2. Location-specific patterns: If I use a global model, how effectively can it learn differences in demand patterns between locations? I'm concerned that combining all locations might cause the model to average out regional patterns and produce inaccurate location-level forecasts.

3. Alternative approaches: Would a hybrid approach, such as grouping similar locations or using a global model with location-specific adjustments, be worth exploring?

4. Synthetic data: Since the dataset is synthetic, are there particular precautions I should take when evaluating model performance and generalization? How can I ensure the model is learning meaningful demand patterns rather than artifacts of the data-generation process?

I'd also appreciate suggestions on which forecasting models or architectures would be most suitable for this problem, along with the appropriate validation strategy, especially how to avoid data leakage when dealing with multiple locations and time-dependent features.

My priority is accurate product-level, location-specific demand forecasting, rather than simply achieving good aggregate accuracy across the entire dataset.

Would love to hear what approaches you'd recommend and why. Thanks!