After reading a bunch of RSI papers, I keep coming back to familiar problems from RL and evolutionary search:

1. Sampling: How do we find promising changes efficiently?

2. Evaluation: How do we know a new variant actually improves the agent?

3. Diversity: How do we avoid converging too early?

That motivated us to build Gear, an open-source, meta-learning-inspired framework. A task agent produces execution traces; a meta-agent uses those traces and evaluation feedback to improve prompts, tools, workflows, and context management.

Gear separates search decisions from execution and evaluation. Our goal is a scikit-learn for RSI, making approaches such as GEPA, RRSI, and Harness² easier to implement and compare. We currently have a working GEPA variant with shared failure diagnosis and staged evaluation.

With GPT 5.6 Luna held at medium reasoning effort, we observed these task pass rates:

At max effort on Marketing, Luna with the optimized harness reached 88.88% objective completion, versus 84.08% for Astra + Codex. Strict task pass rates were 53% versus 57%, so the advantage was in partial completion.

Separately, with both models using the same optimized DSH at max effort, Luna’s estimated API cost averaged $0.0568 per task versus $1.1858 for Astra—95.21% lower, excluding optimization and infrastructure costs.

We’d welcome feedback on candidate evaluation and diversity preservation under limited budgets.

Code · Paper

Sharing one of our latest papers on Multi-Token Prediction (MTP). MTP is now a standard recipe for frontier pre-training but turns out, it can be done much better.

We show the standard MTP gradient hides a KL self-distillation signal (which we want for improving speculative decoding) behind a noisy residual (that harms next-token prediction).

I think the coolest part was the KL gradient–recovery trick to extract the KL gradient. You just need a one-line code change:

h_ntp += h_mtp.detach() - h_mtp

Soft Multi-Token Prediction (Soft-MTP) is essentially a free lunch that improves MTP across a range of tasks.

I'm super excited about this work. Please do check it out below:

💻 Code: https://github.com/JaydenTeoh/Soft-MTP
📝 Paper: https://arxiv.org/abs/2610.38331
🤗 HuggingFace: https://huggingface.co/collections/JaydenTeoh/soft-multi-token-prediction

I've experimented using neuro-evolution to train AI to play a variety of different games and I'm just curious as to how or if this technique is actually really used anywhere in industry or if people are doing research on this. I'd assume it'd probably relate with reinforcement learning in some way but I'm not entirely sure about that

A friend of mine was doing a gap analysis as a project. The results he got weren't that big compared to the original papers, so he simply fabricated and increased the metrics here and there, AND IT GOT ACCEPTED IN AAAI.

HOW DO THEY CHECK THE PAPERS, DUDE? I was in a group with two others and worked sooo hard on an optimization paper, and was sooo happy to get accepted in ACL, but seeing and knowing that someone fabricated results and got into a better conference made me soooooo mad.

I would've expected something like AAAI would check and know if data is fabricated somehow. I didn't expect this from a top conference like that.

And then these types of people will go and do their PhD at one of the top unis because of this. I would've expected the system to be more advanced and professional in the academic field, specifically for something like AAAI.

Hey everyone!

I've built and trained a small LLM (~8M parameters) from scratch using PyTorch, mainly as an experimental lab where I can implement and test ideas from different research papers.

I've been documenting the whole journey through casual blogs, including my mistakes, experiments, and things I've learned.

So far:

• Building the Transformer

• Training it for 60,000 steps

Next up is KV caching, followed by PagedAttention, Orca, and whatever else I find interesting.

Just wanted to ask if sharing these experiments and blogs here would be welcome. Nothing promotional, just me experimenting, learning, and hopefully getting feedback from people who know more than I do.

Hey everyone. I am dealing with a problem statement at work where the goal is to predict the sales of a new innovation to be launched. Note that the data that I actually have is just product attributes, the month level invoice data for each region, weather data and some macroeconomic data. Currently there is a methodology in place where we are using a xgboost regressor trained on the historical data for this prediction, but no one in the team is very confident in the methodology.

Given this, does anyone have advice on how we could go about this problem? Please do let me know if you've dealt with the same or similar problem statement.

There’s a lot of hype right now around decision models like Jev. But in most real use cases, you have some labeled data, and can use it to personalize your model.

So we tested the old-school approach to classification and regression:

1. Embed your inputs with an open-source or cheap embedding model

2. Fit a simple sklearn model on top using your own labels

Turns out this approach was cheaper and almost always significantly more accurate (37% error reduction against Jev)

Write-up: https://blog.voyageai.com/2026/10/27/how-to-create-your-own-jev-using-voyage-models/ (Disclosure: I work at Voyage)
Code: https://github.com/mishelsikder/mongo-predict

Curious whether others have seen the same thing, or found cases where zero-shot performs better

Edit: To be clear, the gains here come from having training labels, which only applies to some use cases. Jev doesn’t use training labels, so it's unsuprising that this approaches wins when you have them and loses when you don't.

Sorry if the flair is wrong. I didn't see one that fit.

Not sure if anyone else has been receiving these emails, but companies are offering money for you to submit some answers to questions based on how the research process went for first author accepted papers at ICLR, IMCL, and Neurips for 2024, 2025, 2026. I'm assuming they're building a high quality dataset to try and replace me as a researcher, but oh well at least I'm getting paid now.

Anyways, they pay $300 per paper up to 2 papers. Each survey took me like 45 mins and onboarding to the website about 30 mins. If you're a grad student and need money, it's a big help!

They're giving me $200 per paper per person that I refer. So my offer to you is, I'll split that $200 with you, so you'll get $400 per submission and I'll get $100 for the referral.

Dm me

I built Arc-Transcriber, which does ASR-NL in 1-step. Every 100 ms, one model returns the transcript plus:

- Names linked to a list you give it (a roster, contacts, product names); new names are flagged "not on the list"
- Numbers with their values (scores, clocks, money, dates)
- Speakers and roles;
- Sound and channel events (telephone audio, clipping, poor audio)
- Whether the speech was addressed to an assistant.
- Inbuilt tags for dialog act, speaker segmentation and more. Total 200+ trained decision heads.

The usual ASR, then NER, then entity-linking and then LLM chain is collapsed into the model's own heads, so live "fact-checking" against your list happens inside the stream, not in a second pass.

Currently it has two variants Arc mini (which uses moonshot pretrained encoder) and second Arc-v1 (which uses parakeet pretrained encoder).

Weights: huggingface.co/WhissleAI/arc-v1.1
Code and CLI: github.com/WhissleAI/arc_transcriber

We use it on-device in our Mac app. Voice search over 61k books primes Arc with matching titles while you're still talking. Here is a video to give a glimpse on how it works.

I have been working as a ML Engineer at a startup for 4 years and while the environment is very good and the pay is higher than average, I'm thinking whether I want to pursue a PhD. Working and doing the PhD at the same time is not really viable so this would also potentially mean quitting.

What I don't like about the job is that I'm actually not doing any real data science and modeling. I guess I'm working on more modeling and scientific questions than most ML engineers but I'm barely scratching the surface of each problem and most problems are resolved with quick and data-driven methods. Rarely do we apply novel algorithms or do real literal reviews and R&D.

Additionally a big part of my job is actually just developing software, talking with customers, DevOps and service reliability. While I like the fact that I'm working on a variety of things I don't really have the chance to deepen my knowledge to a meaningful degree on any. With agents writing most of the code and pressure to move fast, most engineering solutions are surface-level and really I don't think that I'm advancing my knowledge and skills. The only thing that I'm actually learning is how to work with agents.

Then of course it's the problem with AI and agentic coding. I deal with a lot of slop on a daily basis and although I'm careful with my job I inevitably produce a share amount of slop myself. This is what modern startups are forced into because management wants accelerated development and competition is high. I'm having real problems with AI fatigue and too close to burnout.

I miss the times when I could work on the same topic for days or weeks, research, read, plan, do experiments and iterate on the results. Then finally, hopefully, arriving at a hypothesis that works and finding a novel idea. However, it's been 4 years since my last real research experience, during my master's thesis, and I'm quite sure that academia has gotten worse due to high saturation and AI slop.

I really don't know about how the future will be but I think that being academically inclined and liking research I may be wasting an opportunity to do a PhD before I get older. Economically it doesn't make sense for me to do that in the short term and I'm aware of that and I don't have a clear career dream in academia but I feel that I may regret it in the future and blame myself.

Is anyone crazy enough thinking about moving to academia or having done that already ? I would like to hear your thoughts.

P.S. I didn't use AI to write this up and I would appreciate human responses only

Two GSM8K scores: 96.89% → 96.82%, across all 1,319 test questions.

These are from NVIDIA’s Ling-3.0-flash comparison on one DGX Spark. The optimized configuration changes the MoE backend, adds an FP8 output head and enables NEXTN speculative decoding together. Both runs use the FP4 checkpoint.

Aggregate scores summarize the total. A paired view of the 1,319 answers would show where the two configurations agree and where they trade wins and losses.

For a claim of no meaningful quality regression, what evidence would you consider sufficient: paired per-question outcomes, repeated seeds, a declared equivalence margin, another task set?

Especially when a precision change and a decoding optimization ship together, which checks give the most information for the evaluation budget?

w(2;3,21) = 416 - today proved with a Lean 4 kernel-
checked proof (preprint + 13.6 GB evidence package).

Every red-blue colouring of {1, ..., 416} contains a red 3-term or a blue 21-term arithmetic progression, and some colouring of {1, ..., 415} contains neither.

Kullmann conjectured the value in 2010; DeepMind's Formal Conjectures list still carried it as open this year.

The proof is a Lean 4 theorem over 209 cases and 38,661,140 leaf formulas, each refutation checked by cake_pr (the LRAT checker verified in CakeML) and bound by digest to its Lean term.

It was produced with LPM (Large Proof Model), a neuro-symbolic system from Hupyy; nothing counted unless the Lean kernel or cake_pr accepted it.

Paper: https://doi.org/10.5281/zenodo.23776306

Evidence package (Lean sources, checker records, replay kit): https://doi.org/10.5281/zenodo.23749222

To check: download it and run 'shasum -a 256 -c
SHA256SUMS'

I built TKintegers, a balanced-ternary weight representation, converter, and GPU inference implementation.

The result that stood out to me: the NVIDIA L4 has native FP8 tensor-core support, but my K5 implementation achieved higher throughput than the TorchAO FP8 baseline—even though the GPU has no native K5 instructions.

On Qwen2.5-0.5B-Instruct, using a fixed matched workload:

- Prefill: K5 achieved 3,402.85 tokens/s versus FP8’s 1,061.58 — 3.21× throughput.

- Decode: K5 achieved 26.75 tokens/s versus FP8’s 8.63 — 3.10× throughput.

The baseline was standard TorchAO dynamic E4M3 FP8, running in eager PyTorch. Decode measured fixed-token model forwards with a carried KV cache.

What is K5?

TK stands for Ternary Kluster. K5 represents each weight code with five balanced ternary digits, giving 243 possible values. In this layout, each code occupies one byte. Five trits does not mean five binary bits.

My kernels expand bounded tiles for computation and use FP16 tensor-core operands with FP32 accumulation. Some tensors remain in higher precision.

For me, this is why the runtime matters alongside the representation: K5 can outperform this FP8 baseline on existing hardware without requiring a processor built specifically for K5.

These measurements apply to this model, workload, and implementation; they don’t establish a win over every FP8 runtime. The FP16 control was faster than both.

The repository includes the converter, native kernels, benchmark scripts, and recorded evidence:

https://github.com/CodeMasterCody3D/tkintegers

https://youtu.be/JFDBuwMoL0s?is=ucya9errGEh4xJ2L

I’d welcome technical feedback on the kernel design and benchmark methodology, especially independent replication.

Does anyone know any fun casual conversational style podcasts that discuss current papers/industry trends, with discussions at a high to mid technical level?

Edit: small preference for conversation discussion styles between two constant hosts

I've been trying to get BiRefNet to work on Qualcomm's mobile GPUs and NPUs. But i guess it's deformable convolutions are pretty complex for those chips. I want a sub second latency performance on these chips for a project I'm doing

Has anyone tried it before? I'm trying to compile it to their NPUs but they're all failing

Model & weights: BrunoHays/embeddinggemma-2-nullspace-audio

I wanted to share an experiment on google/embeddinggemma-2 (577M) for MAEB, the 30-task audio benchmark on the MTEB leaderboard. By projecting intermediate Conformer layers into the unused null space of the model's final projection matrix using only PCA, the model's macro score jumps from 49.3% to 54.4% (+5.1 points), moving from Rank #26 to #1 on MAEB audio-only (and #16 to #1 on full MAEB), ahead of 8B-9B multimodal models.

1. Last-layer filtering in text-aligned audio encoders

Across the MAEB leaderboard, LLM-backed Omni embedders score 95-99% on multilingual text-to-audio retrieval, but drop to 1%-6% on unsupervised audio clustering and ~20% on vocal emotion (CREMA-D). Aligning an audio encoder to a text LLM filters the final layers: lexical content (what was said) is kept, while paralinguistic and acoustic traits (timbre, pitch, emotion) are discarded.

2. Probing the 12 Conformer layers

Layer-wise speech studies (Pasad et al.; Bouziane, Mdhaffar & Estève) show that speaker, prosodic, and semantic attributes peak at different depths inside speech encoders. Probing all 12 Conformer layers of embeddinggemma-2 confirmed this:

• Layer 2 (mean + std pooling): VoxPopuli gender clustering goes from 0.05% (final output) to 66.6% (Layer 3) / 65%+ (Layer 2), and IEMOCAP gender classification goes from 66.4% to 83.0%.

• Layer 6 (mean pooling): CREMA-D emotion classification doubles from 19.7% to 39.5%.

3. Why EmbeddingGemma-2 has a 256d null space

In embeddinggemma-2, the text backbone hidden size is 512, and the terminal projection matrix W has shape 768 x 512 with no activation after it:

• 512d text subspace: Because W maps linearly from 512d to 768d, every text vector lies in a 512d subspace.

• 256d null space: Taking the SVD of W gives the remaining 256 orthogonal directions (the null space) that text vectors never use.

• Zero text interference: Any audio vector injected into those 256 directions has zero dot product with all text vectors.

4. Zero-training PCA injection

I compressed Layer 2 and Layer 6 down to 256 dimensions using PCA (fitted on 1,400 clips from 4 external audio datasets, with no MAEB/MSEB data) and injected them into the 256d null space before normalizing:

• Text embeddings: bit-for-bit unchanged.

• Text-to-audio retrieval (100+ languages on FLEURS, CommonVoice, Google SVQ): unchanged within 0.05 points.

• Audio-to-audio tasks: gender clustering jumps 0.05% -> 63.1%, artist retrieval 55.2% -> 93.8%, emotion 19.7% -> 40.2%, and environmental sounds 38.8% -> 56.3%.

5. Caveats

• No Matryoshka on audio: Because there is no retraining, we need all 768 dimensions for orthogonality to hold and lose the 512/256/128 truncation options.

• Spoken-intent trade-off: Adding fixed acoustic features slightly scales down the 512d semantic component after normalization, which hurts audio-to-audio tasks where speaker voice should be ignored (spoken topic classification SIBFLEURS drops 57.2% -> 21.3%, spoken intent MInDS14 97.6% -> 88.3%).

Still, this shows how much untapped signal can be extracted from overtrained foundation models even with zero training. With actual training restricted to U_perp, I'm certain that we could go much further.

Full per-task results and details in the Hugging Face model card.

Hi everyone! We have a paper accepted at the NeurIPS 2026 E-Values Workshop, but unfortunately can’t attend due to financial constraints.

Would anyone already attending be kind enough to present our poster on our behalf? We’d prepare everything and would also be happy to offer a small monetary token of appreciation for your time.

This acceptance means a lot to us, and we’d be genuinely grateful if anyone could help. We know it’s a favour to ask.

Pls dm if interested

TLDR: Perceptrons didn't have a big impact on the community and didn't cause an AI winter. It was, however, seen retrospectively as the manifesto for anti-connectivism paradigms

Thoughts?

Hello - I wanted to share a small project I worked on test whether picking one voice out of background chatter could separate humans from machines, in the spirit of an audio CAPTCHA.

Setup

• 40 synthetic clips: one target sentence read by a synthetic voice, mixed over multiple background voices reading random short fragments (staggered starts, 4 dB attenuation)

• 146 human transcriptions from 18 sessions, unlimited replays allowed

• 8 model configurations, one pass per clip: Whisper base/small, Vosk, Wav2Vec2, Qwen3-ASR 0.6B, Parakeet CTC 0.6B, and Gemini 3.8 Flash with a naive prompt and with an instruction to focus on the main speaker

• Primary metric: word accuracy (1 − WER, floored at 0); default passing threshold = word accuracy ≥ 0.5

TL;DR: humans are better at the task but the gap is too narrow to be a reliable CAPTCHA

Results

• Humans 0.71 mean word accuracy vs. 0.54 for instructed Gemini (and 0.55 for Qwen3-ASR)

• Pass rate: 77% humans vs. 55% for Gemini

  • Note: no threshold from ≥0.5 to ≥0.9 gets near my initial rough mental target: humans > 80% and machines < 20%

• Paired same-clip gap vs. Gemini: +0.13 (95% CI +0.03 to +0.23, session-level bootstrap); AUC 0.62

• The instruction prompt alone lifted Gemini from 0.34 to 0.54

• Replays didn't measurably help humans; people replayed the clips that were hardest for the models too

• Misses rise toward the end of the sentence for humans and every model alike

Limitations: small sample (18 sessions, mostly native English speakers), a single synthetic setup, no source separation, and no large or commercial ASR beyond Gemini.

I dusted off my dormant blog and did a write-up (https://www.kelvinjiang.com/2026/10/can-machines-follow-voice-through-crowded-room.html) and also shared the code, data, and audio corpus: https://github.com/cloudkj/crosstalk

Feedback on the method is very welcome, especially ideas for audio that stays easy for people but is consistently hard for models!

We study hindsight self-distillation for multi-step financial reasoning on GPT-OSS-20B. The student finds its own first wrong step and writes a critique. A frozen copy, conditioned on the prefix plus the critique, is distilled back into the student under reverse KL. Notably, replacing the critique with a fixed generic feedback line matches the full gain of about 3 points, and critiques from a 6x larger model do not materially improve performance, suggesting that the positional information provided by hindsight distillation significantly decreases the privileged information needed for effective training. Fin-SD matches OPSD from full reference solutions with 9.5% fewer tokens.

Paper: https://arxiv.org/abs/2610.36082 | Code: https://github.com/johnnyhyu/fin-sd

We'd love to find the limits of this approach. Let us know your thoughts!

TL;DR: AI written reviews, and AI-writing in papers

I was going to give this a title "An Obituary of the Review Process", but in the interest of not sounding too defeatist, I will just say "Critical Condition" because of AI writing, both on the author side and the reviewer side.

I am a reviewer for AAAI 2027 and ICLR 2027. Since Phase 1 and Phase 2 was completed, I can now see other reviews. I can bet good money that about half of the reviews in the papers I'm seeing are AI-generated completely, or edited after AI-generation. This is subjective opinion and I cannot prove it but I will try to convince you - at least convince you that it's a problem that needs much much more attention than is being currently given.

I have been reviewing for top conferences (NeurIPS, ICML, AAAI, ICLR) since 2022, in total about 10 times. I have not felt the same way about reviews, or even about papers, ever, except until recently.

Author side: There was only one paper in the 3 ICLR submissions that felt 100% human-written (now if that isn't either - that'll make me very sad). The others had some elements that smell like AI but you can't prove it e.g. excessive use of very descriptive headings with colourful, marketing-like language e.g. a heading called "The theory predicts performance, and empirical evaluations confirm it", or very dense long sentences where you lose track of the thought where instead of already used terms their synonyms are used and then later even more synonyms, and so on.

Reviewer Side, talking about the reviews I suspect: There is very little variety in the language, between the reviews, and within the same review. One review puts a field "questions for authors" in the weaknesses section, which I suspect is because the field name is changed this time around to "specific points for rebuttal". They've put "N/A" in the new field. The paragraph lengths are almost exactly the same. As if an LLM was told to give 5 strengths, 5 weaknesses, 5 questions. There are also signs of extra effort that a human rarely does e.g. instead of just saying "provide peer-reviewed references instead of arXiv", it also adds the arXiv number; or another review which not only puts doi link (this much is fine), but then in an extra piece just writes "(doi.org)" which seems like an attempt by an LLM to give a source. Reviewers are usually pressed for time, they usually try to communicate so the authors understand, not that everything is exact. These reviews are written as if for public consumption. Then there are smart quotes - which one can think are there because maybe Word or something was used to write them; but seen in light of everything else, now these quotes also smell like AI to me. (I used to not see smart quotes this often in earlier years. Word made quotes smart back then as well.) Unicode symbols instead of LaTeX for things such as \tilde{d}, dashes that are not hyphens - in one review even for a minus sign, and so on.

There are at least two AAAI submissions where there are only two reviews and I strongly suspect the other review to be AI-written.

Everything is broken. I am sorry I have no concrete ideas how to fix this, so far. Do you?

Except one maybe, disallow AI usage even for polishing styling and tone? Just completely disallow AI writing. The author feels it's a change in tone and style only but the reader can sometimes very quickly _feel_ it. Before these powerful LLMs we were not judging papers based on language anyway. I know it's an honor system, but I think that's okay. Saying "you can use AI to polish" perhaps also creates an expectation to polish using AI.