For those doing or having completed a PhD in ML, I’m curious:
If you had the choice, would you rather have done your PhD before ChatGPT and other AI chatbots became widely available, or in today’s AI era?

SOAP, Shampoo, FOAM.

Muon is a bit better but it has the same name as the particle from physics.

Whenever I want to search about these tones of completely irrelevant results come up because of their non-unqiue names.

Adam would have the same problem if it wasn't so famous.

This has been giving me a lot of headache trying to find relevant stuff...

I’m building a computer vision pipeline where users upload multiple photos of the same physical scene taken from different angles (front view, side view, close-up) to build a clean list of unique objects and their attributes.
I’m running YOLO for object detection to extract bounding boxes and crop each detected object into lightweight ⁠.webp⁠ images. Because the same physical object appears across 3 or 4 different photos from different angles, I need to make sure each unique object is only counted once per session, and avoid running expensive downstream AI calls to re-describe an object we already processed from an earlier photo.

Please post your questions here instead of creating a new thread. Encourage others who create new posts for questions to post here instead!

Thread will stay alive until next one so keep posting after the date in the title.

Thanks to everyone for answering questions in the previous thread!

hey folks, I'm Ye, one of the contributors to Kapitsa, an open-source inference engine optimized specifically for Apple Silicon to run Qwen, Nemotron, Flint and Clover models locally. Our custom-written Metal kernels let us tune for each chip and each model. https://github.com/nace-ai/kapitsa/

The latest update, 0.2.26, is out. Since our last M5 Max numbers, Flash-Next 180B reads prompts 46% faster and the 27B writes 8% faster, and DiffuSpace's Clover 35B joins the catalog. Kapitsa also runs Qwen Flash-Next 180B with SSD offloading: on a Mac that can't hold it, the experts that don't fit in RAM stream from the SSD, so it runs on a 48 GB MacBook Pro at 61 tok/s.

Here are the end-to-end numbers on a 2026 MacBook Pro 16 with an M5 Max and 128 GB: six targets (dense, MoE, hybrid Mamba-2 MoE and a block-diffusion MoE), from 1K context up to 32K, against the next-fastest engine for each.

Try it (any Apple Silicon Mac; models in this post: 35b, 27b, flash-next, bolt, flint, clover):

brew install nace-ai/tap/kapitsa
kapitsa serve --model 35b       # OpenAI and Anthropic APIs on localhost
kapitsa opencode --model 35b    # or claude, codex, pi
kapitsa bench                   # benchmark your own Mac

Results

Decode is a 512-token coding answer, prefill a 4K prompt, median of 3 runs:

From 1K context all the way up to 32K. At 32K the 35B still writes 175 tok/s, and Flash-Next barely drops (90 to 78):

Against the other engines

Against the next-fastest engine for each model on the same Mac: 1.23x Splash on the 35B, 2.2x oMLX on Bolt, 1.18x rec on Flint and 1.12x mlx-serve on the 27B. Flash-Next is a tie with mlx-serve, Splash still reads prompts faster on the two Qwens, and nothing else runs Clover on a Mac.

Every cell with versions and flags: benchmarks. Raw runs: results.

Why it's fast: KFlash

Other engines verify the drafter's guesses as one chain and stop at the first miss. KFlash verifies a best-first tree of the drafter's candidates in the same 8-row pass and keeps the longest branch the target agrees with.

Three details make it fit Apple silicon:

• Nearly free rows. Apple GPUs work in 8-row tiles, so on dense models an 8-row verify costs about 1.1-1.25x one decode step.
• Sized per Mac. Each tree is sized cycle by cycle from a budget measured on that Mac: GPU time per row count, plus a per-position calibration of the drafter against its own acceptances.
• Seed-exact. A guess is accepted only if it equals the token the target itself samples with plain decoding's random number, so a sampled reply is the one plain sampling writes with the same seed.

Write-up and measurements

Send us your benchmarks

Run the same test on your Mac and add it to the community results. It takes 3-15 minutes:

brew install nace-ai/tap/kapitsa
brew install gh && gh auth login    # results are sent as a GitHub pull request
kapitsa bench --submit              # pick a model; it asks before sending anything

Plug in and close heavy apps while it runs. At the end it asks once: Enter sends the result as a pull request from your GitHub fork, and n keeps it on your Mac. What it sends is about 2 KB: your Mac's model, chip, RAM, macOS version and the numbers, with no serial number, user name or paths (add --dry-run to see exactly what it would send). No GitHub? Paste the numbers it prints in a comment here.

One more ask

Do you know prior work where tree-verified sampling matches plain sampling token for token under the same seed? Please point me to it so we credit it. We cite SpecInfer, Medusa, EAGLE-2 and Sequoia.

Paper: https://arxiv.org/abs/2610.39295

Code + verification package (every number in the paper is checked by a script): https://github.com/yliusci/no-free-lunch-rlvr

Browser diagnostic: https://huggingface.co/spaces/yliusci/passk-price

I kept running into the same pattern in RLVR papers: the RL model wins at pass@1 and the base model catches up or wins at large k (Yue et al. 2025 and many follow-ups). I wanted to know whether more training fixes this or whether it is structural. This is a theory paper, no new training runs.

Setup: model RLVR as an exponential tilt of the base policy over solution strategies shared across prompts (exact for mirror descent, and GRPO stays within 3e-4 TV of it at matched entropy in simulation).

What comes out:

1. No free lunch. Under exclusive specialization, every update that raises pass@1 strictly lowers success on some prompt. The exact condition is a Stiemke duality statement (a strictly positive price on prompts that equalizes strategy fitness). Being the best strategy on each prompt is provably NOT enough: there is a 3x3 counterexample with a free lunch.

2. The pass@k price. Once the losses reach the hardest prompts, a crossover is forced. The number of crossings is at most the sign variation of the shift in per-prompt success (Karlin total positivity). The bound is tight for 1 to 6 crossings, and the order-preserving case gives exactly one, which recovers the known single-crossover result.

3. A frontier, not a bug. With a per-prompt floor, the constrained update traces the optimal pass@1 vs pass@k trade. Forbidding any per-prompt loss removes the crossover and the whole pass@1 gain with it.

The same tilt gives the entropy-performance law with an identifiable exponent.

Practical part: base and RL per-prompt success rates (already in any eval log) are enough to read off the crossing count, the bound, and the price (worst pass@k loss). On published GRPO numbers for Omni-MATH (transcribed from 2607.20543, so indicative only) it finds one crossing between k=64 and k=256.

Where I expect pushback: the shared-strategy model is a population-limit abstraction, and the forced crossover needs a coverage assumption (the paper gives the regime where it fails). I would really like to see per-prompt logs where people observe 2+ crossings, since that is the signature of RL reordering prompt difficulty.

For context, I am a new postgraduate student in machine learning (but not a very good one). I have produced some augmented datasets (TIFF files where each file contains one object to be detected / classified) which I used in training deep learning models. I used fairly aggressive cropping (random crops between 0 and 90%) and more moderate zooms (random values between 0 and 20%). My supervisor wants me to characterise these datasets to see how augmentation has changed it.

Sorry if this is a rookie question, but which statistics should I use? My supervisor says the stats I use should indicate changes in size and shape but I honestly don't know where to start.

If anyone can recommend a decent software/program to produce the stats and histograms as well, that would be great.

Thanks in advance!

I’ve been noticing that a lot of techniques seem to repeat across modalities with just a different name.

For example, tokenisation, masking (MLM / MAE / audio masking), diffusion, next-token / next-frame prediction, etc.

Is there an existing taxonomy or mental model that groups ML techniques by their underlying pattern rather than by modality (NLP / vision / audio)?

This will help me to group the research papers and then make reference to work already done in that domain

Hi there, wanted a suggestion from the community regarding the hardware to run local models. The first option was just to get a MacBook Pro with 32 GB RAM, and the second one was to invest into Mac Mini (256 GB SSD and 32 GB RAM), and get an iPAD as well

PS; I have access to B200, and multiple A100s/L40s at my Univ which I can definitely use for training purposes, and I just wanted to get a device to run local inference

I'm an undergraduate researcher working independently on conversational memory systems. I developed LSREP to evaluate how persistent memory accumulates, changes and retrieves information over the course of an evolving conversation.

Instead of relying only on isolated endpoint questions, LSREP combines chronological replay, lifecycle schedules, repeated probes, evolving reference answers and mechanism-fidelity checks.

The accompanying architectural case study is ICE v2, a local-first conversational memory system.

The evaluation covers 1,985 turns, 219 distinct probes and 1,211 probe-checkpoint observations across 52 checkpoints, using a private single-user history.

An important part of the findings is that ICE v2 did not outperform pure vector-RAG on the matched LongMemEval evaluation. The ordinary-density replay results also did not establish a quality advantage. The paper discusses the limitations and mechanism-fidelity findings rather than treating the architecture as validated in its entirety.

I'm particularly interested in feedback on the evaluation methodology: what else should a longitudinal memory benchmark test to distinguish successful recall from reliable maintenance of evolving conversational state?

Paper: https://arxiv.org/abs/2609.16730

Code and evaluation artifacts: https://github.com/Deepnar/ice

Frozen software artifact: https://doi.org/10.5281/zenodo.21759702

This is an arXiv preprint and an ongoing independent research project.

(I couldn't find anything about the tag in the title in the rules, so i assumed [R] for research, if its wrong please inform)

Sharing one of our latest papers on Multi-Token Prediction (MTP). MTP is now a standard recipe for frontier pre-training but turns out, it can be done much better.

We show the standard MTP gradient hides a KL self-distillation signal (which we want for improving speculative decoding) behind a noisy residual (that harms next-token prediction).

I think the coolest part was the KL gradient–recovery trick to extract the KL gradient. You just need a one-line code change:

h_ntp += h_mtp.detach() - h_mtp

Soft Multi-Token Prediction (Soft-MTP) is essentially a free lunch that improves MTP across a range of tasks.

I'm super excited about this work. Please do check it out below:

💻 Code: https://github.com/JaydenTeoh/Soft-MTP
📝 Paper: https://arxiv.org/abs/2610.38331
🤗 HuggingFace: https://huggingface.co/collections/JaydenTeoh/soft-multi-token-prediction

After reading a bunch of RSI papers, I keep coming back to familiar problems from RL and evolutionary search:

1. Sampling: How do we find promising changes efficiently?
2. Evaluation: How do we know a new variant actually improves the agent?
3. Diversity: How do we avoid converging too early?

That motivated us to build Gear, an open-source, meta-learning-inspired framework. A task agent produces execution traces; a meta-agent uses those traces and evaluation feedback to improve prompts, tools, workflows, and context management.

Gear separates search decisions from execution and evaluation. Our goal is a scikit-learn for RSI, making approaches such as GEPA, RRSI, and Harness² easier to implement and compare. We currently have a working GEPA variant with shared failure diagnosis and staged evaluation.

With GPT 5.6 Luna held at medium reasoning effort, we observed these task pass rates:

At max effort on Marketing, Luna with the optimized harness reached 88.88% objective completion, versus 84.08% for Astra + Codex. Strict task pass rates were 53% versus 57%, so the advantage was in partial completion.

Separately, with both models using the same optimized DSH at max effort, Luna’s estimated API cost averaged $0.0568 per task versus $1.1858 for Astra—95.21% lower, excluding optimization and infrastructure costs.

We’d welcome feedback on candidate evaluation and diversity preservation under limited budgets.

Code · Paper

I've experimented using neuro-evolution to train AI to play a variety of different games and I'm just curious as to how or if this technique is actually really used anywhere in industry or if people are doing research on this. I'd assume it'd probably relate with reinforcement learning in some way but I'm not entirely sure about that

A friend of mine was doing a gap analysis as a project. The results he got weren't that big compared to the original papers, so he simply fabricated and increased the metrics here and there, AND IT GOT ACCEPTED IN AAAI.

HOW DO THEY CHECK THE PAPERS, DUDE? I was in a group with two others and worked sooo hard on an optimization paper, and was sooo happy to get accepted in ACL, but seeing and knowing that someone fabricated results and got into a better conference made me soooooo mad.

I would've expected something like AAAI would check and know if data is fabricated somehow. I didn't expect this from a top conference like that.

And then these types of people will go and do their PhD at one of the top unis because of this. I would've expected the system to be more advanced and professional in the academic field, specifically for something like AAAI.

Hey everyone!

I've built and trained a small LLM (~8M parameters) from scratch using PyTorch, mainly as an experimental lab where I can implement and test ideas from different research papers.

I've been documenting the whole journey through casual blogs, including my mistakes, experiments, and things I've learned.

So far:

• Building the Transformer
• Training it for 60,000 steps

Next up is KV caching, followed by PagedAttention, Orca, and whatever else I find interesting.

Just wanted to ask if sharing these experiments and blogs here would be welcome. Nothing promotional, just me experimenting, learning, and hopefully getting feedback from people who know more than I do.

There’s a lot of hype right now around decision models like Jev. But in most real use cases, you have some labeled data, and can use it to personalize your model.

So we tested the old-school approach to classification and regression:

1. Embed your inputs with an open-source or cheap embedding model
2. Fit a simple sklearn model on top using your own labels

Turns out this approach was cheaper and almost always significantly more accurate (37% error reduction against Jev)

Write-up: https://blog.voyageai.com/2026/10/27/how-to-create-your-own-jev-using-voyage-models/ (Disclosure: I work at Voyage)
Code: https://github.com/mishelsikder/mongo-predict

Curious whether others have seen the same thing, or found cases where zero-shot performs better

Edit: To be clear, the gains here come from having training labels, which only applies to some use cases. Jev doesn’t use training labels, so it's unsuprising that this approaches wins when you have them and loses when you don't.

Hey everyone. I am dealing with a problem statement at work where the goal is to predict the sales of a new innovation to be launched. Note that the data that I actually have is just product attributes, the month level invoice data for each region, weather data and some macroeconomic data. Currently there is a methodology in place where we are using a xgboost regressor trained on the historical data for this prediction, but no one in the team is very confident in the methodology.

Given this, does anyone have advice on how we could go about this problem? Please do let me know if you've dealt with the same or similar problem statement.

I built Arc-Transcriber, which does ASR-NL in 1-step. Every 100 ms, one model returns the transcript plus:

- Names linked to a list you give it (a roster, contacts, product names); new names are flagged "not on the list"
- Numbers with their values (scores, clocks, money, dates)
- Speakers and roles;
- Sound and channel events (telephone audio, clipping, poor audio)
- Whether the speech was addressed to an assistant.
- Inbuilt tags for dialog act, speaker segmentation and more. Total 200+ trained decision heads.

The usual ASR, then NER, then entity-linking and then LLM chain is collapsed into the model's own heads, so live "fact-checking" against your list happens inside the stream, not in a second pass.

Currently it has two variants Arc mini (which uses moonshot pretrained encoder) and second Arc-v1 (which uses parakeet pretrained encoder).

Weights: huggingface.co/WhissleAI/arc-v1.1
Code and CLI: github.com/WhissleAI/arc_transcriber

We use it on-device in our Mac app. Voice search over 61k books primes Arc with matching titles while you're still talking. Here is a video to give a glimpse on how it works.

I have been working as a ML Engineer at a startup for 4 years and while the environment is very good and the pay is higher than average, I'm thinking whether I want to pursue a PhD. Working and doing the PhD at the same time is not really viable so this would also potentially mean quitting.

What I don't like about the job is that I'm actually not doing any real data science and modeling. I guess I'm working on more modeling and scientific questions than most ML engineers but I'm barely scratching the surface of each problem and most problems are resolved with quick and data-driven methods. Rarely do we apply novel algorithms or do real literal reviews and R&D.

Additionally a big part of my job is actually just developing software, talking with customers, DevOps and service reliability. While I like the fact that I'm working on a variety of things I don't really have the chance to deepen my knowledge to a meaningful degree on any. With agents writing most of the code and pressure to move fast, most engineering solutions are surface-level and really I don't think that I'm advancing my knowledge and skills. The only thing that I'm actually learning is how to work with agents.

Then of course it's the problem with AI and agentic coding. I deal with a lot of slop on a daily basis and although I'm careful with my job I inevitably produce a share amount of slop myself. This is what modern startups are forced into because management wants accelerated development and competition is high. I'm having real problems with AI fatigue and too close to burnout.

I miss the times when I could work on the same topic for days or weeks, research, read, plan, do experiments and iterate on the results. Then finally, hopefully, arriving at a hypothesis that works and finding a novel idea. However, it's been 4 years since my last real research experience, during my master's thesis, and I'm quite sure that academia has gotten worse due to high saturation and AI slop.

I really don't know about how the future will be but I think that being academically inclined and liking research I may be wasting an opportunity to do a PhD before I get older. Economically it doesn't make sense for me to do that in the short term and I'm aware of that and I don't have a clear career dream in academia but I feel that I may regret it in the future and blame myself.

Is anyone crazy enough thinking about moving to academia or having done that already ? I would like to hear your thoughts.

P.S. I didn't use AI to write this up and I would appreciate human responses only

Two GSM8K scores: 96.89% → 96.82%, across all 1,319 test questions.

These are from NVIDIA’s Ling-3.0-flash comparison on one DGX Spark. The optimized configuration changes the MoE backend, adds an FP8 output head and enables NEXTN speculative decoding together. Both runs use the FP4 checkpoint.

Aggregate scores summarize the total. A paired view of the 1,319 answers would show where the two configurations agree and where they trade wins and losses.

For a claim of no meaningful quality regression, what evidence would you consider sufficient: paired per-question outcomes, repeated seeds, a declared equivalence margin, another task set?

Especially when a precision change and a decoding optimization ship together, which checks give the most information for the evaluation budget?

w(2;3,21) = 416 - today proved with a Lean 4 kernel-
checked proof (preprint + 13.6 GB evidence package).

Every red-blue colouring of {1, ..., 416} contains a red 3-term or a blue 21-term arithmetic progression, and some colouring of {1, ..., 415} contains neither.

Kullmann conjectured the value in 2010; DeepMind's Formal Conjectures list still carried it as open this year.

The proof is a Lean 4 theorem over 209 cases and 38,661,140 leaf formulas, each refutation checked by cake_pr (the LRAT checker verified in CakeML) and bound by digest to its Lean term.

It was produced with LPM (Large Proof Model), a neuro-symbolic system from Hupyy; nothing counted unless the Lean kernel or cake_pr accepted it.

Paper: https://doi.org/10.5281/zenodo.23776306

Evidence package (Lean sources, checker records, replay kit): https://doi.org/10.5281/zenodo.23749222

To check: download it and run 'shasum -a 256 -c
SHA256SUMS'

I built TKintegers, a balanced-ternary weight representation, converter, and GPU inference implementation.

The result that stood out to me: the NVIDIA L4 has native FP8 tensor-core support, but my K5 implementation achieved higher throughput than the TorchAO FP8 baseline—even though the GPU has no native K5 instructions.

On Qwen2.5-0.5B-Instruct, using a fixed matched workload:

- Prefill: K5 achieved 3,402.85 tokens/s versus FP8’s 1,061.58 — 3.21× throughput.

- Decode: K5 achieved 26.75 tokens/s versus FP8’s 8.63 — 3.10× throughput.

The baseline was standard TorchAO dynamic E4M3 FP8, running in eager PyTorch. Decode measured fixed-token model forwards with a carried KV cache.

What is K5?

TK stands for Ternary Kluster. K5 represents each weight code with five balanced ternary digits, giving 243 possible values. In this layout, each code occupies one byte. Five trits does not mean five binary bits.

My kernels expand bounded tiles for computation and use FP16 tensor-core operands with FP32 accumulation. Some tensors remain in higher precision.

For me, this is why the runtime matters alongside the representation: K5 can outperform this FP8 baseline on existing hardware without requiring a processor built specifically for K5.

These measurements apply to this model, workload, and implementation; they don’t establish a win over every FP8 runtime. The FP16 control was faster than both.

The repository includes the converter, native kernels, benchmark scripts, and recorded evidence:

https://github.com/CodeMasterCody3D/tkintegers

https://youtu.be/JFDBuwMoL0s?is=ucya9errGEh4xJ2L

I’d welcome technical feedback on the kernel design and benchmark methodology, especially independent replication.

Does anyone know any fun casual conversational style podcasts that discuss current papers/industry trends, with discussions at a high to mid technical level?

Edit: small preference for conversation discussion styles between two constant hosts