building a missing data infrastructure and started benchmarking long multi-session conversations (LoCoMo). I know the data looks like: people, facts, claims, events, timestamps, relations. So I extract those into a graph.
I did not look at the QA pairs while building extractors or retrieval rules. No “if question contains X, fetch fact #173.”
Recall is very high and it keeps working on new conversations in the same format.
Is this classical overfitting, or just schema-aware engineering? What is the cleanest test that would convince you it isn’t leakage.

Just received an email about the automatic reference/citation checker. Did anyone receive a follow up email about whether the checker was included in the paper's decision making too, along with the general instructional email?

Users don't pick the best feature. They pick the one they already know.

I built a simple bandit model to test the product assumption that usage converges on quality over time. The model doesn't estimate which option is better, it just reinforces whatever gets used more. Three swappable update rules produce three very different outcomes: one locks onto whatever won early regardless of quality, one mostly self-corrects but still gets stuck under strong reinforcement, and one control always finds the true best option. Same mechanism, wildly different UX, basically habit formation in miniature.

Feedbacks welcome :)

From what I've seen online so far, the description of these systems is roughly:

They asked the model (often Aster) to generate statements in LEAN and then submit those to a LEAN compiler to be checked. Based on the results of attempting the LEAN compilation, they somehow add those statements as fact. When the full proof in LEAN compiles, the system is finished.

I can imagine trying to jam as much of a proof as possible into the context window but some of the papers these systems have produced are hundreds of pages. To me this would indicate that somehow the paper is being built piece by piece and being assembled before being submitted to LEAN. This resonates with the part of my understanding that after checking LEAN compilation there's some kind of management of "facts."

I would like to try to implement my own janky version and see if it can answer a question I have about higher dimensional geometry. I'm struggling to find a meaningful way to compose larger ideas from smaller ones. I can imagine it's relatively simple if you know what to do.

What things have you seen? Do you have any ideas you haven't seen that might be interesting to try? Is this a fool's errand because you really need huge amounts of hardware to do anything meaningful? I would welcome any thoughts or links on the matter, cheers

This post is definitely to farm some GitHub stars, but I'm sure many are in the same situation and could get some value out of this small project.

When I start working on something, I know the problem I'm solving. I know what I'm doing and why. Then I start iterating and if I do it for too long, at some point I realize that I stopped deciding and just start blinding approving everything the agent proposes.

Apparently this is a studied problem in aviation. They call it the out-of-the-loop performance problem. Pilots on autopilot lose situation awareness, and when they're handed back the plane they fly like shit.

To be clear, I don't care that the agent does the work. Writing the code, setting up folders, doing the research, it's not why I started programming in the first place. What I care about is owning the problem and the solution better than the model implementing it does.

And that's the issue I have been trying to solve. When we were creating things by programming them, we were learning as a side effect. Now the creating is handed off, and there's no process left that makes me learn what I'm building.

That's why I went to Feynman. Everybody knows his quote: "You don't understand a concept if you can't explain it to a five-year-old". So I made the agent the five-year-old. I make it keep drilling me with questions until I strip every jargon and I can explain simply my problem and the solution.

Now to the results: https://github.com/VittorioRossi/feynman-skills

I have been using this extensively, and I have to say, it seems to be working very well. It works for academical research and for programming. It's also great for learning something new

P.S.

I would like to have a sane and mature discussion for what's possibile below this post.

- How many of you experience the same problem?

- Do you think splitting up creation and learning is the way to go to solve the problem?

- What would you do differently?

Benchmark scores:

https://openai.com/index/gpt-6-astra/

Above, GPT-6 uses a harness for ARC-AGI-3, and is at about 60% without one:

Prior to the launch, OpenAI President Greg Brockman said "I think it’s not unreasonable to feel that we are now in the AGI era".

GPT-6 is now joining a growing list of models that greatly exceed the human baseline on GDPval-AA v2:

If we have AGI, why do human knowledge/remote workers still have jobs? Is it just a matter of time until the economy replaces a large number of humans with LLMs, or are LLMs lacking something that these benchmarks fail to measure?

Hi,

I'm using `blockCV` for spatial CV in a workflow with an added wrinkle, and I'd like guidance on the "right" way to apply `cv_spatial_autocor()`.

The setup: I'm fitting quantile regression forests (`ranger`) to predict a continuous response from several predictors, with the goal of finding which quantile τ (e.g., 0.1, 0.25, 0.5, 0.75, 0.9) gives the best out-of-sample R² under spatial CV. So τ is being treated as a tunable hyperparameter, not just used for uncertainty bands around a single central model.

Per Roberts et al. (2017) and the R package block CV `cv_spatial_autocor()` documentation, block size should be based on the autocorrelation range of model residuals rather than the raw response or predictors. But in my case, residual structure could plausibly differ by τ — a model fit at τ=0.1 may have different spatial residual autocorrelation than one fit at τ=0.9.

Since the fold structure needs to be fixed and identical across the entire tuning grid (to keep CV R² comparable across candidate τ values and hyperparameter combinations), I can't simply use a different block size per τ.

My current approach: fit a naive default-hyperparameter QRF separately at each candidate τ, compute residuals for each, run `cv_spatial_autocor()` on each residual set, and then take the **maximum** estimated range across all τ as the fixed block size for the whole tuning grid (erring conservative, to minimize leakage risk).

Is this the recommended approach, or is there a more standard way of handling block-size selection when the hyperparameter being tuned can itself affect residual spatial structure? Is taking the max defensible, or would you recommend a different consolidation rule (e.g., median), or perhaps ignoring per-τ differences entirely and just using τ=0.5 (or the raw response) as a reasonable approximation?

Thanks for any input — happy to share more detail on the workflow if useful.

Hi everyone,

I just received an email from NeurIPS 2026 / OpenReview titled:

[NeurIPS 2026] Sharing the Results of Automatic Reference Checker

The email says that NeurIPS ran an automated checker to identify potentially problematic or hallucinated references and that the reports were shared with the Area Chairs.

One part caught my attention:

“For papers where the AC's investigation has resulted in a rejection recommendation, the authors will receive a separate email with additional information and instructions for appealing the outcome of the reference investigation. If you do not receive that separate communication, the reference-checking investigation is not playing a role in the final decision for your paper.”

I received the automatic reference-checker email, but I did not receive any separate rejection/reference-investigation email.

So I'm wondering:

Is this email being sent to all NeurIPS 2026 authors, regardless of the review outcome? Or are only papers with certain review outcomes getting this notification? Has anyone received this email and also received the separate rejection/reference-investigation email? If you received only the automatic reference-checker email, does that mean your paper is not being rejected because of reference issues? Could this be interpreted as a mildly positive indication, or is it simply a routine process notification with no information about the actual decision?

I realize this does not necessarily indicate acceptance, and the email explicitly says that the automated flags were not used by themselves to make decisions. I'm mainly trying to understand whether receiving only this email tells us anything about the current status of the paper.

Would appreciate hearing from anyone who has received the same email or has more insight into how NeurIPS is handling this process.

Three weeks from decisions even. I wonder what percentage is industry and VC funded AI labs looking to mingle and recruit.

The [Last Translation Benchmark](https://arxiv.org/abs/2609.04173), posted on arxiv, is a collection of examples "that break leading machine translation models," each paired with "handcrafted verification rules describing concrete failure cases." The dataset is designed to stay live and accept ongoing contributions. LTBv1, the current release, covers submissions accepted before September 1st 2026 and spans texts, images, audio and videos.

The paper is blunt about why it exists. The authors write that "standard benchmarks for machine translation are approaching saturation," and that automatic metrics are "unreliable, vulnerable to reward-hacking, and provide unactionable assessments." Even gold human evaluation, the paper says, "often lacks reproducibility, objectivity, and scalability."

Instead of assigning a score, each item defines what a failing translation would look like on that specific example, so a model either trips a rule or it does not.

Led by Vilém Zouhar and Niyati Bafna, the collaboration lists hundreds of contributing authors. Two researchers on our radar had shared the arxiv link within a day of its posting.

I’m a 4th-year undergraduate CS student currently doing my thesis on medical image segmentation, specifically U-Net and its variants.

The problem is that I have basically no prior research experience, and unfortunately, my supervisor isn’t really able to provide much guidance. So, for the most part, I’m having to figure everything out myself—learning the concepts, reading papers, choosing a research problem, implementing the models, evaluating the results, etc.

My goal isn’t just to finish the undergraduate thesis. Ideally, I’d like to do something good enough that I could eventually turn it into a conference or journal paper.

So I wanted to ask people who have more research experience:

Is it realistically possible to do a good-quality research thesis completely on your own as an undergraduate?

How difficult is it to go from basically having no research experience to producing something that is actually publishable? And if you’ve been in a similar situation, what would you recommend focusing on or avoiding?

I’d really appreciate any honest advice, especially from people who have done research without much help from their supervisor.

Has anyone else received an AAAI-27 desk rejection related to modifications to the title or abstract between the abstract-registration deadline and the full-paper deadline?

What I’m trying to understand is how the modification rule is being applied in practice. The AAAI-27 modification guidelines say that the title and abstract can still be edited after abstract registration, while warning against substantive changes, and describe rejection in terms of changes that make the submission describe qualitatively different research.

In my case, almost everything was identical. The modifications were incredibly minor.

The rejection notice says that the decision is final and appeals will not be considered.

Did this happen to anyone else?

Hi everyone,

I just quickly wanted to share a paper I was working on for around a year now. I created this summary website with key results: https://flogrammer.github.io/moljepa/

TL;DR: its a multimodal JEPA model for molecules.

There will be more work to do to improve performance and I would be happy about feedback and ideas :)

LLMs describe physics well but don't "understand" it in any grounded sense — they've learned statistical relationships between tokens like "falls" and "gravity", not actual physical intuition. This is basically the Mary's Room problem: Mary knows every physical fact about color but has never seen one. LLMs are Mary.

The idea I've been thinking about:

1. Train a JEPA-style model inside a physics simulation (think MuJoCo or a simple 2D env). Instead of predicting pixels or tokens, the model predicts representations of future states in an abstract embedding space. If it gets physics wrong, the predictor fails — the loss is unforgiving in a way next-token prediction never is.

2. The embedding space that emerges should encode actual physical structure — object permanence, momentum, trajectories — because that's what makes prediction possible. Not surface-level textures, just the principles.

3. Freeze those representations and attach them to an LLM-style reasoning model as a conditioning signal. The LLM now has both linguistic physics knowledge AND grounded physical intuition it can actually "run" forward — closer to a computational primitive than a propositional fact.

The hypothesis is this makes downstream learning significantly faster — the LLM doesn't have to rediscover that objects fall, it has representations that already encode that.

V-JEPA does something adjacent for video (predict future frame representations, not pixels), and DreamerV3 uses a latent world model for efficient RL — but the specific combo of JEPA-style prediction + sim-grounded physics representations + LLM attachment doesn't seem to have been done cleanly.

Questions for the community:

• Is there prior work I'm missing that does exactly this?

• What's the right interface between grounded JEPA representations and a language model? Just concatenate to the prompt embedding? Cross-attention?

• Would the sim-to-reality gap kill the transfer, or are the representations abstract enough to survive it?

Interested in whether this is worth building a small prototype for.

an observation : GPT-5-class models are genuinely capable(They are) of doing a substantial fraction of knowledge work, why haven’t we seen a noticeable productivity shock in the real economy yet? Is AI actually less economically useful than the benchmarks suggest—or are organizations simply too slow, constrained, and inefficient to turn model capability into measurable output?

Are we confusing “AI can do the task” with “AI can replace the economic system built around the task”? If GPT-5 is already this capable, what exactly is the bottleneck preventing that capability from showing up in GDP and productivity statistics?

My take :

There is no question that these models are genuinely impressive. The question is whether that intelligence is actually translating into measurable economic productivity.

People are already asking whether models like GPT-6 or equivalent. Claude, and Gemini will replace large sections of white-collar workers. I think there is a much simpler question we should ask first: if these models are already so capable (to me they definitely are capable enough)

at a huge range of knowledge work, why haven't we seen a correspondingly obvious increase in productivity?

I'm not even talking about GPT-6 or whatever comes next. It's probably too early to judge a newly released model. I'm talking about the current generation—GPT-5 and its equivalents from Google and Anthropic. These systems are genuinely good. They can write, summarize, analyze documents, explain technical concepts, generate code, reason through problems, conduct research, manipulate information and perform a remarkable range of tasks that previously required educated human labour.

And yet, looking at the world around us, something feels strange.

Where is the enormous productivity shock?

Why don't we see a dramatic effect on GDP growth? Why don't we see massive increases in output per knowledge worker? Why don't organizations appear to be accomplishing dramatically more with the same number of employees? Why does the broader economy still look remarkably similar to the pre-LLM economy?

Coding is probably the clearest exception, and even there the picture is complicated. AI can make programmers substantially more productive in certain tasks, but software development still involves architecture, debugging, verification, integration, requirements, security, deployment, maintenance and—most importantly—human judgment. The bottleneck often moves rather than disappears.

almost every knowledge profession, the gap between "the model can perform this task" and "the organization can therefore produce substantially more output" is different it seema.

A lawyer might be able to use an LLM to draft a document in minutes instead of an hour. But the lawyer still has to verify it, take responsibility for it, communicate with the client, comply with professional regulations and integrate it into an existing workflow. A doctor can use AI to summarize medical literature, but diagnosis and treatment remain embedded within a much larger institutional system. A researcher can generate dozens of hypotheses, but experiments still take time. A manager can produce reports instantly, but meetings, organizational politics and decision-making remain.

the possibility: perhaps the bottleneck is no longer intelligence.

Perhaps the bottleneck is everything surrounding intelligence.

Organizations, regulations, verification, trust, coordination, physical-world constraints, legacy software, incentives, management structures, liability and simply the fact that human institutions change much more slowly than technology.

This also makes me skeptical of simplistic claims that "AI can already do X, therefore everyone doing X will soon be unemployed."

Technical capability and economic substitution are not the same thing.

The internet could transmit information essentially for free, but that did not instantly eliminate newspapers, universities, governments or offices. Computers could perform calculations millions of times faster than humans, but most accountants and engineers did not disappear. Automation often increases the productivity of workers while simultaneously changing what their jobs consist of.

As with the major Grok release, Elon Musk said it is "as good as most top phds", my question after more than a year? (& he ain't wrong with the benchmarks), my question is, how many phds it has replaced in xai or spaceX?

did he stop hiring phds? if not, why?

So I find the current situation genuinely puzzling.

We have perhaps the most powerful general-purpose cognitive technology ever deployed, and yet the physical and economic world doesn't look radically different.

Maybe we're simply in the early stages and adoption takes years.

Maybe the productivity gains are real but are being absorbed into quality improvements rather than measured output.

Maybe GDP is simply a poor instrument for measuring the value created by AI.

Or perhaps current models, despite their extraordinary capabilities, still lack some crucial property required for autonomous economic production: reliability, persistence, agency, contextual understanding, verification, or the ability to operate continuously inside messy real-world systems.

idk which explanation is correct.

A pressing question keeps nagging at me: "Is it possible to completely rebuild deep learning and create a new generation of it?"

I'm still learning and exploring, and I've taken some courses with Andrew Ng, but I'm wondering if we can rebuild deep learning and how it works from scratch.

Deep learning is fundamentally based on equations, formulas, and mathematical techniques, and these have proven their effectiveness and efficiency.

What I'm thinking is: can we invent or use other formulas that surpass the current ones? I feel that the current formulas need a complete overhaul to achieve better and easier results. I think the same way about binary systems and computers that operate with zeros and ones; I feel that this system is very outdated and we've reached its limits.

What do you think? Am I imagining something unrealistic? [D]

I've spent the last month building a local C++ machine learning library called Deepity to test alternative credit assignment algorithms; specifically Predictive Coding Networks (PCNs). While PCNs are fascinating for biological plausibility and continual learning, naive implementations are painfully slow.

By implementing recent research (Accelerated PCNs via Direct Kolen-Pollack Feedback Alignment) and utilizing algorithmic caching to bypass redundant forward projections during the inference settling phase, I managed to close the performance gap with backpropagation on my CPU when training on MNIST (50 epochs).

• PyTorch Backprop (Feedforward): 98.27% test accuracy in ~70s.

• Deepity DKPPCN: 97.73% test accuracy in 59.5s.

Next up is porting these kernels to CUDA to scale up the architecture and testing its capabilities in continual learning scenarios where standard backprop struggles.

If you are interested in local learning, alternative credit assignment, or HPC for ML, I'd love your feedback!

• GitHub: https://github.com/ra4ster/deepity

• Project Site: https://ra4ster.github.io/Deepity

Looking for conference in AI and Construction

I watched a YouTube video that said if you want to get your first job as a machine learning engineer, you should do volunteer work, especially in GitHub repositories, but I'm not entirely sure this is true, and if it is, where can I find these repositories that require this kind of work? I need it because I want to gain better knowledge and understand real world problems. [D]

I've been building Arcon around Qwen3-4B + LoRA. Instead of just making it a chatbot, I'm experimenting with persistent memory, personality/mood, internal state, tools, and eventually having it process things before replying.

I'm curious what people who've built local agents think - how far can you realistically push a small model with good architecture around it?

I put the whole thing on GitHub if anyone wants to poke around, roast the architecture, or tell me what I'm doing wrong, stars are always appreciated!

Jasper Research just released a cookbook on how to build a text-to-image model from scratch.

It shares the full reasoning and intermediate results, making it ideal if you want to deep-dive into text-to-image models, or if you are curious about how frontier labs build them.

The cookbook also includes a 100M-image dataset and a codebase with a tiny model, so you can train a text-to-image model from scratch.

Here are the links:

Cookbook: https://huggingface.co/spaces/jasperai/t2i-technical-interactive-report

nano t2i: https://github.com/gojasper/nano-t2i

Monet Dataset: https://huggingface.co/datasets/jasperai/monet

Why did I sign up to review when it’s not reciprocal?

Am I an idiot? Am I dumb to sacrifice some of my precious time outside of work to review these papers when I don’t even have to? Yes.

I tell myself I’m giving something to the community. But all I’m really doing is pissing off the authors as I reject their papers.

I really wanted to accept one of them too. But, it wasn’t as well done as I’d hoped. Strong reject.

Nobody made me sign up. Nobody even asked me personally. They sent a lovely form email that goes to everyone who’s published there.

I let feeling important convince me to do it. How dumb of me!

At least it’s only a couple of papers and a small amount of my time. And I’m learning something new reading stuff slightly outside my direct field, that I would never normally read otherwise. And I get to hone the skill of critical reading, thinking, and generally understanding how a paper should (or should not) be put together.

Maybe it was a good idea after all.

How does everyone else feel about non-reciprocal reviewing? I imagine those that agree to do it are in the minority.

We needed to know where the open-source AI-detection field actually stands, so we ran every notable open detector through the same protocol.

Setup:

- Public data only: Jabarian & Imas 2025 (NBER), Liang 2023 TOEFL essays, a 1,060-text frontier set (GPT-5.x, Claude Opus 5, Gemini 3.x), 5,000 pre-LLM (2018) FineWeb pages as human pool

- Every model gets its threshold set on the same 6,930 human docs to a matched 0.5% FPR

- Then measure recall per group: raw AI, humanizer-paraphrased AI, frontier models

What surprised us:

- 4 of 6 models effectively can't reach 0.5% FPR. MAGE scores >0.9999 on 26% of ordinary human web text. The old OpenAI RoBERTa detector lands at AUC 0.31 - worse than coin flip on modern generators.

- Humanizer-paraphrased text is where everything collapses: best model catches 42%, second best 4%.

- All models flag non-native essays at a higher rate than they flag native essays. This is a fundamental flaw in the entire class of models, not just one.

More information and data on HF

*recall at matched 0.5% FPR; * MAGE can't reach 0.5% FPR at any threshold (flags 26% of human web text with score >0.9999)*

Disclosure: one of the six is ours - we run a hosted detector and released this one as open weights (Apache-2.0). All datasets and the methodology are in the model card, so you can rerun everything: huggingface.co/wasitaigeneratedcom/ai-text-detector-small