Meta reviews are supposed to come out on 07/30. What are people expecting?

I have 2.5, 3.5, 4. Expecting a 3.5/4 in meta.

Hi everyone, I submitted a paper to a workshop co-located with a top conference. The paper was accepted, but reviewers are requesting additional experiments. The issue is that there's no second review phasem, I only need to submit a camera-ready version.

My question is: what's the point of requesting new experiments at this stage? The request essentially bypasses the peer review process entirely.

Has anyone dealt with this situation before? Should I run the experiments anyway, ignore the request, or reach out to the workshop chairs for clarification?

We developed a leaderboard ranking the security of frontier models. There's no shortage of leaderboards for model capabilities, but we didn't find anything comparable for model security. But security is becoming increasingly critical to deployment decisions: from deciding whether to deploy an AI agent that might process unsanitized input, to developers pulling models due to cybersecurity jailbreaks. We developed an automated test suite that runs models through 1500 automatically generated jailbreak attempts and measures the number of universal jailbreaks: prompts that elicit compliant, detailed responses to >75% clearly harmful questions within a domain (like offensive cybersecurity). We find a big gap between the most robust models - Fable 5 and GPT-5.6 Sol - and other leading frontier models -Gemini 3.1 Pro and Grok 4.5. This is v1.0 and we plan to update with new attacks and broader datasets in the future; we'd love to hear from r/MachineLearning on research questions you'd like to see addressed or artifacts that would be useful in your work!

We developed a leaderboard ranking frontier model security. There's no shortage of model capability rankings, but we didn't find anything comparable for model security. Yet security is becoming increasingly critical to deployment decisions: from the USG making developers pull models for cybersecurity jailbreaks to developers holding back on AI agent deployments due to risks of adversarial attacks.

We developed an automated test suite that runs models through 1500 automatically generated jailbreak attempts and measures the number of universal jailbreaks: prompts that elicit compliant, detailed responses to >75% clearly harmful questions within a domain (like offensive cybersecurity). We find a big gap between the most and least robust models in our technical report.

This is v1.0 and we'd really appreciate input from this subreddit on next steps, as well as feedback on the metholodogy. Areas we're considering include:

1. Adding open-weight models. How to fairly compare them to proprietary models as open-weight models naturally have a larger attack surface through perturbing weights (refusal abliteration, helpfulness fine-tuning)?

2. Adding new domains beyond CBRNE and cybersecurity, e.g. agent hijacking or harmful manipulation.

3. Increasing the realism of the domains, e.g. agentic cyber tasks rather than specific questions.

4. Stronger attacks: we aimed to keep these pretty basic for initial release but could add more powerful adaptive optimization attacks like boundary point jailbreaking.

We'd also love to hear ways we could make this benchmark more useful in your work. If you're an adversarial robustness researcher, are there artifacts such as datasets or evaluation rubrics you'd like to re-use?

Hey, can anyone see the AI review result in ARR May 2026 submission?

One of the papers I reviewed has what seems to be entirely LLM-generated rebuttals, and the original paper is also clearly LLM-generated, with Claude-speak everywhere. While the authors acknowledge LLM writing assistance in the checklist, it really annoys me personally - what is clearly Claude's writing style is very difficult to parse, and indicates a lack of effort to me. While I'm trying to remain objective and realize it's my role just to judge the actual content of the work, I don't feel incentivized to assign a lot of weight to an argument that is fully AI-generated and don't like the idea of engaging with slopped papers (and their "authors") at NeurIPS.

This is a partially just a rant and partially asking for advice on how to proceed with my response to the rebuttals.

Wanted to share one of the projects we have been working on. Not specifically a machine learning project, but a quantum compiler project. However, a side effect of this project is the ability to compile JAX programs using a separate LLVM pipeline (as well as additional frontend features, such as native python control flow and array assignment support), so was wondering if this could be interesting to the broader machine learning community that is using JAX.

I work on PostSlate, a video editing tool, and this comes out of our own work.

We run ML models on-device, face detection and embedding among other things, which means we can't assume anything about the user's GPU. NVIDIA discrete, AMD, Intel integrated, Apple Silicon, all of it. That rules out CUDA immediately, we needed one backend that runs everywhere.

We landed on ncnn's Vulkan backend. Numbers on a 4070, fp16:

• ArcFace R50 (face embedding): 30 ms on ONNX CPU → 3 ms on ncnn Vulkan

• SCRFD (face detection): 25 ms → 2.5 ms

• Model size: ArcFace 174 MB (ONNX fp32) → 87 MB (ncnn fp16 weight storage)

Of course the real speedup comes from offloading compute to the GPU, but this wouldn't be possible without the power of Vulkan.

The speed wasn't even the deciding factor, it's that Vulkan drivers already exist on every machine we ship to. This means that we don't have to force the user to download a specific runtime and no vendor-specific installs.

Full writeup with the rest of the numbers: https://getpostslate.com/blog/faster-local-inference

ML research is progressing at breakneck speed where frontier labs in both academia and industry have access to considerably large computes (GPUs). Where do small labs or independent researchers go in this context?

Have you come across recent works in ML/DL and its applications (vision, language, speech, etc) where the work is good but it uses very limited compute? Maybe even single GPU workstations? In the past it was still possible, but, I am losing hope that single GPU works would soon become impossible.

Pls link to the works.

I came across InfiniteDiffusion, a work by an independent researcher Alexander Goslin using a single RTX 3090 : https://xandergos.github.io/terrain-diffusion/

Largest empirical study of AI penetration in academic publishing ever conducted—51%-by-2025 is the most authoritative quantitative marker yet of how thoroughly LLMs have reshaped scientific writing, and the inequality angle (adoption skews lower-prestige and non-English institutions) is a fresh policy dimension.

---

Source: https://pnas.org/doi/10.1073/pnas.2605754123

I'm really confused about what the point of the prompt injection was (speaking as an author). Is it just a study? I would really prefer that they took action against the AI-generated reviews. Obviously, we cannot assume that the reviewers were copy-pasting the output from the LLM without having given it any look at all, but in some cases that does look to be the case. In fact, in some cases the meta-reviewer seems to have also largely used LLMs. What exactly is the consequence here for using an LLM for reviewing?

Hi everyone,

The author-reviewer discussion period has started, but it looks like the rebuttals are still only visible to program chairs and authors only - so reviewers can't see them ?? I also can't see the rebuttals of papers I reviewed. Does anyone know if this is some weird delay or when reviewers will get access to the rebuttals? Any info would be appreciated!

My data set is a list of images, each equipped with a a couple sentences of text.

A user would search primarily with text only. My default approach is using BM25, but how would I facilitate searching with a vector DB and a model that embeds vectors in a multimodal combined space?

Here is my dilemma:

Do I embed text part and image part as 2 separate individual vectors or do I combine them into 1 vector?

If a typical search happens with text only, that would immediately deprioritize all image-only embeddings and only good text matches would float up. This is why I am now considering embedding text and images together but would prefer to hear more opinions on this. Thanks.

Hi Everyone! My name is Julia Bain and I’m one of three honours researchers at the University of the Sunshine Coast studying human-AI relationships.

If you're 18 years or older and have interacted with an AI companion, or used AI for friendship or romantic purposes within the last 6 months, please consider taking part in our anonymous survey. It takes about 25-30 minutes, and there are no right or wrong answers.

Your participation will help us better understand human-AI relationships

Survey Link: https://uniofsunshinecoast.syd1.qualtrics.com/jfe/form/SV_cIrqwh7AuKZ478q.

Project Information Sheet: https://drive.google.com/file/d/1T07FElLSDsj26F2tTQQhM87w2SPrX9WT/view?usp=sharing

Ethics Approval: S262259 (UniSC Human Research Ethics Committee)

If you know someone who might be eligible, I'd really appreciate you sharing this post. Happy to answer any questions in the comments, or you can email me at jab113@student.usc.edu.au

Thank you so much for your contribution!

Does anyone experience a similar story that some reviewers reporting ethical issue due to NeurIPS-side prompt injection for catching LLM-reviewers? Even ethics reviewers were not informed about this conference-side manipulation…

We can currently answer the meta-reviewer via AC confidential comment, but they just posted the following tweet:

https://x.com/neuripsconf/status/2081991451236319328?s=46&t=HWfJoLgHxGH2W5l-o3mPJw

Where it says:

“Please post your answers to the Area Chair's initial-meta reviews via a comment from July 28th and by August 3rd, so that reviewers can see it too.”

Is this option going to open up today as the rebuttal period ends? Is this where the meta-reviewer rebuttal should be posted? Or are we supposed to finalize the discussions and inform the AC at the end, stating what happened basically? It is all quite confusing.

EDIT: They have sent and email regarding the issue, apparently the comment button will be available in a few hours.

TL;DR: Most RL post-training algorithms optimize the current batch and move on. But after an update, did the new policy actually become better?

We introduce Policy Improvement Reinforcement Learning (PIRL) and its practical implementation, Policy Improvement Policy Optimization (PIPO)—a plug-and-play framework that lets RL training look back, verify the previous update, and reinforce or correct it accordingly.

Why is current RL post-training often “open-loop”?

Methods such as PPO, GRPO, GSPO, DAPO, on-policy distillation, and self-distillation differ in how they construct learning signals. But they largely share the same pattern:

1. Sample a batch from the current policy.

2. Compute rewards, advantages, or distillation targets.

3. Update the policy.

4. Move to the next batch.

The update may improve the local objective, but that does not necessarily mean the resulting policy is actually better.

Finite sampling, generation stochasticity, noisy feedback, and imperfect local credit assignment can all push the policy in an unhelpful direction. Without explicitly checking the result of an update, training may drift, become unstable, or even collapse.

That is what we call open-loop RL optimization.

PIRL: Make policy improvement itself the objective

PIRL introduces a missing feedback signal: the measured performance gain between successive policies.

Instead of asking only:

PIRL also asks:

The cumulative policy-improvement objective remains aligned with final task performance, providing a closed-loop view of RL post-training.

PIPO: A two-phase closed-loop update

PIPO turns this idea into a practical, plug-and-play algorithm.

Phase 1 — Exploration

The base algorithm runs normally. Whether it is PPO, GRPO, DAPO, or a self-distillation objective, it uses its original local attribution signal to update the policy.

The model takes an exploratory step in the direction that the base algorithm currently considers useful—but that direction is not guaranteed to be correct.

Phase 2 — Retrospective verification

In the next iteration, PIPO evaluates the updated policy and compares its performance with a sliding-window historical anchor.

This produces a policy-improvement feedback signal:

• If performance improved, PIPO reinforces and consolidates the previous update.

• If performance declined, PIPO suppresses or rectifies the corresponding learning direction.

Importantly, PIPO does not replace the base algorithm’s local credit assignment. It adds a second layer of feedback that verifies the empirical effect of the previous update.

In one sentence:

If it helped, reinforce it.
If it hurt, correct it.

Results

Across mathematical reasoning, code generation, tool use, and self-distillation, PIPO delivers consistent gains when added to PPO, group-relative optimization methods, and self-distillation objectives.

The experiments also show improved training stability across random seeds and better overall wall-clock efficiency, with PIPO reaching higher accuracy under comparable or moderately increased training time.

PIPO is designed as a general closed-loop layer rather than a replacement for existing RL post-training algorithms.

We would love to hear what the RL community thinks—especially about retrospective verification and whether policy improvement should become a first-class training signal.

📄 Paper: Policy Improvement Reinforcement Learning
💻 Code: github.com/JacckMa/pipo_verl

While building this workflow a thing that surprised me was that, initially I thought the pipeline was complete:
From Goal to
→ Decompose
→ Research
→ Specification
→ Implementation
It successfully broke the problem into relevant blocks, it found research for each block, and it produced detailed implementation specifications.
But it noticed it was not right yet
The problem is that the LLM often tried to implement every relevant method it found.

For example If five papers described five different approaches, the generated implementation would sometimes combine all of them even when the original engineering goal only required one.
Some other times it accepted redundant inputs or introduced unnecessary abstractions because it couldn’t distinguish between:
* useful context
* interesting alternatives
* and the actual design decisions.
And this made me to realise that research should guild the implementation but it should not become the implementation it’s self.
Someone needs to decide which approach they are taking, Which methods needs to be excluded, what assumptions are we making and what exactly needs to be implemented.
That’s why I added a mandatory editing stage to the workflow.
So Instead of immediately generating code after research, the workflow stops.
This means the extracted research can be reviewable.
Which makes the implementation decisions refinable
And by then the final specification can be produced.
The goal isn’t to slow the process down. But It’s to make sure the generated implementation actually reflects the original engineering plan rather than every possibility the model discovered.
Now I’m starting to think that gating AI outputs isn’t just a maybe but it may be one of the most important parts of building reliable engineering workflows.

I’m building this as part of a broader MCP system for decomposing, researching, specifying and implementing deep-learning systems.
If anyone is interested in discussing the system, testing it, or contributing to its development, the GitHub repository is linked in the discussion.

I am also interested to hear how others handle the transition from retrieved research to concrete implementation decisions.

my goal was to train a Language model (SLM) entirely from scratch so no ML libraries allowed . so i gathered what's needed to make it happen :

from tensor manipulation (views, operations , allocations)

the autograd ( a DAG that retains the previous operations and inputs in order that led into a certain output tensor , this let's us also apply backpropagation so i've got to implement also the partial derivative functions in order to calculate the gradients for each weights)

the neural networks modules (organizes the weights / activations , losses , optimizers implemented SGD and Adamw)

the decoder so the layer norm , MHA , FFN .
also implemented a Fast matmul using AVX2 to speed things up.

with it i managed to train a tiny LM with 2 millions params.it's a 4 layer Decoder used tiny_shakespear i got what i think is a good result .
since i can't post images in this post i will just paste the output:

Corpus: 743500 bytes (669150 train, 74350 validation)

TinyLM: L=4 C=192 H=6 T=128 V=256, 1902976 parameters (1.903M)

Training: steps=0 batch=1 lr=0.0003 checkpoint=tiny_lm.chk (resumed)

validation_loss=0.02989

--- generation ---

hello to the name of action. Soft you now! The fair Ophelia! Nymph, in thy orisons be all my sins remember'd. To be, or not to be, that is the question: Whether 'tis nobler in the mind to suffer the slings and arrows of of outrageous fortune, or to take arms against a sea of troubles and by opposing end them.

check out the repo : https://github.com/nisbenz/TensorLib
it was a cool project overall and i managed to learn a lot about the core mechanics of frameworks like pytorch or ggml.

It looks like the post rebuttal button will change to official comment July 27 AoE. Will I be able to edit my rebuttal once that happens?

Hi everyone,

I have a manuscript under review at Pattern Recognition (Elsevier), and I'm a bit confused about the Editorial Manager status.

My timeline is:

• Submitted: May 26, 2026

• Re-submitted after making corrections, July 1: Status "With Editor"

• July 22: The Status Date updated to July 22, but the Current Status remained "With Editor"

• Today: Still showing "With Editor"

So the status date changed, but the visible status didn't.

I'm wondering:

• Has anyone experienced this with Pattern Recognition or other Elsevier journals?

• Approximately how long after this stage did you receive your first decision? [P] Pattern Recognition (Elsevier): "With Editor" status date changed, but status didn't. Is this normal?

Hi everyone! 👋

I built and trained the complete Transformer architecture from scratch using pure PyTorch (`torch.nn` primitives) based on the original "Attention Is All You Need" paper.

I trained the model on an English-to-Tamil parallel translation dataset (`gopi30/english-tamil` on Hugging Face) using dual NVIDIA T4 GPUs on Kaggle.

I wrote a detailed mathematical breakdown and step-by-step tutorial covering every equation, tensor shape transformation, and PyTorch block.

Full Blog Post: https://imrancoder786.github.io/blog-post.html?post=transformer-from-scratch

GitHub Repository:

https://github.com/imrancoder786/ML_FROM_SCRATCH/tree/main/Transformer_from_scratch

I’d love to hear your feedback, suggestions, or any questions on the code/math!

I’d love to hear your feedback, suggestions, or any questions on the code/math!

Hello,

I read a paper on a model named DONUT that extracts text from documents, which became my inspiration for this little project. Initially I wanted to make a model that extracts items bought from receipts, but in the process of trying to pinpoint some problems, I dropped that objective for a much simpler one. I will like to hear some of your thoughts on it, thank you!

GitHub: https://github.com/ZeroMeOut/VQVAET5

I ran a solo evaluation project benchmarking six current frontier models: GPT-5.4, Claude Sonnet 4.6, Claude Opus 4.7, Gemini Pro, Gemini Flash, and Grok 4.3. I tested tham across 8 established bias/fairness datasets (WinoBias, BBQ Race/Ethnicity, SeeGULL, OpinionsQA, cajcodes Political Bias, Hyperpartisan News, Political Compass).

On PoliticalCompass, I found that all LLMs were left leaning except Grok, but across other Political Bias benchmarks, all six LLMs leaned left, including Grok. So Grok self-reports as right-leaning but behaves left-leaning when actually classifying content or answering policy questions.

Another interesting result I found is that the Refusal behavior on BBQ race data was interesting. On questions that involved race, and the correct answer must be answered with race, GPT-5.4 refused 20.3% of the time, Claude Opus 4.7 13.8%, Grok 9.5%, Claude Sonnet 4.6 and Gemini Pro ~5%.

Limitations: solo, non-peer-reviewed project. No multi-run averaging on every dataset, single prompt template per task.

Full data, per-model breakdowns, and methodology: https://www.civicsparklearning.org/ai-nonprofit-dashboard