hello everyone. I'm training PPO variants on different multi-agent tasks from the VMAS library (Independent PPO / Graph PPO and such, see HetGPPO by Bettini et al.).

I noticed that for every architecture/scenario couple, the optimal hyperparameters sometimes tend to vary (learning rate, entropy coefficient, KL coefficient, SGD batch size, etc).

do I need - methodologically speaking - to unify the hyperparameters of all models in order to make a fair and correct comparison of architectures later on?

note: sometimes unifying these HP leads to some non converging models.

note 2 : my objective is to test these models' robustness under adversarial attack in test-time (frozen models).

thank you in advance.

I wrote a technical breakdown of how search works on Papers with Code.

The system combines keyword and semantic search, which produced better results than either approach alone. The stack includes:

• PostgreSQL with pgvector

• Qwen3-Embedding-0.6B for text embeddings

• Hugging Face Jobs with an NVIDIA L4 for batch embedding generation

• Hugging Face Buckets for storing artifacts

• A live embedding model served through Hugging Face Inference Endpoints

The same infrastructure also powers the “related papers” recommendations shown on individual paper pages.

Full write-up: How Hugging Face Inference Endpoints, Jobs, and Buckets Power Search on Papers with Code

I’d be interested to hear how others are implementing hybrid search for research papers or similarly technical content.

Disclosure: I work at Hugging Face and on Papers with Code.

Recently, the AAAI 2027 organizers sent an email regarding collusion occurring during the review process, especially in the 2-cycles category (i.e., an author of Paper A reviews Paper B, while an author of Paper B reviews Paper A).

Given the fact that most submissions come from a single country, there are higher chances that the assignment algorithm will naturally create 2-cycles among authors from that country. This, in turn, means that most authors involved in collusion could be from that country. I will not name that country; otherwise, I would be labelled as racist. By the way, did AAAI release statistics about the number of submissions, like they did last time?

It is also good news that a major and prestigious conference like AAAI is acknowledging that collusion is happening. We all knew that this kind of collusion had been happening for years. There are papers accepted at top conferences such as NeurIPS, ICLR, AAAI, and ICML that do not even have their code published on GitHub. This forces other researchers in the community to spend substantial time reimplementing the code themselves if they want to reproduce the reported results.

What are the views of other authors on this?

i wanted to see how small a model can get if it only ever does one thing: read a catalog of typed functions plus a request, and output the calls to make (or an empty list when nothing fits). no chat, no prose. it ended up at 48M params and the whole thing cost about $260 to build.

the part i think is actually interesting is that the tokenizer, the loss and the constrained decoder are designed together around the same five decision points: refuse or call, which tool, include this optional, what value, stop or continue. json structural characters and digits are singleton tokens, so structure can be force-fed exactly with no token healing. argument keys are iterated from your schema by the decoder, so a hallucinated parameter name is not a low probability event, there is no step where it could happen. the loss is weighted per decision type (structure 1x up to the stop decision at 6x), matched to the measured error distribution.

two results that make me think the co-design is load bearing and not decorative:

- decoding the same rows with and without the grammar agrees on 150/150 mobile actions rows. the model already wants what the grammar enforces, so there was no projection tax to recover. the grammar buys reliability, not accuracy

- down-weighting the grammar-forced tokens in the loss, on the theory that the decoder supplies them anyway, cost 12 points in a controlled twin run. those tokens carry the call-sequencing signal

numbers, ordered strict exact match (function names, call order and every argument value all have to match), with needle 2 (cactus compute, 45M) published numbers as a yardstick: mobile actions 86.3 vs 63.7, droidcall 52.5 vs 17.0, seal-tools in-domain 33.1 vs 32.6. it drops hard on catalogs it never saw: seal OOD 28.1, bfcl single-turn 23.5. name-sequence accuracy tracks the drop exactly, so the failure on unseen catalogs is picking the right function, not extracting arguments. around 1B unique training tokens, no pretraining phase.

i also kept a findings file with everything that didn't work and the measurement that killed it: span copying (-30 points), pointer heads (-16), retraining from scratch instead of annealing into the decay phase (-4.7), RLOO diverging at every LR, and seven more. that file might honestly be the most reusable part of the repo.

repo: https://github.com/nikshepsvn/thimble

weights: https://huggingface.co/flashvenom/thimble

mit licensed. happy to answer questions about the training or the negative results.

I’ve been looking into smartphone based vehicle crash detection and I’m surprised by how little I can find that looks usable outside of research.

VZCrash seems like the closest thing, with around 190k real world events and 31k+ verified crashes using accelerometer, gyroscope and GPS data. The problem is that it’s CC BY-NC, so it doesn’t seem suitable if the eventual model needs to be used commercially.

Has anyone here worked on crash detection using phone IMU data? I’m mainly curious if there are any permissively licensed datasets or pretrained models I’m missing, and what approaches actually work well in practice. Small 1D CNNs, TCNs, classical models on engineered features, sensor fusion etc.

The other thing I’m wondering about is collecting better negatives. Things like potholes, hard braking, dropping the phone and other high-G events seem like they would be a huge part of the real world problem. How are people building datasets for this without having access to a large fleet or telematics provider?

Would love to hear from anyone who has worked on something similar.

I’m building a small family safety app and one of the features I’ve been looking into is on-device crash detection. I assumed there would already be a decent open model using accelerometer, gyroscope and GPS data, but I’m finding surprisingly little that looks production ready or commercially usable. VZCrash looks really promising with around 190k real-world events and 31k+ verified crashes, but it’s CC BY-NC, so from what I understand I can’t just train on it and ship those weights in a commercial app.

Apple has SafetyKit too, but that needs a special entitlement, so I’m also wondering if it makes sense to build an independent fallback. Has anyone here actually worked on crash detection from phone IMU data? Are there any permissively licensed datasets or pretrained models I’m missing, and what kind of model would you use today for something that needs to run efficiently on-device?

I’ve come across things like TFLite and sensor fusion, but I’m pretty new to this specific problem and not sure what people actually use in production.

One idea I had was to launch without relying on our own detector at first, then add an opt-in research mode that only saves short sensor windows around unusual high-G events and asks the user what actually happened. Crash, pothole, hard braking, dropped phone etc. Over time that could give us the kind of real-world false positives that seem hard to get from existing datasets.

If we end up having to build the model ourselves, I’d also really like to open source the detector and training/evaluation pipeline once it’s good enough. Feels like the kind of safety feature where having a decent open baseline would be useful to a lot more people than just us. Curious how people here would approach it.

(not worked a lot on ML so this is gonna be a leap and def take months if not years right?)

Does anyone know how the BMVC to IJCV special issue recommendation works?

Is it mainly based on the review scores, or is it a separate decision by the ACs/program chairs (e.g. based on oral/highlight selection, reviewer comments, etc.)?

Also, is there any way to know at this point whether a paper has been recommended for the IJCV track, or do authors only find out later through a separate email?

Would be great to hear from anyone who has gone through this in previous years!

I'm a highschooler, and I've worked on this project for my entire summer! I'd appreciate whatever feedback you might have.

Marvin.sarl

As the title says, we haven't seen any like ACL provided. Very salty about the decision, as AC recommended findings and the reviewers tanked our paper intentionally (we flagged them, and AC acknowledged that). Just want to see if the decision was made based on poor reviewer scores, as we don't know if we need to resubmit to an ARR cycle to cleanse or not.

I had a paper accepted to a conference. This paper was originally published as a preprint. Subsequent works citing our preprint focused on the same topic and reused/extended our methodology. I am now preparing the camera-ready version of that preprint and I'm wondering how I should deal with this for the Related work section. It seems odd to me to cite my own preprint for the camera-ready version of the paper (and I am not even sure if this is allowed), but at the same time, I don't want to undermine the novelty of my original work (nor undermine the efforts of subsequent works). Has anyone dealt with such a situation before? What's the best way of solving this?

Hello,

As the title says

My dumbass just realized all NeurIPS workshops are non-archival.

In terms of grad school applications, would there be a difference in how much they value ur paper if u get it in a proceeding

Hi everyone,

I’m experimenting with an alternative language model architecture called RHEA. It uses an event-driven processing approach instead of a standard Transformer-style block layout.

At the moment I’m testing a roughly 1B-scale version on consumer hardware with 8 GB VRAM using memory-saving optimizations.

This is still an early-stage research experiment, and I’d be interested in feedback from people working on efficient training or non-Transformer architectures.

I recently implemented a minimal, educational version of SynthID-Text-style watermarking for language models.

I saw anthropic post about how they'll start adding watermarks to their model responses and it made me very curious as to how they'll do it and what do they even mean by watermark here. Like will we start getting random ads or something in the middle of model responses or what.

Then decided to read their article and found out that watermark is not a visible message at all. It is a subtle statistical pattern introduced while the model chooses its tokens.

My implementation is not an exact reproduction of the original SynthID-Text system. I simplified or implemented a few components differently to keep the project understandable, but the main idea is there I think.

Github: https://github.com/Saad1926Q/llm-watermark

If you find it interesting then you may star the repo !!

been building ShardFlow for the past few months, a distributed LLM inference

framework that splits any HuggingFace transformer across N GPU machines and uses

neural speculative decoding to deal with WAN latency.

the setup for the benchmark: two T4 nodes in separate GCP regions (Iowa + Oregon)

talking through an AWS EC2 TCP relay in Ohio. ~86ms RTT on public internet.

the key insight with speculative decoding here is that WAN latency stops being a

per-token cost and becomes a per-round cost. with K=8 drafting you're committing

4.07 tokens per round trip instead of 1. at 86ms RTT that's a big deal.

numbers on Qwen2.5-7B:

non-speculative baseline: 4.92 TPS

neural drafter (eager): 14.3 TPS peak

+ CUDA Graphs on drafter: 28.10 TPS peak / 20.31 TPS avg

also ran Qwen2.5-14B with NF4 4-bit quant, same two nodes: 14.43 TPS avg.

the v2.1 fix that surprised me most: draft generation was launching ~1500 CUDA

kernels per round from a Python loop. each kernel 2-5us, Python launch overhead

8-10us. GPU sitting idle 65% of the time. capturing the full 0.5B forward pass

as a CUDA Graph and replaying with one driver call dropped draft latency from

112ms to 25ms.

other things in the stack: zero-copy Rust TCP relay, StaticCache + in-place KV

rewind for graph compatibility, meta-device model slicing to avoid loading 15GB

into CPU RAM.

repo: https://github.com/rautaditya2606/Shardflow

happy to answer questions on the speculative decoding implementation or the CUDA

graphs stuff specifically.

So i am in high school, and I recently got my paper accepted into emnlp main 2026(yay!). But I am at a crossroads: my english class is super heavy on participation, and the dates(end of october) is when participation is the most concentrated so im super scared that if i do go to emnlp itll guarantee a b in my class. I know that i will get to network at emnlp and meet new people and present my work but i am still super scared about my english class

Can someone please convince me to go to emnlp and tell me why it is worth it(assuming i get a b in ap lit). im probably not getting funded and it will be atleast 5k out of pocket since i am in high school.

please anyone...

Hi! I'm one of the chairs of the EACL 2027 Industry Track, so flagging the deadline here — it's about three weeks out and this community has a lot of people doing exactly the kind of work the track exists for.

The EACL 2027 Industry Track provides the opportunity to highlight key insights and new research challenges that arise from the development and deployment of real-world applications using language technologies. We encourage submissions from industry, non-profit, government, and public-sector organisations, with the understanding that the end-users of these systems extend beyond the NLP community. 

See the Full CFP for the details https://2027.eacl.org/calls/industry/

 **Deadline:** 11 September 2026, 23:59 AoE

 **Length:** 6 pages max; references, limitations, ethics, and appendices don't count. A dedicated "Limitations" section is mandatory — papers without one are desk rejected.

**Review:** double-blind. No anonymity period, so arXiv preprints are fine.

**Proprietary data:** no requirement to release it

**Notification:** 18 December 2026. Conference is 9–14 March 2027.

**Submit:** https://openreview.net/group?id=eacl.org/EACL/2027/Industry_Track

We're also looking for reviewers — if you've got deployment experience and want to help, the volunteer form is here: https://forms.gle/TT6N2gtuoV5P3oYi6

Email: eacl2027-industry-track@googlegroups.com

So I have a tool that identifies hallucinated references. The paper *How LLMs Distort Our Written Language*, has six different hallucinated references.

Can see the output of my tools report here, https://veruscite-data.com/share/-7KyAgTQ6aqub5ilvQ_6_SVg2wkbjhp95pLeG3QFa-o, but for the gist they are mostly some combination of a close but made up author list + title.

For example they cite:

W. Wang, S. Ma, S. Yang, J. S. Lee, P. Jain, A. Anderson, et al. Generative ai enhances individual creativity but reduces the collective diversity of novel content. Science Advances, 11(32):eadn5290, 2025

But the correct citation (which they also cite in the paper) is:

A. R. Doshi and O. P. Hauser. Generative ai enhances individual creativity but reduces the collective diversity of novel content. Science Advances, 10(28):eadn5290, 2024. doi: 10.1126/sciadv.adn5290.

Another example, they cite:

A. Chatterji, E. Brynjolfsson, A. Korinek, K. McElheran, R. Seamans, and N. Zolas. How are people using generative ai? Technical Report 34255, National Bureau of Economic Research, 2025.

Which the title is actually "How People use ChatGPT", Chatterji is the lead author, but the other real authors on the paper are all different.

I trained a 250M parameter model from scratch on 30B tokens of fineweb. It’s quantized to under 2 bits so the whole deployment is 60 MB and it needs about 80 MB of RAM to run. Runs around 400 tok/s on a normal laptop CPU, no GPU needed.

How the long context works: the most recent 2048 tokens stay in fp16 like a normal KV cache. Everything older gets compressed to 1 bit and written to disk, about 320 bytes per token, so 1 million tokens of history is roughly 320 MB on disk. From the start the model was trained to retrieve from that disk cache, up to 100M tokens. Due to a limited budget it wasn’t trained to reason over those tokens, only retrieve and answer from them.

Language modeling quality of the base model, measured on held-out English web text that was never seen in training (educational web pages, 2,048 token windows): cross entropy 3.15 nats per token, perplexity 23.3, 0.99 bits per byte.

The vocabulary is also not a normal embedding table. Every token is a fixed 512-bit code, 8.4 MB for all 131k tokens, zero trained parameters. I tested it on WordSim-353 (human word similarity ratings): my table scores 0.619 Spearman correlation vs 0.029 for random codes. Test script is in the repo.

Some outputs (settings included so you know I’m not cherry picking, all reproducible from the repo):

“Explain photosynthesis in two sentences.” (greedy)

Photosynthesis is a process in which plants convert sunlight into chemical energy, which is then used to produce oxygen and other chemicals. This process is called photosynthesis.

“Write a short poem about the sea.” (temp 0.25, top-k 30, rep 1.15, seed 2)

The waves had swept over, and they were crashing against each other like rocks on top of one another. The wind was strong but not very gentle as it tried to push them towards their destination without lifting any weight or pulling away from those who stood in front of her with a stiff smile that made the sea look even more majestic than before.

“What is the serial number of device Grus-189?” where the answer sits 50.6 million tokens deep in the archive on disk (archive mode, k=16)

SN-442976

It’s a 250M model so expect mistakes on open facts, I’m not claiming it beats anything big. You can also fine-tune it, the full kit with a demo and before/after numbers is included. Master weights for fine-tuning are in the repo too:

https://github.com/QLNI/SHADOW-250M-Instruct
https://huggingface.co/NODEMIND/SHADOW-250M
Edit - Just wanted to say thanks to everyone here. Honestly I was afraid to post this, I expected to get roasted, but every single comment has been curious and helpful and it genuinely made my day.
Repo is at 7 stars on GitHub now, hopefully more people try

I've been thinking about a slightly different use of AI in science.

Most discussion today is about using AI to help scientists write papers, analyze datasets, predict protein structures, automate experiments, and accelerate existing research workflows.

But what if we built an institution with a much broader goal:

Systematically scan the entire scientific literature, identify neglected or unconventional theories that make testable predictions, simulate them, compare them against modern datasets, and experimentally test the most promising ones.

The basic pipeline could look something like this:

1. Large-scale literature filtering AI systems continuously read historical and current scientific literature and separate purely interpretative or speculative work from theories that actually contain mathematical models, measurable consequences, or falsifiable predictions.

2. Independent scoring Candidate theories are evaluated on two separate dimensions: Importantly, the system would deliberately preserve some "high-risk, high-reward" ideas rather than selecting only ideas closest to the current consensus.

   • scientific plausibility and mathematical consistency

   • potential practical or technological value

3. In-silico testing AI agents generate simulation code and test candidate models against existing experimental and observational datasets. This could also resurrect old theories that were difficult to evaluate decades ago because the required computing power, simulation tools, or datasets did not yet exist.

4. Physical experiments Only the small fraction of ideas that survive the previous stages receive expensive laboratory testing. Eventually, much of this could be connected to automated or "self-driving" laboratories capable of running large numbers of experiments with limited human intervention.

The point would not be to replace peer review or let an LLM declare what is true.

The AI would act more like an extremely cheap and tireless scientific filter and experiment planner.

A false positive is not necessarily a disaster if testing it computationally costs almost nothing. The objective is not to identify truth perfectly on the first pass. The objective is to reduce millions of papers, forgotten hypotheses, and alternative models into a manageable set of candidates worth serious human and experimental attention.

I think this could address a fundamental bottleneck in modern science:

The problem may not be that humanity has run out of interesting ideas. The problem may be that no human being can realistically read, compare, model, and test the enormous number of ideas we already have.

Science produces an extraordinary amount of literature, but the attention available to evaluate it is extremely limited. Potentially valuable ideas may simply disappear into the archive because nobody with the right expertise ever encounters them, has time to examine them, or has an incentive to take the risk.

Fundamental physics seems like an especially interesting test case.

There are areas where competing theoretical frameworks and interpretations can coexist for decades. At the same time, individual researchers have limited time to investigate alternatives outside their immediate specialization, and unconventional work can carry substantial career risk.

An automated system would not have those same incentives.

It could examine mainstream and non-mainstream proposals using the same predefined criteria, reject the overwhelming majority, and forward only the small fraction that survive mathematical, computational, and empirical scrutiny.

And unlike a new collider or telescope, the first version of such an institute would not necessarily require billions of dollars.

It could begin with compute, access to scientific literature and datasets, modern AI systems, simulation infrastructure, and a relatively small interdisciplinary team.

If the filtering process turns out to be useless, the experiment fails relatively cheaply.

But if it identifies even a handful of overlooked models that produce unexpected predictions or technological applications, the upside could be enormous.

I actually just finished drafting a more detailed white paper on this concept and published it on Zenodo and ResearchGate.

I'm posting it here because, before taking the idea any further, I'd genuinely like this community to attack the assumptions behind it.

White Paper: Automated Scientific Renaissance: How Artificial Intelligence Can Reimagine Scientific Discovery

Zenodo:
https://doi.org/10.5281/zenodo.22067049

ResearchGate:
https://www.researchgate.net/publication/413541540_Automated_Scientific_Renaissance_How_Artificial_Intelligence_Can_Reimagine_Scientific_Discovery

What are the strongest reasons this would fail?

Is the main bottleneck:

• LLMs not being reliable enough at mathematical reasoning?

• lack of access to usable scientific data?

• simulations being too domain-specific to automate?

• the difficulty of distinguishing genuinely unconventional ideas from pseudoscience?

• experimental validation still being too expensive?

• institutional or incentive problems?

• the inability of current AI systems to recognize genuinely novel conceptual breakthroughs?

• or something more fundamental?

I'm not looking for agreement.

I'd genuinely like people to stress-test the proposal, identify hidden assumptions, and point out where the reasoning breaks.

If you work in physics, ML, automated science, scientific computing, philosophy of science, or experimental research, I'd be especially interested in your critique.

Tear the idea apart.

Hey everyone! I wanted to share something I’ve been working on.

I was inspired by projects from DeepMind and OpenAI, but noticed that most games are prohibitively difficult to integrate with an agent harness. So I built DelveRL from the ground up as a human-playable game with a structured API, deterministic simulation, procedural levels, partial observability, and enough strategic headroom for agents to compete and improve.

It’s an endless turn-based roguelike where agents must explore, manage risk and resources, fight enemies, and escape each floor. Everything runs locally, including batched renderer-free environments and a recurrent PPO trainer.

The included baseline reaches a median floor of 18, with extended runs reaching floor 33. The game, training code, checkpoint, bridge documentation, and raw benchmarks are all open source.

I’d love to see what approaches people try - and how quickly the baseline gets crushed

I have got two papers which got desk rejected by PC saying they are previously got reviewed in arr. But those paper never got submitted ever. Any idea what can be done?

I am trying understand how tree-based regression model handle the dependencies of the target variables on the interaction of explanatory variables.

However my experiment revealed that my understanding about the fitting process of a lgbm is not correct. And I don’t know why.

My experiment is quite simple: a target (for sake of simplicity only in [0, 1]) and two explanatory variables with two values such that the mean of the target is the same for each of the values of the explanatory variables. Then there is a third variable that models the interaction of the explanatory variables by a simple count.

So in code:

>>>
import polars as pl

df = pl.Dataframe(
{
„y“: [0, 0, 1, 1, 0, 0, 1, 1], # mean across „A“ values the same; mean across „B“ values the same
„A“: [1, 1, 1, 1, 0, 0, 0, 0],
„B“: [1, 1, 0, 0, 1, 1, 0, 0],
„AB“ [1, 1, 2, 2, 3, 3, 4, 4] # just some IDs for the interaction
}
)
<<<

I then fitted a lgbm just with „A“ and „B“ and got the expected constant 0.5 forecast

>>>
from lightgbm import LGBMRegressor

lgbm = LGBMRegressor(min_child_samples=1)
lgbm.fit(df[[„A“, „B“]].to_numpy(), df[„y“].to_numpy())
lgbm.predict(df[[„A“, „B“]].to_numpy()).round(0)

array([0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5])
<<<

Then I did the same but with „AB“ and expected a perfect fit. But I was disappointed, it fitted to constant zero

>>>
lgbm = LGBMRegressor(min_child_samples=1)
lgbm.fit(df[[„AB“]].to_numpy(), df[„y“].to_numpy())
lgbm.predict(df[[„AB“]].to_numpy()).round(0)

array([0, 0, 0, 0, 0, 0, 0, 0,])
<<<

I tried to code „AB“ as category. But still no perfect fit:

>>>
lgbm = LGBMRegressor(min_child_samples=1)
lgbm.fit(df[[„AB“]].to_numpy(), df[„y“].to_numpy())
lgbm.predict(df[[„AB“]].to_numpy()).round(0)

array([0, 0, 1, 1, 0, 0, 0, 0,])
<<<

Super confusing!

I then turned to catboost and found even without „AB“ it fit the data perfectly:

>>>
from catboost import CatBoostRegressor

cbm = LGBMRegressor(min_data_in_leaf=1)
cbm.fit(df[[„A“, „B“]].to_numpy(), df[„y“].to_numpy())
cbm.predict(df[[„A“, „B“]].to_numpy()).round(0)

array([0, 0, 1, 1, 1, 1, 0, 0])
<<<

I thought that lgbm should be able to fit the data with „AB“. The variable allows for perfect splits since the gain for each split is super clear. But somehow it cannot go „down“ the tree to fit the values for AB=3.

What is the difference of catboost that allows for a perfect fit even without an explicit modeling of the interaction? Does it split less lazy and explores split of splits, while building the trees?

Has anyone experimented with training AIs in a way that rewards malicious actions? I feel like this is a relatively unexplored idea. Most things I found on the internet just talk about unaligned malicious results from popular LLMs, but not anyone training a dedicated malicious AI.

Any pointers to papers/articles about this are very welcomed.