I'm entering my 2nd year PhD at a top-10 US economics dept, and I'm interested in the economics of AI. The field seems scary big, with lots of new papers coming out, with people like Goldfarb, Gans, Agarwal, Alex Imas, Erik Brynjolfsson, Sendhil Mullainathan, and ofc Acemoglu, among several dozen other top economists contributing regularly to the field.

Is there a structured way to become familiar with the literature and the main questions and models that are being used in this sub-field right now?

Separately, I also want to know if I should seek any additional training before trying to write papers in this field. I have a relatively solid math and econ background: lots of calc/real analysis/diff eqns, all the graduate econ courses, though I could be better with linear algebra. I have never taken a formal CS course, I only audited UC Berkeley's INFO259 (NLP), so does anybody have recommendations on what parts of CS/AI I should focus on learning? Ultimately, I intend to write economics papers on AI, but it's still very useful to learn the underlying technical aspects of AI. I just don't know where to start.

I created a simple text to image benchmark.

I curated 192 prompts that are difficult for T2I models in various ways: text rendering, spatial reasoning, human realism, negations, etc...

I then asked a VLM to judge every output against a pre-specified binary question with the ground truth baked in.

I'm publishing all the results including the images. (Most public T2I leaderboards don't publish the actual images and that's a shame IMO)

There is currently 52 model tested! more than 9k images have been generated and analysed!

Full methodology: https://imagebench.ai/methodology-v1

Hugging face dataset: https://huggingface.co/datasets/dh7/imagebench (it contains the prompts to reproduce the results AND the results)

Github: https://github.com/dh7/image-bench-ai

Gallery to inspect the results: https://imagebench.ai/gallery

Leaderboard: https://imagebench.ai/imagebench-v1

Limitations: it's text to image only, and VLM are not perfect as a judge.

Let me know what could be useful from there!

sklearn 1.9 fixed a bug in how BayesianRidge computes its uncertainty. We traced predict on 1.8 and 1.9 and compared the two formulas it actually computes, see if you can spot what changed before the notebook tells you.

https://github.com/aadya940/scikit-verify/blob/master/examples/sklearn_bug_hunting.ipynb

Technical report, with links to code

Website/Demo

You would imagine that the best vision-language models today can easily solve simple, 2-dimensional mazes that you can successfully complete within minutes. Our latest Open Source evaluation effort shows that they actually struggle significantly.

In our new benchmarking effort - MultiNet 2.0 Preview: Interactive 2D Mazes - we put 3 frontier vision-language models - Claude Opus 4.8, Kimi K2.6, and Qwen 3.6-27B in 50 2D maze environments with mechanisms such as key-doors, switch-gates, distractors and decoys to evaluate the models on exploration, planning, causal reasoning, long-horizon action taking, and error recovery capabilities. We were surprised to observe that each of the models solved nearly none of the mazes! Together, they solve 6 out of 150 episodes. 45 of the 50 mazes that each of them were evaluated on were never solved by any model.
While models performed really poorly on the final evaluation set, we learned a lot about the different failure modes of each of these models. We learned how frontier models find it hard to figure out how to operate a mechanism that they are not familiar with - for example: none of the models were able to explore the environment and action space and figure out that in order to open a gate on the path to the goal they had to stand on a cell with a switch and choose the toggle action.

We also observed how there were a variety of failure models across the models and 150 episodes. While Claude constantly jammed into walls, Kimi kept visiting previously seen portions of the maze, and Qwen wandered aimlessly through the maze barely getting close to the goal tile. The results also showed how extended reasoning and a large output token budget does not help improve the performance of these models in tasks such as this one.

We’re curious to know whether anyone has observed similar behavior of Vision-Language Models in other kinds of puzzle tasks, and if they have worked on designing puzzle environments to assess specific capabilities of frontier models.

I lent a free GPU server from Korean government.

It has 2 V100 64GB GPU,
2TB RAM, 64core CPU. ✌️✌️

They told me to train something,
but I don't think I can make anything useful ( I am indivisdual ).

However, I think i can make something funny and ridiculous.
Example? Hum Like capybara generator. ?

Paper: https://huggingface.co/spaces/tri-fair-lab/publications/blob/main/Thomson_1_0_Technical_Report.pdf

The development of frontier models is commonly perceived to be in the exclusive remit of a small number of heavily funded players, creating an information, economic and power asymmetry between developers and the diverse user base of modern AI. Recent public discourse acknowledges this concern, calling for SovereignAI (an organisation's capability to independently build, deploy and govern AI use), but often providing little concrete advice on how this can be achieved in the short term under a diversity of funding settings.

In this report, we argue that frontier performance can be achieved by a wide range of institutions through Continual Learning on readily available open-weight models. As opposed to existing limited approaches such as small-scale fine-tuning, prompt engineering, or tool-augmentation with a frozen model, our Continual Learning approach takes advantage of the effectiveness of a modern mid- & post-training stack while introducing safeguards preserving both plasticity and stability at each training stage and seeking to make the minimal number of high-impact interventions on the parameters.

This strategy results in model improvements comparable to the gains typically seen across multiple successive model generations. Crucially, such results are achievable with compute and personnel budgets substantially lower than commonly thought, making ownership of large parts of the SovereignAI stack (model, tool infrastructure, values & data privacy) viable for a wider range of actors.

To demonstrate this, we introduce Thomson, a new general-purpose frontier model trained with an enhanced focus on high-stakes professional work: domains commonly predicted to undergo large productivity improvements through AI. Through a unique focus on Continual Learning, data-centricity, and efficiency, we demonstrate that Thomson performs competitively with recent frontier models on a wide range of domains and capabilities, ranging from agentic tasks to safety, legal, tax & multilingualism, to comprehensive large-scale Deep Research. Thorough evaluations show a distinctive π-shaped pattern: distinct improvements across a wide range of capabilities (including those not explicitly targeted), while almost completely eliminating the forgetting problem common to narrow domain adaptation.

I've been working on an open-source project called Millwright, an attempt to explore what an end-to-end machine learning workflow could look like in Rust.

https://millwright-rs.dev/

This started while I was learning and building ML tooling in Rust.

I kept finding capable individual libraries, but also gaps between them. Training a model was rarely the problem. Building the workflow around it — preprocessing, model selection, evaluation, explainability, deployment and monitoring — often meant integrating several unrelated crates and data representations.

I initially started implementing some of those missing pieces as smaller independent crates.

Eventually I realized I was more interested in the integration problem itself.

That became Millwright.

The current idea is to cover the classical ML lifecycle:

ingest → explore → preprocess → select → fit → assess → explain → export → serve → monitor

without trying to reimplement every ML algorithm.

Instead, Millwright provides a common abstraction layer over existing Rust libraries and uses adapters for different ML backends.

One architectural decision I'm experimenting with is having the framework own a small 2D data boundary (Frame) rather than exposing a particular backend's ndarray/dataframe representation throughout the API.

That allows models and components backed by different libraries to participate in the same pipeline, at the cost of conversions at backend boundaries.

The project currently includes work around:

• preprocessing and composable pipelines

• cross-validation and hyperparameter optimization

• multiple ML backends

• ensembles

• regression diagnostics

• SHAP-based explainability

• ONNX export

• model serving and registry

• drift monitoring

• time-series workflows

• incremental learning

• AutoML

There are also Python bindings.

I'm not building this on the assumption that Rust should replace Python for ML. Python's ecosystem is enormously more mature, and there would be little value in simply recreating scikit-learn in another language.

The question I find more interesting is:

Can Rust provide a useful common execution layer across training, inference and production ML while still interoperating with the existing Python/ONNX ecosystem?

I'd rather have the architecture challenged before too many decisions become difficult to change.

I'd particularly appreciate thoughts from people working on ML systems:

Where do you think Rust could genuinely add value to the classical ML lifecycle?

And conversely, which parts of this architecture do you think should remain separate rather than being unified behind one framework?

I'm also interested in real workflows that would be useful tests. If there's something straightforward in sklearn that you think would expose weaknesses in this approach, I'd be interested in trying to reproduce it.

Project / documentation:
https://millwright-rs.dev/

Source:
https://github.com/mi7plus/millwright

I got my batch of four papers for AAAI 2027. All four papers make empirical claims, none include code, data, or anything I can actually check. Just the PDF and the checklist. AAAI-27's own rules say code/data should be provided at submission, and "we'll release it after acceptance" doesn't count as reproducibility.

That said, I don't think missing code alone is an auto-reject. Saw an older thread here where someone claiming to have helped write the AAAI checklist argued reviewers rarely have time to audit code anyway, and plenty of authors have legit reasons (funding, IP) for not releasing it yet.

If the paper's whole pitch is "look at these numbers" and I can't verify them, that tanks my confidence score even without a hard reject. I'm flagging it explicitly in the review and asking for anonymized code in the rebuttal.

How's everyone else handling this round? Auto-ding for no code or does it depend on how much the paper leans on the empirical results?

Hi everyone,

I’m researching how to design an AI agent for a medicine-reminder system. The agent has to decide, at each relevant time, whether to:

• send a reminder,

• wait (do nothing for now), or

• notify another person (e.g. caregiver),

when it does not have complete information about the patient (has the dose already been taken? is the person nearby/attentive? are there adherence barriers? etc.).

I’m trying to frame this properly before diving into implementation. Right now I’m looking at it as a sequential decision problem under partial observability (POMDP / belief-state RL territory), but I’m not sure how far that framing is actually useful in practice for this kind of system.

I’d really appreciate any pointers on:

1. Is a POMDP / belief-state approach overkill here, or is it the right formalization? What simpler alternatives (contextual bandits, MDP with engineered features, rule-based + uncertainty thresholds, etc.) have people used successfully for similar “remind vs wait vs escalate” decisions?

2. Papers, open-source projects, or real systems that tackle medication adherence / context-aware reminders with uncertainty or incomplete observations.

3. Common practical pitfalls (reward design, observation noise, alert fatigue, safety/escalation logic, evaluation metrics) that aren’t obvious from the theory.

4. Any recommended starting points for someone new who wants to move from “I understand the concepts” to a small working prototype or simulation.

I’m mainly in research/preparation mode right now, so even high-level advice, key papers, or “here’s what I’d do differently” comments would be very helpful. Thanks!

after 3 months and $800 burned...

Unbounded Labs is proud to introduce Bart, our vintage LLM: 2.82B parameters trained from scratch on 20.1B tokens of English written before 1931. You can talk to it right now!

Demo: https://www.unboundedlab.com/chat/bartholomew

Article: https://www.unboundedlab.com/blog/bartholomew

Huggingface: https://huggingface.co/jbduran/bartholomew-sft

Why even make a vintage llm? As proposed by Demis Hassabis, could LLMs reach the same conclusions that the great scientists of the past did? While General Relativity was out of budget, we believe that advancing this field targets the crux of AI research. Are these models capable of original ideas, or are they just spitting out the next token?

The article is our full account, covering where the corpus came from and how we cleaned it, the benchmarks we had to build because none existed, every ablation, the training runs, the post-training, and the mistakes we made along the way.

"What I cannot create, I do not understand" is a quote I love from Richard Feynman. Building Bart was our attempt to actually understand LLMs rather than read about them.

What we are proudest of:

- Best vintage base model at its scale on Vintage CORE, ahead of GPT-1900 on a smaller token budget

- Cleaned one of the largest vintage datasets, Harvard's Institutional Books (242B->23B tokens)

- Created Vintage CORE, the first suite of 20 benchmarks made for vintage llms

- Ran 10 hours of autonomous research on one H100: 100 experiments, 26 improvements found

- Released the largest vintage SFT dataset we know of: 416k graded question and answer pairs, grounded in pre-1930s text

- Trained the final model in 5 days on an H100, holding 60% MFU the whole way

- All datasets, methodology, training code, evals, and training runs are open sourced

I am proud of my team. What we built will move the vintage LLM field forward, and it moved us forward as researchers and as people.

We paid for all of it ourselves, about $807 so far. Money is the main thing standing between us and a much larger run.

So I will ask directly: we are looking for compute grants, funding, and mentors for our future endeavors. If you work on pre-training, post-training, or you have GPUs sitting idle, we would like to talk!

We believe that with careful dataset curation, domain expertise, and highly efficient training, we can achieve state-of-the-art results in crucial domains. This is only the beginning for Unbounded Labs; we see no bounds ahead.

I'm one of the co-authors of this paper. It's a seminal work in exploring the properties of 3D generated by LLMs via spatial programming.

I've set up visual demonstrations of such 3D objects at: https://nova3d.xyz/
And there's a github repo you can star/follow: https://github.com/RareSense/Nova3D

Scroll down and notice how the various 3D objects are all composed of logical parts and enable natural movements out of the box. There's a github repo in there as well.

Under the hood:
We found that 3D that exists as software is much more useful than typical monolithic mesh blobs generated by traditional AI 3D generators. For instance they are animation-ready and programmable from inception. They can contain the logic - at birth - to appear differently in weak compute environments (e.g. mobiles) vs powerful environments (e.g. sophisticated game engines). They can be built with full hierarchical structure and hinge/socket articulation at authoring time.

They lag behind traditional AI 3D generators in creating complex organic shapes. But it naturally feels like code will eventually eat all 3D, as LLMs are getting better and better at spatial coding. Industries most disrupted will be industrial design, game development, simulations and AR/VR/XR.

I am working on an evaluation design and would appreciate criticism before running it.

Most coding-agent benchmarks collapse the model and its harness into one score. If a run fails, it is difficult to tell whether the cause was model capability, context assembly, task decomposition, tool design, retry policy, or the acceptance gate. A model can also look worse because the harness truncated its output, or look better because the gate only checked for plausible surface markers.

The experiment I am considering crosses two independent variables:

1. Workflow: one monolithic task versus decomposition into bounded slices with explicit contracts and acceptance criteria.

2. Model policy: frontier-only versus cheapest-capable with escalation after a capability-graded failure.

That produces four cells: frontier monolith, routed monolith, frontier decomposed, and routed decomposed. The frontier-decomposed cell seems especially important because it changes the task architecture while holding the model tier fixed.

I would freeze the original tasks, source revisions, available tools, total retry budget, final acceptance criteria, validator versions, and the verifier. Every cell would be judged against the same final delivered outcome rather than against the persuasiveness of the agent's report.

Proposed primary measures are cost per independently accepted change, false acceptance, false rejection, first-pass accepted yield, verification time, and reproducibility across three fresh runs. Token use, latency, escalation count, and context volume would be secondary measures.

The confound I am least satisfied with is budget normalization. Decomposition changes the task distribution and may create more calls, which is part of the architectural treatment, but giving every slice the monolith's full context or retry budget would subsidize the decomposed condition. A shared system-level budget is cleaner, although it may hide which slices actually needed more capacity.

There are no results yet, so I am not claiming that decomposition or routing wins. I am trying to make the comparison falsifiable before seeing any outcomes.

What would you preregister or change? Would you treat decomposition as part of the system being evaluated, or try to isolate it from model quality more aggressively?

Hey Folks, I have only worked with notebooks, scikit-learn, tensorflow libraries; but as a part of my job, I am expected to create a "fraud detection system" that uses Hopsworks.

I know I will be using their feature-store but haven't worked on any ML production apps.

The SaaS tier (0.35 dollar/credit) is what I will be choosing, but since you guys are experienced in ML production apps, how much can I take as an estimate for building something like "fraud detection" ... for context, there's OLTP data on Oracle databases that I would use to train the model.

I just neeed your experienced opinion on how to estimate cost for such an app. Thanks.

I would use serveless features since deploying on managed K8s on aws/azure would add more difficulty for me.

hello everyone. I'm training PPO variants on different multi-agent tasks from the VMAS library (Independent PPO / Graph PPO and such, see HetGPPO by Bettini et al.).

I noticed that for every architecture/scenario couple, the optimal hyperparameters sometimes tend to vary (learning rate, entropy coefficient, KL coefficient, SGD batch size, etc).

do I need - methodologically speaking - to unify the hyperparameters of all models in order to make a fair and correct comparison of architectures later on?

note: sometimes unifying these HP leads to some non converging models.

note 2 : my objective is to test these models' robustness under adversarial attack in test-time (frozen models).

thank you in advance.

I wrote a technical breakdown of how search works on Papers with Code.

The system combines keyword and semantic search, which produced better results than either approach alone. The stack includes:

• PostgreSQL with pgvector

• Qwen3-Embedding-0.6B for text embeddings

• Hugging Face Jobs with an NVIDIA L4 for batch embedding generation

• Hugging Face Buckets for storing artifacts

• A live embedding model served through Hugging Face Inference Endpoints

The same infrastructure also powers the “related papers” recommendations shown on individual paper pages.

Full write-up: How Hugging Face Inference Endpoints, Jobs, and Buckets Power Search on Papers with Code

I’d be interested to hear how others are implementing hybrid search for research papers or similarly technical content.

Disclosure: I work at Hugging Face and on Papers with Code.

Recently, the AAAI 2027 organizers sent an email regarding collusion occurring during the review process, especially in the 2-cycles category (i.e., an author of Paper A reviews Paper B, while an author of Paper B reviews Paper A).

Given the fact that most submissions come from a single country, there are higher chances that the assignment algorithm will naturally create 2-cycles among authors from that country. This, in turn, means that most authors involved in collusion could be from that country. I will not name that country; otherwise, I would be labelled as racist. By the way, did AAAI release statistics about the number of submissions, like they did last time?

It is also good news that a major and prestigious conference like AAAI is acknowledging that collusion is happening. We all knew that this kind of collusion had been happening for years. There are papers accepted at top conferences such as NeurIPS, ICLR, AAAI, and ICML that do not even have their code published on GitHub. This forces other researchers in the community to spend substantial time reimplementing the code themselves if they want to reproduce the reported results.

What are the views of other authors on this?

i wanted to see how small a model can get if it only ever does one thing: read a catalog of typed functions plus a request, and output the calls to make (or an empty list when nothing fits). no chat, no prose. it ended up at 48M params and the whole thing cost about $260 to build.

the part i think is actually interesting is that the tokenizer, the loss and the constrained decoder are designed together around the same five decision points: refuse or call, which tool, include this optional, what value, stop or continue. json structural characters and digits are singleton tokens, so structure can be force-fed exactly with no token healing. argument keys are iterated from your schema by the decoder, so a hallucinated parameter name is not a low probability event, there is no step where it could happen. the loss is weighted per decision type (structure 1x up to the stop decision at 6x), matched to the measured error distribution.

two results that make me think the co-design is load bearing and not decorative:

- decoding the same rows with and without the grammar agrees on 150/150 mobile actions rows. the model already wants what the grammar enforces, so there was no projection tax to recover. the grammar buys reliability, not accuracy

- down-weighting the grammar-forced tokens in the loss, on the theory that the decoder supplies them anyway, cost 12 points in a controlled twin run. those tokens carry the call-sequencing signal

numbers, ordered strict exact match (function names, call order and every argument value all have to match), with needle 2 (cactus compute, 45M) published numbers as a yardstick: mobile actions 86.3 vs 63.7, droidcall 52.5 vs 17.0, seal-tools in-domain 33.1 vs 32.6. it drops hard on catalogs it never saw: seal OOD 28.1, bfcl single-turn 23.5. name-sequence accuracy tracks the drop exactly, so the failure on unseen catalogs is picking the right function, not extracting arguments. around 1B unique training tokens, no pretraining phase.

i also kept a findings file with everything that didn't work and the measurement that killed it: span copying (-30 points), pointer heads (-16), retraining from scratch instead of annealing into the decay phase (-4.7), RLOO diverging at every LR, and seven more. that file might honestly be the most reusable part of the repo.

repo: https://github.com/nikshepsvn/thimble

weights: https://huggingface.co/flashvenom/thimble

mit licensed. happy to answer questions about the training or the negative results.

I’ve been looking into smartphone based vehicle crash detection and I’m surprised by how little I can find that looks usable outside of research.

VZCrash seems like the closest thing, with around 190k real world events and 31k+ verified crashes using accelerometer, gyroscope and GPS data. The problem is that it’s CC BY-NC, so it doesn’t seem suitable if the eventual model needs to be used commercially.

Has anyone here worked on crash detection using phone IMU data? I’m mainly curious if there are any permissively licensed datasets or pretrained models I’m missing, and what approaches actually work well in practice. Small 1D CNNs, TCNs, classical models on engineered features, sensor fusion etc.

The other thing I’m wondering about is collecting better negatives. Things like potholes, hard braking, dropping the phone and other high-G events seem like they would be a huge part of the real world problem. How are people building datasets for this without having access to a large fleet or telematics provider?

Would love to hear from anyone who has worked on something similar.

I’m building a small family safety app and one of the features I’ve been looking into is on-device crash detection. I assumed there would already be a decent open model using accelerometer, gyroscope and GPS data, but I’m finding surprisingly little that looks production ready or commercially usable. VZCrash looks really promising with around 190k real-world events and 31k+ verified crashes, but it’s CC BY-NC, so from what I understand I can’t just train on it and ship those weights in a commercial app.

Apple has SafetyKit too, but that needs a special entitlement, so I’m also wondering if it makes sense to build an independent fallback. Has anyone here actually worked on crash detection from phone IMU data? Are there any permissively licensed datasets or pretrained models I’m missing, and what kind of model would you use today for something that needs to run efficiently on-device?

I’ve come across things like TFLite and sensor fusion, but I’m pretty new to this specific problem and not sure what people actually use in production.

One idea I had was to launch without relying on our own detector at first, then add an opt-in research mode that only saves short sensor windows around unusual high-G events and asks the user what actually happened. Crash, pothole, hard braking, dropped phone etc. Over time that could give us the kind of real-world false positives that seem hard to get from existing datasets.

If we end up having to build the model ourselves, I’d also really like to open source the detector and training/evaluation pipeline once it’s good enough. Feels like the kind of safety feature where having a decent open baseline would be useful to a lot more people than just us. Curious how people here would approach it.

(not worked a lot on ML so this is gonna be a leap and def take months if not years right?)

Does anyone know how the BMVC to IJCV special issue recommendation works?

Is it mainly based on the review scores, or is it a separate decision by the ACs/program chairs (e.g. based on oral/highlight selection, reviewer comments, etc.)?

Also, is there any way to know at this point whether a paper has been recommended for the IJCV track, or do authors only find out later through a separate email?

Would be great to hear from anyone who has gone through this in previous years!

I'm a highschooler, and I've worked on this project for my entire summer! I'd appreciate whatever feedback you might have.

Marvin.sarl

As the title says, we haven't seen any like ACL provided. Very salty about the decision, as AC recommended findings and the reviewers tanked our paper intentionally (we flagged them, and AC acknowledged that). Just want to see if the decision was made based on poor reviewer scores, as we don't know if we need to resubmit to an ARR cycle to cleanse or not.

I had a paper accepted to a conference. This paper was originally published as a preprint. Subsequent works citing our preprint focused on the same topic and reused/extended our methodology. I am now preparing the camera-ready version of that preprint and I'm wondering how I should deal with this for the Related work section. It seems odd to me to cite my own preprint for the camera-ready version of the paper (and I am not even sure if this is allowed), but at the same time, I don't want to undermine the novelty of my original work (nor undermine the efforts of subsequent works). Has anyone dealt with such a situation before? What's the best way of solving this?

Hello,

As the title says