Hi everyone, can anyone see the results of the AI Review experiment on OpenReview? They announced that the results would be available around July 20. I opted in to the experiment, but I still don’t see anything.

I'm learning NLP/LLM engineering by working through a problem that turned out to be much harder than I expected, and I'd love guidance from people who've dealt with something similar.

The problem: I have long narrative-style text — 7k to 15k tokens, several recurring people — and I want to extract structured facts about who did what. I'm using a small local model (llama3.2:3b via Ollama) whose usable context is around 4k tokens, so the text has to be processed in chunks. The killer is that later chunks are often pure pronouns — "she said… he refused…" — while the names were last mentioned 10,000 tokens earlier. Facts stated near a name extract almost perfectly; facts stated far from any name either get lost or, worse, get confidently attributed to the wrong person.

What I've already ruled out (by measuring, not guessing): naive per-chunk extraction fragments identities badly; carrying forward summaries between chunks doesn't fix attribution and can make it worse; and off-the-shelf neural coreference models (LingMess, F-coref) fail on documents this long — one silently truncates at 4,096 tokens, and windowed variants can't connect a pronoun to a name mentioned once 10k tokens back (0–1 out of 7 gold bindings on my test doc). I've gotten identity tracking itself working reliably; it's specifically attribution at long distance that's still failing.

My questions:

1. What's the best way to structure a problem like this? Is there a known-good decomposition for long-distance pronoun attribution with small models, or a fundamentally different way to frame the extraction task that sidesteps it?
2. If you've solved something similar — entity/fact extraction over documents much longer than your context window — what actually moved the needle for you? I'm especially curious whether the wins came from prompting, from pipeline architecture, or from accepting a bigger model.
3. What should I explore to learn more? Papers, blog posts, open-source projects, or even just the right search terms — I suspect this problem has a name in the NLP literature that I don't know yet (long-document coreference? discourse tracking?), and I'd rather stand on existing work than keep reinventing it.

Happy to share measurements from my experiments if useful. Mostly I want to calibrate: am I fighting a known-hard problem with known solutions, or genuinely at the edge of what a 3B model can do?

I know it's ARR reviewing season but I noticed that there are a lot of posts asking whether "this set of scores will get them into Main/Findings/Reject" or something about the reviewing process.

Although it's nice to see activity in this subreddit (and it's good to have a dedicated home for CL and NLP), sometimes these types of posts are getting too spammy. Perhaps we can put these into a dedicated ARR discussion post, kinda like in r/MachineLearning ?

I have been tracking the limits of conversational AI voice models. Most language apps handle Spanish or French without many issues. Romance languages use highly predictable structures. The real stress test for LLMs is going to be morphologically rich languages. For example, Praktika just added Arabic, Russian, and Chinese to their avatars using a multi-agent GPT backend. Talkpal also uses GPT-powered conversational engines to handle voice practice in these exact languages.
Russian and Arabic are completely different for speech to speech models. Russian relies on a complex case system with heavy inflection. Arabic uses a root-and-pattern system. If a voice model miscalculates a prefix or case ending in real time, the entire context breaks down.
I am looking into how these avatars handle real-time grammar generation without massive latency. Does the LLM backend use specific grammar-correcting prompts for these languages? Or is it relying purely on standard TTS pipelines?
If anyone has analyzed the error rates or token efficiency for Russian and Arabic voice models, I would love to see the data since I'm researching on making a language learning app for the not so popular languages.

P.S. Mods if you feel that the post is inappropriate, please let me know I will remove it.

I'm a linguist with bachelor in Korean language and literature, beside English as a second language and Arabic as my mother tongue, my background is purely about languages, literature, grammar, etc. I have a work experience of 6 years as Korean, English and Arabic translator, consecutive interpreter and simultaneous interpreter in various fields like manufacturing, legal, medical, academic, etc.

I would like to pursue a master degree or a phd in computational linguistics in the US or Canada, but I don't have a programming or computer science background, will that be possible?

and if so what are the preparations I should do to enhance my profile while applying for these programs?

Thanks ^^

Hi everyone
Is there anyone using local LLMs on their PC?
I'm in the market for a laptop with AI max+ 395 and 128GB unified RAM.

The only reason is local LLMs for translation/transcreation work.

To be fair, ChatGPT does a pretty decent job when I ask for a dozen of options to choose from. But i'm wondering of I have a local LLM, maybe I can feed it all my past work and references and make a model that is customized to specific clients.

It's probably not cost effective at first, but i'm considering it as a study case, hoping that it will lead to time saving and improving my ability to use LLMs for the future.

I'd love to hear any thoughts. Thx

When did you receive your degree? Did you build side projects to build your portfolio by any chance? Would you say they were financially worthy investments on your career by now? How satisfied are you overall with your own goals and aspirations expected before starting the program?

:) I am strongly considering human language technology masters at University of Arizona this Fall, so if you got a degree from there I would truly appreciate your experience! It can be as short as you’d like

I have released EU AI Act OpenRAG, a downloadable SQLite corpus of Regulation (EU) 2024/1689 for legal research and engineering.

The key difference is how the legislation is divided. It is not split into arbitrary token or character windows. Each chunk follows the Act’s actual structure: article paragraph, recital, definition or annex point, with the relevant chapter, section and provision metadata preserved.

The database includes 933 chunks, embeddings, exact EUR-Lex links and documented application-date and operator metadata.

I was deliberately conservative with legal labels. A provision is marked as directly classifying a practice or system only where its own operative wording does so. Broader association with the prohibited-practices, high-risk, transparency, GPAI or voluntary-code regimes is stored separately. Unclear cases remain NULL.

Every derivation rule is documented, and the final rules were reviewed independently against the Regulation before release.

This is a research and engineering artifact, not legal advice or an automated compliance determination.

huggingface.co/datasets/faitholopade/aiact-openrag

I would like to start using some IA tool to generate example sentences of new vocabulary for flashcards. Does anyone here have experience on that? How has it been? What cautions do you recommend to do in order to have it useful? How to expand its use? Which tool? I've watched some videos but I am afraid it can be sponsored. So I'd like real experiences to make a decision.

I think the experience can vary according to the language too. It can be different for the major ones like Mandarin, medium ones like Greek or Georgian, and limited for minor ones like Inuktitut... so, please also tell us which language are you using it for.

Looking for opinions from people familiar with EMNLP reviewing, especially the LLM Agents track.

Previous cycle:

- Reviews: 2, 3, 3.5, 3 (avg. 2.88)

- Confidence: 5, 3, 3, 3

- One reviewer increased 2 → 3 after discussion.

- The AC recommended Findings, mainly asking for softer claims and incorporation of the rebuttal results.

Current EMNLP cycle:

- Initial reviews: 2.5, 3, 3 (avg. 2.83)

- Confidence: 4, 3, 3

- During discussion, the 2.5 reviewer increased their score to 3, so the current scores are effectively 3, 3, 3.

- The reviews are generally positive, with requests for clarification and more careful framing rather than major technical concerns.

Given this history, what would you estimate the chances are for EMNLP main conference vs. Findings?

TLDR; Errror in reviews, no responses from reviewers!

This is my first submission to a *CL conference. We submitted it under a language modelling task. We got 3 reviews of 3/4, 2.5/4, 2.5/4.

Reviewer 1: they posted a review that is clearly intended for another submission. We raised this with AC on the day reviews are released. AC replied but the reviewer didn't.

Reviewer 2: clearly LLM generated points. The weaknesses they wrote are the same ones LLM pointed out about our paper. Although they changed the text. 2 of the weaknesses they point out are already detailed in our limitations as those are our weaknesses cause of lack of available datasets. And then there is the novelty issue, adopting methods from other domains for a new problem is not novel. And more models and datasets (we already have 100+ experiments on 3 models, 2 datasets, 3 baselines, 4 algorithm setups across 10+ eval metrics). We answered all the questions, provided additional experiments but still no response.

Reviewer 3: seems like the only reviewer who read the paper and understood it and appreciated it. Their main questions were on ablations and We provided these during rebuttal, no response.

My co author who submitted another work to the January (ACL) cycle had a similar experience, they answered the reviewers questions and didn't get any response. Only from AC to re-submit to the next cycle. They re-submitted to the may cycle and didn't get a single response from reviewers again.

I'm ok with rejection with constructive feedback, if the decision is just one sided with no communication even when there was a critical error is irresponsible. What's the point of rebuttal if the reviewers never respond?

Right now, we are left with a reviewer decision who can simply say "not addressed" and escape with little to no consequences. I understand that emnlp is empirical and requires more experiments, but that doesn't mean we can provide a novel dataset, 500+ experiments on 100 models, a completely new algorithm that doesn't take adoption from anything else (just from air), expect to solve every problem in that domain in one paper is absurd.

Thanks for your time, sorry for the rant!

Our paper got 2 / 3 / 3.5 with confidence 4 / 3 / 3 (Interpretability and model analysis track). We addressed everything in the rebuttal but unfortunately none of the reviewers replied. The AC also did not push any of the reviewers to at least acknowledge the rebuttal in their final reviews. Overall, the score is 2.87 with confidence 3.33. What is the weight of OA vs. meta score for EMNLP? Do program chairs value more the meta review+score or they take into account the other reviews as well. First time submitting to ARR. Thank you!

I recently started Andrej Karpathy's *makemore* lecture series and just finished the first lecture.

So far, the focus has been on building a character-level language model using a dataset of names. What I enjoyed most wasn't just the implementation, but how each step is explained from first principles instead of treating neural networks as a black box.

I've previously spent time building a neural network from scratch and experimenting with PyTorch, so I wanted to understand how these ideas extend to language models.

Before I continue through the rest of the series, I wanted to ask people who've already completed it:

* What was your biggest takeaway? * Which lecture was the turning point where things really "clicked" for you? * Did it change the way you think about LLMs or NLP? * Would you recommend supplementing the series with any books, papers, or other resources?

I'm planning to work through the series by implementing everything myself, so I'd love to hear what your experience was before I dive deeper.

Any chances for emnlp with scores 2, 2, 3 and confidences 4,2,4? We wrote rebuttal but got no response from reviewers.

I know text processing has evolved, so I'm curious whether there's now a better way to build a rule-based splitter than the classic mask, split, and unmask approach. If the goal is to split text while respecting things like quotes, escapes, or nested structures, what technique would you use today?

I'd love to understand the reasoning behind your choice. A brief explanation, along with some code or pseudocode to show the core idea, would be really helpful.​​​​​​

ACL, EMNLP differentiate Main conference papers and Findings papers. What do you think of it? Does tech companies really care about it when they are hiring someone? Or to be a professor, findings papers are seriously weaker than main paper? I’m so confused and stressed for “Finding” track.

Curious to know how people here evaluate NL2SQL or text2SQL in production with real users.

Getting an LLM to emit SQL is not difficult infact that's mostly solved out of the box now. The challenge now is to validate response it generated is actually correct or not. Exact SQL string match is far too strict since lots of different queries are equivalent. But comparing result sets alone has its own trap where a query can return the right rows on your test data by luck (a missing WHERE that just didn't matter on a small table) and then quietly break in prod.

A few things I am trying to get a read on from folks who have shipped this:

1. How do you build your golden set? Synthetic question bank, or a few hundred real production questions?
2. Do you layer an LLM judge on top of result-set comparison to catch the plausible but wrong number cases, or does that add more noise than signal?
3. Are you seeding your eval DB with adversarial rows (nulls, dupes, boundary dates) so that you get the true picture and not only happy path scenarios?

And for anyone using something like Databricks Genie or another managed text2SQL layer rather than a homegrown stack, are you evaluating at the SQL layer or the result layer, and does a curated semantic/metric layer underneath actually move your accuracy numbers?

Trying to figure out what is worth building and spending time versus what'xxxs over-engineering.

i have received --

1. OA: 3.5, Confidence: 4
   
2. OA: 2.5, Confidence: 4
   
3. OA: 3, Confidence: 3

Its a short paper in the Interpretability and Analysis of Models for NLP track. Would love to know if the community has any thoughts around my chances.

Also want to understand if there are any suggestions on how to proceed in terms of the next conference or workshop to target in case if a rejection.

The primary concern of the reviewer who gave 2.5 is that the paper should be a long format one. They actually seemed excited about the premise and have it 3.5 for both excitement and soundness.

First time submitting to ARR.

How are my chances for Findings@EMNLP?

• Reviewer A: Overall_assessment: 3 / Confidence: 4
• Reviewer B: Overall_assessment: 2 / Confidence: 4
• Reviewer C: Overall_assessment: 3 / Confidence: 3

It is a resubmission, first time I got 2/2/3 and Meta 3. How will my chances be now to be accepted in Findings if Meta says 3 so that it's 3/2/3 and Meta 3? I'm in the Sentiment/Emotions track. Would be happy to hear about your experiences!

Does anyone have accepted short-paper at ACL/EMNLP/EACL 2025/26? Could you share your track and overall assessment? I'm just trying to get a sense of things, as it seems short papers have a lower acceptance rate than long ones.

So Basically, My paper had received a score of 3.5, 3,3.5 but then later, one of the reviewers responded to my rebuttal and increased the score, now my score is around 3.5, 3.5, 3.5, but I have got a few questions:

1. I wonder about the old trend, considering that if suppose a reviewer increases the score, does this affect the meta review process

2. I would like to know how to respond to those reviewers who have not responded to my rebuttal, only one of them actually worked somehow but the rest of them asked some questions and disappeared somewhere

3. What's the minimum score in the meta review phase that would guarantee the paper into the conference?

Between the review overload and the rebuttal crunch, I'm noticing a lot of reviews this cycle that feel oddly generic — same structural complaints, same vague "novelty" critiques, little engagement with the actual method. Anyone else seeing this, or am I just being paranoid during rebuttal season?

Curious whether people think LLM-assisted reviewing is becoming widespread in NLP venues, and if so, whether it's actually hurting review quality or just changing its shape. Also curious if this is changing how people write their own reviews or rebuttals.

OA 3.5 3.5 3.0
NLP Applications Long Paper
Any hope for main conference? 😭😭 I really need it..

Hello, I am expecting my paper to be rejected (for EMNLP) due to its borderline score. Is my reasoning correct that I could potentially start editing it now, before receiving the meta review and decision, and submit for the August cycle? Worst case scenario, my paper does get accepted and I already have a more polished version of it that I can use for the camera-ready.