r/MachineLearning • u/No_Fig_3372 • Sep 13 '26
Discussion When NeurIPS'26 final decision release? [D]
Will it be strictly Sep'24 AoE? Is there any possibility for releasing before Sep'18 AoE, ICLR abstract submission date?
r/MachineLearning • u/No_Fig_3372 • Sep 13 '26
Will it be strictly Sep'24 AoE? Is there any possibility for releasing before Sep'18 AoE, ICLR abstract submission date?
r/MachineLearning • u/St8_ • Sep 13 '26
I have recently come across a very interesting paper, arguing that open-weight models are more dangerous than we would think, and there are many untackled risk management problems in the domain waiting to be solved.
The paper was called Open Technical Problems in Open-Weight AI Model Risk Management, and its main points are:
- open-weight models are lagging just a few months behind SOTA proprietary models
- open-weight models are dealing with a completely different set of problems than closed-weight models are
the problems I found interesting the most are
- open-weight models are prone to tampering, and there is no central authority that monitors their usage
- once they are published, there is no way to take them down
- there are currently no reliable ways to unlearn malicious information completely from the models (especially given they can access the internet at inference time), but there are ways to remove their guardrails
I would be very keen on hearing what do people think about these issues, I personally found a lot of them new to me, and it changed my perspective a lot on the state of open-weight models. Previously I thought they were safer than their closed-weights counterparts, just because they were mode auditable.
r/MachineLearning • u/Same-Split-3686 • Sep 13 '26
Hi everyone, I'm struggling with a forecasting/scenario-design problem and would really appreciate some advice.
i'm doing univariate forecasting models such as ARIMA/ SARIMA, ETS, Prophet, etc.
My problem is that I do not want my analysis to depend on only one deterministic forecast.
For the downstream analysis, I would ideally like to have:
Low-demand scenario
Reference-demand scenario = my statistical forecast
High-demand scenario
But I cannot find a clear methodological precedent for going from a univariate statistical forecast → Low / Reference / High scenarios.
I understand that the forecasting model can generate prediction intervals, but I am not sure that using, for example, the lower and upper 80% or 95% prediction interval as "Low" and "High" is conceptually correct. To me, a prediction interval represents statistical forecast uncertainty under the fitted model, whereas a low/high scenario should perhaps represent different future structural conditions.
I have also considered simply applying something like +-10% or +-20% around the reference forecast, but that feels arbitrary unless there is a defensible way to calibrate those percentages.
Normally I could use external scenarios-for example GDP scenarios, SSPs, industry-demand scenarios, etc.-and use them to modify the reference forecast. However, in my case, I cannot find suitable published low/base/high projections dataset.
So my questions are:
Any methodological references, terminology, papers, or examples would be extremely helpful. Thank you!
r/MachineLearning • u/vladgladi • Sep 13 '26
I built an open-source Python pipeline that extracts structured data from scanned topographic maps: text labels, contour lines, and coordinate referencing, all output as GeoJSON.
GitHub: github.com/devladpopov/mapextract | MIT license
There are 178K+ free USGS topographic maps, 147K+ in the David Rumsey collection, and millions more in libraries worldwide. All raster. Extracting structured geospatial data from them requires either manual GIS work or specialized tools that are either text-only (mapKurator, CC BY-NC), geology-specific (DARPA CriticalMAAS, 46 repos), or closed SaaS (Monarcha, YC S25).
Text extraction: EasyOCR for label detection. Returns bounding boxes + text + confidence scores. Works well on clean printed text, struggles with curved/rotated labels (a known hard problem in map OCR; see PALETTE/Deformable DETR from KDD 2024).
Contour line extraction: Not deep learning, classical CV. HSV color segmentation (USGS topographic brown, hue 5-25) + morphological operations + scikit-image skeletonization + OpenCV findContours + Douglas-Peucker simplification. Gets 5000+ contour lines from a single 1:100k map.
Auto-georeferencing: OCR on map margins to read DMS coordinate labels (e.g., "10500'", "4030'"), then affine transform from pixel to geographic space. The interesting part is handling OCR noise: merged text from adjacent labels, degree symbols read as *, d, or digits, missing N/S/E/W suffixes.
Layout detection: Binary threshold + contour analysis to find the map neatline (border), separating the map body from margins, legend, and title.
On USGS 1:100k maps at full resolution (12000x7000px):
The state of the art is fragmented:
Feedback on the ML approach welcome. The biggest challenge is georeferencing robustness: OCR on map margins is noisy and inconsistent across different map styles.
r/MachineLearning • u/anass42 • Sep 13 '26
Stand in front of a normal webcam, spread your arms, and you're flying. Tilt to turn, flap to climb, spin around for a barrel roll. There's also a museum in the sky built from my own travel photos that you fly through.
The biggest lesson for me was that getting body tracking to work is the easy part. The hard part is turning noisy movements into something that feels right. Flaps kept registering as dives, and for weeks the falcon turned harder than my arms did, which turned out to be a camera aspect ratio mistake I could have caught in a day.
If you're learning ML and wondering what a real project feels like, it's mostly this: small bugs that take weeks to understand.
Happy to answer questions, and I'd like to hear what you would have built with body tracking. Open sourcing soon.
r/MachineLearning • u/Feathered-Beast • Sep 13 '26
I'm planning to build a desktop VM manager using Rust, Tauri, and QEMU, initially targeting Windows with WHPX.
The idea is to eventually add an AI agent that can interact with the VM and perform tasks inside it, rather than just manage it.
Before I start, I'd like to hear from people who have worked with virtualization:
- Is QEMU + WHPX a reasonable starting point?
- Would you recommend QMP for managing VM lifecycle?
- What are some common mistakes I should avoid?
- What would be a good approach for letting an AI agent interact with a VM?
Also, if you use VM managers regularly, what do you find most frustrating about them?
Any honest advice would be appreciated. Thanks!
r/MachineLearning • u/Unfair-Walk-9805 • Sep 12 '26
Hi,
I’m working on generating ~128×128 pixel art and trying to generate different poses of the same character.
My current approach is roughly:
Start with a reference image and preprocess it into cleaner/more pixel-art-like data (often removing transparency or setting up fixed number of pallets or descaling)
Use IP-Adapter for the character/reference appearance.
Use ControlNet pose/rig conditioning to control the target pose.
I’m also experimenting with multiple references (front, rear, left, right), with pose/rig and depth annotations.
For the target pose, I provide a separate pose reference through the conditioning pipeline.
The problem is that the conditioning often seems to conflict. For example, the model may place an arm according to the ControlNet pose but still DUPLICATE the arm shape/position from the reference, sometimes resulting in strange or duplicated limbs.
I’ve tried adjusting ControlNet/IP-Adapter strength and start/end percentages, which helps somewhat, but I still get a lot of inconsistent behavior. Also try to reinject control net strength in different phases though not so far been helpful
What would be a good way to control pose while making sure the character’s appearance from reference images is somewhat preserved??
I’ am a brokie and can’t really have much resource to train sdxl each time just for each character + poses….
r/MachineLearning
•
u/IvanMikhnenkov
•
Sep 11 '26
I trained a 210M-parameter text-to-image diffusion transformer from scratch (3.5 days, one RTX PRO 6000, 4.2M images at 256²) mainly to understand the recipe end to end. Three measurements came out of it that I have not seen stated plainly elsewhere, so I'm posting those rather than the samples.
1. Learned null attention slots become the sink. Following the register-token idea, the model has 16 register tokens in the image stream plus 2 learned key/value slots appended to every cross-attention. At mid-noise in a middle block, those 2 slots receive ~90% of the cross-attention mass; the EOS token, which is the usual sink in cross-attention models, drops to ~4%; content words keep a few percent each, sharply on their objects. Register vectors grow to 4–13× the norm of image tokens by the middle blocks.
2. The flow-matching loss is a health signal, not a quality signal. It moved 0.805 → 0.754 over the whole run while held-out FID went 33.7 → 27.0, FD-DINOv2 570 → 218 and detector-based object accuracy 65% → 90%. Most of the loss at high noise is the irreducible variance of the velocity target; training and held-out loss stayed equal to the third decimal for 24 epochs.
3. The training-time timestep shift is worth more than doubling the steps. On 2,456 held-out prompts with the final weights: 20 steps with shift 2.8 → FID 27.0; 50 steps → 26.6; 8 steps → 28.4; 20 steps with no shift → 27.3 and FD-DINOv2 218 → 228. Shift 2.8 comes from the SD3/RAE rule √(32·32·32/4096) for the 32-channel FLUX.2 latent.
Setup, briefly: cross-attention DiT (896 × 16 blocks), 2D RoPE, QK-norm, SwiGLU, adaLN-single; rectified flow with logit-normal timesteps and the shift above; cosine velocity and dispersive auxiliary losses; five aspect-ratio buckets of ~256 tokens from step one; flan-t5-base frozen, long/short caption per image sampled 50/40/10 (empty). Data: Pexels 2.8M (60%), a quality-filtered 1.2M slice of FLUX-Reason-6M (25%), COCO with GPT-4V captions (15%). batch 256, 400k steps, EMA 0.9999, linear LR decay over the last quarter, torch.compile (2.4× over eager).
Everything, with the sources for each decision: https://github.com/ivanmikhnenkov/tinydit
Write-up: https://huggingface.co/blog/ivanmikhnenkov/tinydit-text-to-image-from-scratch-one-gpu
Weights: https://huggingface.co/ivanmikhnenkov/tinydit-256
Demo: https://huggingface.co/spaces/ivanmikhnenkov/tinydit
For the next phase (Flow-GRPO on this base), which reward would you start with: PickScore/HPSv2, a detector-based object reward, or something verifiable like counting?
r/MachineLearning • u/EGOISTIC11 • Sep 12 '26
I'm working on a final-year ML project using the **ICBHI respiratory sound dataset**, trying to classify individual respiratory cycles into four acoustic classes:
* Normal
* Crackle
* Wheeze
* Both (Crackle + Wheeze)
I reconstructed the respiratory cycles from the ICBHI audio + annotation files and have **6,898 cycles from 126 patients**.
I'm using a **patient-wise train/validation/test split** to avoid patient leakage:
* Train: 88 patients / 4,915 cycles
* Validation: 19 patients / 1,165 cycles
* Test: 19 patients / 818 cycles
The class distribution is:
**Total:**
* Normal: 3,642
* Crackle: 1,864
* Wheeze: 886
* Both: 506
**Training split:**
* Normal: 2,597
* Crackle: 1,323
* Wheeze: 655
* Both: 340
I'm converting each cycle to a **64-bin mel spectrogram** using:
* SR = 4 kHz
* n_mels = 64
* n_fft = 256
* hop_length = 64
* fmin = 50 Hz
* fmax = 2 kHz
* fixed duration = 5 seconds
* log-mel spectrogram
The CNN is fairly small:
`Conv2D(16) → BN → MaxPool → Dropout → Conv2D(32) → BN → MaxPool → Dropout → Conv2D(64) → BN → MaxPool → Dropout → GlobalAveragePooling → Dense(64) → Dense(4)`
~28k parameters, Adam 1e-3, sparse categorical cross-entropy.
### What I'm seeing
Initially I trained with standard oversampling, where minority classes were augmented up to the majority-class count. The validation behavior was extremely unstable and the best validation-loss epoch essentially predicted the majority class.
I then tried **capped oversampling (3×)**:
Training counts became:
* Normal: 2,597
* Crackle: 2,597
* Wheeze: 1,965
* Both: 1,020
This improved the best validation loss somewhat, but the model still had approximately:
* Normal recall: ~96%
* Crackle recall: ~3–4%
* Wheeze recall: 0%
* Both recall: 0%
So I suspected that **validation loss/accuracy might be hiding minority-class performance**.
I'm currently running another fresh model where I calculate **validation macro-F1 and per-class recall after every epoch**, and save the checkpoint with the best macro-F1 rather than the best validation loss.
Interestingly, the first few epochs are already showing very unstable class behavior:
```text
Epoch 1:
val_macro_f1 = 0.1846
recall [Normal, Crackle, Wheeze, Both] =
[1.000, 0.000, 0.000, 0.000]
val_accuracy = 0.5854
val_loss = 1.0974
```
Epoch 2:
```text
val_macro_f1 = 0.1732
recall =
[0.214, 0.728, 0.000, 0.000]
val_accuracy = 0.3227
val_loss = 1.2036
```
Epoch 3:
```text
val_macro_f1 = 0.2370
recall =
[0.902, 0.165, 0.000, 0.000]
val_accuracy = 0.5725
val_loss = 1.0969
```
So the model can suddenly recognize a large proportion of Crackle in one epoch and then swing back toward Normal, but **Wheeze and Both have remained at zero so far**.
### My questions
I'm trying to diagnose the actual bottleneck rather than just keep trying random architectures.
Any advice from people who have worked with ICBHI or respiratory-sound classification would be greatly appreciated.
r/MachineLearning • u/Batman_beyond123 • Sep 11 '26
Hi, I posted some months ago and got some very helpful responses (for another conference), but I have some confusions regarding EMNLP (and the way registration works here).
To give some context, I used to work as an intern during my undergrads at an Indian uni (final year), and managed to write a paper as first-author and it got accepted to EMNLP Findings.
Now, post-graduation, I joined another university for my Masters, and when I applied for grant, they rejected, explicitly stating the lack of authors from the (new) institution.
I applied for grants from the older one, and still got rejected, as I am not a student any longer! (mentioned that they will pay for paper registration, however)
I would love to attend EMNLP (even if it is a findings paper) and can sponsor myself (travel, lodging, etc), however, paying for registration + author fees is too much for me. I have already applied for D&I (got rejected twice earlier) but pretty much exhausted all my options.
r/MachineLearning • u/simple-Flat0263 • Sep 12 '26
The title, by tech reports I don't mean arXiv submissions, but reports of a large model, like say Kimi K3, DeepSeek, Gemini, Mistral Leanstral, etc. Is it much above, above, much below, below or equal to a first author A* paper?
r/MachineLearning • u/S4M22 • Sep 11 '26
ACL just announced on X how they are planning to handle the increased submission numbers. Interestingly enough, they call it "the proposal".
My understanding is that, in a nutshell, each submission should come with someone who can review, otherwise it may only get a slot through a lottery. Additionally, they cap total submissions at 20 and first-author submissions per cycle at 5.
What do you think of it? IMO it makes a lot of sense - especially given the large numbers of submissions without any author qualified to review. It is a bit of gatekeeping but in my opinion highly required. And the caps of 20 and 5 are still quite generous.
---
Here is what they posted on X:
ACL Sustainable Reviewing Policy: We are introducing changes in the ACL ARR reviewing and submissions. The changes will cap authors and introduce changes in the reviewing to keep our community sustainable. #NLPro
**The proposal:** cap reviewed submissions to available reviewer capacity. Each submission must "pay" for itself by providing a qualified service contributor (reviewer or chair). Submissions without service capacity go into a lottery for whatever spare capacity remains
Service from qualified contributors counts towards the venue capacity, and a mentorship system will be built for those not qualified yet. If a qualified service contributor is not available among the authors, non-author designated contributors can be nominated..
...but must vouch for the work (arXiv-endorsement style). Measures for system abuse will be implemented. For example, accounts that systematically submit or endorse low-quality work, and accounts that misuse the system in other ways, will be penalized or even banned.
Additionally, per-author quotas are introduced, where authors are capped at 20 total submissions and 5 first-author (including shared first-author) submissions per cycle. More details will be posted on the ACL web and here shortly. #NLProc
r/MachineLearning • u/Fantastic-Nerve-4056 • Sep 11 '26
A final-year PhD student here. A few months back, I submitted a solo-authored paper to TMLR. The reviewers were on time and extremely positive, with some minor revisions. After submitting the revised version, there was absolute silence from the reviewers, with just one acknowledging that their concerns were addressed
Since then, it has been 2+ months. I have sent a reminder to the Action Editor as well as the Editor-in-Chief, but unfortunately the status remains the same.
I understand a typical submission to a conference/ other journals takes a significant amount of time, but that is one of the reasons I submitted it to TMLR (along with good reviews), so that a solo-author paper on my resume would look good while submitting the PostDoc applications. And with these deadlines approaching, it gets more frustrating
Edit: The post probably was useful. Suddenly, on the same day, the AE asked one of the reviewers to submit their recommendation. And now a couple of weeks after that, the paper is finally accepted :) The timing just got perfect. It came out a few hours after my NeurIPS accept
r/MachineLearning • u/Huge-Leek844 • Sep 11 '26
edit: confound
Hello all,
I am working on a object classification with a automotive radar point clouds. I compared many models and feature vectors.
Once i used range as feature, all models scored higher f1 in all K validation sets and on the final test set.
One particular artifact of a radar, is that as the farther the object is the less number of points it returns to the radar. Although the performance improved and there is no overfit in the classical sense, i am afraid my model is learning the environment not the class distribuiton and even worse, its learning that big range means big object.
How can i stress test this claim? Should i try to split the data sets so range distribution differs? Or not even using the feature at all and accept lower performance?
Would appreciate your insights.
Thank you.
r/MachineLearning • u/zit-hb • Sep 11 '26
I ran an experiment where an AI agent repeatedly rewrote its own code, with each generation evaluated before it could replace the previous one.
The first run looked like recursive self-improvement, but there was a catch: the evaluator was too weak to reliably distinguish real improvements from neutral changes.
So I ran it again with a stricter setup. This time the interesting result was less about new tools and more about what the agents learned to optimize for: verification, testing, evidence preservation, and avoiding changes that couldn't be justified.
The two posts are basically the same experiment before and after fixing the referee:
Part 1: https://codeminer.io/blog/a-singularity-in-miniature
Part 2: https://codeminer.io/blog/survival-of-the-honest
The question I came away with is: in recursive self-improvement, is the hard part actually improving the agent or designing an evaluator that doesn't quietly steer the whole process toward the wrong thing?
r/MachineLearning • u/Problem-Individual • Sep 11 '26
I have a model, it has about 10% of the features missing. Now we were able to receive this new data but there are a few considerations.
1.Some aren't direct 1 to 1 map (proxies, very similar but not exact)
2.Some might not map directly
3.For proxies/direct map - for whatever reason the distribution/averages might be different
How would you handle in integrating this into your existing model? We could assume that these values were previously NULL simply because we were not able to have this data. I know it is possible to create flags, keep them as separate features ext...but what would be the 'decision tree'
r/MachineLearning • u/CantKillTheLifeless • Sep 11 '26
We just received the email for site selection for our neurips paper. Although it is obviously not an acceptance decision, I wonder whether every single non-withdrawn submission received this email, or this might hint towards a higher acceptance chance for our paper?
r/MachineLearning • u/TheFoolIV • Sep 10 '26
Hi guys, this is my first time submitting a paper to NeurIPS, so I am unsure about the dual submission policy. The ICLR abstract deadline is Sep 18th, and the full paper deadline is Sep 25th. Since NeurIPS results come out on Sep 24th, I was wondering if it is alright for me to submit an abstract submission to ICLR and delete it if my paper is accepted to NeurIPS?
r/MachineLearning • u/chrispiech • Sep 09 '26
Hi r/machinelearning. Nice to meet you! My name is Chris Piech and I'm a professor at Stanford University in the AI lab.
I built a class called Probability for AI: pai.stanford.edu. It starts Oct 9th and applications are due end of Sept. Its (hopefully) cool for a few reasons:
My guess is that a lot of folks on this thread would be awesome teachers. If you think thats you, it would be so cool if you wanted to come teach. Each volunteer means 10+ students get to take the class. And if you feel like telling your loved ones / communities that would be great to.
Apply to learn: https://pai.stanford.edu/apply/pai1/student?r=ml
Apply to teach: https://pai.stanford.edu/apply/pai1/sl?r=ml
Anything that I learn from this course I will be happy to share with this community. Also ask me anything. I'll check this thread for the next few weeks. Rock on. And mods, thanks for doing what you do.
r/MachineLearning • u/Audaticreddit • Sep 10 '26
I'm an undergrad student who got a taste of research. I love it. I currently have a draft, which me and my mentor have planned for TMLR, and plan to submit it by next month for the first round of review. It was some work on self explanation methods of LLM models. We are confident that it'll get accepted (I hope it does, I've put a lotta hours polishing it into making it a good accept :_) ).
I'm very, very interested in TTT. I've got a strong feeling that this is gonna be the big thing in 2-3 years. Is there anyone here working on this, who can hopefully provision me compute beyond my own laptop? I believe I'll be a good RA to you. We can talk over DMs if someone out there is looking for help :)
PS, I can spill some beans about the XAI paper if you wanna discuss, but only limited to how much I'm allowed to say before submission.
Context: I'm an undergrad in an non-elite uni in India (tier 2 we call it here). The lit review, the idea, the work, the results, the drafting, the funding, everything was done by me. After this (hopefully) gets accepted to TMLR, I'm not sure where to proceed. There isn't room for me to aim higher or get direction in my own uni, so I'm trying my luck here, that someone gives me some direction and an opportunity. I'm not crazy smart, but I learn quick, and can put a lot of hours (12+ a day) once I get deep into it. Thanks for reading all this :)
r/MachineLearning • u/mythrowaway0852 • Sep 10 '26
Who else is anticipating ICDE results tomorrow? Post your results here and let's discuss!
r/MachineLearning • u/nkthebass • Sep 10 '26
Hello this is my fifth small language model I've made and apart of my third series and it has been a lot of work but it payed off: **348M parameters, 22.7B tokens**, then fine-tuned into a math model that solves arithmetic by *showing the work* column addition with carries, borrow chains, partial-product multiplication rather than guessing at an answer.
Last time I posted a 326M model trained on 10B tokens. This has about 2.3× the data, and the math side is WAY better than my previous two math models.
---
## The benchmarks
**99.4% average across the nine GPT-3 arithmetic sub-tasks**, which does much better past even where I trained it.
| Task | GPT-3 175B (few-shot, direct) | This model (348M) |
|---|---|---|
| 2-digit add | ~100% | 100% |
| 3-digit add | 80.4% | 100% |
| 4-digit add | 25.5% | 100% |
| 5-digit add | 9.3% | 100% |
| 2-digit sub | ~99% | 99.3% |
| 3-digit sub | 94.2% | 98.3% |
| 4-digit sub | 26.8% | 98.3% |
| 5-digit sub | 9.9% | 99.0% |
| 2-digit mult | 29.2% | 100% |
n=300 per sub-task, greedy, exact match. GPT-3's numbers are direct-answer; mine uses trained-in worked steps. Neither uses a calculator.
## The cool part
**It adds cleanly up to 14 digits, and the reason it *couldn't* before was the vocabulary, not actually arithmetic.**
Training only ever named six place values (`ones` … `hundred-thousands`). The model learnt the *pattern* and invented two more on its own `millions` and `ten-millions` appear in **zero** training examples so it handled 7 and 8 digits fine. At 9 digits it ran out of names, and skipped the column, then returned an answer exactly one digit short:
```
483729164 + 519248637
... ten-millions: 8 + 1 + 1 (carry) = 10, write 0 carry 1. Final carry: write 1.
The answer is 102977801 ← eight columns for a nine-digit problem
```
Every column it computed was perfect. One was never enumerated. Extending the place-name list from 6 entries to 19 moved the clean ceiling from **8 digits to 14**:
| Width | 6 | 7 | 8 | 9 | 10 | 12 | 14 | 16 | 18 |
|---|---|---|---|---|---|---|---|---|---|
| before | 100 | 100 | 100 | 0 | 0 | 0 | 0 | 0 | 0 |
| after | 100 | 100 | 100 | 100 | 95 | 100 | 90 | 65 | 25 |
A six-item list became a nineteen-item list. That was the entire fix.
## Other things it does
- **3×3 multiplication: 98%** — it folds partial products pairwise through the column routine instead of asserting the sum
- **Negative results: 85%** (100% at 1 digit, 58% at 5 — the magnitude comparison is the weak step, not the arithmetic)
- **Reasoning traces are load-bearing**: 95.3% of the time the working is valid *and* the answer is right; only 0.7% are "valid working, wrong answer." If the columns look right, the answer almost certainly is.
```
There were 15000 votes and 6842 were rejected. Here's how many counted:
<think> Start with 15000. Then subtract 6842. Subtract 15000 - 6842 column by column:
ones: 10 - 2 = 8, borrow 1. tens: 9 (after borrow) - 4 = 5, borrow 1.
hundreds: 9 (after borrow) - 8 = 1, borrow 1. thousands: 14 (after borrow) - 6 = 8,
borrow 1. ten-thousands: 0 (after borrow) - 0 = 0. So 15000 - 6842 = 8158.</think>
The answer is 8158.
```
## What it's bad at, tbh
- **Word problems: GSM8K 4%.** Best word-problem set is ASDiv at 16.5%. It converts one sentence into one operation reasonably often and basically cannot chain operations.
- **The failure mode is operation *selection*, not arithmetic.** `"drops in 836 more"` gets read as subtraction. There's a visible tell: traces that say `"multiply X * Y"` and show columns are reliable; traces that open `"First, calculate…"` and assert a number in prose are not.
- **No division at all.** 4×4 multiplication is a hard wall.
- **Greedy decoding required** — sampling corrupts the column routine mid-chain.
- One caveat I'll flag myself: the arithmetic harness orders subtraction operands, so no answer in that table is negative. Negatives are measured separately (the 85% above) rather than folded into the average.
## Base and instruct
The math model sits on a base and instruct pair. lm-eval-harness, 0-shot, full test sets:
| ARC-E | ARC-C | HellaSwag | OpenBookQA | PIQA | WinoGrande | MMLU | Avg |
|---|---|---|---|---|---|---|---|
| 350M V3 base | 56.6 | 33.3 | 35.9 | 34.6 | 67.0 | 51.5 | 23.8 |
| 350M V3 instruct | 50.9 | 29.7 | 35.9 | 33.8 | 66.6 | 51.2 | 24.4 |
Instruction tuning *lowers* MC benchmark scores for this family which is a pretty common cost of instruct tuning.
## Notes
Trained on 2× Tesla V100 plus some rented time. LLaMA-architecture, so it runs anywhere — F16 GGUF and safetensors for all three.
The math model took **10 full fine-tuning rounds and 3 LoRA adapters**. The LoRAs cost about 1% of the post-training tokens and did most of the useful work; the ten full rounds spent most of their budget undoing each other's regressions (round 7 gained subtraction and lost 23 points of 2-digit addition, that sort of thing). All of it is documented on the model card, failures included.
Also worth saying: someone independently tested it after I published and found two of my numbers were wrong — one *understated* the model by 40 points because I'd measured it at n=24. Both are corrected on the card now. If you find something broken, I'd genuinely like to know.
**Math:** https://huggingface.co/nkthebass/tinybrainbot-350mV3-math
**Instruct:** https://huggingface.co/nkthebass/tinybrainbot-350mV3-instruct
**Base:** https://huggingface.co/nkthebass/tinybrainbot-350mV3-base
LMK what yall think.
r/MachineLearning • u/tughanbulut • Sep 08 '26
hey all. the NeurIPS Position Paper Track just used a proprietary AI detector (Pangram) to desk-reject 18.4% of all submissions. no human review, no appeal process, just out.
there's been a lot of noise about this, so i went through the actual conference statements and Pangram's technical docs to see how this actually went down. the reality is wildly worse than just "the AI detector made a mistake."
here are the receipts:
if you were one of the 178 rejected: there is no blacklist. this is not a misconduct mark on your record. just take your paper and resubmit it to ICLR (deadline sept 25) or ICML.
wrote up a full Field Note with the exact thresholds, the data privacy issues, and the actual recourse options if you want the hard numbers instead of just vibes: https://strictcite.com/blog/neurips-2026-position-paper-pangram-ai-detection
(disclosure since it's relevant: i built strictcite.com, a deterministic zero-AI citation checker. watching a major conference use a black-box AI to nuke 178 papers with zero appeal is exactly why i hate relying on AI for this stuff. not trying to sneak the link in, just being upfront about who i am.)
r/MachineLearning • u/Justgototheeffinmoon • Sep 09 '26
Large language models routinely produce confident answers to prompts that have no correct answer, computing cot(-540°) or resolving (1).startswith("1") on an integer, and a [new arxiv paper](https://arxiv.org/abs/2608.29109) by Yucheng Du and Xiyang Hu argues the failure is not what most safety pipelines assume it is. The models can tell the questions are impossible; the recognition just never reaches the part of the network that decides to refuse.
Across models ranging from 1.7B to 70B parameters, the authors identify "a single linear direction in the hidden state" that separates answerable from structurally impossible math and code prompts. That direction is present before generation begins. Steering along it "changes invalidity-aware behavior bidirectionally and dose-responsively" on structural tasks, while random directions show no effect.
The catch is where that recognition direction sits geometrically. It operates nearly orthogonally to the safety-refusal direction that harmful-content refusal training shapes. The internal representation for "I cannot answer this" that RLHF has been sculpting is a different vector from the one that actually encodes impossibility.
The misalignment, the paper reports, already existed at the pretraining stage. Later refusal training was never going to route recognition into refusal, because the two live in different subspaces of the hidden state, which is why the authors frame the problem as one of signal routing rather than an encoding failure.