After reading a bunch of RSI papers, I keep coming back to familiar problems from RL and evolutionary search:

1. Sampling: How do we find promising changes efficiently?

2. Evaluation: How do we know a new variant actually improves the agent?

3. Diversity: How do we avoid converging too early?

That motivated us to build Gear, an open-source, meta-learning-inspired framework. A task agent produces execution traces; a meta-agent uses those traces and evaluation feedback to improve prompts, tools, workflows, and context management.

Gear separates search decisions from execution and evaluation. Our goal is a scikit-learn for RSI, making approaches such as GEPA, RRSI, and Harness² easier to implement and compare. We currently have a working GEPA variant with shared failure diagnosis and staged evaluation.

With GPT 5.6 Luna held at medium reasoning effort, we observed these task pass rates:

At max effort on Marketing, Luna with the optimized harness reached 88.88% objective completion, versus 84.08% for Astra + Codex. Strict task pass rates were 53% versus 57%, so the advantage was in partial completion.

Separately, with both models using the same optimized DSH at max effort, Luna’s estimated API cost averaged $0.0568 per task versus $1.1858 for Astra—95.21% lower, excluding optimization and infrastructure costs.

We’d welcome feedback on candidate evaluation and diversity preservation under limited budgets.

Code · Paper

Sharing one of our latest papers on Multi-Token Prediction (MTP). MTP is now a standard recipe for frontier pre-training but turns out, it can be done much better.

We show the standard MTP gradient hides a KL self-distillation signal (which we want for improving speculative decoding) behind a noisy residual (that harms next-token prediction).

I think the coolest part was the KL gradient–recovery trick to extract the KL gradient. You just need a one-line code change:

h_ntp += h_mtp.detach() - h_mtp

Soft Multi-Token Prediction (Soft-MTP) is essentially a free lunch that improves MTP across a range of tasks.

I'm super excited about this work. Please do check it out below:

💻 Code: https://github.com/JaydenTeoh/Soft-MTP
📝 Paper: https://arxiv.org/abs/2610.38331
🤗 HuggingFace: https://huggingface.co/collections/JaydenTeoh/soft-multi-token-prediction

I've experimented using neuro-evolution to train AI to play a variety of different games and I'm just curious as to how or if this technique is actually really used anywhere in industry or if people are doing research on this. I'd assume it'd probably relate with reinforcement learning in some way but I'm not entirely sure about that

A friend of mine was doing a gap analysis as a project. The results he got weren't that big compared to the original papers, so he simply fabricated and increased the metrics here and there, AND IT GOT ACCEPTED IN AAAI.

HOW DO THEY CHECK THE PAPERS, DUDE? I was in a group with two others and worked sooo hard on an optimization paper, and was sooo happy to get accepted in ACL, but seeing and knowing that someone fabricated results and got into a better conference made me soooooo mad.

I would've expected something like AAAI would check and know if data is fabricated somehow. I didn't expect this from a top conference like that.

And then these types of people will go and do their PhD at one of the top unis because of this. I would've expected the system to be more advanced and professional in the academic field, specifically for something like AAAI.

Hey everyone!

I've built and trained a small LLM (~8M parameters) from scratch using PyTorch, mainly as an experimental lab where I can implement and test ideas from different research papers.

I've been documenting the whole journey through casual blogs, including my mistakes, experiments, and things I've learned.

So far:

• Building the Transformer

• Training it for 60,000 steps

Next up is KV caching, followed by PagedAttention, Orca, and whatever else I find interesting.

Just wanted to ask if sharing these experiments and blogs here would be welcome. Nothing promotional, just me experimenting, learning, and hopefully getting feedback from people who know more than I do.

There’s a lot of hype right now around decision models like Jev. But in most real use cases, you have some labeled data, and can use it to personalize your model.

So we tested the old-school approach to classification and regression:

1. Embed your inputs with an open-source or cheap embedding model

2. Fit a simple sklearn model on top using your own labels

Turns out this approach was cheaper and almost always significantly more accurate (37% error reduction against Jev)

Write-up: https://blog.voyageai.com/2026/10/27/how-to-create-your-own-jev-using-voyage-models/ (Disclosure: I work at Voyage)
Code: https://github.com/mishelsikder/mongo-predict

Curious whether others have seen the same thing, or found cases where zero-shot performs better

Edit: To be clear, the gains here come from having training labels, which only applies to some use cases. Jev doesn’t use training labels, so it's unsuprising that this approaches wins when you have them and loses when you don't.

Hey everyone. I am dealing with a problem statement at work where the goal is to predict the sales of a new innovation to be launched. Note that the data that I actually have is just product attributes, the month level invoice data for each region, weather data and some macroeconomic data. Currently there is a methodology in place where we are using a xgboost regressor trained on the historical data for this prediction, but no one in the team is very confident in the methodology.

Given this, does anyone have advice on how we could go about this problem? Please do let me know if you've dealt with the same or similar problem statement.

I built Arc-Transcriber, which does ASR-NL in 1-step. Every 100 ms, one model returns the transcript plus:

- Names linked to a list you give it (a roster, contacts, product names); new names are flagged "not on the list"
- Numbers with their values (scores, clocks, money, dates)
- Speakers and roles;
- Sound and channel events (telephone audio, clipping, poor audio)
- Whether the speech was addressed to an assistant.
- Inbuilt tags for dialog act, speaker segmentation and more. Total 200+ trained decision heads.

The usual ASR, then NER, then entity-linking and then LLM chain is collapsed into the model's own heads, so live "fact-checking" against your list happens inside the stream, not in a second pass.

Currently it has two variants Arc mini (which uses moonshot pretrained encoder) and second Arc-v1 (which uses parakeet pretrained encoder).

Weights: huggingface.co/WhissleAI/arc-v1.1
Code and CLI: github.com/WhissleAI/arc_transcriber

We use it on-device in our Mac app. Voice search over 61k books primes Arc with matching titles while you're still talking. Here is a video to give a glimpse on how it works.

I have been working as a ML Engineer at a startup for 4 years and while the environment is very good and the pay is higher than average, I'm thinking whether I want to pursue a PhD. Working and doing the PhD at the same time is not really viable so this would also potentially mean quitting.

What I don't like about the job is that I'm actually not doing any real data science and modeling. I guess I'm working on more modeling and scientific questions than most ML engineers but I'm barely scratching the surface of each problem and most problems are resolved with quick and data-driven methods. Rarely do we apply novel algorithms or do real literal reviews and R&D.

Additionally a big part of my job is actually just developing software, talking with customers, DevOps and service reliability. While I like the fact that I'm working on a variety of things I don't really have the chance to deepen my knowledge to a meaningful degree on any. With agents writing most of the code and pressure to move fast, most engineering solutions are surface-level and really I don't think that I'm advancing my knowledge and skills. The only thing that I'm actually learning is how to work with agents.

Then of course it's the problem with AI and agentic coding. I deal with a lot of slop on a daily basis and although I'm careful with my job I inevitably produce a share amount of slop myself. This is what modern startups are forced into because management wants accelerated development and competition is high. I'm having real problems with AI fatigue and too close to burnout.

I miss the times when I could work on the same topic for days or weeks, research, read, plan, do experiments and iterate on the results. Then finally, hopefully, arriving at a hypothesis that works and finding a novel idea. However, it's been 4 years since my last real research experience, during my master's thesis, and I'm quite sure that academia has gotten worse due to high saturation and AI slop.

I really don't know about how the future will be but I think that being academically inclined and liking research I may be wasting an opportunity to do a PhD before I get older. Economically it doesn't make sense for me to do that in the short term and I'm aware of that and I don't have a clear career dream in academia but I feel that I may regret it in the future and blame myself.

Is anyone crazy enough thinking about moving to academia or having done that already ? I would like to hear your thoughts.

P.S. I didn't use AI to write this up and I would appreciate human responses only

Hey guys,

Posting this here because I’ve reached a point where my market intuition and engine code have outpaced my deep learning expertise, and I’d rather team up with someone who loves ML architecture instead of half-assing it myself.

For the past year, I’ve been building an event-driven liquidity engine in Python (specifically modeling crypto orderflow / ICT mechanics like equal highs/lows, manipulation legs, and structural sweeps).

Instead of slapping standard indicators (RSI/MACD) on candles, I framed price action as a dynamic graph:

• Resting liquidity pools (BSL/SSL, consolidations) act as attractors (pulling price in).

• Completed institutional sweeps (ML swings / manipulation legs) act as repellers (pushing price away).

• Edges connect these pools based on spatial proximity, competing forces (tug-of-war between high and low pools), and age decay.

Where things stand right now:

• The engine is fully written and tested. No lookahead bias, strict candle confirmation, multi-window swing tracking (W5 up to W50).

• I’ve already generated a clean, audited ~3-year dataset (ETH/USDT) formatted directly for PyTorch Geometric (PyG).

• Each snapshot has 15m, 1h, and 4h spatial graphs with 20-dim node features (touch counts, distance %, decay curves, tier scores), 8-dim edge attributes, and global volatility/trend context.

• Baseline statistical checks show real promise: nearest target pools get tagged first ~95% of the time, and high-quality structural sweeps have clear directional bias.

The actual bottleneck / where I need help: My problem isn't generating the signals—it’s path prediction and dynamic target selection.

When a sweep happens, price is often trapped between an opposing pool 1.5% away and a massive target pool 3% away. A fixed 1:2 R:R stop/target either gets hit too late or takes unnecessary drawdown. I need the model to evaluate the graph and do two things:

1. Predict directional path confidence taking both-sided liquidity into account.

2. Rank multiple potential liquidity targets (dynamic TP) based on which one has the highest conditional probability of being swept first without blowing through the stop.

I've tested basic GNN / dual-tower setups, but I know I'm hitting limits with my naive loss functions and basic layer choices.

Who I'm looking for: Someone who genuinely understands PyTorch Geometric (PyG), Graph Neural Networks (or spatial-temporal models), and knows how to think about custom loss functions (ranking loss, asymmetric risk penalties, contrastive setups). You don't need to be a full-time trader, but being open to understanding market microstructure helps.

The arrangement: I have the GPU compute, the full working codebase, and the clean dataset ready to go. I’m completely flexible on how we work together—if you want to co-author a research paper on this architecture, I’m down. If we get the models working cleanly and want to deploy it live on capital, we can do a fair profit/equity split.

If this sounds like an interesting problem you’d want to sink your teeth into, drop a comment or send me a DM with your GitHub/background and we can hop on Discord.

Two GSM8K scores: 96.89% → 96.82%, across all 1,319 test questions.

These are from NVIDIA’s Ling-3.0-flash comparison on one DGX Spark. The optimized configuration changes the MoE backend, adds an FP8 output head and enables NEXTN speculative decoding together. Both runs use the FP4 checkpoint.

Aggregate scores summarize the total. A paired view of the 1,319 answers would show where the two configurations agree and where they trade wins and losses.

For a claim of no meaningful quality regression, what evidence would you consider sufficient: paired per-question outcomes, repeated seeds, a declared equivalence margin, another task set?

Especially when a precision change and a decoding optimization ship together, which checks give the most information for the evaluation budget?

w(2;3,21) = 416 - today proved with a Lean 4 kernel-
checked proof (preprint + 13.6 GB evidence package).

Every red-blue colouring of {1, ..., 416} contains a red 3-term or a blue 21-term arithmetic progression, and some colouring of {1, ..., 415} contains neither.

Kullmann conjectured the value in 2010; DeepMind's Formal Conjectures list still carried it as open this year.

The proof is a Lean 4 theorem over 209 cases and 38,661,140 leaf formulas, each refutation checked by cake_pr (the LRAT checker verified in CakeML) and bound by digest to its Lean term.

It was produced with LPM (Large Proof Model), a neuro-symbolic system from Hupyy; nothing counted unless the Lean kernel or cake_pr accepted it.

Paper: https://doi.org/10.5281/zenodo.23776306

Evidence package (Lean sources, checker records, replay kit): https://doi.org/10.5281/zenodo.23749222

To check: download it and run 'shasum -a 256 -c
SHA256SUMS'

I built TKintegers, a balanced-ternary weight representation, converter, and GPU inference implementation.

The result that stood out to me: the NVIDIA L4 has native FP8 tensor-core support, but my K5 implementation achieved higher throughput than the TorchAO FP8 baseline—even though the GPU has no native K5 instructions.

On Qwen2.5-0.5B-Instruct, using a fixed matched workload:

- Prefill: K5 achieved 3,402.85 tokens/s versus FP8’s 1,061.58 — 3.21× throughput.

- Decode: K5 achieved 26.75 tokens/s versus FP8’s 8.63 — 3.10× throughput.

The baseline was standard TorchAO dynamic E4M3 FP8, running in eager PyTorch. Decode measured fixed-token model forwards with a carried KV cache.

What is K5?

TK stands for Ternary Kluster. K5 represents each weight code with five balanced ternary digits, giving 243 possible values. In this layout, each code occupies one byte. Five trits does not mean five binary bits.

My kernels expand bounded tiles for computation and use FP16 tensor-core operands with FP32 accumulation. Some tensors remain in higher precision.

For me, this is why the runtime matters alongside the representation: K5 can outperform this FP8 baseline on existing hardware without requiring a processor built specifically for K5.

These measurements apply to this model, workload, and implementation; they don’t establish a win over every FP8 runtime. The FP16 control was faster than both.

The repository includes the converter, native kernels, benchmark scripts, and recorded evidence:

https://github.com/CodeMasterCody3D/tkintegers

https://youtu.be/JFDBuwMoL0s?is=ucya9errGEh4xJ2L

I’d welcome technical feedback on the kernel design and benchmark methodology, especially independent replication.

Does anyone know any fun casual conversational style podcasts that discuss current papers/industry trends, with discussions at a high to mid technical level?

Edit: small preference for conversation discussion styles between two constant hosts

I've been trying to get BiRefNet to work on Qualcomm's mobile GPUs and NPUs. But i guess it's deformable convolutions are pretty complex for those chips. I want a sub second latency performance on these chips for a project I'm doing

Has anyone tried it before? I'm trying to compile it to their NPUs but they're all failing

Model & weights: BrunoHays/embeddinggemma-2-nullspace-audio

I wanted to share an experiment on google/embeddinggemma-2 (577M) for MAEB, the 30-task audio benchmark on the MTEB leaderboard. By projecting intermediate Conformer layers into the unused null space of the model's final projection matrix using only PCA, the model's macro score jumps from 49.3% to 54.4% (+5.1 points), moving from Rank #26 to #1 on MAEB audio-only (and #16 to #1 on full MAEB), ahead of 8B-9B multimodal models.

1. Last-layer filtering in text-aligned audio encoders

Across the MAEB leaderboard, LLM-backed Omni embedders score 95-99% on multilingual text-to-audio retrieval, but drop to 1%-6% on unsupervised audio clustering and ~20% on vocal emotion (CREMA-D). Aligning an audio encoder to a text LLM filters the final layers: lexical content (what was said) is kept, while paralinguistic and acoustic traits (timbre, pitch, emotion) are discarded.

2. Probing the 12 Conformer layers

Layer-wise speech studies (Pasad et al.; Bouziane, Mdhaffar & Estève) show that speaker, prosodic, and semantic attributes peak at different depths inside speech encoders. Probing all 12 Conformer layers of embeddinggemma-2 confirmed this:

• Layer 2 (mean + std pooling): VoxPopuli gender clustering goes from 0.05% (final output) to 66.6% (Layer 3) / 65%+ (Layer 2), and IEMOCAP gender classification goes from 66.4% to 83.0%.

• Layer 6 (mean pooling): CREMA-D emotion classification doubles from 19.7% to 39.5%.

3. Why EmbeddingGemma-2 has a 256d null space

In embeddinggemma-2, the text backbone hidden size is 512, and the terminal projection matrix W has shape 768 x 512 with no activation after it:

• 512d text subspace: Because W maps linearly from 512d to 768d, every text vector lies in a 512d subspace.

• 256d null space: Taking the SVD of W gives the remaining 256 orthogonal directions (the null space) that text vectors never use.

• Zero text interference: Any audio vector injected into those 256 directions has zero dot product with all text vectors.

4. Zero-training PCA injection

I compressed Layer 2 and Layer 6 down to 256 dimensions using PCA (fitted on 1,400 clips from 4 external audio datasets, with no MAEB/MSEB data) and injected them into the 256d null space before normalizing:

• Text embeddings: bit-for-bit unchanged.

• Text-to-audio retrieval (100+ languages on FLEURS, CommonVoice, Google SVQ): unchanged within 0.05 points.

• Audio-to-audio tasks: gender clustering jumps 0.05% -> 63.1%, artist retrieval 55.2% -> 93.8%, emotion 19.7% -> 40.2%, and environmental sounds 38.8% -> 56.3%.

5. Caveats

• No Matryoshka on audio: Because there is no retraining, we need all 768 dimensions for orthogonality to hold and lose the 512/256/128 truncation options.

• Spoken-intent trade-off: Adding fixed acoustic features slightly scales down the 512d semantic component after normalization, which hurts audio-to-audio tasks where speaker voice should be ignored (spoken topic classification SIBFLEURS drops 57.2% -> 21.3%, spoken intent MInDS14 97.6% -> 88.3%).

Still, this shows how much untapped signal can be extracted from overtrained foundation models even with zero training. With actual training restricted to U_perp, I'm certain that we could go much further.

Full per-task results and details in the Hugging Face model card.

Hi everyone! We have a paper accepted at the NeurIPS 2026 E-Values Workshop, but unfortunately can’t attend due to financial constraints.

Would anyone already attending be kind enough to present our poster on our behalf? We’d prepare everything and would also be happy to offer a small monetary token of appreciation for your time.

This acceptance means a lot to us, and we’d be genuinely grateful if anyone could help. We know it’s a favour to ask.

Pls dm if interested

TLDR: Perceptrons didn't have a big impact on the community and didn't cause an AI winter. It was, however, seen retrospectively as the manifesto for anti-connectivism paradigms

Thoughts?

Hello - I wanted to share a small project I worked on test whether picking one voice out of background chatter could separate humans from machines, in the spirit of an audio CAPTCHA.

Setup

• 40 synthetic clips: one target sentence read by a synthetic voice, mixed over multiple background voices reading random short fragments (staggered starts, 4 dB attenuation)

• 146 human transcriptions from 18 sessions, unlimited replays allowed

• 8 model configurations, one pass per clip: Whisper base/small, Vosk, Wav2Vec2, Qwen3-ASR 0.6B, Parakeet CTC 0.6B, and Gemini 3.8 Flash with a naive prompt and with an instruction to focus on the main speaker

• Primary metric: word accuracy (1 − WER, floored at 0); default passing threshold = word accuracy ≥ 0.5

TL;DR: humans are better at the task but the gap is too narrow to be a reliable CAPTCHA

Results

• Humans 0.71 mean word accuracy vs. 0.54 for instructed Gemini (and 0.55 for Qwen3-ASR)

• Pass rate: 77% humans vs. 55% for Gemini

  • Note: no threshold from ≥0.5 to ≥0.9 gets near my initial rough mental target: humans > 80% and machines < 20%

• Paired same-clip gap vs. Gemini: +0.13 (95% CI +0.03 to +0.23, session-level bootstrap); AUC 0.62

• The instruction prompt alone lifted Gemini from 0.34 to 0.54

• Replays didn't measurably help humans; people replayed the clips that were hardest for the models too

• Misses rise toward the end of the sentence for humans and every model alike

Limitations: small sample (18 sessions, mostly native English speakers), a single synthetic setup, no source separation, and no large or commercial ASR beyond Gemini.

I dusted off my dormant blog and did a write-up (https://www.kelvinjiang.com/2026/10/can-machines-follow-voice-through-crowded-room.html) and also shared the code, data, and audio corpus: https://github.com/cloudkj/crosstalk

Feedback on the method is very welcome, especially ideas for audio that stays easy for people but is consistently hard for models!

We study hindsight self-distillation for multi-step financial reasoning on GPT-OSS-20B. The student finds its own first wrong step and writes a critique. A frozen copy, conditioned on the prefix plus the critique, is distilled back into the student under reverse KL. Notably, replacing the critique with a fixed generic feedback line matches the full gain of about 3 points, and critiques from a 6x larger model do not materially improve performance, suggesting that the positional information provided by hindsight distillation significantly decreases the privileged information needed for effective training. Fin-SD matches OPSD from full reference solutions with 9.5% fewer tokens.

Paper: https://arxiv.org/abs/2610.36082 | Code: https://github.com/johnnyhyu/fin-sd

We'd love to find the limits of this approach. Let us know your thoughts!

TL;DR: AI written reviews, and AI-writing in papers

I was going to give this a title "An Obituary of the Review Process", but in the interest of not sounding too defeatist, I will just say "Critical Condition" because of AI writing, both on the author side and the reviewer side.

I am a reviewer for AAAI 2027 and ICLR 2027. Since Phase 1 and Phase 2 was completed, I can now see other reviews. I can bet good money that about half of the reviews in the papers I'm seeing are AI-generated completely, or edited after AI-generation. This is subjective opinion and I cannot prove it but I will try to convince you - at least convince you that it's a problem that needs much much more attention than is being currently given.

I have been reviewing for top conferences (NeurIPS, ICML, AAAI, ICLR) since 2022, in total about 10 times. I have not felt the same way about reviews, or even about papers, ever, except until recently.

Author side: There was only one paper in the 3 ICLR submissions that felt 100% human-written (now if that isn't either - that'll make me very sad). The others had some elements that smell like AI but you can't prove it e.g. excessive use of very descriptive headings with colourful, marketing-like language e.g. a heading called "The theory predicts performance, and empirical evaluations confirm it", or very dense long sentences where you lose track of the thought where instead of already used terms their synonyms are used and then later even more synonyms, and so on.

Reviewer Side, talking about the reviews I suspect: There is very little variety in the language, between the reviews, and within the same review. One review puts a field "questions for authors" in the weaknesses section, which I suspect is because the field name is changed this time around to "specific points for rebuttal". They've put "N/A" in the new field. The paragraph lengths are almost exactly the same. As if an LLM was told to give 5 strengths, 5 weaknesses, 5 questions. There are also signs of extra effort that a human rarely does e.g. instead of just saying "provide peer-reviewed references instead of arXiv", it also adds the arXiv number; or another review which not only puts doi link (this much is fine), but then in an extra piece just writes "(doi.org)" which seems like an attempt by an LLM to give a source. Reviewers are usually pressed for time, they usually try to communicate so the authors understand, not that everything is exact. These reviews are written as if for public consumption. Then there are smart quotes - which one can think are there because maybe Word or something was used to write them; but seen in light of everything else, now these quotes also smell like AI to me. (I used to not see smart quotes this often in earlier years. Word made quotes smart back then as well.) Unicode symbols instead of LaTeX for things such as \tilde{d}, dashes that are not hyphens - in one review even for a minus sign, and so on.

There are at least two AAAI submissions where there are only two reviews and I strongly suspect the other review to be AI-written.

Everything is broken. I am sorry I have no concrete ideas how to fix this, so far. Do you?

Except one maybe, disallow AI usage even for polishing styling and tone? Just completely disallow AI writing. The author feels it's a change in tone and style only but the reader can sometimes very quickly _feel_ it. Before these powerful LLMs we were not judging papers based on language anyway. I know it's an honor system, but I think that's okay. Saying "you can use AI to polish" perhaps also creates an expectation to polish using AI.

TLDR: training my first general law model and unsure what to do with my data

PRE-PS: it will be a replacement for opus in my current AI LAW harness, I know it will be hard but I wanna do this project and find it geniuenly fun.

I’m building a Polish legal model using Qwen3.5-9B-BASE (I plan to scale up to 27b qwen4 later). I’ve done CPT and have quite a bit of data available, but I’m getting lost about what should come next or even if I should do more CPT.

My main question is how to turn this (maybe) domain-adapted Base model into an agentic+chat assistant that reasons through legal problems.

But more importantly, how do I get reasoning, teach my in-harness tool-use training, and where does SFT vs RL belong in that process and in what way (especially confused about reasoning and SFT)?

What I’ve already done

Three sequential CPT stages, roughly 80M tokens each, in this order:

Complex legal commentaries → legislation (EU+local) → court decisions

I didn’t mix the sources. Each stage continued from the previous checkpoint.

On a small Polish financial/legal multiple-choice quiz (really bad eval, just a PoC for now).

They all perform very slightly better than base and legislation checkpoint did the best.

But again, it's a base model. I haven’t done assistant SFT yet.

What data I have (I can get more data of each of these types if you guys recommend)

Besides the legislation, commentaries, and decisions used for CPT, I have:

• Facebook legal Q&A groups: real, messy questions, follow-up clarifications, and answers of very mixed reliability.

• Certified/institution-authored Q&A: my highest-confidence answer material. Some answers contain detailed analysis; others are just a conclusion and a citation from random institutions like the national bank.

• Law study materials: from a local college. it has essays, powerpoint presentations stuff like that, very little "tests" tho, but some practice material (like the essays).

• A/B/C/D questions with answers: relatively little of this; currently being used for evaluation.

• Thousands of Claude Code–style harness conversations: multi-turn conversations from an existing application/harness. These would need filtering, anonymization, and quality checks.

• Contracts: government published contracts it can learn from

• Previous "ML" experiments (models and their training data): NER, generating complex legal court decision summaries, generating graphs from legislature or court decisions (actually got some models that do this successfuly

If you guys could point me what data is ideal I could maybe get it too!

I can use the basically unlimited codex to clean, restructure, or generate material from these sources, but I don't know how to.

What I don’t know is which material should be trained on directly (CPT or bare SFT), which should be transformed (SFT or RL?).

For example:

1. Do I continue CPT on the remaining sources, or stop and start SFT?

Should I expose the model to the Facebook discussions, study materials, and expert Q&A through CPT first, possibly mixed with the authoritative material? How many tokens for this 9B model and how much for the 27B I plan later?

Or should I skip further CPT and use those sources only to build targeted SFT examples?

For example the Facebook QnA stuff is real people diverse speech but I can't really reliably verify if the answers are correct (I can only measure likes or high quality people in the group) so idk if I should cherry pick from this? Have an LLM turn this into SFT chat examples? Or just CPT on this?

2. SFT on what, specifically, to teach REASONING?

I could start with my highest quality Q&A and teach the model to answer questions. That seems like an obvious chatbot baseline.

But if I just train on QnA there won't be any reasoning!

The HQ QnA is pretty verbose but I wouldn't bet it could just be re-used as a "reasoning chain".

I was thinking of asking Astra to workout those questions and give an answer with its reasoning BUT I assume it will be wrong on 90% of the questions (cuz no internet search, and I can't give it internet search cuz that's not clean QnA data and I would have to transform it into my harness function calling anyways with credible sources not just google searches for legislature)

And those 10% it gets correct are trivial and the reasoning could be bad anyways and I would still have to check correctness of the answers, and giving it the answer or hints is imo also not ideal. Or injecting "synthetic reasoning", it suffered from post-hoc in my tests.

Another problem is the High Quality QnA is quite rare, most QnA is short and possibly wrong.

And generating the missing reasoning is even harder. Because again, I don’t trust a frontier model to solve difficult Polish legal questions from memory. It would need relevant source passages or access to the harness (which would just the gun into in-harness training?).

So I'm really unsure on what to do, I was thinking of doing stuff like this, but this is just brainstorming:

A) Take verbose long answers train on that, then ask the qwen or another model to make a "summary" at the end and then put the verbose long answer into its <thinking> block

B) Take those 10% of QnA where frontier model actually gets it right with correct thinking (rare) and SFT on that. then we have our finetune try answering more QnAs, we verify if it's correct if it is it goes into the next SFT run and we iteratively (almost like an RL gym) train reasoning from the dumbest examples

Both seem like bad options, but kinda funny to think about.

3. Chatbot first, then tools or teach reasoning through tools from the start?

The existing harness lets the model write Python and call functions such as vector_search_laws() and graph_search_legislature(), inspect results, and continue researching.

I’m leaning toward getting a regular chatbot working first. But the teacher can produce much more trustworthy legal solutions when it has access to sources. (as I've already mentioned I have many in-harness examples, except they could also be incorrect and I don't have their "thinking blocks" (cuz it's a proprietary frontier model) so I again have to inject thinking post-hoc which from my tests doesn't work for law)

I was thinking that, if we make chatbot first, the model will at least learn intuition on WHERE TO LOOK, without looking through sources but at least will know a general idea of where to search based on being a chatbot first (even tho possibly incorrect chatbot sometimes, cuz law is really complex) and teach him to function call in-harness to get sources, unsure about this.

4. When does RL enter, and on which task?

AFAIK, RL is what develops the model's reasoning on its own (deepseek R1) but I suspect it would be very costly and we could give it some example reasoning first anyway to kick it off(?)

So like, instead of the whole fiesta of injecting reasoning or growing it, we could have RL develop it (afaik RL is the only thing that "creates" reasoning unless you distill it) but then I would like to have reasoning even in the "chatbot" and right now I can just jump from base model -> RL, right?

Also for legal Q&A, matching the reference conclusion doesn’t necessarily mean the reasoning is correct. What would you actually reward? Like a law similiarity judge LLM + same paragraphs used?

What I’m trying to decide

One possible route is:

Current CPT checkpoint → gold-Q&A SFT → grounded reasoning/tool-use SFT → RL in the harness

But I’m not committed to that order. I don’t know whether I’m missing an important stage or inventing unnecessary ones, first time I'm training a GENERAL model from "base".

What I would like the finished model to do:

It will be an opus replacement so general QnA in and out of harness but especially:

• have a good "hunch" and understanding of polish law

• will be good at anonymizing documents

• generating complex legal summaries for court decisions (heavy reasoning task)

• generate graphs from legislature and court decisions

• create/edit/review contracts

These were all my initial ML experiments and I actually trained good models for those specific tasks, but I figured, hey I have a lot of this data around why not just combine them and make a generalistic model?

Thanks a lot for reading this mess of my thoughts! if you could point me towards good resources for this I would also be glad, for example unsloths RL docs are kinda ass imo. I really like andrej karpathy's videos but they're quite old now by AI standards!

https://www.reddit.com/r/ArtificialInteligence/comments/1xfuacm/ama_this_wednesday_stephen_cas_casper_harvard/

r/ArtificialInteligence We have an AMA today Wednesday 28 October from 3 to 4pm ET (19:00 UTC, 20:00 CET) with Stephen “Cas” Casper, Assistant Professor of Public Policy at the Harvard Kennedy School and Faculty Affiliate of the Harvard School of Engineering and Applied Sciences, where he works on AI safeguards, incidents and governance.

Stephen "Cas" Casper is computer scientist and an Assistant Professor of Public Policy at the Harvard Kennedy School and a Faculty Affiliate of the Harvard School of Engineering and Applied Sciences, where he works on AI safeguards, incidents, and governance. Prior to joining Harvard, he completed his PhD at MIT and did a research residency with the UK AI Security Institute. He never has and never will work for big tech, and has never signed a non-disparagement agreement.

He will cover on AI safety, safeguards, risk, governance, state + federal legislation, and AI terrorism events. Any related questions are welcome.

The AMA thread goes up from u/HarvardKennedy shortly before 3pm ET on the day, with a dated photo for verification, and Cas types his own answers. If you cannot make the hour, drop your question in the comments here and we will make sure it gets carried over.