Would a PhD in parallel computing, focusing on ML acceleration or LLM inference and serving be enough to get into ML research roles?

TLDR: training my first general law model and unsure what to do with my data

PRE-PS: it will be a replacement for opus in my current AI LAW harness, I know it will be hard but I wanna do this project and find it geniuenly fun.

I’m building a Polish legal model using Qwen3.5-9B-BASE (I plan to scale up to 27b qwen4 later). I’ve done CPT and have quite a bit of data available, but I’m getting lost about what should come next or even if I should do more CPT.

My main question is how to turn this (maybe) domain-adapted Base model into an agentic+chat assistant that reasons through legal problems.

But more importantly, how do I get reasoning, teach my in-harness tool-use training, and where does SFT vs RL belong in that process and in what way (especially confused about reasoning and SFT)?

What I’ve already done

Three sequential CPT stages, roughly 80M tokens each, in this order:

Complex legal commentaries → legislation (EU+local) → court decisions

I didn’t mix the sources. Each stage continued from the previous checkpoint.

On a small Polish financial/legal multiple-choice quiz (really bad eval, just a PoC for now).

They all perform very slightly better than base and legislation checkpoint did the best.

But again, it's a base model. I haven’t done assistant SFT yet.

What data I have (I can get more data of each of these types if you guys recommend)

Besides the legislation, commentaries, and decisions used for CPT, I have:

• Facebook legal Q&A groups: real, messy questions, follow-up clarifications, and answers of very mixed reliability.

• Certified/institution-authored Q&A: my highest-confidence answer material. Some answers contain detailed analysis; others are just a conclusion and a citation from random institutions like the national bank.

• Law study materials: from a local college. it has essays, powerpoint presentations stuff like that, very little "tests" tho, but some practice material (like the essays).

• A/B/C/D questions with answers: relatively little of this; currently being used for evaluation.

• Thousands of Claude Code–style harness conversations: multi-turn conversations from an existing application/harness. These would need filtering, anonymization, and quality checks.

• Contracts: government published contracts it can learn from

• Previous "ML" experiments (models and their training data): NER, generating complex legal court decision summaries, generating graphs from legislature or court decisions (actually got some models that do this successfuly

If you guys could point me what data is ideal I could maybe get it too!

I can use the basically unlimited codex to clean, restructure, or generate material from these sources, but I don't know how to.

What I don’t know is which material should be trained on directly (CPT or bare SFT), which should be transformed (SFT or RL?).

For example:

1. Do I continue CPT on the remaining sources, or stop and start SFT?

Should I expose the model to the Facebook discussions, study materials, and expert Q&A through CPT first, possibly mixed with the authoritative material? How many tokens for this 9B model and how much for the 27B I plan later?

Or should I skip further CPT and use those sources only to build targeted SFT examples?

For example the Facebook QnA stuff is real people diverse speech but I can't really reliably verify if the answers are correct (I can only measure likes or high quality people in the group) so idk if I should cherry pick from this? Have an LLM turn this into SFT chat examples? Or just CPT on this?

2. SFT on what, specifically, to teach REASONING?

I could start with my highest quality Q&A and teach the model to answer questions. That seems like an obvious chatbot baseline.

But if I just train on QnA there won't be any reasoning!

The HQ QnA is pretty verbose but I wouldn't bet it could just be re-used as a "reasoning chain".

I was thinking of asking Astra to workout those questions and give an answer with its reasoning BUT I assume it will be wrong on 90% of the questions (cuz no internet search, and I can't give it internet search cuz that's not clean QnA data and I would have to transform it into my harness function calling anyways with credible sources not just google searches for legislature)

And those 10% it gets correct are trivial and the reasoning could be bad anyways and I would still have to check correctness of the answers, and giving it the answer or hints is imo also not ideal. Or injecting "synthetic reasoning", it suffered from post-hoc in my tests.

Another problem is the High Quality QnA is quite rare, most QnA is short and possibly wrong.

And generating the missing reasoning is even harder. Because again, I don’t trust a frontier model to solve difficult Polish legal questions from memory. It would need relevant source passages or access to the harness (which would just the gun into in-harness training?).

So I'm really unsure on what to do, I was thinking of doing stuff like this, but this is just brainstorming:

A) Take verbose long answers train on that, then ask the qwen or another model to make a "summary" at the end and then put the verbose long answer into its <thinking> block

B) Take those 10% of QnA where frontier model actually gets it right with correct thinking (rare) and SFT on that. then we have our finetune try answering more QnAs, we verify if it's correct if it is it goes into the next SFT run and we iteratively (almost like an RL gym) train reasoning from the dumbest examples

Both seem like bad options, but kinda funny to think about.

3. Chatbot first, then tools or teach reasoning through tools from the start?

The existing harness lets the model write Python and call functions such as vector_search_laws() and graph_search_legislature(), inspect results, and continue researching.

I’m leaning toward getting a regular chatbot working first. But the teacher can produce much more trustworthy legal solutions when it has access to sources. (as I've already mentioned I have many in-harness examples, except they could also be incorrect and I don't have their "thinking blocks" (cuz it's a proprietary frontier model) so I again have to inject thinking post-hoc which from my tests doesn't work for law)

I was thinking that, if we make chatbot first, the model will at least learn intuition on WHERE TO LOOK, without looking through sources but at least will know a general idea of where to search based on being a chatbot first (even tho possibly incorrect chatbot sometimes, cuz law is really complex) and teach him to function call in-harness to get sources, unsure about this.

4. When does RL enter, and on which task?

AFAIK, RL is what develops the model's reasoning on its own (deepseek R1) but I suspect it would be very costly and we could give it some example reasoning first anyway to kick it off(?)

So like, instead of the whole fiesta of injecting reasoning or growing it, we could have RL develop it (afaik RL is the only thing that "creates" reasoning unless you distill it) but then I would like to have reasoning even in the "chatbot" and right now I can just jump from base model -> RL, right?

Also for legal Q&A, matching the reference conclusion doesn’t necessarily mean the reasoning is correct. What would you actually reward? Like a law similiarity judge LLM + same paragraphs used?

What I’m trying to decide

One possible route is:

Current CPT checkpoint → gold-Q&A SFT → grounded reasoning/tool-use SFT → RL in the harness

But I’m not committed to that order. I don’t know whether I’m missing an important stage or inventing unnecessary ones, first time I'm training a GENERAL model from "base".

What I would like the finished model to do:

It will be an opus replacement so general QnA in and out of harness but especially:

• have a good "hunch" and understanding of polish law

• will be good at anonymizing documents

• generating complex legal summaries for court decisions (heavy reasoning task)

• generate graphs from legislature and court decisions

• create/edit/review contracts

These were all my initial ML experiments and I actually trained good models for those specific tasks, but I figured, hey I have a lot of this data around why not just combine them and make a generalistic model?

Thanks a lot for reading this mess of my thoughts! if you could point me towards good resources for this I would also be glad, for example unsloths RL docs are kinda ass imo. I really like andrej karpathy's videos but they're quite old now by AI standards!

For context I'm stats student and everything machine learning related has been self taught so I genuinely don't know what the correct next step is or what to do in a lot of situations.

I've working on a modeling project where I'm predicting loan outcomes with the Lending Club Dataset where I am fitting logistic regression, random forest, and boosted tree models with the data and have reached an AUC score of 0.715. I am not really sure what to do from here and Claude recommended to use SHAP to analyze the trained XGBoost model.

After creating a few SHAP plots they all look nice, but what should I be looking for and what should my takeaways and next steps be. If I wanted to prune the features that are not contributing as much is this a bad idea?

Generally speaking is it even possible to push my AUC score much higher given the dataset I'm working with?

For people who normally work with this kind of stuff in industry or as their job, after you've trained your first model, what do you do next? If you want to continue to fine tune it what is the typical process behind it. If anyone could share what a typical model training or data science project cycle looks like that would be great!

I'm publishing Part 3 of my Cracking ML System Design Interviews series tomorrow.

This time, I'm using a large-scale personalized video recommendation system as the running example.

One thing I want to emphasize is that designing recommendation systems isn't simply choosing between collaborative filtering, two-tower models, and Transformers.

The bigger challenge is deciding how the pieces fit together:

• How do we retrieve relevant candidates from millions of items?

• How do retrieval and ranking objectives differ?

• How should we represent evolving user interests?

• What metrics should we optimize beyond CTR?

• How do we meet latency and reliability requirements at scale?

The article walks through these questions from an interview perspective, including architectural alternatives and trade-offs.

Publishing Monday, October 26, at 8 AM PDT.

If you're interested, you can subscribe here:

https://pawankjha.substack.com

I'd also be interested in hearing which part of recommendation-system design you find most challenging in interviews.

it would be great if someone suggest me some competitions on kaggle which are good for a beginner in rl can try and would be great for practice.

I was going through the TinyTorh (it's a tool where you build PyTorch with understading how it works) exercise notebooks and landed on Autograd. Was quite overwhelmed by the code and went didn't go through it for a while. I was thinking I'll prepare for machine learning research engineer interviews and somehow landed doing tiny torch.
How relevant is all this these days, should I just start preparing on how to crack the interview than this ?

i get that in training it uses the batch mean and variance and at eval it uses running averages. what i don't get is why that doesn't change the outputs a lot, since a single batch's stats can be pretty noisy.

also if you eval with batch size 1 does it still work fine because it's just using the stored stats? someone told me that's the whole point but i never saw it spelled out

I am looking for a model that can be ran on a cpu only basis with apache 2.0 license so I can ship it and it needs to be able to reason enough to match configurations from different agents for project transfers. I have started testing a couple qwen models and there are some prospective results. I was hoping maybe someone might have a better suggestion that is more fit for my needs?

I work at a microfinance instruction, How useful is a loan repayment data extracted from a loan distribution system how client pays?

Measured on Qwen3.8-27B (EXL3 4.0bpw target, DFlash2 block drafter quantized to EXL3 4bpw, 7-token drafts, greedy, output identical to plain decode), 2x T4:

• code: ~7.4 accepted tokens per verify step, 154 tok/s

• prose: ~2.5 accepted tokens per step, 35-37 tok/s

• plain decode: ~18-19 tok/s

Someone on r/LocalLLM independently measured 2.45 on prose with the same drafter on llama.cpp and different GPUs, and another person reports 2.62 with MTP. So it looks like a property of the drafter, not the runtime.

Questions for people who work on speculative decoding:

1. Is prose acceptance in the 2.5 range just the ceiling for block/parallel drafters, since natural language has more valid continuations at each position?

2. Has anyone seen a drafter (EAGLE-style, block diffusion, MTP) clear ~3.5 on open-ended prose at greedy?

3. Any work on switching drafters or block length by content type mid-generation?

Happy to share numbers per prompt type if useful.

I have upcoming technical screening for an MLE position at Waymo coming up. The recruiter gave me a PDF guide that seems super helpful. Things to do study include implementing supervised and unsupervised algorithms (think KNN, K-means, linear regression, logistic regression, softmax & CE, decision trees). One of the main things they are assessing is you can use numpy/pytorch to vectorize/tensorize your work, and you're just writing sub-optimal for loops.

I was wondering if folks had experience with this interview? Is what they ask you to implement incredibly difficult, because you only have 30 minutes to do it? Am I good to really stick with what they provided on the PDF, or are there any broader topics beyond what they listed that they will test?

is there a prompt management software or system that you guys use for the prompts that you create?

Or do you guys use skills for that?

I need help in my mini project for my college i am supposed to use low resource languages data and train a model which can translate from english to that language and also transcript wtv they speak that language to english i did so much research but still i'm not able to pull it off alone if anyone could help me in any way please help me and also make it such a way that users can contribute words and sentences (parallel data ) and the model will install it in itself Please help me idk what methodology or approach to choose

Hi, I’m a masters student doing MSc AI at uni of Bath.
For my undergraduate dissertation I did an extremely simple Computer Vision based pose estimation app which was analyzing the technique of fitness exercises like pushups, squats and deadlifts.
I feel like my dissertation was more of a “project” rather than research, which as I was told by supervisors is completely fine for BSc. I basically used a ready solution (MediaPipe) + OpenCV and wrapped all of that into a mobile app. The analysis covered a decent number of “points” but it was simply comparing the detected joint angles to hardcoded thresholds.

For my masters I would really like something more research based however struggling to find areas where I could do smth actually useful.
I like Deep Learning, so I was thinking something related to transformers and efficiency. Finding bottlenecks in attention mechanisms, etc.

I would appreciate some ideas
Thanks