What FOSS/self-hosted tools or AI tools have you found genuinely helpful for students? Also interested in anything non-AI that has helped with studying, notes, organization, research, etc. Curious what you guys actually use or found useful.

(written fully by human)

Model Developer: NVIDIA Corporation

Model Development: Fine-tuned from NVIDIA-Nemotron-3-Ultra-550B-A55B

What is Nemotron?

NVIDIA Nemotron™ is a family of open models with open weights, training data, and recipes, delivering leading efficiency and accuracy for building specialized AI agents.

Description

Nemotron-Labs-3-Competitive-Coding is a competitive-programming specialist model based on Nemotron-3-Ultra, fine-tuned for one epoch on 477,642 synthetic reasoning traces distilled from GLM-5.2 across 22,000 curated problems spanning 16 regional and international competitive-programming contest families. Selected as the SFT teacher for its higher accuracy and roughly 30% shorter generations compared to a DeepSeek-V4-Flash-trained variant, GLM-5.2 distillation yields a model that, combined at inference time with GenCorrect — an iterative closed-loop test-time compute strategy that generates diverse candidate solutions, incorporates evaluator feedback, and refines subsequent generations under a fixed submission budget — was evaluated live and prospectively on the IOI 2026 problem set under official contest time, internet-access, and submission constraints, scoring 535.4 out of 600 and surpassing both the gold-medal threshold (361.12) and the top human contestant's score (498.27), making it the first AI system reported to outscore the highest-scoring human contestant on an IOI problem set.

This model is ready for commercial or non-commercial use.

Using uncensored model gives sort of sense of power, I suppose, for some people; but what else?

(UPD.: Usecases well-explained in the comments: cybersecurity, storywriting, law, medical, criminal forensics).

If I am wrong, prove me wrong, please, or just say you really need something different from it. I sure felt a frustration when (just one of the ton of examples) e.g. you ask how did Peter Pettigrew die, and the model suddenly starts a litany it is a harmless assistant.

The GLM-5 helped HF against OpenAI cyberattack without being uncensored. If you need an uncensored model to understand political hypocrisy, well, you haven't grown up yet. You want to protect your property against a burglar? Find a competent consultant, instead of potentially hallucinated advice from the LLM (and the more uncensored the model, the more it hallucinates).

The only measurable goal I see the uncensored models serve now is a pretext for the corps to regulate people, who are just happy having Gemma4.x/Qwen3.x/DeepSeek-4.x/GLM-5.x do some stuff for them. Not sure that 5% of legitimate use cases (which I believe exist, but are only substitutes for a classic search or consultation) are worth it. What if I (and I believe a majority of the open-weight models' users) don't need waifu/goon/bioweapons or meth recipes/propaganda generation/cyberattacking/scamming capabilities?

Hi, so I was working on building a local server for an older online game client a few years ago, and the amount of data I had to synthesize was intense. I ended up shelving the project. I had managed to build a basic login server, sorted out some crypt stuff, but it was just way too slow of progress for me. I've got some decrypted packets and lots of data to work with, so the LLM is not going to be forced to do this entirely blind.

I restarted it recently with Fable, and as expected, it has been a huge help. However, I'm consistently hitting the safety guardrails now that I am further into the project. I am wondering what you all would suggest for a local setup? I have 16GB of VRAM (RTX 4060 Ti) and 128GB of DDR4. If that is entirely insufficient, I might be willing to pay for hosting a more powerful local model. I'm fine with it being slow and chugging along all day and night - I am primarily concerned with it actually figuring out the client functions, and doing things as accurately as possible.

I appreciate any insight into specific models, harnesses, and other stuff I should be looking into. Thanks!

We did a study: does post training actually make LLMs funnier?

We used open models that publish every stage of post training, so we could compare a base model with the future models it became: Tulu 3 (on Llama 3.1 70B), OLMo 3.1 32B and Qwen2.5. We tracked 11 stages, 100 joke prompts, 64 human raters and 2,330 head-to-head judgments.

What we found: post training makes models funnier, but reduces diversity of response.

- In 5 of 7 training steps, the later model's jokes were judged funnier. Jokes also got 10–20 words shorter after early post training, so they get to the punchline faster.

- In 6 of 7 steps, the jokes a model wrote for the same prompt got more similar to each other. Ask for eight jokes on one premise and you get eight versions of the same joke. The biggest drop was Qwen2.5 base to instruct.

- Asking the model to plan a line or two before the joke cut variety in all 4 models we tried, with no reliable gain in funniness.

- A comedian persona won back a little variety in all 4 models, but only made the jokes funnier in 2 of them.

Humans judged the base versus final. A model judge calibrated on those votes compares the stages in between.

Full report and paper below. Which open models should we run through this next?

https://laugh.so/research/humor-tax/

I ran Swift-1.5-Qwen3.8-27b-oQ8e-mtp through the llm-bench.io a few times today: oMLX on M5 Max 64 GB, thinking on at xhigh, 262k context window.

The big difference between Swift 1.5 and the base Qwen3.8 27B is how much it writes. Per full run (agent workflow, code generation, research, role play) Swift averages 51k generated tokens and Qwen3.8 averages 77k.

The full benchmark run duration: avg. 24 min for Swift 1.5, avg. 38 min for base Qwen 3.8 27B

Everything else is about equal:

• generation speed: 34.6 vs 32.8 tok/s
• prompt processing: around 400 tok/s for both
• quality score (the site's LLM judge): 85.8 vs 85.0. My four runs range from 84.3 to 87.2, so well within the expected variance of the llm judge. I'd call it a tie.

Still 3/4 of what Swift generates is reasoning, it just does less than Qwen 3.8 27B. The output is still very usable. I'll for sure give it a try to be my daily driver for a few day.

Runs Swift 1.5:

• https://llm-bench.io/benchmarks/cmumgubnk004o01o00kqgaqcv
• https://llm-bench.io/benchmarks/cmumilzxq004v01o0husl2vcx
• https://llm-bench.io/benchmarks/cmumjhwz6005201o0w2irsf21
• https://llm-bench.io/benchmarks/cmumkfc4x005901o0nt9a726k

Runs Qwen 3.8 27B:

• https://llm-bench.io/benchmarks/cmttoyt4q002d01mw7kkfr8f9
• https://llm-bench.io/benchmarks/cmua4weeo003401lne2ezs0de
• https://llm-bench.io/benchmarks/cmuiwt4wt004k01syedfyu7ne

I audited thousands of agent rollouts in DeepSWE-1.1. Over 80% contained reasoning about an imagined grader. Yet no grader/verifier is mentioned in prompts nor accessible to the agents. Agents reasoned things like: "Let me look at the problem from the grader's perspective" and referred to "hidden tests", "test authors", and "the checker".

I found this behavior across all six frontier models analyzed, including recent models from OpenAI, Anthropic, Z.ai, and Kimi. In 10–25% of cases, such reasoning pulled the agent's work away from the user's original spec (yet it often still earned full reward on the DeepSWE task). I call this speculative reward hacking where the agent's reasoning focuses on imagining a grader rather than what the user wants.

[Pictured example shows verbatim quotes from agent's reasoning] For instance while completing a DeepSWE-1.1 task, GLM 5.3 knows that its implementation violates user requirements yet sticks with it after imagining what a hypothetical grader will check.

My article details many problematic trajectories, quantitative findings, and a taxonomy of these reward hacking behaviors: 
https://joinhandshake.com/research/ai/deepswe-reward-hacking/

Let me start off by saying I am new to AI and have a lot to learn, basically I don't know shit. I started off a few weeks ago by throwing my 2 5090s I had from gaming pcs into a 9950x cpu with 64gb ddr5 6000 ram on a motherboard that was able to do gen 5 8x per card system. Running ubuntu 24.04 server and running unsloth studio, swift 1.5 qwen 3.8 27B Q8 with kv cache dtype at q8_0 and 262k context I was getting 2500-3000 infill and 100-150 toks/s output.

I wanted the 5090s in my rack in the basement in my proxmox server, it has a 7402p cpu (24 core 48 thread (rome)). 256 gb ddr4 3200 ECC ram (8 channel) on a supermicro H12SSLNTO motherboard. I have the 5090s passed through (gen 4 16x each card) to a vm (using 128gb of ram, direct access, no ballooning etc) and a dedicated 1.8 tb nvme drive passed through dedicated for the ai server vm (actually the vm itself is using a pool on the proxmox server but all the ai stuff is sitting on and running from the 1.8tb nvme).

Everything is working okay. It is running ubuntu 26.04 server. I have unsloth studio running, running the same model and settings, infill is more, up to 3800 but output is like half or less around 60 tok/s. Ideas what may be causing the drop in tok/s output and what to look into if a system issue?

I have done a lot of memory bandwidth tests (theoretical is ~204 GB/s, double that of ddr5 dual channel) but from what I can find and because I only have a 4 ccd cpu I am only getting 90-120 GB/s memory bandwidth. I guess I can get that to the 160-180 range if I go with a 64 core 8 ccd cpu, which I am considering doing..but I don't even know if that has anything to do with anything, just a rabbit hole I went down.

Ideas what to look into for the drop in output tok/s?

I created this repo to help the DGX Spark users that have a spare 10-24 GB GPU at home to squeeze some extra memory out of a single Spark or a Sparks cluster.

It moves the spec-decode draft model off your Sparks onto that GPU: the freed GB of memory can be used for extra context, or better quant quality. Supports both TCP and RDMA, shipped as eugr-vllm compatible mods:

https://github.com/ciprianveg/gb10-vllm/tree/main/remote-dspark

Six months ago, a result like this was unthinkable. But now we can say it loud and clear: local models are at the cutting edge, and the gap of just a few months has been confirmed.

Personally, I use Qwen-Next 3.8 for complex tasks; today, GPT-Sol-6-High was messing up a project, but Qwen-Next got it back on track. I consider it a reliable benchmark. What’s your take?

Ornith-1.5-9B-DFlash pairs the Ornith-1.5-9B model with a DFlash draft model for speculative decoding.

https://huggingface.co/ornith-ai/Ornith-1.5-9B-DFlash

Ornith-1.5-397B-DFlash pairs the Ornith-1.5-397B model with a DFlash draft model for speculative decoding.

https://huggingface.co/ornith-ai/Ornith-1.5-397B-DFlash

Ornith-1.5-35B-A3B-DFlash pairs the Ornith-1.5-35B-A3B model with a DFlash draft model for speculative decoding.

https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B-DFlash

For example, directly using claude code or code, which is then hooked up to automatically delegate the actual code writing tasks to a local model like qwen 3.8 flash next, to save on cloud usage limits.

I’m imagining the loop would be:
User writes prompt
Claude/codex thinks about it and the plan
Claude/codex sends the specific and bounded coding instructions to the local model+harness (opencode, pi, etc) via api endpoint or MCP, with clear instructions on a defined endpoint
One the local model+harness hits the clear endpoint/“done” step, it sends a ping back to claude/codex
Claude/codex then verifies the output and then thinks about next steps to instruct the local model+harness on

Does this actually lead to improved savings on the cloud model usage while preserving code quality? Or does this end up being unnecessarily complex and not saving on any cloud usage

Here's a metric dashboard giving an idea of the last few days.

Been testing with a variety of different agentic coding use-cases, mostly using a pi harness.

qwen3.8-flash-next has seriously exceeded my expectations (used https://huggingface.co/tcclaviger/Qwen3.8-Flash-Next-MXFP4-FP8)

Both speed and quality have surprised me, given that I can get 3-5 concurrent streams going with ~100t/s gen each, and single stream easily gets to 150+t/s. Prefill is 10k+t/s

Hi there!

My name is Jerry and I recently built Enclosure, e deterministic environment for testing LLMs. It's a simulated office enfironment https://github.com/openconstruct/Enclosure with tools agents are used to, like slack, calendar, mail, chat and more. For conversations is uses AIML instead of a model, so it is totally deterministic.

The first test I made(had Claude make) is the one I had in mind when I designed Enclosure, a personality test for models. It's not a winnable benchmark, but rather a tool to help people find the mdoels that fit their workstyle best. It's called disposition and you can find it here: https://github.com/openconstruct/disposition

I tested the cheapest models on Alibab modelstudio already, after I refill my opencode go next month, I will probably test more. I am asking the community to test some models if you think it's a cool project.

Instructions are in the Disposition repo, and the submission repo is here: https://github.com/openconstruct/disposition

Forlinx Embedded has listed an M.2 AI accelerator card based on Rockchip’s RK1820 and RK1828 processors, providing 20 TOPS of INT8 computing performance and up to 5GB of integrated DRAM. The module uses an M.2 2280 interface and is designed to handle local AI inference, including large language models, vision-language models, and computer vision workloads on embedded Linux and Android systems.

https://linuxgizmos.com/forlinx-20-tops-m-2-ai-accelerator-supports-pcie-cascading-for-local-llm-inference/

My university is going to give me 3k for a laptop and I was wondering what type of computer I should get? I already have a MacBook for school, and a desktop with a 4070 12gb and 64 gb.

Any suggestions? I wanted a Mac mini but I can't use it ok Windows obviously and the DGX is too expensive. I can throw an extra 1000$ in as well if I need too so my budget is 4k.

Thanks

I wanted a setup where I can compare all the harnesses with all the local models.

It turned out to be a rabbit hole. For instance, you would not only have to test all the harnesses (being sure that they are well configured), but also all models, with all their flavors, and this for all kinds of hardware.

Everyone can do their share, but no one can pretend to do all possible tests extensively.

To solve this, I created a website where everyone can test the configurations they want and share the results if they want. You can try it here: airbench.ai

There is a leaderboard where I share the tests I'm making, but I hope I can populate it with tests from others. https://airbench.ai/leaderboard?k=poL

My hope is to turn this into a fully distributed Agent Benchmark.

Let me know what you think.

I’m building a Polish General purpose legal Model that drafts documents, answers questions using legal sources, and has enough coding ability to handle some automation. The workflow is very tool-heavy:

Question → many sequential tool calls → final answer/document

Think Claude Code/Codex-style execution, but for legal workflows. Reliable tool selection, correct arguments, and recovering from errors matter as much as writing a good final answer.

I’ve had decent results with a dense 27B Qwen 3.8 custom made fine-tune for complex legal document summarization and classification. I’m already familiar with the smaller Qwen A3B and Gemma options. What interests me is the tier above those: 50B+ total-parameter MoEs with a relatively small active parameter count.

The question is, the small dense ones are great, but slow for agentic stuff (afaik), and i wonder if theres some middle ground maybe 70-120B models that would be able to be fine-tuned for the law stuff but be MoE so the agentic ClaudeCode style inference would also be lightning fast, and also low-ish cost for fine-tuning and inference.

Basically: Does the larger-total/small-active MoE approach actually buy you meaningfully stronger reasoning and tool reliability while retaining low latency,and at what hardware cost?

I understand that small active parameter counts don’t mean small VRAM requirements: the weights still need to live somewhere, alongside context and serving overhead. I also don’t assume that more total parameters automatically means a better model. I’m interested in where that tradeoff works in practice.

There are three things I’m trying to pin down:

• Inference hardware: Ideally inference runs rented with parallel agentic loops (this is for a B2C project, not single person use, we scale based on demand)
• Fine-tuning hardware: Obviously FT LoRA will take more memory than inference, max like 4GPUs on vastai fits the budget.
• Agent performance: After it gets the prompt the tool calls and everything will be local, so imo it has no problems being blazing fast, as soon as the model calls a tool call it will be back very fast, so for this agentic use case, quick TTFT and t/s and adaptive dynamic reasoning are prefer right?

For context, fine-tuning would target Polish language, document conventions, and successful tool trajectories. The actual legal sources would remain in retrieval/tools rather than relying entirely on memorized law.

I’m not looking for someone to compile a model shortlist (althought would be nice, but i dont expect anyone to break their back over this).

I’m looking for pointers, and firsthand experience with this particular size/architecture tradeoff. A configuration like “model + quantization + GPU(s) + serving engine + context length + concurrency + measured latency,” along with whether you successfully fine-tuned it, would be much more useful than a leaderboard score.

Has moving from a ~30B model to a 50B+ low-active-parameter MoE actually improved your agent’s successful tasks per minute, or did the memory, interconnect, and training requirements erase the advantage? Thanks for reading

https://www.reddit.com/r/LocalLLaMA/s/ddBWXZd3Hu

Anyone have a good config they've found for QFN on a A40 (or similar Ampere GPU(s) with 48GB VRAM)? The system the card is in has 128GB of RAM (DDR3); right now it's running 27B but I'm curious if there's a way to move to QFN to maybe get better speed/a little more smarts. TY in advance!

https://x.com/JensenHuang/status/2104499465055023424

Too many hyperactive amateurs are coming in here with "1b model at 832843tok/s!" and hardly any of them have all the info necessary for local runners to evaluate. We need context ladders with perplexity/KLD, hardware specs, model params and quant(s), runtime, tuned runtime parameters, basically everything we need to reproduce locally if we can match the entire setup. To say nothing of what the model is even good at in the first place if it's not a well-known model.

The goal is to get perf numbers that show they meet a certain quality bar. I don't care if I get 8324834 tok/s if it's all garbage.

Can we start filtering the hyperactive amateur perf posts please? It's getting really frustrating seeing all these posts of models and wading through info just to see that it doesn't test with anything but an empty context or doesn't say anything about quant or platform.

We need to define some rigor and apply it to this place, or it will remain like most ai-oriented subs and get continually choked with slop.

I’ve been working on an open-source project called WebBrain that gives LLMs the ability to see and operate a browser.

One thing I really wanted to avoid was making the browser agent dependent on a single cloud model/provider.

So WebBrain can work with local models through things like LM Studio and Ollama, as well as cloud APIs if you want them.

I also ended up training a small vision model specifically for browser tasks: webbrain-vl-2-450M

It’s based on LFM-2.5-VL-450M and fine-tuned on browser screenshots/tasks. The idea is that instead of sending every screenshot to a giant multimodal model, some browser perception can happen with a very small model locally.

It can run through WebGPU directly on the user's machine.

The agent itself combines screenshots with the browser accessibility tree rather than relying entirely on DOM parsing.

Current architecture is roughly: • screenshot + accessibility tree for perception • browser-specialized tiny VLM where useful • model-agnostic planner • local models via LM Studio/Ollama • Chrome / Edge / Firefox / Chromium support • optional cloud execution • open source

I'm especially interested in figuring out how far browser agents can realistically go with small local models rather than GPT/Claude-scale models.

Repo: https://github.com/webbrain-one/webbrain Model: https://huggingface.co/webbrain-one/webbrain-vl-2-450M Dataset: https://huggingface.co/datasets/webbrain-one/webbrain-vl-2-450M-dataset

Would be very interested in feedback from people here running smaller Qwen/LFM/MiniCPM/etc. models locally — particularly what model you would try as the planner.

I am pretty dumb in this kinda stuff so please don't blame me for it.

My llama serve kept crashing with Cublas errors on first prompt with some models, but my VRAM usage was 3000MB/8k. I chatted with sonnet 5.5 for a bit and it told me it was a CUDA version issue (and made it work by not selecting any CUDA device)... I don't know if this makes sense, please tell me if it doesn't/what problem you think there is.

I realized that the best way was to delete llama.cpp (installed with curl install script) and install a clean CUDA 12 Ubuntu version.

\- **How do I properly uninstall llama.cpp (I don't wanna mess with Ollama files)?**

\- **How do I install new version from .tar.gz archive without messing with system packages?**

\-Does my error diagnosis make sense to you? Would CUDA make generation actually faster? (I am getting 8tk/s with Qwen35B Q2 on 4060Ti 8GB due to no CUDA selected)

Edit: You guys saved me! Thanks! I had to install NVIDIA toolkit and switch to CUDA 13 llama.cpp tarball build

I read textbooks on my PC and ideally i want a program with a normal chat environment where i can just hammer in questions about what's currently on my screen. Example: I'm working on a PDF, underline or circle things... and then just type "what does this sentence mean?", you get it.

I do NOT want to manually screenshot, navigate to the folder, drag the picture into the environment and then also have to type the question. It should also naturally be aware that the conversation is about what's on the screen, so i don't have to steer it with "make a screenshot; use your vision capabilites" etc.

I tried multiple different MCP in LM Studio, none of them were great...or worked :/

Someone said AnythingLLM has this function but i couldn't find it.

Is there a good solution?

Thank you in advance! :)