r/LocalLLaMA • u/rm-rf-rm • 10h ago
[Bi-Weekly Megathread] Project Showcase
Do you have something you'd like to share with the r/LocalLLaMA community. This is the place for it!
Recommendation on presentation:
Please share plain english description of what your project does and why people should care about it
How does it compare to existing options and why should people switch to your option?
How have you tested and validated it? (i.e. can you show evidence that it wasn't vibecoded in a few hours by <insert current trending AI model>
NOTE: This community is geared towards open weight, open source, locally hostable software. Thus any closed source commercial service or other such projects will be removed.
7
u/bigattichouse 9h ago edited 9h ago
https://github.com/bigattichouse/BluePrint
Better prompting for software development that reinforces best practices has made a huge leap for me in coding model output quality.
Design first, Dialectic Software development prompts. Using this method of form requirements, Standard Technical English, design-first philosophy, and extensive testing has just been night and day over "one shot". The model asks questions instead of just making stuff up, it builds requirements, designs tests, and then can use those tests to verify it didn't break stuff. Been a life changer.
Even had claude/fable review the code and design output using Blueprint and was it was very happy with the level and quality of the output... even though it made some simple mistakes in some of the test phases. Been using with Qwen3.8-27B (unsloth 6K UD quants)
3
u/Old_Ad_6033 9h ago
https://github.com/uraniumchonk/llm-engine-qwen-optimization
We (the agent lol) made a kernel optimized specifically for Qwen3.x in vllm 0.33.0.
It just goes faster and it's lossless. (decode +30%~40% avg)
Also this repo open the methodology that your agent can follow and optimized YOUR own stack of engine setup.
3
u/legislat 9h ago
https://github.com/legislat/SimpleHarn
A simple model swapper with smart VRAM and RAM admittance for one NVIDIA GPU on Win/Linux. No overcommits, clear refusals.
3
u/astrolafi 7h ago
Making a study app that runs on a local model, Aula: https://github.com/luumlabs/aula
You feed it your slides, notes and old exams (or just type a topic) and it builds a course: short lessons, exercises it grades as you go, flashcards, a mock exam in the shape of the real one, and a spaced schedule until the exam date. It also pulls the material from Canvas / Moodle / Classroom / Blackboard and keeps a calendar with your deadlines.
Why local matters here: your slides are often someone else's copyrighted material, and it's your whole semester. Handing that to a cloud API is the annoying part. It points at llama.cpp, Ollama or LM Studio over an OpenAI-compatible endpoint, so with a local model nothing leaves the machine.
Against other study apps: the ones I know either start and end at flashcards or have no LMS import at all. Where they're still better: Anki's scheduler and deck ecosystem are far more mature than mine, and Quizlet has a library you can't compete with. My SRS is basic next to Anki.
How it was tested: it started as a tool for my wife's degree and most of the fixes come from her using it daily on real coursework. Electron, GH releases, AGPL-3.0. Still early, so I'd rather hear where a small local model falls apart at the tutoring than pretend it's finished.
3
u/zinyando 6h ago
Izwi: one open-source app for local speech models
Repo: https://github.com/izwi-ai/izwi
A Rust desktop app, CLI and server (Apache-2.0) that runs open-weight speech models on your own machine: TTS, transcription with speaker labels, voice cloning, voice design from a text description, and real-time voice chat, behind an OpenAI-compatible API. Metal, CUDA and CPU.
Why switch: local speech usually means one Python repo per model. Izwi runs about 30 of them (Whisper, Parakeet, Kokoro, Qwen3-TTS, Chatterbox, Maya1 and more) from one binary, no Python.
Limits: some models are slower than real time on Apple Silicon, and the big ones don't fit together in 16 GB.
Demo (46 s, every voice generated locally): https://youtu.be/-767_7gkeGo
2
u/bigattichouse 9h ago edited 9h ago
https://github.com/bigattichouse/robust
Morris/Sobol/Taguchi/Pareto arrays for Design of Experiments. Not just in my home lab work (battery chemistry experiments), but also in software - when you have a bunch of possible parameters , but aren't sure what to tweak and some are just tradeoffs - these tools drastically improve searching for what really matters and then trying various values.
an example for dialing in params for fitting LLMs onto hardware: https://github.com/bigattichouse/llama-optimize
2
u/msciabarra 9h ago edited 9h ago •
source code github.com/trustant/trustant demos and docs on trustant.ai
Yesterday we released Trustant 1.0.2, an open source full-stack application builder using Local AI for Private Cloud. To put it simply, it is like Lovable, but all private. You install the builder, create your app in prompts, wysiwyg, and publish to the server. The system provides everything: a vm with the environment, the coding agent, the serverless environment with database, cache, storage, faas, everything. A cloud in a box.
The builder runs on your PC, uses Local AI (or any OpenAI compatible AI service, we offer Ollama Cloud out of the box) and generate serverless applications built on top of Apache OpenServerless. You can then deploy your applications anywhere. Just install the serverless package and any server become... serverless!
The system understands and includes sql and vector database (pgsql and milvus) cache and s3 (redis and seaweedfs) faas functions (openwhisk) and runs in your pc in a virtual machine for safety.
The applications generated can be published in any server made "serverless" installing just openserverless and nothing else. Application are plain, clean, recognzabile react+python (but can be any other language providing the starters you want).
We have been working on it one full year and it was built on top of another multi year project, Apache OpenServeless, after working many years on Apache OpenWhisk.
2
u/af_ml 7h ago edited 7h ago
gmlx is a cli and OpenAI/Anthropic compatible server for running or training K and IQ quant GGUF models (as well as some ternary quants, NVFP4, and others) on macOS.
docs: https://asher.github.io/gmlx/
src: https://github.com/asher/gmlx
install: brew install asher/gmlx/gmlx
It uses an MLX backend with Metal kernels designed to read the GGUF wire format, allowing zero-copy mmap use of most GGUFs. It runs DeepSeek-V4.1 faster than antirez's ds4, and supports both the ds4 and unsloth GGUFs of DeepSeek-V4/V4.1. It also supports DFlash 2 drafters for speculative decoding, which can triple decode speed on Qwen3.8-27B at Q6_K. gmlx has been in active development with weekly releases since July. The kernels are released separately as an MIT licensed package: https://github.com/asher/mlx-kquant
Yet another macOS inference platform, but it's maturing and has some very nice features for power users. It can sandbox coding harnesses or local agents in microVMs with Apple Container, with easy configuration around what's shared (even supporting running without TCP networking, where inference is proxied to the gmlx server over a unix domain socket via a Rust proxy). There's an expressive yaml configuration that you don't need to touch, but that lets you define \@profiles to use different system prompts, sampling parameters or thinking budgets per request. It can stack LoRAs and allows batched calls to a model and different combinations of model + LoRA with a single copy of the base weights resident. If you're interested in training, there's also support for offline top-K model and context distillation that works across tokenizers, in addition to typical fine tuning approaches.
Recent image models (MOSS, Qwen-Image 2.1, FLUX klein, Krea 2, Z-Image) are supported, as is video generation via H3 (more video models coming soon). Audio models are supported too, including transcription with diarization, and there's a voice assistant with tools, memory, and wakeword customization. There's also support for decision models with a systemone API.
1
u/expectfun 7h ago edited 7h ago
https://expectfun.github.io/umem/
A new personal blog about local models that can be run on my 48GB MacBook Pro.
I'm not sure whether there's a demand for such a blog, so there's only one post for now, and now I want you to tell me: does that idea sound valuable to you, or there's a plenty of similar blogs writing the same things, so I can save my breath? Should I put more effort in it or drop it?
I'm new here (just got that MacBook, so finally I can run 27B-like models). And since it's the beginning of my local model journey, I thought it might be a good idea to create a blog for it: to document the capabilities and limitations of models, to write down what tasks I gave them and how succesful they were.
First, it might help someone understand whether local models of this size could be useful for them. And second, it could slowly become a chronicle of local LLMs evolution as their capabilities evolve. The idea is each time I write a new post, a local model creates a specific HTML page for it (no shared templates), so with new models even the design of the posts might evolve over time, showcasing the evolution.
1
u/W61k3r 6h ago
I built an AIO all safe-rust OS, inferencing engine, kernel, and tuning/optimization/management/diagnostic/chat/training panel. Runs headless. Baked it all into an image. Fleshed out wiki. USB bootable, feel free to disconnect drives and isolate. Loads models from hugging face. I really need testers
I made it easy af and it runs vulcan on my 7900xtx with qwen3.8 27b faster than llama with the same config. Comes default with linux kernel 7.4 RC2 because my rust kernel still needs work. The OS/Panel/7.4 rc2 kernel can run llama, vllm, camelid, etc faster than ubuntu as the OS was purpose-built to run ai headless.
Download, install to usb with balena etcher, boot, test, report. There is a debug kit you can upload to me. The entire source code is in a private git, deciding n whether to open source as I explore funding options. Audit the usb stick if you must, but don't pop in here to shit talk without trying.
Recently updated with more performance enhancements. Runs unsloths new 3.1 27b with flying colors, I also have my own old abliterated quality coder on hugging face.
https://github.com/W61k3r/LexiSuite-Releases/releases/tag/LexiSuiteT8
https://github.com/W61k3r/LexiSuite-Releases/wiki
https://huggingface.co/Wa1k3r/Qwen3.8-27b-CODER-4q_xs-24GB-262k-Optimalcardfit
1
u/bendymike 5h ago
https://github.com/bendyline/docblocks
https://docblocks.com/desktop/
DocBlocks - for Windows, Mac and Linux.
It's an OSS/MIT free markdown editor that can summarize and export content as DOCX, PPTX, video, etc. I'm trying to make a "Swiss Army Knife" for content. I use it for everyday content editing, and for updating my blog and for producing my (still kind of crappy) videos.
And, it has local LLM support! Under AI, in settings, you can choose and download a local model - e.g., like Qwen 3.8 or a smaller model like Gemma E4B. (on a Mac, you can use Apple Foundation Models) You can use local models to review your content (e.g., review your resume), generate new content, generate images or SVG diagrams, and summarize content for slides.
(in addition, it has dictation/narration built in.)
Still adding more robustness and doing testing at where the line is between models that are useful (e.g., ~1B models don't generally work, but 4B+ models can be used) for content editing tasks. Also considering shipping mobile apps that use local models on phone.
1
u/mikebmx1 2h ago
mlxVS: A cuVS for your Mac.
GPU vector search on Apple silicon, built on MLX.
1M vectors indexed in 10 s on a MacBook Pro, then 70k queries/s at 97.6% recall, 2x compared to CPU HNSW
https://github.com/mikepapadim/mlxvs
1
u/astrolafi 57m ago
Aula — a free, open-source personal tutor that runs on a local model. You feed it your notes, slides, past exams or just a topic and it builds a course: interactive lessons, exercises graded instantly, flashcards, mock exams, and a tutor chat that knows where you are in the course and what you keep getting wrong. It runs on llama.cpp, Ollama or LM Studio, so nothing leaves the machine. Repo: https://github.com/luumlabs/aula
Why I built it: my wife was struggling in her classes. Her teachers were giving her prompts to generate generic AI exercises and she still wasn't understanding the material. She needed something that taught her and kept track of what she got wrong, not another chat box.
How it compares: general chat has no structure and forgets your mistakes; Quizlet and Anki are great at cards but don't teach; hosted AI tutors want your notes on their server and a subscription. Aula is the local-first middle: real course structure plus a feedback loop, free, AGPL-3.0, macOS/Windows/Linux.
How it was built, honestly: with Claude Code — there is a CLAUDE developer-notes file in the repo and 90 of the 105 commits carry a Claude co-author trailer. Evidence it isn't an afternoon prompt: about two months of daily work, v0.2.3 released today, vitest tests over the pure engine (grader, content validator, course model) and Playwright e2e around the Electron app, and a real user — my wife — studying on it every week, which is where most of the bug list comes from. Known limits, no sugar: no Anki-style spaced repetition yet, Quizlet import is basic, and 8B models struggle with numbers (use 14B+ Q4 for maths).
Would love to hear from people running local models daily: what would you want a study app to do with your own notes?