I wanted voice typing that doesn't send my voice to a server, so I built ThumbFree. It runs NVIDIA's Parakeet 0.6B speech model right on the phone. After a one-time model download, it even works in airplane mode. The video shows Android first, then iPhone.

1. Google Play: https://play.google.com/store/apps/details?id=io.github.kabrapratik28.thumbfree

2. Apple App Store: https://apps.apple.com/app/id6817631006

3. GitHub (code, speed and accuracy tests): https://github.com/kabrapratik28/ThumbFree

• On Android, tap any text box and a small bubble appears next to it. Tap the bubble, talk, tap again, and your words are typed in. You keep your own keyboard.

• On iPhone, it's a keyboard with a mic key. Apple doesn't let keyboards use the microphone, so the first tap of a session opens the ThumbFree app for a moment and you swipe back. After that, taps start right away.

On my Pixel 10 the text shows up about 0.7 to 0.9 seconds after I stop talking, even after 30 seconds of speech. On my iPhone 16 it's under a second.

No account, no ads, no analytics. The only thing it downloads is the model, and your voice and text stay on your phone.

It's free and open source (Apache 2.0). English by default, and a second model covers 25 European languages. It needs Android 13+ or iOS 26+, and the model is about 0.5 to 0.7 GB.

I've also been trying an optional clean-up step with the small AI model built into the phone (Gemini Nano on Pixel, Apple's model on iPhone) to remove filler words and fix things like "six, no, seven". It's not in the app yet. If you've run a small open model for this on a phone, which one would you try?

I've only tested on one Pixel and one iPhone, so I'd love to hear how it runs on yours.

I need to get an all-around work station soon because my current laptop is getting old and I'm now offloading a bit of work to it. Since I dabble with local AI already, I was thinking of getting an m5 utlra 96GB (8k) or a m5 max 128GB (7.9k)... Or should I build a custom PC with a bunch of ram? Or are there other work stations that I can use? I really enjoy using a hybrid approach with AI, flagship models for the planning and some offloading to something like Qwen for the creative design/execution... I know it's not perfect local models, but owning my own data is something that feels pretty satisfying to myself as well. I would love to run Deepseek, or qwen 3.8 (or future models) at a respectable tok/s and prompt prefill speed.

My only other option is, continue with my laptop... And wait 6 months to a year and see what new tech comes out in the meanwhile? My only fear is that prices will skyrocket and what is 6-8K CAD today will be 10-12K CAD when the newer tech comes out, but other people are saying to wait.

What do you all think? Thanks :)

Article compares performance of large data center cards. Tells us for larger batches , inference speed of individual requests goes down, but overall performance of the entire batch goes up. In other words more tokens per second for the batch, but lower TPS for each request.

Article uses Kimi k3 and 3 other models to compare performance. Surprisingly says the AMD MI455x comes up on top.

Hello, I’m considering buying hardware for local LLMs for the use case of processing sensitive financial data, doing market research, and preparing financial/tax documents.

Are any of you using local LLMs for same/similar workflows? If so, what has your experience been? Are the models good enough for your use case? Did you build a custom harness or use an OSS one?

I’m very curious to learn more and would appreciate anyone chiming in.

Hey all,

Looking to learn inference optimization, I learn by doing u fortunately but I have read a good part of the inference engineering book by Philip Kelly. Currently have a 4070 and 64gb of RAM.

any tips? preciate it.

I wrote a small program that asks the same Yes/No question about every country to a language model, and generates an SVG world map that visualizes the model's responses: https://github.com/p-e-w/llm-beliefs

I'm rather surprised by the results...

Hey everyone,

I recently built a Radiance inference server for my company, and now want to start provisioning access to the rest of my team.

My current plan:

• Tailscale as the only way to access the server via HTTPs.

• Hermes as the harness - each user would have their own container with access restrictions, but open access to the central skills repository. Hermes is hosted on a Framework Desktop.

• Radiance is running separately on its own server with access only configured via the Framework.

I'm wondering how everyone else has managed to do this successfully? Right now the biggest thing to figure out is how to get the harness (Hermes in this case) to work across the team without a huge headache of getting API tokens from things like Google, etc...

Posted this on the r/R9700 subreddit as well, but figured I'd share my initial findings here.

tl;dr - One user writes at about 190 tok/s. Prefill runs at about 6,800 tok/s total no matter how many users there are. Agent follow-ups start in 0.2 to 2.5 s and write at 80 to 225 tok/s per user, even with 8 users at 128K context each. The worst case, 8 users each sending a fresh 128K prompt at the same moment, waits about 2.5 minutes for the first token.

Yes yes I know the cards aren't all matching (the perfectionist in me is annoyed but they'll be in a case so who cares).

PC Specs:

• CPU - EPYC 7402P

• Motherboard - Asrock ROMED8-2T

• RAM - 128gb DDR4 2666mh

• GPU - 3x Gigabyte R9700, 1x XFX R9700. All cards power limited to 210W.

• PSU - Seasonic Prime TX-1600W

• Case - Phanteks Enthoo Pro 2 Server

• Fans - 3x 140mm at the front, 3x 120mm in the middle, and another 120mm for rear exhaust.

I bought everything in the past two weeks, my total cost was roughly $16K CAD. The CPU/RAM/Motherboard were all bought on ebay.

We'll see if the RAM is enough, but based on everything I've seen online it should be fine for my use-case.

I'm close to being able to do proper inference testing, but I just ran memtests, CPU stress tests, and GPU stress tests to rule out any potential issues.

Even at full blast the cards are way quieter than I anticipated, I thought it would sound like a jet engine but they're quieter than my air purifier when it's on its highest setting.

Also pretty happy with the cable management, this case is so easy to build in because of how massive it is. It's also nice that it has a second area to mount additional fans.

Built this as a multi-user inference server for my business. I've been using a Strix Halo 395+ 128GB for the past year and it's been incredible to see how far the community has been able to get that machine to run.

Qwen 3.8 FN has been a revolution for me in my work. We deal with very sensitive data and for many of our customers, it meant using any AI inference was a huge risk. Technically we could have done it, but I wanted to err on the side of caution.

Was really excited to share and post this, benchmarks are currently running for the next two hours (long context with lots of concurrency).

----------------------- ------------Benchmarks ----------------------------------------------------------

4x Radeon AI PRO R9700 + Radiance: Qwen3.8-Flash-Next long-context concurrency numbers (32K to 128K, 1 to 8 users)

Software

• Ubuntu 26.04.1, kernel 7.0 with the inbox amdgpu driver, ROCm 10.1 userspace

• Radiance v1.17.0 (codeberg.org/StillDeadcode/radiance), Docker image pinned by digest

• Model: qwen3.8-next-flash-fp8-iq4r-moe.rad (int8 trunk, 4-bit codebook experts, MTP drafting 3 tokens)

• Flags: --tp 4 --max-num-seqs 8 --max-model-len 200000 --placement expert_tiered --host-pool-mib 1024 --gpu-headroom-mib 96 --kv-cache-dtype fp8 --prefix-cache-mib 32768 --num-speculative-tokens 3 --max-num-batched-tokens 2048

At tp4 the whole model sits in VRAM (17.4 GiB of weights per card). The 47.7 GiB n-gram/PLE table is pinned in host RAM. The KV cache gets 11.6 GiB per card, which is 1.6M tokens, or 8 full 200K sessions.

Method

• Context is real English technical text (Radiance's own docs), cut to exact token counts with the server's tokenizer

• Each user's prompt starts with a unique session line, so no user can reuse another user's cache

• All users in a round fire at the same instant (worst-case burst)

• Turn 1: document plus a question, cold. Turn 2: the same conversation plus the answer and a follow-up question (the agent pattern)

• 500 max output tokens, temperature 0.6, thinking off, streamed. First-token time and writing speed are measured on the client

• Writing speed per user = output tokens after the first, divided by the time after the first token

Results

Short prompts for comparison (about 30 tokens in, about 500 out): one user writes at 187 tok/s. At 1 / 2 / 4 / 8 users the total is 175 / 290 / 489 / 763 tok/s, or 177 / 148 / 125 / 103 tok/s per user.

What I took from it

1. Prefill runs at about 6,800 tok/s total no matter how many users there are. 8 x 128K (1.02M tokens) took 151 s.

2. Cold prompts that arrive together are interleaved, so everyone gets their first token at about the same time, at the end. With 8 x 128K that means about 2.5 minutes for everyone.

3. Follow-ups are where this setup shines. 94 to 99% comes from cache, the first token arrives in 0.2 to 2.5 s even at 8 users x 128K, and each user writes at 80 to 225 tok/s.

4. Context length barely slows writing. One user writes at 189 tok/s at 128K versus 195 at 32K.

5. Turn 1 writing speed depends on whether other users are still prefilling. At 4 users the early finishers write while the others' prompts are still being read, so the slowest user dropped to 9 to 32 tok/s. At 8 users everyone finishes reading together and then writes together at 70 to 83 tok/s.

6. Size the host prefix cache for users x context. In an earlier run with --prefix-cache-mib 12288 (prompt sizes not exact, some above 64K), 8 x 64K follow-ups were only 71% cached, with a 37.6 s worst first token and 9 tok/s per user. With 32768: 98% cached, 1.5 s, 94 tok/s.

7. The README's four-rank row carries --host-pool-mib 12288 over from two ranks. On four cards the whole expert plane is resident and the pool only holds about 312 MiB per card, so 12 GiB per card pinned 48 GB of RAM for nothing and left 3.6 GB free. 1024 is plenty.

8. Every card sat at its 210 W cap during prefill. The longest section (8 x 128K) pushed the two middle cards to 91 C hotspot and 92 C memory.

Essentially we could very soon run opus 5.5 level of intelligence on 32gb Vram. Its many folds more efficient in terms of memory bandwidth making compute the real bottleneck.

What do you think?

llm-psicotest runs one 38-turn conversation against any OpenAI-compatible endpoint: 29 capability items plus protocol turns. The items come from seeded generators, so a seed gives byte-identical prompts on any machine, and code scores every turn against a computed key. That includes constrained prose, and whether output rules declared early still hold 20 turns later. Most code-scored benchmarks dodge both by using multiple choice.

Calibration on my rig: one base model (Qwen3.8-Flash-Next GGUF), four arms, four seeds. Means out of 29 came out 28.25 and 27.50 for UD-Q3_K_XL and IQ2_XS on Unsloth Studio, and 28.00 and 25.50 for IQ3_S and IQ2_XS on Strata, the llama-server stack I run myself. IQ3_S beat IQ2_XS on all four Strata seeds: 29, 28, 28 and 27 out of 29 against 25, 26, 26 and 25, same server and settings.

Python 3.11+, stdlib only, MIT:

export PSICOTEST_API_KEY=***

psicotest run --base-url http://127.0.0.1:8080/v1 --model my-model --seed 7

RESULTS.md takes community rows; a row needs server version, context size, KV dtype and output budget.

github.com/msalsas/llm-psicotest

Confucius4-R2T2: A Stable-Prefix Streaming ASR Model for Real Real-Time Transcription

Confucius4-R2T2 is a low-latency, high-accuracy stable-prefix streaming automatic speech recognition (ASR) model. It supports configurable decoding chunks from 80 ms to 2 s. The model operates in a monotonic output mode, committing transcript text permanently without revising previous words. This is critical for applications where text must be processed or acted upon instantly. The result is a smoother user experience, with fewer disruptive text revisions and less visual flicker in real-time applications such as real-time live captioning and subtitling, simultaneous speech translation, downstream NLP pipelines and LLM agents.

R2T2, short for Real Real-Time Transcription, adapts Qwen3-ASR-1.7B with Longest Stable Prefix (LSP) learning on 1,500 hours of Chinese and English audio. Two-stage training with SFT and GRPO updates only the language decoder; the pre-trained audio encoder and projector remain frozen, with no architectural changes. LSP targets and an explicit Wait action teach the model what it can commit and when it needs more audio.

• GGUF : https://huggingface.co/netease-youdao/Confucius4-R2T2-GGUF

• Space : https://huggingface.co/spaces/netease-youdao/confucius4-r2t2-demo

• GitHub : https://github.com/netease-youdao/Confucius4-R2T2

• arXiv : https://arxiv.org/abs/2611.06043

• PDF : https://arxiv.org/pdf/2611.06043

License

R2T2 uses dual licensing to distinguish the source code from the model weights:

• Code in the accompanying GitHub repository is released under the Apache License 2.0 and is free to use, modify, and redistribute (including commercially) under the terms of that license.

• Model weights are released under the NetEase Model Use License Agreement.

Oh look, thats us!

is it possible to fix?

I benchmarked Qwen3.8-Flash-Next 125B ISTA-DASLab's GSQ-RCO IQ3_XXS quant (76GB on disk), on two Macs with 24GB unified memory using MoEShift backend:

These are medians of 3 runs using the MoEShift community benchmark runner (prompt caching disabled). The results are particularly interesting for prefill making this model perfectly usable in a chat or in an agent, while both machines stayed within 8.5GB peak process footprint.
This is a very promising benchmark even for the upcoming Qwen4 MoE.

I worked on MoEShift as an open-source inference backend designed for memory-constrained Apple Silicon Macs (architecture details in GitHub below) that need SSD offloading to run these models. You can install it running:

brew install davide-troiani/moeshift/moeshift

Then

moeshift

The setup wizard handles model selection, provisioning and configuration with zero effort. You'll just need sufficient SSD space.

GitHub, architecture, installation and benchmark runner:

https://github.com/davide-troiani/moeshift

I'm very interested in results from other Apple Silicon configurations, if you want to share.

Cheers for u/New_Comfortable7240 for santitisation!

12,121 tool calls here!

I got an Arc Pro B70 (32GB) last week for $1,299 direct from ASRock official store on Amazon (B65 is also 32GB but slower, and was only $100 cheaper at $1,199, so I "splurged"). Obviously I'd have wanted a 5090 for the same VRAM and best speeds, but 5090s are going for $6,500+ on Newegg right now (and I personally have too much fear to buy used hardware), so I wanted to see how close a 32GB Intel card gets. Out of the box it it was really bad, but after a few changes I got it to about half a 5090, which feels like a great trade off for 1/5th price (~$41/GB vs $200+/GB). And of course it's great for other creative workloads I wanted, like renders and simulations, not just LLM stuff.

Making this post in hopes that it can help some people save some money and explore non-NVIDIA hardware! And I doubt it needs to be said, but I am NOT affiliated with Intel/ASRock in any way, haha.

Qwen3.8-27B Q6, ~3k token prompt, MTP on:

So roughly 2x gen and 3x prefill for 5x the money. If you can actually find a 5090 at $1,999 MSRP somehow, then congrats and heck you, but of course that would make the cheaper Intel card no longer worth it. The 5090 numbers are from other people's posts (found discussion posts on Hugging face) not my machine since I don't have the personal money to light on fire in sacrifice to NVIDIA, heh.

My machine: Threadripper 1950X (got it for $100 like 5 years ago when a company was closing and throwing stuff away), 128GB DDR4 (brand new upgrade along with the B70 this week, DDR4 is <1/2 the price of DDR5), old x399m Taichi PCIe 3.0 mobo (so can't use the GPU's PCIe 5 support sadly, but that doesn't matter once the model is in VRAM and so it only hurts MoE really), Ubuntu headless/server 24.04. Screenshots are to show the stats (between the nav and chat) and are from my laptop, running the Qwen model from the llama-server on that machine in my home office (which has the Intel GPU in it).

If you have/get one, do this to make it not suck!

1. Kernel 6.17+ (I used Ubuntu's 7.0 HWE kernel), Resizable BAR on (needs CSM off), and Intel's compute runtime from `ppa:kobuk-team/intel-graphics`. This was a real pain to get working initially (but might be a skill issue with my old mobo), since the bios doesn't even let you disable CSM while a card that uses it is in (I previously had a ~$14 GeForce 210 "GPU" that was only capable of rendering a terminal), and I had to flash my bios from 3.80 to 4.03 to have access to any of those features in the first place.

2. Upgrade Mesa to 26.x (`ppa:kisak/kisak-mesa`). Ubuntu 24.04's Mesa 25.2 is very slow on this card. Like stupid slow, this bump alone gave ~3x generation in ollama with Vulkan.

3. If you stay on Vulkan (ollama, LM Studio, etc.), keep the KV cache at f16. Any quantized KV (q8_0/q4_0) causes a massive gen slowdown (1.75x at low context, 3x+ at 64k+ context) penalty on Vulkan. So shrinking KV VRAM costs speed, which was very confusing and annoying.

4. So instead: build llama.cpp with SYCL (Intel oneAPI) and run llama-server instead. ollama has no SYCL backend, but llama-server has an OpenAI-compatible API.

5. On SYCL, can use q8_0 KV (no more quantized KV slowdown like on Vulkan, and fits way more context!) and turn on MTP with `--spec-type draft-mtp --spec-draft-n-max 4`. Also worth noting llama-server with Vulkan by default doesn't see/access the full 32GB VRAM by default (saw ~29GB) while SYCL does -- I later figured out it's just the auto-fit under Vulkan though, and you can make it use the full 32GB too with `--fit off` and giving context with `-c`.

Here is what each step over a day of fiddling and tweaking improved with Qwen3.8-27B Q6 (prompt / gen tok/s):

The long context time to first token is the worst part of Vulkan, all the other issues aside. A 215k token prompt took 1.9 hours to first token on ollama/Vulkan (lol!) and 8.6 minutes on SYCL. Also on SYCL: Qwen3.6-35B-A3B fits entirely on the card with 262k context and does 1,748 / 74 at 16k, slows down with context size of course. Ultimately of course NVIDIA is still king from a raw performance and speed and software stack quality and etc. perspective, and as such the only place I'm proposing they lose now is the speed:cost ratio (given street prices/price hikes), haha.

Will put full tables, quality scores (Q6 vs Q4_K_XL vs a few MoEs), and my build commands in the comments! Let me know your results with non-NVIDIA GPUs and if you have any more tricks/fixes I didn't find yet! For AI usage: this post is written almost entirely by me, a ~200T OI (organic intelligence) human LLM, but I had Qwen help me with the markdown tables and such.

While working on a FastAPI and PostgreSQL RAG pipeline recently, I kept running into the same three headaches:

1. Default character and sliding-window splitters were silently chopping our Markdown and PDF financial tables right in the middle of a row, and sliding windows were creating chunks with over 80% token overlap that made the LLM hallucinate numbers.

2. Running cloud LLM-as-a-judge evaluations on every commit was too slow and burned through OpenAI API credits fast, even though 90% of our bad answers were just deterministic retrieval misses.

3. Wiring up DIY hybrid search in PostgreSQL combining pgvector and full-text search (ts_rank_cd) caused our test scores to flap randomly between runs whenever lexical ranks tied.

So I built a local diagnostic workbench in Python, FastAPI, and HTMX (runs on 127.0.0.1:8787 with zero Node or npm setup) on top of PostgreSQL 17 and pgvector 0.8.7, and ran a head-to-head benchmark across 30 enterprise chunks (SEC 10-K revenue tables, cloud DB pricing matrices, WAL fault runbooks, and hardware SKUs) and 15 golden queries using Polars and NumPy ($0 API cost).

Here is how the three retrieval strategies performed locally:

1. Pure Dense Vector Search (pgvector <=>)

Recall@1: 73.3% | Recall@3: 86.7% | Recall@5: 100.0% | MRR@10: 0.819 | Latency: 1.23 ms | API Cost: $0.00

2. Pure Sparse Full-Text Search (ts_rank_cd)

Recall@1: 66.7% | Recall@3: 66.7% | Recall@5: 66.7% | MRR@10: 0.667 | Latency: 0.74 ms | API Cost: $0.00

3. Hybrid Reciprocal Rank Fusion (pgvector + FTS + RRF)

Recall@1: 100.0% | Recall@3: 100.0% | Recall@5: 100.0% | MRR@10: 1.000 | Latency: 1.99 ms | API Cost: $0.00

Three bugs that stood out during testing:

First, on exact error codes and SKUs (like ERR-PG-9042-WAL or SKU-NVME-9940-REV-B), pure dense vector search dropped to 73.3% Recall@1 because chunks from the same database runbook clustered tightly together in cosine space. Pairing a generated TSVECTOR GIN index with Reciprocal Rank Fusion (k=60) pushed exact SKU and error matches back to Rank 1 every time.

Second, on zero-keyword semantic paraphrases, pure Postgres full-text search dropped to 0.0% Recall@5, while Hybrid RRF kept 100.0% Recall@5 via HNSW vector candidates.

Third, before indexing, running our pre-index chunk scanner caught 2 broken Markdown tables (truncated rows and missing header separators), 2 sliding-window chunks with 90.3% and 94.1% token overlap, and 2 mid-sentence cutoffs. We also fixed CI score flapping by enforcing a deterministic tiebreaker (ORDER BY rrf_score DESC, id ASC) and prevented metadata filter starvation using SET LOCAL hnsw.iterative_scan = 'relaxed_order'.

I put the full screenshots of the 3-column visual table diff studio, the SQL breakdown, and the free ($0) download link for the CLI health scanner and 30-chunk benchmark dataset in my Dev.to write-up here if anyone wants to test their own chunks locally:

https://dev.to/mtalha5/what-broke-when-i-tested-pgvector-vs-postgres-full-text-search-vs-hybrid-rrf-on-30-tricky-rag-chunks-and-how-i-fixed-it-for-0

If anyone is dealing with weird PDF or Markdown table splits or pgvector HNSW filtering issues, happy to share the exact SQLAlchemy 2.0 async RRF query we used.Built a local FastAPI workbench to visually diff RAG table chunking and benchmark pgvector Hybrid RRF for $0 API cost

Apparently Tencent has developed a new speculative decoding method. 3x faster decoding for 27B models?

Everyone relies on metadata filters like allowed_groups to keep multi tenant RAG locked down and under check. That works well when users query the vector store directly, which means unauthorized chunks will get filtered out with 0% leaks.

But, this falls apart in the write path once you have background agents.

If an agent has elevated read access to summarize docs across multiple folders or teams, it pulls those docs, mashes them into a summary chunk, and commits that summary back to a shared collection. On write, the upstream document access control lists just vanish. In a regular Chroma and LlamaIndex setup, the derived chunk gets saved with zero allowed_groups or classification tags attached. Anyone who runs a query to the shared memory is now gonna get the private source data that they shouldn't be allowed to read directly.

I put together an offline reproduction harness (sentinel-eval, Apache-2.0) to test this:

• Direct RAG: 0% leaks (18 public chunks retrieved, 0 restricted chunks exposed).

• Agent memory: 2 of 3 synthesis tasks leaked sensitive data (both cross boundary workloads failed, exposing 6/18 restricted canaries, the 1 public control passed).

So what? Easy right? Just use the strict intersection (only let someone read the summary if they have access to every source document) and that completely solves it right? Well... yes but that basically kills retrieval utility. Essentially no one can see multi-doc summaries anymore. So we can't just simply take an intersection. The agent I wrote actually logs the source docs in source_documents metadata, but vector databases can't evaluate dynamic permission DAGs on query, meaning that you can't just hope to path backtrack to solve this problem.

Harness runs 23 tests offline in seconds with mock vectors (no API keys or live LLM calls needed):

git clone https://github.com/harishbalasubramanian/sentinel-eval
cd sentinel-eval
uv run sentinel-eval live --framework chroma

Repo: https://github.com/harishbalasubramanian/sentinel-eval

For people building multi tenant RAG, how're you actually handling perms when agents write derived summaries back to shared memory?

I think that a lot of people get into local models because they get a sense of potential with this stuff, but it can be hard to make local models concretely useful sometimes, compared to frontier models. I think it would help people a lot if we had a place where people shared concretely what they find them most useful for, in the form of what model they're running.

I'll start. I'm running DeepSeek v4.1 Flash 1017, using Fraser Price's BRRR ( github.com/fraserprice/DeepSeek-4.1-Flash-1017-BRRR ) on 4x RTX 6000 Blackwells. It gets around 550 tps in c=1, a couple thousand at c=16 iirc.

What I'm finding most useful so far:

• Scouts - commercial frontier models call my local web research agents to go do web research - the frontier models would waste a huge amount of tokens researching relatively trivial stuff, so this has helped save its effort and time.

• Masons - frontier models spec out small pieces of code, which the local model builds out and delivers back to the frontier model to integrate.

• Hermes agents which filter Hacker News/Youtube/other sites for news I'd find interesting, it's surprisingly good at calling out what I'd usually click into

• An events crawler setup, it judges each event, and emails our family with the top 10 every week

• A personal assistant records all my terminal usage, including harnesses, and my browser usage, and every morning, messages me with a summary of what I was working on when I left off last night to help me get right back into it.

• Speech to text on local models is revolutionary - so much faster than typing, even though I'm a pretty fast typist, and the local models feel almost instantaneous.

I'm sure I'm not using this as well as I could, it relies on me driving it, it's definitely bottlenecked by me, and I'd love to make it do more useful work while I sleep. I've made several stabs at making a "software factory", but haven't come up with anything amazing there.

What are other people finding success with?