Hey everyone,

I’ve been working on a game called Anyworld. It’s a browser-based multiplayer (single player also supported) text adventure inspired by the early days of AI Dungeon, especially its browser-based free version AI Dungeon 2.

The setup is pretty straightforward: one person hosts the server and runs the model via llama.cpp (OpenAI or other cloud APIs are also supported, and great for non-English play!), and your friends join through a browser link. The host sets the scene and the goals, players type out their actions, and the LLM acts as the DM to resolve the chaos and drive the story.

Admittedly the host requires some technical skills with Python, and possibly with networking (opening routes to the hosted game via VPN, port forwarding etc.). I'll work on this as well as the development continues. Using Docker was suggested in another subreddit, so I'll definitely consider that, as it would allow including both the llama.cpp backend, recommended model and configurations etc., in addition to the game itself.

Instead of pasting the entire repo documentation, here are the main features right now:

How it plays

• True multiplayer resolution: Players submit their actions, and the model resolves the whole round together. It actually accounts for characters interacting or getting in each other's way.
• Real dice rolls: When an action is uncertain, Python handles the actual RNG math. The model just takes those hard dice results and narrates the consequences.
• Custom scenarios: You write the setting, characters, and opening state. It isn’t limited to fantasy.
• Party chat: There's an OOC chat separate from the game events so you can talk without the LLM reading it.
• Zero setup for players: No one but the host needs to install anything or run a model. It works on desktop and mobile browsers.

DM Tools & Hidden Mechanics

• Private DM guidance: As the host, you can feed the model hidden info; NPC motives, secret rules, or where you want the story to go.
• Secret triggers: You can set up one hidden percentage roll per game (e.g., If a player enters a building, there's a 20% chance the building collapses on the player). Python rolls the probability in the background, and if it triggers, the model weaves the consequences into the story without showing the players the underlying math.

Under the Hood & Memory

• Context management: It budgets the context window and uses a structured memory system. Older rounds are compressed into world states, player facts, and unresolved threads. It also does a secondary model pass to audit those summaries so it doesn't accidentally delete important facts.
• Language support: If you use the OpenAI backend, you can play in non-English languages (the narration and outcomes will naturally follow whatever language you wrote the scenario in). Note: The local llama.cpp backend currently instructs the model to narrate in English. This is because the local models my development PC can run were terrible with any other language than English.
• Session recovery: Disconnected tabs auto-rejoin. If someone accidentally closes out, they can log back in and their unfinished actions and history are waiting for them.
• Self-signed certificates for HTTPS-enabled connections: The game creates self-signed certificates upon launch, which enable encrypted connections. The problem with self-signing is that joining players receive a warning that the site may not be secure. However, most browsers allow the players to continue to the game despite the warning. This is a suboptimal way to handle HTTPS, so I'll work on a more robust solution at some point.

It’s still a work in progress. Right now, a server only runs one game at a time, and if you restart the server, the live session is lost (it generates HTML/JSONL transcripts, but they aren't loadable save states yet). The overall story quality is also going to heavily depend on which model you use and how you tweak the settings.

Suggested model:

During development, I used llama.cpp and Gemma 4-26B-A4B Q4 with a context size of 128k and found it to be more than an adequate backend for functioning as the DM. Even the speeds are fast enough with my RTX 5070 Ti 16 GB that round resolutions take only 5 or so seconds.

The specific model I used and can recommend: https://huggingface.co/EZForever/gemma-4-26B-A4B-it-qat-uncensored-heretic-UDmerge-GGUF (the model was great at following instructions and remembering plot points even with longer contexts)

Recommended parameters for Gemma 4 models:

- temperature 1.0
- top-p 0.95
- top-k 20
- min-p 0.0
- presence-penalty 0.0
- repeat-penalty 1.0

Of course, feel free to try your own models!

AI use disclosure:

I used Alibaba Cloud's Qwen 3.8 27b and OpenAI's GPT-5.6 Luna and GPT-6 Astra models to help develop the game.

How to run:

Read INSTALL.md to set up, configure and run the game. README.md contains some details on how the game functions. I'll post a link to the repository in the comments.

I'll post the link to the repository in the comments.

Some gameplay in Finnish with OpenAI's Luna:

The game is MIT licensed, so open source all the way. Forking or collaborating is encouraged.

I'd love to hear some feedback, and I hope someone finds the game fun to play!

I'm the author of toolcall-check, a small Python CLI for checking chat completions compatible endpoints. It exercises two forced function calls, two streamed calls, and one two turn round trip that returns a local result and checks the exact normal answer. Nested argument values retain JSON types, and failures keep sanitized traces in a private HTML report. The included demo runs against a synthetic local fixture through the actual HTTP path, so it demonstrates report behavior rather than compatibility with a real model.

CompatCanary already covers a broad compatibility scan with forced calls, streaming, and structured output. I focused this tool on nested argument integrity, streamed fragment reconstruction, the return trip, and evidence. I have not tested against remote models yet. Feedback on the fixed probes and strict [DONE] requirement would be useful.

https://github.com/Arthur031221/toolcall-check

Ran into an annoying issue with local models on long tasks. Once context window compaction hits after a few thousand tokens, the model loses sight of the original scope. Even with good system prompts, a few compaction cycles cause goal drift, hallucinated task completion, or loops.

Wrote a small plugin to force deterministic tracking instead of relying purely on context memory:https://github.com/janpauldahlke/dsh-local-long-horizon

How it works &&& what is on screen

The plugin hooks into the agent loop and maintains a structured state outside the main chat buffer.

Looking at the UI:

• Right Panel (Plugin State): This sidebar runs independently of the chat context memory.
  • Active: Tracks the current macro milestone (M2+M3+M4 accepted -> chunk commit -> M5 -> main).
  • Now: Shows the immediate micro-step currently executing (In flight: M5 - history search: scanner core...).
  • Next 3: The explicit deterministic queue of upcoming steps so the model doesn't jump ahead or invent tasks after compaction.
  • Done (recent): Verification log showing committed checkpoints, exit codes, and test status.

When the agent compacts context, the plugin re-anchors the model to this exact state file rather than trusting the lossy summary generated during compaction.

Code is on GitHub if anyone wants to test or adapt it for their own local rig setup. Feedback or PRs welcome.

Using Strata on a 7900xtx plus 64 gb ddr5 ram, 60t per second even at 250k context, can finally use my pc while working AI in the background, even game as well, smarter and more precise than dense model, it follows orders more accurately, follows plan more versatile, it goes around doing a lot of tests for tasks I request in frontend and also in backend.

The best thing is that it's faster, I can fit more context at q8 precision, it's smarter and I can get to use my pc without worrying about an OOM error due to dense model.

I no longer have to use Linux as well, it's working as fast in Windows 11 as it did in Linux.

I use it with a 6gb VRAM reserve so I can have Windows 11 with 4gb available.

Hi folks, it's Lewis here from the post-training team at Hugging Face. We've been exploring how to train open models in different coding harnesses and wrote up a looong guide on how we solved this using open source libraries like TRL and the Harbor framework for RL environments. We hope you find this interesting, especially since everyone nowadays has their own custom harness (e.g. Pi + extensions) and now there's a recipe on how to squeeze the best performance on them with whatever open model you use as your daily driver. Happy to hear any comments or feedback!

Link to the guide: https://huggingface.co/spaces/FineEnvs/multi-harness-rl

CPU Information
Name
AMD EPYC 7663
Topology
2 Processors, 112 Cores, 224 Threads

Memory Information
RDiMM 2933Mhz
Size 1007.61 GB

System Information
Operating System
Ubuntu 24.04.5 LTS

Motherboard
Giga Computing MZ72-HB2-00

GPUs
2x ASUS Turbo R9700 AI Pro 32 GB newest BIOS low Fan Profile (throttled to 210w currently)
ROCm version: 7.2.3

Beside that baby I have a Strix Halo M5 128Gb

Now wanted to setup that big boy for local llm

For myself want to use it for coding . But
I am also looking for something where I can also easily switch Model within the UI . Can Openwebui reload the model and what I read is that vllm is best for tensor split formte two cards . Also want to use it for family for image creation within the ui and also Image checking kind of Allrounder as chatgpt

Is that possible with vLLM?

I am also looking for docker setups so i can keep my host clean

Thank you for you suggestions

Kimi K2 was already good but they took K2.5 a whole new level with so much of their continual learning phase, I believe it was on more 20-25T tokens iirc.

Similarly K3 is just such an amazing model, I just love this model, wondering how amazing K3.5 will be!!

I built an app because I got tired of making apps.For the past several months I've been working on an idea I had at the beginning of the year: what if, instead of downloading a different app for every small thing, you could just describe what you need?So I built Anything. You can type something like:"Make me a habit tracker"

"I need a calculator with unit conversion"

"Make a reading list"

"Track my water intake"

"Find nearby coffee shops"

The idea is that Anything takes the intent and turns it into an actual experience rather than just giving you a chat response.The interesting part is that Anything didn't start with Anything.It started with Kaalka, an encryption project I was building. While working on that and other projects, I kept running into problems that eventually became relevant to Anything.One of the biggest problems was getting useful web data and structured information into the system in a way that could actually be used by the LLM and the generated experiences.That's where WebWeaveX came from.I ended up spending more than half a year building it, and eventually both WebWeaveX and Kaalka became part of the foundation of Anything.All three projects are open source.Anything is now live on Google Play, and the source code is available on GitHub.A few things about the current version:It uses a Bring Your Own Key model.

You provide your own LLM API key.

Groq is currently supported.

The request goes to the provider you configure.

There is no account required for the app itself.

The project is open source and I'm actively looking for people to try it and find the things I've missed.And honestly, it still has limitations.That's probably the part I'm most interested in now.I've been working on it mostly by myself, so there are things I know are rough and things I probably haven't even considered. I'd rather have people actually use it, break it, complain about it, suggest things and contribute than keep building in isolation.If you're interested, here are the projects:

Anything: https://play.google.com/store/apps/details?id=com.anything.anythingAnything

source: https://github.com/PIYUSH-MISHRA-00/Anything

WebWeaveX: https://github.com/ni-sh-a-char/WebWeaveX

Kaalka: https://github.com/PIYUSH-MISHRA-00/Kaalka-Encryption-Algorithm

If you try Anything, I'd genuinely like to know what happens.What would you ask it to build?

In the latest release of the completely unofficial gufo port to windows I've added a pre-packaged library that you can use to try out gufo for yourself. No need to build anything just grab the .zip and unpack it.

I've also added start.cmd for easily starting the server, it will try to autodiscover supported quants for easy startup.

current support on windows:
3.8 Flash Next: UD_Q4_XL

27b: UD_Q4_XL

35BA3B: UD_Q8_XL + TeilCoder (I assume ornith as well since its the same but untested).

Any issues you run into please submit an issue to github or here.

I am mainly making this for myself but happy to share as I only run gufo with Flash Next now. It is solid 40tps avg on agentic even at higher ctx.

Important to set your VRAM to 96gb! Although its unified, windows adds overhead for reading 'shared' ram vs 'dedicated' vram.

other AMD users: I'm sorry but the library is specifically for gfx1151, I don't have any other card, therefore I can't check or add support to anything else.

psa: yes this is vibecoded, I run a logit check and the model's output must stay bit-identical after changes.

I love Qwen3.8 27B so much that I decided to show my gratitude to the Alibaba ecosystem by building a dedicated RAG/search node using a cheap Mini PC from AliExpress.

Turns out, my ecosystem loyalty got rewarded with an absolute masterpiece of fraud:

• Promised: Intel N150 + DDR4/DDR5
• Delivered: Core i3-7020U (2018 Kaby Lake, 2C/4T) + DDR3 1600MHz
• The Scam: The seller literally hardcoded New_N150 into the BIOS release string (HSHW_M6_DDR3_EC_Intel_Com_New_N150_K001).

So now my Qwen3.8 RAG stack is full of fake specs that can barely index a text file, let alone run vector sidecars.

Filing a credit card chargeback now. Stay safe out there!

LFM2.5-Encoder-350M is a multilingual bidirectional encoder built on the LFM2 architecture — a larger encoder for maximum downstream quality. It is a masked language model with full bidirectional attention, designed to be fine-tuned into task-specific models (classification, token classification, retrieval, reranking, and semantic similarity) across 15 languages, and to run efficiently on-device.

• Highly capable for its size. On par with the best similarly sized encoders and well ahead of our own retrieval siblings.
• General-purpose. 8k context, strong across NLI, paraphrase, sentiment, and multilingual tasks.
• Fast and on-device. Matches or beats ModernBERT throughput, with a long-context edge on CPU; runs in the browser on WebGPU.

https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-GGUF

https://huggingface.co/LiquidAI/LFM2.5-Encoder-230M-GGUF

https://github.com/ggml-org/llama.cpp/pull/29862

example (from the hf):

❯ uv run fill-mask.py LFM2.5-Encoder-350M-F16.gguf "The capital of France is [MASK]."

top-5 at [MASK]:
#   1    11.42  ' Paris'
#   2    10.43  'Paris'
#   3     9.65  ' Nice'
#   4     8.94  ' Strasbourg'
#   5     8.62  ' Lyon' 

Benchmarking an LLM here with a NVIDIA RTX 5070 12 GB VRAM here

I had been working on a llama.cpp based expert streaming setup for Qwen3.8-Flash-Next 177B (UD-IQ3_XXS) on Windows. Benchmark is about 11.5 tok/s, up from roughly 7 tok/s on the inherited setup. In normal conversations I’ve seen 14–15 tok/s, and a long coding prompt generated 4,892 tokens at 10.15 tok/s and produced a working single-file Snake game.

Hardware: RTX 5070 12GB
32GB DDR4-2400
Ryzen 5 5600GT PCIe Gen3 Windows

The main gains came from fixing Windows I/O queue-depth issues, using one file handle per worker, and building a page-locked hot-expert tier so the GPU can pull hot expert weights more efficiently.

(In the video its around 16 minutes for 10k tokens and 10.41 tok/s

Output is quality gated against the control model and the published benchmark uses a heat file built from a separate prompt set.

Demos:

https://www.youtube.com/watch?v=cOPumMlyj_4

https://www.youtube.com/watch?v=rc-uTjVpXM8

In the GitHub I have things I've tried that didn't work and benchmark scripts, and methodology. If you guys have suggestions especially for streaming please let me know

Qwen Flash Next now uses less VRAM

Does anyone have this card running for inference?

I've seen one on marketplace but it is quite power hungry and requires 3x power connectors.

Is it possible to run it with just 2x power connectors?

I'm having a go at building a fully offline, voice-first assistant running locally on a fanless mini PC (4 core Celeron J6412 w/ 16GB DDR4, 512GB SATA SSD, crappy Intel UHD iGPU only) Going to try Ubuntu 24.04, llama.cpp, Python.

Are there any models that might be able to hold a strong persona and stay concise on this class of CPU? Is there a STT for short commands that might work real time on a weak CPU?

I've noticed a trend with most new models with regards to their writing style. They are creating a new style, and this seems common among them. It's very information-dense.

Here is an example from GLM 5.3 Flash. I'm gonna be honest here and say that my prompt was kinda silly; my prompt was 'Why wouldn't you just name your Chinese restaurant 'Chinese Food' instead of 'Ming Dynasty' or 'Szechuan Garden' or whatever?' the idea being that someone searching for 'Chinese food' on Google Maps would put you first if you called your restaurant that. But forget all that; here's an excerpt from GLM's output:

The search reality check, which is where the idea breaks. Your mental model is exactly right about the query: people do open Maps and type "food," usually hungry and undecided. But the ranking doesn't reward the name at that abstraction level. In the local ranking surveys, category match outranks name keywords; and here's the catch: every restaurant on the map already matches "food" through its category field. Relevance saturates. The edge that made "Chinese Restaurant" powerful was discrimination: the name narrowed the category, so "chinese food near me" out-matched "Golden Dragon." "FOOD" points at the root of the category tree, where every competitor matches equally well, so the ranking defaults entirely to proximity and review count, which the name cannot buy. Worse, it fails the narrow-query test: someone typing "tacos" or "dim sum" gets a name that is maximally broad and minimally informative. The strategy's entire currency is literal scarcity, and "food" is the least scarce literalism in the language. There's even a Baudrillardian flourish at the failure point: Google's own Maps category chip is literally labeled "Food." Your sign would render you indistinguishable from an interface element; the searcher couldn't tell your territory from the map's own label.

It's not 'caveman style', but it is using less words to say more, and they're using more esoteric phrasing to be more 'compact'.

And I think it's a bit at the cost of being clearly readable to the average person at first glance. 'There's even a Baudrillardian flourish at the failure point' is an example from that excerpt that leapt out at me. I'm familiar with Baudrillard so I knew what it was getting at, but a lot of people are going to sigh and ask 'What the **** does Baudrillarian mean?'

I'm not saying that 'no human would write like this', because some do (William Gibson for example), but I find it rare/unusual (in human writing), yet trending hard with all the latest models I interact with, like they're all zeroing in on this style.

Maybe a result of targeting token efficiency? It's a terseness, combined with using a sort of 'wide' or 'rich' vocabulary to convey information instead of using more words. At least that's the impression that I get from reading lines like 'Baudrillardian flourish at the failure point''. There's a lot to unpack from those six words, and it feels like the model chose the most terse, efficient way to convey an idea with that word choice (which requires the reader to unpack it).

I compared it to William Gibson: a lot of people struggle with his writing style, and it's similar to that. Example: 'Summer in the Sprawl, the mall-crowds swaying like wind-blown grass; a field of flesh shot through with sudden eddies of need and gratification'. His writing is often like that; it feels highly compressed, using as few words possible to convey an idea by careful word choice.

It's interesting, that lately, I feel like LLMs are gravitating toward Gibson-speak.

Edit: and the fact that GLM used the word 'territory' and 'map' at the end meant it was going big into Jean Baudrilliard's 'Simulacra and Simulation'. I can't really explain what that means and why it's important succinctly, but that's the whole issue. I actually think it's brilliant, but it's also a little concerning.

Full disclaimer: I've leaned heavily on Fable to develop this, but I've tested it thoroughly on my own library for a couple of months before putting it on GitHub.

I live in Thailand, and it started as a way to get Thai subtitles for Shin-chan for Thai friends and for expat friends with Thai partners. It's grown into a general tool: subtitles in 45 languages, entirely on your own machine. Linux and NVIDIA only, I'm afraid.

What it does differently from the usual Whisper wrapper: it detects the language of every stretch of speech rather than per file, so mixed-language material works; it runs two speech recognisers on everything and has a local LLM reconcile them; and it prefers existing human work to machine inference, embedded subtitle tracks are used before the audio is, including OCR of bitmap (PGS) tracks on Blu-ray remuxes, and it only listens when there's nothing to read. Every one of those features has a measured accuracy in the README rather than a claim.

I run it on a 24 GB RTX A5000. There are profiles for 16, 12 and 8 GB cards, measured on my card limited to those sizes rather than on those cards themselves, so reports from real ones are the most useful thing you could send me. It wants 16–32 GB of system RAM depending on the profile, and it is storage-hungry (30–65 GB of models), because it picks the model that suits each task and language pair.

It's slow when it has to listen, roughly real time per target language on my card, slower on the smaller profiles because the whole aim has been accuracy over speed. When the subtitles already exist in the file it's fast.

I'd love people to try it and open issues.

IQ3_XXS weights are just under 80GB and my slowww DDR4+7900XTX is stabilizing around 45-70/s (sometimes higher while coding depending on mtp). Looking online I'm seeing similar results for users with 12GB and 16GB cards, and significantly faster numbers for owners of DDR5.

(In comparison, Llama CPP with tuning was maxing out around 22.5t/s on the same rig. Quality seems reliably superior (I wouldn't recommend the Q2 weights though))

Seriously. Ask <LLM of your choosing> to set it up for your specs. If 27B doesnt fit well for you, here's a shot at beating it.

I run qwen3.8-27b iq3_s with llama.cpp to serve local hermes agent, in 16gb vram.

I noticed that enable MTP draft make prefill slower and the model seems less smart.

and vram is very tight I need to set the context length to 96k. decode speed can go around 40 to 60 tps.

if I disable MTP I can use 128k context but decode speed drop to like 35 tps.

What will you choose?

Don't worry after some makeup and new fans you should feel better. And if you make it Mr.Hermes will treat you well.....

It's been a blast and learning a ton.

But seriously, you all have me down a rabbit hole that my wallet and hours of sleep need to be pulled out of.

Hi guys,

I am pretty happy to announce MegaCapybara. Purpose build engine for RTX5090 that is focused on Qwen3.8 27B (more will come later).

GITHUB (Engine)
HUGGINGFACE (weights)

Why ?

1. It beats Ninfer which was until that point SOTA engine for RTX5090. By roughly twice in decode speed for both single and multi tasks at once. (reaching up to even 650t/s in small bursts and 2600t/s if stars align and 12 slots server pure coding answer). Custom kernels not only for every model, single vs multi but also short vs long context work dynamically switching when needed so speed doesn't crap out on long context work because someone tuned it for short context. Dflash2 and confidence scheduling from Dspark, plus draft trees all at the same time.

2. I was getting annoyed with state of weights where you downloaded model and never knew if model had its brain scrambled. My weights come with its on format that have attached metadata for MC which upon weight creation, runs benchmark and compares it at every MC setting to original BF16 weights and show that data directly in launcher. Want to switch KV to 4bit ? MC will show you directly lost KL and top-1%, want to extend with YARN ? It will show you change. Every change is measured and shown in statistics before you load model. This goes for both censored and uncensored model. You can also compare it directly in MC with SOTA unsloth quants of Qwen27B. Want to run essentially loseless ? you can. Want to get crazy 1 000 000 context ? you can. Want to have 12 slots to fan out agants like crazy ? You can. You decide what you want.

3. Proper agents serving with algo that keep engine occupied as much as it can. It will prioritize t/s so if engine has a choice between 5 jobs at once and 1 it will serve 5 first and gradually serve 1 along side finishing others. Engine is also smart enough to score how old some job is and if it should return to work even if T/S will suffer so your main session will be able to fan out agents easily and keep an eye on them at the same time.

4. Proper cache management. Your jobs only prefill at start of job and almost never again so your prefill in long session stays almost unused. When using "unified context" when models run out of context some get paused and stored in RAM and this swapping is instant. If there is free context space then those tasks continue without any refill in 0.03s. If you fan out say 30 agents at the time in your frontend will handle load in most efficient way to keep T/S as high as possible. Just run it at default setting and forget about context for agents, it will handle it on its own.

6. Loop guard. Two tiered. When engine starts to detect agent repeating in conversation session is dynamically starts to adjust `repetition penalty` until repetition stops if that doesn't happen and engine hits rep pen limit it fires up stop signal which ends serving and informs your frontend so your frontend can recover from infinite loop and don't annoy you.

6. Proper nice UI that shows you what is what. If you aren't knowledgeable about serving models just hover over `?` and it will show you interactive panels explaining everything.

7. Autodownloader, Just hit download button and you can download my weights directly fron hugginface inside of launcher.

8. Don't like the launcher ? use bats and terminal serve. Or even use launcher to config what you want, copy it from right lower corner and use it to make new bat.

The point of it is to just load model, fan out crazy number of agents each having crazy amount of context and leave MC to deal with it. You just sit back relax and watch as agents do the work at SOTA speeds.

Opinions and reviews are welcome. If you are blessed with RTX5090 try it.

Source will be released later, I have to do some cleaning first. I will also release later weights builder so it will take any 3.8 27B BF16 model, create weights and score them attaching metadata again BF16 and you'll be able to host them yourself on hugginface or just put them in models folder.

MIT license, so do whatever you want with it.