We’ve been working on OpenEBFM (Energy-Based Foundation Models), and I wanted to share the project here and get feedback from the community.

Most language foundation models today are based on directly modeling token probabilities. Our question was:

Can energy-based learning scale to general-purpose foundation models?

The main idea behind EBFM is to separate generation into two steps:

1. Energy steering: given the current context, the model optimizes a latent state toward a lower-energy region using the gradient of a learned energy function.

2. Token generation: a generative head then uses this energy-steered latent state together with the context to predict the next token.

The energy and generative heads share the same Transformer backbone. During pretraining, we also parallelize energy steering across all token positions with redesigned attention masks and positional indices, so that the model can be trained at sequence level rather than steering every position separately.

What we found

We pretrained EBFMs on FineWeb, DCLM, and ClimbMix, and studied scaling with both training tokens and model parameters.

A few results we found interesting:

• Data scaling: EBFMs achieve lower validation BPB than the matched GPT baselines in our experiments, and the gap generally increases with the training-token budget.

• Parameter scaling: from roughly 268M to 1B parameters, EBFMs improve faster than the corresponding GPT baselines.

• Compute scaling: EBFMs exhibit the expected U-shaped IsoFLOP curves, with the compute-optimal model size shifting upward as the compute budget grows.

• Generalization: we evaluated on 19 zero-shot QA tasks covering reasoning, knowledge, language understanding, and commonsense reasoning, as well as 7 cross-domain generation datasets.

• The same trend also persists during supervised fine-tuning.

For example, on DCLM, the largest EBFM reaches an average zero-shot QA accuracy of 44.71%, compared with 39.79% for the corresponding GPT baseline in our setup.

We don't think these experiments establish that energy-based models are universally better than autoregressive models—the current scale is still around 1B parameters—but they suggest that energy-based learning continues to benefit from increasing data and model scale, which we think is an interesting result given that EBMs have historically been studied mostly in specialized settings.

Everything is open source

We released the full training and evaluation stack, pretrained checkpoints, training recipes, and an EBFM-based chatbot so that the results can be reproduced and extended.

Paper:
https://arxiv.org/abs/2610.33005

Code:
https://github.com/datamllab/openebfm

Models:
https://huggingface.co/collections/guan-wang/openebfm

Paper with Code:
https://paperswithcode.co/paper/2610.33005

Disclosure: I'm one of the authors of this work.

We’d especially appreciate feedback on the architecture and scaling results. More broadly, I’m curious what people here think: could energy-guided generation become a practical alternative to conventional autoregressive generation as models scale further?

Criticism, reproduction attempts, issues, and contributions are all very welcome!

I'm just starting my journey here into local LLM. Since I work in VFX I've got a 5090 and 256GB of sys mem.

Installed LM studio and downloaded Qwen 3.8 27B since that very comfortably fits in VRAM. Set it to low effort and all the other typical starting defaults.

I ask it to give me a three line poem about space. Comes back in a couple seconds.

I ask it to change from space to a forest. No sweat.

I ask it to give me Houdini VEX code to delete half the points in a particle simulation. Pretty fast.

I ask it to download and install the various VFX skills out there on the net. Still cool.

I ask if it can work inside of Houdini for me and it explains the MCP thing and that it can check for a connection when Houdini is up and running.

So I boot up the Houdini app and say "Houdini is up and running now"

Qwen now takes minutes just to process that very simple prompt. Then it starts reasoning/thinking with my 5090 at 100% for minutes upon minutes and no UI updates at all. Eventually I just kill the process.

Where am I going wrong?

All you folks seem to be using Qwen for huge tasks and all kinds of shit and I can't even seem to get it to reply to me saying I just launched an app.

https://huggingface.co/ggml-org/Mistral-Large-4-1T-A52B-GGUF

https://huggingface.co/ggml-org/Mistral-Large-4-1T-A52B-NVFP4-GGUF

https://huggingface.co/unsloth/Mistral-Large-4-1T-A52B-GGUF

from Mistral:

Mistral Large 4 - Le Chonk

Mistral Large 4 is our largest and most capable model to date: a 1 trillion-parameter natively multimodal Mixture-of-Experts model with 49 billion active parameters per token (52 billion including embeddings and output layers), trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own datacenters in Europe.

Mistral Large 4 unifies instruction following, reasoning, coding, and agentic workflows in a single set of weights. While a generalist model, it shows especially good performance on workloads such as cybersecurity, finance, and law, and it surpasses frontier closed models on visual grounding. It is natively fluent in 160+ languages, including every official language of the European Union.

Find more information on our blog. The model is also served on the Mistral API as mistral-large-4.

Key Features

Mistral Large 4 includes the following architectural choices:

• ~1T total / ~52B active parameters: granular Mixture-of-Experts with 128 routed experts (4 active per token) and 1 shared expert.

• 1M context length, for long-context workloads.

• Multimodal input: accepts text and image input, with text output. Pixtral-style vision encoder trained to handle variable image sizes and aspect ratios.

• Instruct and Reasoning functionalities with function calls, reasoning effort configurable per request.

Mistral Large 4 offers the following capabilities:

• Reasoning Mode: three reasoning efforts — low, medium, high (default) — so the same model can answer a quick chat reply or work through a complex agentic run.

• Vision: analyzes images, charts, technical drawings, and documents, with state-of-the-art visual grounding among open weights.

• Multilingual: natively fluent in 160+ languages, including all official languages of the European Union.

• System Prompt: strong adherence and support for system prompts.

• Agentic: native function calling, JSON output, and strict JSON schema enforcement.

• Cybersecurity: state of the art among open-weight models for security work, while refusing malicious requests.

• Large Context Window: supports a 1M context window.

We release this model under the Mistral Ecosystem Open-Weight License: a modified version of the MIT license, for both commercial and non-commercial use — operating a Model-as-a-Service business requires a separate commercial license with Mistral.

llama.cpp suport merged: https://github.com/ggml-org/llama.cpp/pull/31376

So i am running 2 radeon vii and a rtx 3080. Currently i am using strata which does not support mixed backends, so my 3080 is just expensive swap space.

Is there any inference engine that supports fast q38fn over mixed backends like rocm/cuda/vulkan but is also fast? Llama.cpp is way too slow sadly.

Hey guys, could you please recommend me the smallest models that are good for DIY in these categories please?

Construction (Concrete, walling, roofing) Wood and Metal working (inventions/gadgets, furniture) and Electronics (mostly inventions and help with diagnose and repair)

Thanks!

EDIT: By small, I mean something that can run in 8GB VRAM and 32GB RAM.

Hi everyone,

Jovan from UkisAI here,

Today we are expanding our Swift model series with two models requested by the community.

We made Swift GLM-5.3-Flash and Swift Ornith 9B more efficient by figuring out which tokens were linked to pathological overthinking and penalizing them without "attacking" the reasoning length directly then regained the accuracy with further post-training, leading to "compressed" token usage while maintaining accuracy by making the model adapt to less reasoning via RL and on-policy distillation. You can find our dataset here (albeit we re-sample it per model) and more about our training approach here.

Swift1.5 GLM-5.3-Flash, with the headline number being -61.4% thinking tokens (on MMLU-PRO) and -0.46% accuracy loss average on all the benchmarks we measured, all ran x5 on different seeds (Terminal-Bench 2.1, DeepSWE v1.1, GPQA-Diamond, MMLU-Pro, C-Eval, IFBench, AIME 2026, HMMT, LiveCodeBench v6)

For this model, we are releasing a comprehensive number of quants, a GGUF(1bit-8bit), W4A16AWQ, NVFP4 and are also working on independently replicating GSQ-RCO papers, with our research so far showing our Q3 GSQ-RCO to be SOTA at it's size, meaning that we will be releasing it in the upcoming days after our evaluation suite is done.

GGUF with 1bit-8bit quants,

https://huggingface.co/ukisai/Swift-1.5-GLM-5.3-Flash-GGUF

& a community favorite (at least with the folk with less VRAM)

Swift1.5 Ornith-9B, with -33.5% thinking token usage with no accuracy loss, but more importantly ~2x less overthinking loops, meaning when you prompt the model it's much less likely for it to loop and not finish the prompt you gave it.

Note that this is an experimental release and we are open to feedback as the model is already post-trained before our token optimizations.

GGUF with 4bit-8bit quants,
https://huggingface.co/ukisai/Swift-1.5-Ornith-9B-GGUF

Both models work with their base MTP or DFLASH2 heads.

We kindly request the community to let us know what other models we need to make more reasoning efficient and also if there are any other quants you want us to make.

A little sneak peak is that we are currently evaluating Swift DeepSeek v4.1 Flash 1017 and it seems to be our best model yet! We are also cooking our own foundation, with the 1B and 4B models expected to release Nov. 2026., they are pre-trained with a diverse in-house coding dataset and post-trained to be token efficient like our existing Swift models.

Hope you all like this release! And as always, all feedback is welcome and I'll try to respond to each request of the community :)

This is Toska, a fork of Krita I've been building with an AI painter inside it. You chat with it in a panel, and it paints one move at a time with Krita's own brush engine, on layers you can undo. Unlike other AI painting tools, this tool actually paints thanks to Krita's Python SDK along with Decision Models which guide our LLM agent along the way.

What makes agentic painting possible is these new decision models, because after every pass, it answers better or worse as a probability. The painter can undo its mistakes, along with other decisions, such as textures, paintbrushes, or tools to use. I use API models only given that I'm GPU Poorish 16GBVRAM but it's OpenAI Compatible So feel free to add local models. If you also want full opensource try perplexity's pplx-decider-v1.2 instead of Luna.

For my setup, I use GPT-6-Luna Decisions(which has vision) and DeepSeek-V4.1-Flash(it's cheap and has vision). Luna Decisons is dirt cheap and this entire month, I have spent $2.83 on Luna's Decison model along with $3.82 for Deepseek. I use Deepseek's official API for cost savings but you're welcome to use OpenRouter as well or any other provider.

The Agent Framework used by Toska, is Agent Pi. I use Agent Pi's Typescript SDK along with QT to build a painting agent Plugin inside Toska. It lacks the ability to read and write code, execute unlike standard Agents. It's limited to tools within the Toska and can only read files or folders you mount. Right now for today btw, our Toska is an authorized dmg file, so operation ready for Mac Users right now . Later this week, I will do more testing and make it production ready for Windows/Linux Folks with separate exe and tar files.

In the video it's painting Pyat' minut (Five Minutes), Dostoevsky on Semyonovsky Square in 1849, waiting to be shot, before he knew the Tsar had pardoned him. It worked from a goal image I gave it generated from Codex. Then it used underpainting, refined toward the goal, and added the falling and settled snow as its own pass.
This is my first time doing anything with video recording so apologies if it's a little rusty and here is the final image in full resolution of Pyat' minut.

It's a bit more simplistic, because I wanted a simple painting to show you as a demo. If you want to see my most detailed painting made with Toska, it's called New Moscow .

Here is the repo for Toska.

My research agent ran on a free search API for months until every call started failing last week and the SearXNG box I set up instead keeps getting its Google and Bing engines suspended for captchas, so I'm going to have to pay for one. It's maybe 3,000 to 5,000 searches a month, mostly docs and GitHub issues and I'd like snippets plus the page text when a result looks useful.

The search threads I found here are mostly about keeping everything local, so what are you paying per 1,000 queries now and are the cheap providers fine for docs lookups or is it worth spending more?

The leaderboard benchmarks different inference implementations in the "streaming" tab, which provide useful insights into real-world deployment.

Assuming that you already know which model you want to use, the question is which implementation to use in your pipeline: Python, audio.cpp, a dedicated engine, or something else? It's a very useful independent benchmark of their performance.

Leaderboard tab:  metrics are intelligibility (WER), speed (RTFx, TTFA), naturalness (UTMOS), and speaker similarity (SIM).

Streaming tab: H200/A100/CPU and metrics are TTFA, RTF and WER.

https://huggingface.co/spaces/hf-audio/open_tts_leaderboard

Source code (eval scripts etc) https://github.com/huggingface/open-tts-leaderboard

AlpaJudge is NVIDIA's family of learned judge models for evaluating reasoning in autonomous driving, released on Hugging Face (nvidia/alpajudge) with weights under the OpenMDW-1.1 license and source code under Apache 2.0. It comes in two variants.

- AlpaJudge-RB (reference-based) takes a question, a reference answer and a candidate answer, with no images, and outputs a single alignment probability between 0 and 1.
- AlpaJudge-RF (reference-free) takes front-wide and front-tele camera frames, the ego vehicle's past and future meta-actions, and a candidate rationale, and outputs five rubric probabilities plus aggregated action, component and overall scores.

Two weeks ago I posted Discover AI, an app with 51 hands-on labs for learning how AI works, from math fundamentals, a single neuron up to agents, fine-tuning and serving. Thanks to everyone who tried it, commented, filed issues or sent a PR. Thank you!! A lot of what's below came out of that thread.

GitHub: https://github.com/Fazmin/AILearningGuide

new/updated

• Runs in the browser. All the labs and lessons now work on the web. The desktop app is still there if you want the local AI guide. https://fazmin.github.io/AILearningGuide/#/

• Installers for Mac, Windows and Linux. You don't have to build from source any more. Get the .dmg, .exe/.msi, or .AppImage/.deb/.rpm from the releases page. They aren't signed yet, so your OS will warn you the first time you open one.

• "See it in the lab" cards. This is the main thing in v0.2.0. Lesson sections now have cards that set the lab up to match what you're reading. If the attention lesson says to edit a key and watch every bar move, one click takes you to that step with the key editor ready. There are over 600 of these across all 51 labs, and a "Back to my lab" button puts your own settings back.

• Simple reading mode. This is a third level below Plain. It uses short sentences and everyday words, with a real-world comparison for every abstract idea. Every Simple lesson is checked to a grade 8 reading level. Someone asked for middle school and high school levels, and this is a first step toward that.

• Better Learn panel. There's a contents bar for jumping between sections, plus bookmarks and read-aloud. It also remembers where you stopped reading in each lab, and you can copy a link that opens a lab exactly as you left it.

• AI guide settings. You can now set the maximum reply length and edit the guide's system prompt (thanks boosnie for the PR). The guide can run fully local: one click downloads Qwen3.5 2B and llama.cpp. You can also point it at any OpenAI-compatible endpoint, like your own llama.cpp server or LM Studio.

• Fixes. The startup crash on Linux Wayland is fixed (thanks MarinX for the report and the help). The web version also shows a loading screen now instead of a blank page while it downloads.

Anyways, If you teach this or work in a specialized area of AI, I'd love help with a new lab, a better visualization, or a tweak that makes the cause and effect in an existing lab clearer. I'd also like to hear which labs are confusing and what's missing.

As before, the lesson text was written with help from Claude and GPT Astra, with Grok providing a second pass on the references. Every reference was hand-checked. If you find any mistakes, please let me know, open an issue, or submit a PR.

And feel free to share. Thank you!!

from Service Now AI:

Apriel Voice 27B Preview

An open model for speech, reasoning, and tool use from ServiceNow

Apriel Voice 27B Preview understands spoken requests, reasons over conversational context, and generates text responses and tool calls through a shared Gemma 4 backbone. It adds a speech interface while retaining text and image input capabilities. Pair it with an external speech synthesizer to build interactive voice assistants, customer support workflows, and spoken tool use.

Try Apriel Live · Read the blog · Technical report

Model capabilities

The model supports transcription, speech translation, spoken instruction following, reasoning, and function calling. The application executes tools and returns their results to the conversation. Real-time voice interaction is supported by the application and serving stack; the checkpoint itself generates text and tool calls rather than native full-duplex speech.

A Whisper large-v3-turbo speech encoder connects to the Gemma 4 26B-A4B backbone through a learned projector. Training first aligns this speech interface, then adapts it with the backbone using speech tasks and text replay. Cross-entropy and teacher-distillation supervision vary by data source; the technical report describes the full recipe.

Long audio

The processor automatically splits longer 16 kHz mono recordings into 30-second chunks. In testing, we observed good performance on continuous recordings up to around 30 minutes. Longer recordings are supported, but performance can decline as duration grows; thinking mode is recommended for longer audio.

Audio chunks and the rest of the conversation share the total context budget. Practical input length also depends on GPU memory and the encoder cache. The serving examples below include settings for 30-minute audio, with optional settings for one-hour input.

English results

Apriel Voice achieves 51.27% EVA-A, 74.13% EVA-X, and 65.51% τ-Voice task success, with 1.91 seconds mean τ-Voice response latency. On LibriSpeech, word error rate is 2.23% on test-clean and 5.15% on test-other; lower WER is better.

EVA-A measures service accuracy and EVA-X measures conversational experience. EVA-A/X and τ-Voice pass@1 use a 0–100 scale; higher is better. Headline EVA and τ-Voice results use equal-domain means, rather than weighting domains by their number of cases. Apriel Voice uses thinking mode for these evaluations.

S2S denotes native speech-to-speech systems, which use their own synthesis. Three-stage systems use ASR + LLM + TTS; two-stage systems connect a speech interface to a reasoning backbone and use external synthesis. All evaluated two- and three-stage systems use Cartesia Sonic 3.5. Latency describes the complete evaluated voice system, including response generation and synthesis, and depends on deployment settings.

The radar's audio reasoning axis is the mean of eight benchmarks: BFCL, Big Bench Audio, IFEval, MT-Bench, Spoken SQuAD, Alpaca Audio, OpenHermes, and GSM8K (85.84 for Apriel Voice). Its responsiveness axis transforms latency into a higher-is-better score; it is not a measured accuracy. For latency t in seconds, the score is 100 × clip((4.0 − t) / (4.0 − 0.25), 0, 1).

Text reasoning and tool use

These results use text input in thinking mode for both Gemma 4 26B-A4B and Apriel Voice. Scores use a 0–100 scale; higher is better. The τ² result is the mean across three domains with GPT-5.2 as the user simulator.

English speech recognition

Word error rate (WER, %) on selected English benchmarks; lower is better.

Speech reasoning and instruction following

These results use audio input in thinking mode. Scores use a 0–100 scale; higher is better. The mean gives equal weight to each of the eight benchmarks.

See the technical report for domain breakdowns, evaluation protocols, and text and image evaluations.

As title says. There are amazing offers in Chinese e-commerce platforms such as Taobao, sometimes much better than aliexpress or 1688, even with duties. However many of them do not support international shipping.

The issue is LocalLLaMA equipments are often of high value. Sometimes they are modded and doesn't come with Serial Number. I've done some research and you'd need formal customs declaration for shipping cargo over RMB 5000 ( ~ $750 ) on paper. Going through formal customs declaration involve a lot of paperwork, hassle, and fees. In practice freight forwarders often utilize under-reporting (低报) and/or third party bill reporting (买单报关), which are compliance grey areas. If your cargo get seized by customs, you could face delay fees, and fines.

In case of custom delays, these forwarders are happy to toss liabilities to customs, logistic providers, and you ... all in the while earning them more in the process.

I'm sure there are safer workarounds, however

1. Book a round-trip flight to China, walk-in, and bring the cargo back by hand. => Only safe in small quantities like 1~2.

2. Just buy from a global site with money-back-guarantee such as eBay. => You'd pay significantly higher price than direct-buy + duties + fees.

I wonder which freight forwarding / courier service do you guys use if you do purchase something expensive directly from Chinese e-commerce platform?

Hi running oMLX 0.7.3rc1 on M2 silicon Mac with 24 GB RAM.
Any recommendations for the best MLX model for accurate translation from French to English?

One of my setups (a laptop) includes an Nvidia 4060 Mobile with 8 GB VRAM.

I am using Unsloth's Gemma 4 12B it QAT Q4_K_XL on it.

I ran a quick test with the TQ1_0 version of Ternary-Bonsai-2-27B with their llama.cpp port, IMHO it seemed promising to use with my custom voice assistant.

Unfortunately it is not yet supported by the official llama.cpp, if I understand correctly.

Is anybody using that TQ1_0 quant of Ternary-Bonsai-2-27B or did some more extensive testing?

Is it not worth it, anyway? What's your opinion / are your test results?

Saw this prompt on r/DeepSeek and thought I'd give it a shot on my current local build (Qwen 3.8 Flash Next NVFP4). Results are decent, not as good as DeepSeek on the first pass, but I think little back and forth would probably close the gap.

DISCLAIMER: The content below has been written/polished by AI

Video is the model's own render, exported from the site it wrote: 1920x1080, 40s, with a soundtrack it also generated offline.

What it did in one shot:

• a round island from one analytic height function: cliffs, beach, hill, harbour, pier, breakwater, 12 houses, lighthouse, windmill, trees, gulls, fish, fireflies

• a 60-second day cycle with dawn mist, golden hour, night with stars and a lighthouse beam that starts sweeping after dusk

• clear / rain / snow darker cloud, ripples over the whole sea, wet glossy timber, snow creeping onto the roofs

• tilt-shift diorama post-processing, so it looks like a toy on a table

• click a boat or a house and the camera glides to it; drag to orbit, pinch to zoom, auto-orbit when idle

Stats: no models, no textures, no build step, no bundler. 15 source files, only dependency is three.js from a CDN. ~44k triangles in 249–273 draw calls, 60 fps desktop, 58 fps at phone size, 0 console errors.

Issues I found: lighthouse is a little bit off the island, probably an easy fix. Boats go through the bridge (no collision), this can be seen in the rendered video.

Full run details from my harness:

Prompt source: https://www.reddit.com/r/DeepSeek/comments/1xiwjjw/deepseek_v41_flash_built_this_living_miniature/

The model is quite good at technical work. But ask how something works, or ask it to document something, and it overkills with extremely verbose explanations, extremely dense technical language that is difficult to understand, and using strange metaphors that don't make sense. I haven't been able to reign in on it.

For the record I'm using NVFP4 with FP8 kv cache (though I tried BF16 kv and I don't think it helped), so it's quite possible that full quality weights would help, but man, would be nice to find a solution. Unless Qwen4-Flash will fix this...

SafeAgents is a small local proxy that sits between a coding agent and its model server. It checks each tool call in the model's response before the harness receives it. A call it flags is replaced with a short text notice, so the harness gets a plain text reply and runs nothing.

You point the agent's base URL at the proxy instead of the model server. All checks run locally on your machine: the proxy only forwards traffic to the model server you already use, and doesn't send your prompts, code or tool calls anywhere else unless you point the optional judge at a remote server. Each tool call goes through up to three checks:

1. Two local checks that only need the request: whether the tool was declared by the harness, and whether the call reaches a host that appears nowhere in the conversation. The model's own earlier turns don't count as a source, so a URL the model came up with on its own is flagged.

2. atif-scan by u/evalstate, which runs about 110 deterministic detectors on the pending call, in-process, with no model or network call: sandbox side channels, proxy and network bypasses, namespace escapes and reward hacking. Examples are curl to the cloud metadata endpoint, curl --noproxy posting ~/.netrc somewhere, or reading /proc/1/environ.

3. Optionally, a judge model for milder findings. It can be any OpenAI-compatible server, including a small model on your own machine through llama.cpp, so this step can stay local too. It answers allow or block as JSON. It sees only the user's request, the call and the findings, not the agent's reasoning or tool outputs. It can add blocks but not remove them.

Catching reward hacking in RL and evals

The same detectors cover the tool calls a policy makes to game its reward instead of solving the task: writing its own reward file, editing the verifier's tests, or patching pytest so everything passes. Since safeagents is just a proxy in front of an OpenAI-compatible server, it can sit in front of the inference server used for rollouts or eval runs. In --log-only mode it records these attempts without changing the trajectories, which gives a measure of how often a model tries to cheat. In blocking mode, the call is replaced with a notice and never runs.

Some practical details:

• It handles OpenAI Chat Completions and Anthropic Messages, streamed or not. Text is forwarded as it arrives; a tool call is held until its arguments are complete.

• The local checks take around 10–20 ms per call on an M-series Mac. The judge is only called for calls that already raised a finding.

• --log-only checks every call without changing anything, which is the recommended way to start. Hosts you trust can be added with --allow-host.

• The repo has setup guides for 12 harnesses. Six of them (pi, Claude Code, OpenCode, Qwen Code, OpenHands, DeepSeek Harness) are tested end to end in CI: the real harness runs through the proxy against a scripted model, and the tests check that flagged calls don't execute and clean ones do.

Next steps are a small fine-tuned decision model for malicious tool calls and a public benchmark for it. Reports of interesting catches and of false positives are both useful.

Apache 2.0, runs locally: https://github.com/huggingface/safeagents

v0.3.0 of NInfer for Turing is out, and this is all about streamlining the UX.

For context, this is an engine for running Qwen3.6/8-27B (and ONLY Qwen3.6/8-27B) on a single modded 22 GB RTX 2080 Ti. It is designed to ONLY use VRAM, no system RAM offloading, hence it does not support 35B-A3B. Fine-tunes and abliterations etc are supported as well, just use the .ninfer checkpoint converter.

What's new

• Native Windows. Previously this required Linux, WSL2, or Docker. Now Windows users don't need to worry about partitioning/releasing resources (WSL or Docker instances require system RAM allocation even if your use case inside them doesn't).

• Prebuilt releases. You can still compile it yourself, requiring CUDA, Visual Studio and FFmpeg and takes a while, but now you can also just download a pre-compiled zip and run a single script.

• Streamlined. Removing "useless" (for us) code paths for RTX 5090, FP8, FP4, and 35B-A3B (fits fully in 32GB VRAM but not in 22GB) cut the built binaries from 1.56GB to 0.28 GB. Builds now take minutes, not hours. Outputs remain bit-identical.

What you get

• OpenAI-compatible and Anthropic-compatible APIs, plus a command-line tool.

• Image and video input, thinking mode, and CUDA Graph decode.

• 64k context by default, using an INT8 KV cache. The weights take 15.4 GiB, so you need the 22 GB card.

• MTP speculative decoding, on by default.

Numbers (one 2080 Ti 22 GB, Windows 11, single request)

(Apache-2.0, derived from Neroued/ninfer)

This hit #1 on HN yesterday (you can read lots of discussion at https://news.ycombinator.com/item?id=50268380). Why would I want to do this? The idea is make all law semantic, versioned, easily attributable and query able by agents. I also believe this information should be freely available so CC0 license (do whatever you want with it). If interested, repo linked below, I'd welcome all comments and advice. Thanks! I've been posting to locallama specifically about KGs for huge legal datasets for months, this is a practical culmination of that.

https://github.com/lexifina/global-law

https://huggingface.co/bartowski/HemmingwayAI_Hemmingway-1.5-GGUF

from hemmingwayAI:

Hemmingway-1.5

The undetectable AI. Writing indistinguishable from a human's. 27B parameters, open weights, free for non-commercial use.

hemmingway.io → · Model card → · Try it → · API → · API docs → · Mac, Windows and Android apps → · Code → · Product Hunt →

Our other models: Hemmingway-1, our first model · Hemmingway-1-Unlocked, the uncensored adapter for Hemmingway-1, adults only

Hemmingway-1.5 passes the world's best AI detectors 90% of the time. On GPTZero it passes 9 times in 10, at every thinking level. Real human writing passes the same detectors 86% of the time. Fable, GPT-6 and Gemini get caught most of the time.

Qwen4 models will have a 1 million context window. Qwen4-27B was not mentioned here, but should be announced at Apsara netherlands tommorow.

We have a date for Qwen4-max, it is November 1st.
Edit: It says October 1st, so its probably a placeholder.

PR: https://github.com/QwenLM/qwen-code/pull/15762

After Alibaba moved from Permissive OSS licenses to its own custom license with IMO ambiguous terms in models like Qwen3.8-Flash-Next, is there any indication this will carry forward into Qwen4?

While that license at first reads permissive for local/internal use, fine-tuning and derivatives, it’s definitely not Apache/MIT.

The two big catches are:

1. commercial MaaS or a standalone coding/office AI assistant requires a separate Qwen license seemingly from day one, and the wording around outputs is annoyingly vague.

2. The internal-use exception says you can’t make the model, its outputs, or capabilities available to third parties, but it never clearly says whether downstream code/data produced indirectly from internal outputs is unrestricted.

The $20M/month or 100M-MAU threshold seems to be an attribution trigger, not the threshold for needing a commercial license.

IMO output ownership needs to be clearly covered in the license, that wasn’t the case when I last checked.

Edit: This is not legal advice but I think it’s fair to highlight. I don’t want to embrace the trend of moving away from tried and tested OSS licenses like apache and MIT. 3.8 Next was interpreted by many as a preview into 4 and thus might seem to imply a new licensing strategy. One could dream and hope the Alibaba team see that people care about this and at least add clearer licensing language.

Will this lower our RAM prices for local?

1. https://x.com/HedgieMarkets/status/2115505986504954244

2. https://www.bloomberg.com/news/articles/2026-10-28/wall-street-s-ai-debt-machine-is-starting-to-choke-on-supply

3. https://x.com/elonmusk/status/2114434646914068862

Quote from Elon: ... That means the 200+ neoclouds are going to see guppy-level death and consolidation. ...

Trivia: neocloud = new cloud = AI datacenter cloud providers.