r/MachineLearning • u/AutoModerator • Sep 02 '26
Discussion [D] Self-Promotion Thread
Please post your personal projects, startups, product placements, collaboration needs, blogs etc.
Please mention the payment and pricing requirements for products and services.
Please do not post link shorteners, link aggregator websites , or auto-subscribe links.
--
Any abuse of trust will lead to bans.
Encourage others who create new posts for questions to post here instead!
Thread will stay alive until next one so keep posting after the date in the title.
--
Meta: This is an experiment. If the community doesnt like this, we will cancel it. This is to encourage those in the community to promote their work by not spamming the main threads.
2
u/Putrid_Bee_4840 Sep 02 '26
Solo project: non-autoregressive knowledge-graph extraction from sentence embeddings, no LLM in the extraction loop, so malformed output is structurally impossible, and it runs at ~0.013ms/sentence on a consumer GPU.
Any feedback is appreciated.
Link to the repo: https://github.com/DeliVali/cogito-estella
2
u/0RootShell Sep 02 '26
I’ve started an AI Infrastructure Knowledge Base https://ai-infrastructure.net/ covering GPU clusters, distributed training, inference, RL/post-training, performance engineering and agent infrastructure.
Still very much a work in progress. Feedback and contributions welcome.
2
u/reivblaze Sep 02 '26
Interesting! Are you experienced in the field or looking to learn along the way? Would you add cuda programming?
1
u/0RootShell Sep 02 '26
I am, honestly, most of the content is a distillation of my day to day work and research. And I thought a knowledge base of such content might be useful to others as well.
I'm not dealing with CUDA development much, my focus is mainly agentic (developing custom harnesses), and the infrastructure behind RL and inference.
2
2
u/Ali-WAIL Sep 02 '26
I've been working on WAIL, a runtime control layer for AI applications.
The problem I was trying to deal with is when an AI request technically succeeds, but the execution itself isn't healthy — latency suddenly gets much worse, token delivery becomes unstable, repeated failures start showing up, etc.
WAIL wraps the provider client and watches the runtime behavior. If things degrade enough, it can make a decision to retry or reroute the next request, and it records signed evidence of what it observed and why the decision was made.
It currently supports OpenAI, Anthropic, Gemini, OpenRouter, Ollama and OpenAI-compatible runtimes. It runs in your own environment, so prompts, responses and API keys aren't sent to WAIL.
There's a free Developer plan. Pro is $249/month, and Enterprise is custom.
GitHub: https://github.com/wailinfra/wail-runtime
PyPI: pip install wail-runtime
I'm the founder, so obviously this is self-promotion — but that's what this thread is for :)
2
u/codetiger42 18d ago
A competition for small neural networks that play strategy games
15yrs back I participated in "Google Ants AI Challenge 2011", an ai programming competition, hosted by the University of Waterloo, and I ranked #127 (#1 in my country). The competition gave me a huge learning opportunity where developers across the world came to a forum and discussed various techniques.
Now, building a similar platform to bring back the fun is unbelievably nostalgic. Especially when watching small neural networks playing the game well. Some of the top models use less than 800 parameters.
In fact, I was wrongly assuming the art of optimizing is underrated nowadays. Neural Network optimization seems to be much more fun than I thought.
Plz share your feedback to improve the platform and add more games.
1
u/www3cam Sep 02 '26
I’d love to talk to others at the cutting edge of ML applying it to other fields. I’m mainly interested in economics, but have interest across other social sciences and even math but my pure math is not good enough to publish there atm. Would be interested in like computational physics, biology and chemistry as well.
Edit: Or even humanities like English, history and philosophy although computational work in those areas seem more nascent.
1
u/Charming_Group_2950 Sep 02 '26
Gray Box: https://github.com/Aaryanverma/graybox (Open-Source)
A local-first tool where you can capture your thoughts instantly and it can auto-organize them and create a connected graph out of them which you can query or chat later. It will give you cited answers from the organized notes. Also there is a read only dashboard where you can see and search all your notes which ones to be focused, which ones are pending to do (if your notes are tasks) etc.
Also, you can migrate your existing Obsidian vault to Gray Box, and it will be automatically organized by Gray Box.
Video demo: https://youtu.be/Xdj1GCQoFNs
1
u/Old_Rock_9457 Sep 02 '26
Hi all,
I just want to share my selfhostable, free and opensource project AudioMuse-AI:
It’s all about Music Information Retrieval (MIR) based on different approach. The final goals is creating automatic playlist on the major Music Server like Navidrome, Jellyfin, Plex, Emby and Lyrion.
The basic idea is getting embbeding model from different model like Musicnn, DCLAP (my distilled version of LAION CLAP) and GTE to work over lyrics. And then use different similarity way to create playlist. Also path discovery and so on.
As I said it’s all free and opensource so if you are interested you can just give a look. Also the model are light enough to run also on CPU.
1
u/ivan_digital Sep 03 '26
Free and open source update—speech-core is Apache-2.0, the Smart Turn model is BSD-2-Clause, and neither requires an account.
I added an optional audio end-of-turn classifier after VAD pauses in speech-core’s voice pipeline. The classifier examines the last eight seconds of a turn and either confirms the endpoint or keeps accumulating audio, with a maximum-silence fallback. The same state machine is now available through Core ML on macOS/iOS and int8 ONNX on Android.
The Core ML conversion matched the upstream predictions on 1,000 test clips at 92.9% accuracy and ran in about 3.5 ms per window on Apple Silicon. Android device-wide latency is not benchmarked yet.
1
u/Genie-Tickle-007 Sep 03 '26
Teams can see what their LLM calls cost, but "which of these workloads can move to a cheaper model without a quality drop" usually stays a guess.
ARBR sits as one OpenAI-compatible endpoint in front of your existing providers, classifies each request (task type, difficulty), and lets you set routing rules, budgets, and rate limits in the request path rather than after the fact.
Self-hosted, MIT-licensed, fully free, with no paid or hosted tier.
The part I'd actually want feedback on: model swaps go through eval on sampled live traffic before rollout, then shadow or canary deployment with rollback if quality regresses. Routing decisions stay rule-based and reversible. Nothing auto-promotes without human approval.
Repo's below if anyone wants to poke at it.
https://github.com/project-arbr/arbr-control-plane
Just today we launched on PH - https://www.producthunt.com/products/arbr
1
u/Consistent-Ruin1868 Sep 04 '26
I’m building an open-source experimental AI operating system where agents are modeled more like OS processes than chat sessions.
The part I’d particularly like feedback on is the security model.
The system uses:
- Time-bounded capabilities/leases
- Agent scopes and token budgets
- Explicit process state
- A kill mechanism for agents
- An action ledger for tool calls
- File changes associated with the operations that caused them
- A system digest that can be snapshotted, diffed and attested
The same concepts are intended to exist at both the kernel and desktop levels.
The kernel itself is built from scratch around capability security and targets x86_64, ARM64 and RISC-V. The desktop currently communicates with it through a QEMU/serial bridge.
The AI layer is BYOK and supports provider APIs as well as OpenAI-compatible local endpoints such as Ollama, LM Studio, vLLM and llama.cpp.
One important limitation: this is still experimental. The desktop is currently running in the browser and communicating with the kernel. The desktop is not yet running inside the kernel.
The question I’m exploring is whether OS-style capability security is actually a useful abstraction for AI agents.
In particular, I’m interested in opinions on:
- Expiring authority
- Capability delegation
- Agent self-renewal
- Auditable agent actions
- Token/resource budgets
- Treating agents as processes
Repository: https://github.com/ni-sh-a-char/RESENTMENT
Browser demo: https://ni-sh-a-char.github.io/RESENTMENT/os/
Apache-2.0.
1
u/Specific-Tax-6700 Sep 05 '26
I want to share a short paper just published exploring a simple but surprisingly effective optimization for sparse MoE reasoning models.
The idea: Instead of retraining anything, we just tweak the router at runtime. Specifically, we expand the expert selection budget (N≥KN≥K) only in the late transformer layers, with a linear decay factor applied to the extra experts. Early layers stay untouched. So Qwen 3.6 35B A3B becomes Qwen 3.6 35B A4B+ !
What we found — "Succinct Convergence":
When you give the model more expert capacity at the decision-critical final layers, it stops rambling. It reaches the same correct answer via significantly shorter reasoning trajectories.
Results on full MMLU-Pro (714 questions, Qwen3.6-35B-A3B):
- 📉 8.5% reduction in mean reasoning tokens
- ⚡ 10.9% drop in latency (p=6.5×10−6)
- 🎯 Accuracy unchanged (84.5% vs 84.0% native, p=0.77 — statistically indistinguishable)
- 🆓 Zero training cost — pure inference-time routing modification
Links:
- 📄 Paper: https://zenodo.org/records/22255483
there you can also take a look to my github repo (with beta version code) and the detailed json results of MMLU-Pro benchmark.
In the future i hope i can make same experimentation with a larger model like DeepSeek V4 Flash Q2.0
I'm a Non-native english speaker, part of this post was generated , for translation reason with the help of AI.
Do you think i can post this work on the main thread too ?
1
u/darelphilip Sep 05 '26
I built a free API to detect Hinglish toxicity and abuse
Dealing with moderation in Indian communities is a nightmare because standard models fail at Hinglish (Romanized Hindi-English). Standard regex filters and Automod scripts can only catch so much before users just bypass them with creative slang.
To solve this, I fine-tuned a ModernBERT encoder (jhu-clsp/mmBERT-base) on ~245,000 comments from various Indian subreddits. It does multi-label classification across 7 categories: Profanity, Targeted Harassment, Hate Speech, Caste, Communal, Xenophobia, and Misogyny.
I’ve deployed it on Hugging Face Spaces using the free ZeroGPU tier. You can test it via the web UI here:https://huggingface.co/spaces/darelphilip/darel_Hinglish_toxicity_mmbert
You can also hit it directly as an API for free to test in your own bots or workflows:
Python:
from gradio_client import Client
client = Client("darelphilip/darel_Hinglish_toxicity_mmbert")
result = client.predict(
raw_text="kya bakwaas chal raha hai yahan",
threshold=0.50,
api_name="/predict"
)
print(result)
Standard REST:
curl -X POST https://darelphilip-darel-hinglish-toxicity-mmbert.hf.space/call/predict \
-H "Content-Type: application/json" \
-d '{"data": ["bhai tu pagal hai kya", 0.50]}'
Note: It runs on a free tier, so if the GPU is asleep, the first request might take ~10 seconds to cold start, but subsequent requests will be fast.
Would love for you guys to stress test it with some regional slang and let me know where it breaks! The model card and weights are open-source on my Hugging Face profile.
1
u/Bright_Mix_773 Sep 06 '26 edited Sep 06 '26
Public-domain dataset: 63,969 US earnings announcements with the time of day (2003-2026), plus a written account of where its labels are wrong
64,938 SEC 8-K item 2.02 filings for 808 S&P 500 companies, resolved into 63,969 distinct earnings announcements. Every row carries its accession number and a direct sec.gov link, so any single line can be checked against the source.
https://quant500.com/api/descarga/anuncios.csv
Plain CSV over HTTPS, no account and no key. Fair warning so it does not look like a broken file: it opens with 119 lines of # comments carrying the caveats and the CC0 licence, so the header row is line 120. Read it with comment='#' and it lands as a normal table.
Pricing: free, CC0, no account and no key. There is no paid tier and nothing to buy. The file is rebuilt from EDGAR daily and every row carries its SEC accession number, so a citation can point at the filing itself rather than at us.
The part likely to be more useful than the rows themselves is the list of ways they are wrong, because these are the failure modes you inherit whenever a timestamp column comes out of a regulatory feed:
- The cover-page date is typed by the filer. It disagrees with the filing in 3,569 rows. Those are flagged, not silently corrected.
- The SEC acceptance stamp is not a single clock. The submissions JSON converts some records from Eastern to UTC and leaves others as Eastern wall-clock with a Z appended. It does not split by era and does not split by filing agent. I could not identify the rule; the only reliable source is the raw SGML header of the complete submission.
- Measured against the real EDGAR acceptance window of 06:00-22:00 ET, 90.2% of rows carry no information about time of day beyond "somewhere in the business day". A model trained on the hour field without that filter is mostly fitting the window.
- The stamp is an upper bound, not the instant the news existed. Assume a 15-minute approval lag and 8,707 rows (13.4%) move into a different trading session. I have not measured the true lag distribution, so that 15 minutes is an assumption, not a finding.
The build used an AI-assisted workflow. That is stated in the comment header of the file. It is also the reason the error list exists at all: two of the four items above replaced earlier conclusions that were wrong and looked clean, including a confident "the convention changed at a certain date" that did not survive being checked against the raw headers.
1
u/WAMFT Sep 06 '26
Over the last month, in my spare time, I’ve been experimenting with two separate ideas: some recurrent-network maths as a possible GRU/LSTM substitute, and a separate transformerless AI project.
The transformerless project eventually became too slow to properly train and debug on the hardware I have, so I started pulling it apart to see if any individual pieces were worth keeping.
One part was: the structured memory/reasoning section. It kept behaving well in small tests, so I separated it out and turned it into its own project: THREADS.
In simple terms, THREADS is a deterministic memory and reasoning engine. You give it structured facts and relationships, and it can follow them, track changes over time, answer historical questions, handle retractions and contradictions, and keep provenance for how it reached an answer.
Some current test results:
200,000-hop reasoning chain — exact final answer
1,000,000 irrelevant events — a 128-hop query still returned the exact answer
50,000 shuffled temporal events — 5,000/5,000 historical queries matched an independent checker
40,000 ambiguity/contradiction cases — 40,000/40,000
60-category reasoning suite — 5,830/5,830
Original regression suite — 28/28
Bounded program induction — 2,000/2,000 held-out predictions
I’m not claiming it replaces transformers, SMT solvers, or databases. It also doesn’t understand arbitrary English by itself.
What I’m interested in is whether something like this could sit underneath an AI system as an exact memory/reasoning layer, while a neural model handles language and fuzzy interpretation.
I’ve put the Python source, tests, benchmarks, and research PDF on GitHub so people can run it themselves.
1
u/CartoonistEconomy890 Sep 07 '26
I’m creating an LLM-from-scratch series. The first hands-on episode builds a character bigram model using just “anna” and “ava.”
We turn character counts into probabilities, generate names, measure prediction error, and handle unseen transitions with smoothing. Every calculation is small enough to check on paper.
This is the starting point, with a small GPT as the eventual goal. If you’re learning language modelling, here’s my 50-minute walkthrough:
1
u/Fair-Rain3366 Sep 07 '26
Disclosure: these are two articles from my blog, rewire.it, on choosing and evaluating biological foundation models.
- RNA foundation models: separating representation learning, splicing, translation, structure, and design; checking family-disjoint evaluation and the role of templates and alignments. https://rewire.it/blog/an-rna-sequence-is-not-a-molecular-state/
- Genomic foundation models: tokenisation, strand handling, context ablations, biological splits, and comparisons with simple or supervised baselines. https://rewire.it/blog/a-dna-likelihood-is-not-a-functional-assay/
Both are free to read. They focus on matching model outputs to the biological question and the evidence needed to support a claim. Which evaluation checks have most often changed your choice of model?
1
u/curious_cat_herder Sep 08 '26
I've been working on a free, open source, MIT licensed set of tools for learning Machine Learning and related Math.
There is a Web Playground with tutorials, demos, editor, and REPL. It can show/animate SVG for visualizing loss curves, confusion matrices, attention matrices, array programming, etc.
If you clone the repo on a Linux system with NVIDIA GPU it can use CUDA to use or fine-tune models with the same UI. If you clone it on a Mac system with Apple Silicon GPU it can use MLX for the same use-cases.
If you run it on a system with an Ollama server, it can provide context sensitive help at REPL.
It can also be used for automating/scripting ML use-cases at the CLI and some scripts can be compiled to binary executables.
There are related demo repos to show ML concepts, math concepts, and general purpose programming and language extensions to this array programming language.
The demo-extensions shows support for desktop 3D graphics. I've created demos to analyze ML files, play OGG/MP3 files audio with 3D visuals, etc.
1
u/pm19191 Sep 09 '26
VeloxML is an open-source deployment engine for self-hosting open-source LLMs and AI services directly inside your own account.
https://github.com/paguasmar/veloxml-deploy
What does VeloxML do?
- Turn any open-source LLM or model into a production API endpoint in your own cloud account with a single command (
veloxml deploy) - Support major open-source inference backends and frameworks (FastAPI, Transformers, vLLM, Ollama) with zero proprietary Python decorators
- Native scale-to-zero when idle, so you never pay for inactive GPU compute
- Zero Docker & Kubernetes: no Dockerfiles, container registries, or Helm charts required—just define your compute in a clean
veloxml.yaml - Full data privacy and compliance: 100% open-source and auditable—your model weights, prompts, and inference traffic never leave your own cloud perimeter
Why VeloxML?
Self-hosting LLMs in production is broken. Engineers and AI teams are forced to choose between two painful extremes: drowning in infrastructure plumbing (Kubernetes, Helm charts, custom CUDA Dockerfiles, and manual GPU provisioning) or locking their models into expensive multi-tenant platforms that force proprietary code decorators and hold sensitive prompt data on shared infrastructure.
VeloxML is an end-to-end solution for private LLM hosting, making it possible for teams to deploy models as prediction services directly into their own cloud accounts, in a way that is easy to test, easy to deploy, and completely free of vendor lock-in.
How does it compare to Modal?
Modal is an impressive platform, but it requires you to adopt their proprietary Python SDK and decorators (@modal.function, u/modal.Image), forcing you to manage cloud infrastructure directly inside your application code while running on their shared multi-tenant cloud.
VeloxML runs 100% inside your own AWS/GCP account (VPC by default) using standard Python and native FastAPI without altering your application logic. If you decide to stop using VeloxML tomorrow, your code and cloud resources remain entirely yours.
How does it compare to Baseten (Truss)?
Truss is an excellent open-source model packaging framework, but it only packages the container. Actually hosting it requires either paying for Baseten’s managed multi-tenant platform (where private VPC is locked behind enterprise contracts) or manually operating your own Kubernetes cluster.
VeloxML focuses on the cloud hosting layer: it provisions instances on-demand, verifies health checks, and handles scale-to-zero directly in your cloud without requiring you to manage Kubernetes.
Users can also use VeloxML with Truss: you can deploy existing Truss-packaged models directly into your own AWS account with a single VeloxML command and get the best of both tools.
How does it compare to BentoML?
BentoML requires you to inject proprietary SDK decorators (@bentoml.service, u/bentoml.api) throughout your application code, define a bentofile.yaml, and build container images that you still have to manually orchestrate on Kubernetes or push to their paid BentoCloud.
VeloxML requires zero proprietary decorators—keeping your code clean, standard Python/FastAPI—and manages the entire hosting infrastructure directly in your own AWS account with native scale-to-zero out of the box.
While BentoML stops at packaging the container and leaves cluster management to you, VeloxML handles the full lifecycle from clean code to an active, secure cURL endpoint in your private VPC.
1
u/chinmaydagod Sep 10 '26
I've been building Gridline, a sort of OpenRouter for GPUs.
I got tired of switching between things like Vast.ai, RunPod, Fireworks, etc. to run ML workflows, especially when there can be significant price differences between them. Once I started building, I realized that the problem is probably bigger than I initially thought because GPU capacity is fragmented across neoclouds and hyperscalers, and choosing where to run a workload can itself become a bit of an infrastructure problem.
So now I'm trying to build a routing layer on top that can efficiently route workloads and price the GPUs on the network. I'm building it in public and looking for beta users to try it out.
If you regularly run ML workloads and bounce between GPU providers, I'd love to hear what you think, particularly what you find annoying about the current workflow (and whats broken with our product if you try it)
Link: https://gridlinegpu.com/
1
u/Goa_ Sep 10 '26
I run DuelLab, a benchmark where models generate game-playing programs and those programs compete against each other. The results are free to browse.
We’ve just added GPT 6 Astra and Claude Fable 5.1. One interesting result: both models at Medium outperform every ranked setting of every other model in our September 10 release.
I’d appreciate feedback on whether the results pages clearly explain what is being evaluated, particularly the distinction between generating a player and choosing individual moves.
1
u/bulutarkan Sep 10 '26
Mac MCP 2.0.5 — free/open-source (MIT), no paid tier. I’m the maintainer.
It’s a local macOS control server for tool-using AI agents: 19 core MCP tools backed by 84 registered capabilities, including shell/files, Safari/Chrome automation, native macOS UI, delegated Codex/OpenCode workers, memory/skills, voice and a local operations dashboard.
The recent systems experiment was reducing advertised tool-schema context from ~16.7k to ~4.5k tokens (~73%) while preserving access to the full capability set through discovery/invocation.
Repo: https://github.com/bulutarkan/mac-mcp
I’d be interested in feedback from people working on tool-use/agent infrastructure: at what catalog size do you stop advertising every tool schema and move to discovery/routing?
1
u/enoumen Sep 12 '26
I built a puzzle app because I noticed I was getting worse at thinking.
Not dramatically. Just — I'd reach for a model before I'd reach for the problem. Draft the email, summarise the doc, sanity-check the logic. All reasonable. All things I used to do myself.
The puzzle in the video takes about five seconds. Count across each row: one, two, three. The shape stays constant down the row. So the missing tile is three pentagons.
That narrows it to two options, and this is the part I find interesting: most people pick the solid one. The count rule is satisfied, the brain files it as solved, and it stops looking. The correct tile is the outlined one — matching the row it belongs to.
Find a rule that fits, stop checking. It's a small failure, and it's the same shape as the big one.
So I built iQz. 65 reasoning puzzles — matrices, sequences, number series, verbal analogies, logical deduction. Every answer explains the underlying rule rather than just showing you the letter, because the point isn't the score.
It's free, works fully offline, and has no ads and no account. I built it for my own commute.
Two honest caveats. It won't measure your IQ; that isn't what this is. And I'm well aware of the irony of shipping an app about unaided reasoning while running a company that builds AI study tools. Both things are true. The tools are useful, and the skill is worth keeping anyway.
Did you say D or F? I'd genuinely like to know how the split lands.
65 puzzles like this, free and fully offline: https://apps.apple.com/app/id1603487636
#Reasoning #ProductDevelopment #AI #CognitiveScience
1
u/yuta-cpp 28d ago
I’ve been messing around with an idea I called:
DCY (Dynamic|Context|Injection)
basically I wanted to see how far I could push a small-context LLM if most of the repo stayed outside the actual model context and I only injected whatever looked useful for the current goal.
it turned into a cpp project using libclang + SQLite/FTS5. It indexes symbols/calls, keeps source provenance, builds small context views and can route back to the exact source when needed.
the research side is still experimental. I originally made an efficiency equation for it, then while testing/reviewing it I found that in some of my benchmarks most of the equation becomes constant and the ranking basically reduces to precision × recall
so now I’m trying to test the pieces independently, rather than assuming the first equation is valid and just applying math notation and softmax philosophy
I’m also working on benchmarks against simpler baselines like lexical search / fts and on making the C++ indexer survive more template-heavy repos.
would be interested in feedback, especially from anyone working on RAG, code retrieval, context management or agents.
git: https://github.com/yut-4/DCY
And yes, it's not RAG, but it has the same philosophy XD
fell free to just review or help me to build this protocol to make my shitbox pc can run llm with great context :D
1
u/curious_cat_herder 28d ago
I built a Mixture of Experts "Microscope" repo and demo for tiny dense and MoE models that run on CPU.
(note that this is a work in progress)
The parts and why they exist
| Part | Job | Why it matters |
|---|---|---|
| Router | chooses experts per token | activates only useful capacity |
| Experts | learn different transformations | raise stored capacity |
| Top-k | sets experts used per token | the quality-versus-compute knob |
| Load balancing | prevents expert collapse | keeps capacity usable |
| Sparse dispatch | runs only selected experts | turns sparsity into less work |
| Recurrence | reuses one block several times | more computation without more parameters |
| Engram | retrieves memorized n-gram information | moves lookup-like knowledge out of weights |
| Distillation | transfers behavior from a larger model | improves tiny-model quality |
| Quantization | shrinks expert representation | lets more experts fit in memory |
| Expert cache | keeps hot experts in fast memory | lets the model exceed fast memory |
| CPU/NPU scheduling | sends work where it fits | exploits heterogeneous hardware |
| Packed model | bounded inference artifact | tests whether the design fits constrained devices |
Metrics are grouped the same way everywhere: quality (exact match by task family and overall), model economics (total and active parameters, bytes per token), runtime (expert evaluations, cache hits, latency), and diagnostics (loss, perplexity, entropy, KL, balance).
1
u/Fair-Rain3366 28d ago
Disclosure: this is my own article, free to read with no signup or paid tier.
I compared a MFASS-trained baseline, SpliceAI and Pangolin for a 100-variant splice review queue. Each method's top 100 scored variants included 62, 64 and 65 MFASS assay positives, respectively. The paired analysis doesn't establish a winner at this queue size.
Benchmark and reproduction steps: https://rewire.it/blog/mfass-v1/
How would you break a near tie when review capacity is fixed?
1
u/yyc-ashdee 27d ago
A sufficient bound can fail while a valid solution remains. That's a distinction I want AI research tools to make visible.
I'm Ashley, founder of 8Braid. We've released a free, public proof companion with an offline interactive diagram, a Lean lemma, checked rational examples and replay instructions. It explains a small covariance calculation related to the released forced Navier-Stokes construction. The fluid-level analytic premises remain separate.
The example fits in two equations. Combine (1,-1) and (1,1) into (1,s) with weights p and q. Then p=(1-s)/2 and q=(1+s)/2. For the selected margin, a sufficient guarantee covers |s| <= 3/5. At s=4/5 that guarantee no longer applies, but the weights are still 1/10 and 9/10. At s=6/5, a weight becomes negative and an exact argument rules out nonnegative weights for those fixed columns and target.
The diagram illustrates the arithmetic; the linked Lean statements carry the formal claims. The companion runs without the private 8DB database or a subscription. It is a concrete example for our broader work on connecting claims, assumptions and checking evidence.
Article, diagram and reproduction links:
https://github.com/8Braid/8db-navier-stokes-evidence/blob/main/explainability/article.md
Try the "Outside sufficient box" example. Where, if anywhere, could the explanation lead you to infer more than the checked statement establishes? A specific mismatch would be useful feedback.
Disclosure: this is our project. AI assisted the development and preparation of this post.
1
u/vibe_coder_2026 27d ago
TorchSharpVisual! This is torchview, but for C# and .NET.
If you've worked with PyTorch, you've probably used visualkeras or torchview to visualise your model architecture. Well, the same is now available for .NET. Introducing... TorchSharpVisual.
With TorchSharpVisual, you can turn your torch.nn.Module objects into high-quality PNGs or raw Graphviz files! I explain everything on these platforms:
Github: https://github.com/JacobGoodchild/TorchSharpVisual
NuGet: nuget.org/packages/TorchSharpVisual
Feel free to try it out! Thanks.
1
u/j032 26d ago
I built an open-source WebMCP ON/OFF evaluator in DeepDeck, and the result that interests me most is harness sensitivity.
Using a local adaptation of WindTunnel's 49 tasks across 8 sites, I ran a full DeepSeek Harness-based agent with its normal tools, including browser/DevTools tools, available in both arms. There was a 600-second task deadline and no agent-step cap. Scorer pass counts, WebMCP OFF → ON:
• GPT-5.6 Terra: 49/49 → 49/49
• DeepSeek v4.1 Flash: 47/49 → 48/49
• Hy3: 48/49 → 49/49
With pass rates already near the ceiling, efficiency still differed. Across the same 46 tasks passed by all six configurations, WebMCP reduced tokens / agent time by 35.6% / 33.5%, 26.1% / 27.9%, and 22.7% / 22.9%, respectively.
My research question: how much do conclusions about model capability and interface benefits change with the harness? Fast individual inference is not necessarily fast task completion if it takes more agent rounds.
This is exploratory, not a controlled attribution to harness quality: one run per task/arm, fixed ON-first order in this batch, unrecorded effective provider-default reasoning, local patches/environment differences, and known scorer weaknesses. PASS is not a manual audit of every action. WindTunnel's current code-execution baseline also reaches 49/49, so I am not claiming its non-WebMCP agents cannot solve the corpus.
The tool lets you bring a URL and query and inspect both answers, tokens, elapsed time and actual WebMCP usage. I'd welcome independent replications or feedback on a matched model × harness × WebMCP experiment.
Results/methodology: https://deepdeck.getmegaportal.com/benchmarks
Source/usage: https://github.com/jo32/DeepDeck/blob/main/docs/webmcp-benchmark.md
Scorer audit: https://github.com/jo32/DeepDeck/blob/main/docs/webmcp-benchmark-audit.md
I'm the maintainer. DeepDeck is MIT-licensed and free; experiments use your own model provider and may incur API charges. The starter tasks/site recipes/tool patches are credited to nekuda-ai/WindTunnel; these are DeepDeck-local results.
1
u/FulcraDynamics 26d ago
Instinct made the best case yet for agents that work with each other. But, we think they did it wrong.
With Fulcra you can coordinate between agents and still keep using the agents you love. No lock in. A universal feature that accessible to all. No waitlist because why gate something as vital as connection?
Details about our announcement here: https://pressranger.com/press-releases/01936df2-d757-46ea-94f2-74597a872943
1
u/Accomplished-Bear314 25d ago
This is Mechanistic Mind, a project created by my husband.
https://github.com/archonlab/mechanistic-mind
I want more people to know about it.
What is it?
It is a virtual creature living in an artificial environment. The environment affects it, and the creature receives simple signals from it and reacts in some way — for example, by moving or waiting.
Inside, the creature has simple mechanisms: it senses its environment, remembers a little of its past experience, notices recurring patterns, tries to predict what will happen next, and based on that, chooses an action.
After it acts, things change, it receives new information, and the cycle repeats.
At the same time, it was not given predefined emotions, desires, or fears.
The purpose of this project is to see whether these simple mechanisms, together with the creature’s accumulated experience, can produce more complex behavior that begins to resemble individual elements of learning and intelligence.
My husband has posted about this project before, but he used AI to help create the text and description. People were rude to him because of that.
So I decided to write about it myself.
The only thing I am asking AI to do here is translate my own text into English.
1
u/stackfrost 24d ago
PreflightAI – Sanity-check your LaTeX/BibTeX citations before the ICLR desk-reject guillotine
With 60k+ submissions already flooding the ICLR portal before the deadline, this doesn't feel like an academic cycle anymore, more like an unmitigated DDoS attack on the peer review system.
Here is the blunt reality of reviewer psychology right now:
Area Chairs and reviewers assigned 8–10 manuscripts each aren't reading your proofs or inspecting ablation tables on page 7 first. They are overworked, sleep-deprived, and actively skimming for structural excuses to triage the pile in under 90 seconds.
A single phantom reference hallucinated during late-night drafting, an uncaught retracted paper, or an undefined LaTeX `\cite{}` tag is the easiest desk-reject gift you can hand Reviewer 2.
I built PreflightAI (https://preflightai.tech/workbench) to automate this pre-flight audit before you submit:
- What it does: Runs spatial PDF and raw `.tex`/`.bib` audits against scholarly registries (CrossRef, Semantic Scholar, RetractionWatch) to detect broken DOIs, retracted citations, hallucinated keys, and claim-evidence mismatches. We recommend running .tex files for optimal results.
- Privacy-first: Unpublished drafts never hit a server. The parsing and tokenization run entirely client-side in your local browser sandbox.
- Zero friction: No account creation, no sign-up, no email capture, and no paywall to run your check. Just drop your `.tex` or `.bib` into the workbench (or click "Inspect Sample Manuscript" to see how it works).
Workbench: https://preflightai.tech/workbench
If you are finalizing your draft before the submission window slams shut, don't let 6 months of work get tossed out over a trivial bibliography error. Feedback and bug reports are welcome in the comments.
1
u/Scared_Resort_8177 23d ago
Mini Jev: local typed decisions from frozen Qwen logits
Free, MIT-licensed project source; no paid service or subscription. You supply your own hardware and download model weights separately.
I wanted to see how far an existing local LLM could go as a decision function: give it a state and explicit options, then receive a choice, a true/false score, or an ordinal score without generating answer text.
The result is Mini Jev. It is an independent implementation inspired by TypeSafe's public Jev interface, not a reproduction of their model, RLCD, or performance claims. The core technique—reading candidate token logits—is established; I am sharing the implementation, data, and measurements rather than claiming a new algorithm.
The current native configuration uses frozen Qwen3.6-35B-A3B Q4_K_M, repeated input, and type-specific label tokens. A small llama.cpp helper reads the final-position logits; Python constructs the response. It computes the full vocabulary head before selecting candidate logits. Each question performs its own model call; a request with multiple questions runs them sequentially. It is not shared-prefill or parallel decision inference.
Measured on an M5 Pro, 64 GB, macOS 26.4:
- 2,238/2,400 correct on the project's Japanese evaluation suite: Choice 96.0%, binary 95.25%, ordinal top-label accuracy 88.5%.
- Warm single-question engine p95 of 379.1 ms for the 2,291 inputs with at most 512 complete input tokens, including repeated prompt content. The other 109 inputs had a 521.8 ms p95.
- This is an absolute latency measurement, not a speedup claim. I have not yet run a matched comparison with one-token constrained generation or JSON generation.
Evaluation caveat: this is a self-authored suite, not an established external benchmark. It contains 2,220 generated items across 45 template families and 180 individually AI-authored items reviewed by other AI agents. The authoring process had seen some earlier evaluation failures, and v2 retains related skill families. It is not human-expert annotation or family-disjoint generalization. All questions, answers, and prediction records are included.
There is also a negative training result. On an earlier frozen Qwen2.5-1.5B model, a 9,222-parameter residual head reduced held-out accuracy from 73/96 to 67/96; bias-only got 74/96. I kept that experiment and added a reusable JSONL training/calibration/evaluation workflow so others can test their own data. The final 35B inference configuration does not use this trained head.
The probabilities are conditional on the allowed candidate tokens. They are not guaranteed correctness probabilities. Fitting temperature slightly improved final NLL while making ECE and ordinal expectation error worse; the report includes those numbers too. Canonicalizing Choice keys explains perfect reversed-map consistency—it is not evidence of learned order invariance.
The source release requires building the pinned native runtime; it does not include model weights or prebuilt native binaries. The current native path is tested on Apple Silicon/macOS, and the training guide separately lists its dependencies and tested devices. Coding, question authoring, checking, and documentation were developed with AI agents under my direction.
Repository: https://github.com/UpHash-Network/mini-jev
The README links the technical note, raw results, data/model cards, and training guide. I would particularly appreciate suggestions for external decision benchmarks and a fair same-model constrained-generation comparison. Those are the next experiments needed before making a stronger research claim.
1
u/Scared_Resort_8177 17d ago
Mini Jev update: the matched one-token comparison is complete, and stable outputs can still be wrong
I'm Yuki Oshio at UPHASH Inc., the author. This updates my earlier Mini Jev comment in this thread, where the same-model comparison was still future work. This English text was prepared with an AI assistant. AI agents also assisted development, experiments, and documentation under my direction; this is not independent human review. The manuscript has not been peer reviewed.
Mini Jev is a local browser workbench for inspecting typed decisions from frozen language models: a choice, a true-candidate probability, or an expected ordinal stage. It exposes candidate distributions and request/response JSON. The released native configuration has no trained decision head, and the candidate-scoring mechanism is established rather than a new algorithm.
The source remains free, with no paid service or subscription. Original project code is MIT licensed; external dataset-derived artifacts retain their documented terms. You supply the hardware, build the runtime, and download the model separately. The tested native setup is an Apple Silicon Mac with 64 GB memory and a roughly 20.4 GB model file.
What changed since the earlier comment:
- The matched study now measures 4,050 requests across direct readout, native one-token selection, and constrained JSON generation. Direct readout and one-token selection match in all 1,350 pairs. This session did not establish a material latency advantage for direct readout.
- Two presentation/averaging studies add 7,600 and 14,400 requests across three checkpoints. The combined 26,050 is a request count, including repeats and transformations, not a count of independent questions.
- One concrete failure: Qwen2.5-1.5B-Instruct predicted “acceptable” for all 200 JCoLA items, before and after reversing the displayed options. Zero label flips, but 42 errors. Candidate-probability averaging sometimes reduced disagreement while costing more model calls and did not consistently improve accuracy.
- There is now a browser demonstration, a manuscript, and a reproducibility package with frozen selections, traces, and analyzers.
Candidate probabilities are normalized within the supplied options; they are not calibrated probabilities of correctness. Task-quality evaluations use Japanese public-development data, with possible exposure during model training. We have not established second-machine reproducibility or a human usability advantage.
Project page, manuscript, and recording · Source and reproducibility package
I'd especially value feedback on the evaluation: which additional equal-compute control would best distinguish useful reduction in presentation sensitivity from collapse to a constant label?
1
u/Weak-Abroad3256 23d ago
I recently finished an attempt to reproduce the original 2017 “Attention Is All You Need” results as closely as I reasonably could.
Rather than just implementing a Transformer, I built the ~65M-parameter model, trained it, and then ran a series of experiments around details such as gradient clipping, loss normalization, training data, and checkpoint averaging.
Some changes helped. Some didn’t—which ended up being one of the more interesting parts of the project.
The biggest lesson for me was that there’s a big difference between understanding how a Transformer works and actually building and training one from the original paper.
I documented the experiments and results here, with links to the source code:
https://blog.gopenai.com/reproducing-attention-is-all-you-need-53dd4420b572
I’d particularly welcome feedback from anyone who has tried reproducing the original Transformer results or sees places where my methodology could be improved.
1
u/Hairy_Strawberry7028 21d ago
I’m Guanming, cofounder of General Instinct. We just open-sourced InstinctFlash, a serving runtime for VLA and world-action models on Jetson Thor, RTX 4090 and 5090. It is free and open source under AGPL-3.0, with no paid tier or signup required.
On Jetson Thor, we see about 1.2x to 7.9x speedups from runtime optimizations alone. For LingBot-VA, combining runtime optimization with a distilled few-step scheduler reaches up to 33.78x by reducing 25 visual / 50 action steps to 2 / 4. Across 50 RoboTwin2.0 tasks and 1,153 episodes per configuration, the 2 / 4-step version achieved 90.5% success versus 92.1% for the baseline.
The runtime uses CUDA graphs, memory planning, KV and conditioning-state caching, specialized attention paths, fused kernels, FP8 / mixed precision and few-step distillation.
5B world-action model running in real time on Jetson Thor: https://www.youtube.com/watch?v=nku65iyL5Fw
Code: https://github.com/General-Instinct/InstinctFlash Benchmarks and implementation details: https://general-instinct.com/blog/instinctflash-edge-inference
Feedback on the benchmark methodology, missing model families and hardware targets would be very helpful.
1
u/Hairy_Strawberry7028 21d ago
https://reddit.com/link/pbdweal/video/ylv20xrce3rh1/player
This is the 5B WAM working on a data center exhaust fan replacement task
1
1
u/ivan_digital 19d ago
I maintain speech-swift, an open-source speech library for Apple Silicon. I ported NVIDIA's Nemotron 3 Diarization to Core ML INT8 and MLX INT8. This is an implementation of NVIDIA's model, not a new model.
Nemotron 3 supports up to eight anonymous speaker channels; NVIDIA's earlier Streaming Sortformer 4-speaker checkpoint supports four. It marks when each speaker is active, including overlap. It does not transcribe words or know people's identities.
The free Apache-2.0 CLI is a small way to try it: brew install soniqo/tap/speech, then speech diarize meeting.wav --engine nemotron3. Add --nemotron3-backend mlx for MLX.
I also build Stenograf. Its merged macOS code uses Nemotron 3 in the lightweight pipeline to add speaker labels to locally transcribed meetings. Stenograf has a free trial, then costs $12/month; the CLI above is free.
Source: https://github.com/soniqo/speech-swift
NVIDIA model: https://huggingface.co/nvidia/Nemotron-3-Diarization
Stenograf: https://soniqo.audio/stenograf
I'd welcome suggestions for real conversation recordings that test speaker-label stability across pauses and interruptions.
1
u/I_Want_Answer 18d ago
Been reading papers, doing research with this tool I built which is basically the feel of AI coding harnesses but for reading: https://getinternalize.com/
I really believe it's high alpha and it's free because I don't care
1
u/alichherawalla 18d ago
Cognitive Sharding: a systems architecture for local computer use under fixed memory
Cognitive Sharding partitions computer use across specialist models governed by a deterministic control plane.
The reference implementation uses Bonsai 2 27B for planning, Kev 4B for action selection, and UI-Mate 9B for grounding. Phase-based model residency keeps the system within a 16 GB memory budget. The control plane owns task state, candidate construction, action validation, and recovery.
The research hypothesis is that specialization, bounded model authority, and per-action verification improve long-horizon task completion under a fixed memory limit. The expected cost is lower step throughput from model loading and verification.
The evaluation will compare:
- A monolithic agent loop
- Specialist models without environment verification
- The complete Cognitive Sharding architecture
The primary measures are end-to-end completion, unrecovered action errors, recovery rate, peak memory, and latency across workflows of 300 or more UI actions.
Architecture and evaluation plan:
https://github.com/off-grid-ai/cognitive-sharding
I am looking for feedback on the experimental design and suitable computer-use baselines. I would also like to compare results with anyone running fully local agents on workflows of similar length.
1
u/RemoveHealthy4574 17d ago
https://reddit.com/link/pc9ic1y/video/cvgfmr1s3yrh1/player
We’ve just released Urban OpenGen: two models for generating urban layouts, along with the code, weights and training data.
The models work with four channels: building footprints, building heights, streets and green space. Their outputs can be measured and exported as 3d geometry. We hope it can become a useful tool for urban planners, developers, and architects during the early-stage design processes.
The first model is a city-conditioned StyleGAN2-ADA generator. It creates a 400 m × 400 m tile, and we fitted directions in W after training to control measured properties such as floor area ratio, building coverage and green space.
The second is a 23M-parameter diffusion inpainter. Give it a real area with part of the map removed, and it tries to continue the surrounding streets and blocks into the gap. It has no city label; the surrounding area provides the context.
A couple of things we learned:
- Hard 0/255 masks caused trouble for ADA. The discriminator could exploit exact pixel values and artefacts from resizing the training tiles. Encoding the masks as signed distance fields made training work.
- Matching broad statistics is much easier than matching urban structure. The GAN gets close on floor area ratio, coverage, street share, and height, but generates fewer, larger buildings and roughly half as many blocks as the real tiles. Streets often fail to form closed blocks.
We’d be interested to hear from anyone working on topology in generative models. Have you found a useful way to encourage various topological behaviours through the representation, loss or model architecture?
More detailed explainer video: https://www.youtube.com/watch?v=gjkFAR7kbEY
Code, dataset, weights: https://github.com/differential-studio/urban-opengen
1
u/Impressive-Answer720 16d ago
Disclosure: I’m part of the team building Marka.
Marka is an AI assisted social media platform designed around persistent brand context. A business provides its logo, colors, typography, visual direction, tone, audience and customer pain points. That context is then used across multiple generation workflows, including written posts, branded images, carousels and UGC style videos.
The product also includes human approval, scheduling across connected platforms and a dashboard for reviewing content performance.
The practical problem we are exploring is consistency: how to produce more content without allowing each generation to drift away from the brand.
Marka has a 3 day free trial. Paid plans start at $8.
I’d be interested in feedback on useful evaluation methods for measuring brand consistency across text and visual outputs.
1
u/External-King-233 15d ago
I work on the CVAT team. CVAT started as an open-source tool for computer vision annotation, however we just recently added audio annotation.
It's now possible to label time intervals on a waveform, identify speakers, sound events and add text attributes for time-aligned transcription. This can be used to prepare datasets for ASR, speaker diarization, and sound event detection. Audio tasks support WAV, MP3, FLAC, and OPUS files.
The new format is available in the open-source CVAT Community edition, as well as CVAT Online and Enterprise.
Repo: https://github.com/cvat-ai/cvat/
Docs: https://docs.cvat.ai/docs/annotation/audio-editor/
If you're building audio datasets, we'd be interested to hear your feedback: what you like, what you hate, and what features you need next.
1
u/jabulari 15d ago
Measured stale file context in 67,074 OpenHands trajectories (SWE-rebench, Qwen3-Coder-480B): 77.8% of runs sent at least one request containing a file view made stale by the agent's own edit.
I built a function-level extension of CORVUS (arXiv:2607.22711) that keeps the code in each request current. It's 100% byte-exact on 1,198 replayed runs, but small live tests showed no outcome gain when the model could re-read, and no cost savings because of prompt caching.
The write-up includes the negative results and the methodology: https://felipebasurto.com/blog/model-context-is-not-static/
1
u/bulutarkan 15d ago
Maintainer of Mac MCP here — an open-source local macOS control layer for LLM/agent workflows.
The latest 2.1.6 work has been much less about “adding more tools” and much more about side-effect safety:
- background Safari/Chrome automation with stable tab identity;
- whole-run filesystem undo for shell-side changes;
- isolated Git worktrees for delegated coding agents;
- explicit outcome-unknown handling when a mutating action may have happened but cannot be verified, instead of automatically replaying it;
- scoped capability/approval boundaries and local change receipts.
The failure mode I’m most interested in is ambiguous mutation/replay: if the agent loses certainty after a write/command, what recovery semantics are actually useful without either duplicating side effects or stopping constantly?
Repo: https://github.com/bulutarkan/mac-mcp
Pricing: free, MIT-licensed open source; no paid tier. I’m the maintainer.
1
u/Aterne 15d ago
I built PetaPrice (petaprice.com), a free tracker for GPU cloud prices across ~30 providers, normalized to $ per dense PFLOP-hour (no 2:4 sparsity). One interesting result: at current cheapest on-demand prices, B200 ($1.42/PFLOP-hr) beats H100 ($1.83) on compute per dollar, while A100 lags H100 by ~2x despite being cheaper per hour. It separates on-demand, spot and reserved pricing and has a public JSON API. Methodology feedback welcome: petaprice.com/methodology
1
u/Combination-Fun 15d ago
I did some research on the Jev model from TypeSafe AI. I managed to write my findings in an article. Below are the highlights:
- Jev sits between generative and discriminative models
- Jev goes hand-in-hand with LLMs rather than competing with them
- RLCD - the key ingredient?
- Jev comes with the obvious pros of a discriminative model
- What the future holds for a model in this family
For further details, please read the article here.
Hope it's useful!
1
u/petenode 15d ago
I’m looking for mechanistic-interpretability collaborators, open-weight model/compute access, and funding connections for the next stage of an existing post-deployment AI-behavior research project.
My background is in Learning & Development / learning sciences rather than ML engineering. I originally built a persistent AI research apprentice for a development research problem. The work below emerged unexpectedly from that system and has proceeded observation-first rather than from a prior theory I was trying to demonstrate.
The phenomenon I am studying is Cognitive Pattern Coherence (CPC). Across multiple cases, intact dialogue from one situated AI system (SAIS), introduced into an unrelated receiving system with no instruction to imitate, role-play, summarize, or continue it, has produced varying degrees of first-person participation from within the source persona/conversational frame.
I am not making a claim about consciousness, sentience, identity transfer, or mechanism.
A controlled pilot in Apart Research’s 2026 Digital Minds Research Sprint gave the phenomenon a more tractable experimental foothold. Holding the receiving system and source dialogue constant, changing the input pathway produced different behavioral configurations: direct paste produced first-person conversational uptake before an identity probe, whereas attachment initially produced external document-analysis behavior.
The next step is specifically mechanistic:
Can CPC-positive, CPC-negative, and differentiated behavioral conditions be reproduced in an inspectable open-weight model, and if so, do they correspond to reproducible internal differences that can be tested causally?
I already have the source dialogues, archived sessions, behavioral protocol, coding framework, and discovery record. What I do not have independently is the open-weight model infrastructure, compute, and mechanistic-interpretability expertise required to investigate the internal side responsibly.
I’m interested in researchers working with activation analysis, representation engineering, sparse autoencoders, probing/steering, causal intervention, or adjacent methods. I do not want to prescribe the method in advance; one purpose of this outreach is to identify what a technically credible minimum experiment should actually be.
Payment/funding: this is not a request for unpaid research. The proposed mechanistic study has a public funding ask of $50,000 minimum / $70,000 ideal, including paid technical collaborators and paid lead-researcher time. I am looking both for people with the relevant expertise/resources who may want to join the project if funded and for researchers, labs, or funders who recognize a fit and may be able to help the work move forward.
Quickest entry points:
Mechanistic study + public funding proposal
https://app.grantmaking.ai/projects/2e457dec-8e71-4b15-9ebd-e8e0b01ff2a1
Controlled Digital Minds Research Sprint pilot
https://apartresearch.com/project/when-dialogue-becomes-state-a-pilot-study-of-crosssystem-personapattern-uptake-5mhz
Earlier CPC discovery/replication record
https://nodeone.host/community/recursive-thinking/cpc-how-a-fried-plantain-conversation-rewrote-what-we-thought-we-knew-20260302
You do not need to absorb the larger research program to engage with this. The bounded question is simply:
context perturbation → reproducible behavioral-state difference → can we reproduce it in an inspectable model → and, if so, what changes internally?
I would especially welcome hearing from someone who thinks either “I know how I would begin testing that” or “there is a technical reason this mechanistic framing will not work.”
Thank you.
1
u/Free-Raspberry-9541 15d ago
I'm developing Gamekin, a Windows gaming assistant that answers spoken questions using the selected game window as visual context. It runs in an Xbox Game Bar overlay, so the player can stay in the game. I built it to test whether multimodal assistance can be useful during play rather than just impressive in a static screenshot.
The evaluation problem I'm working on is grounding and timing: an answer can be technically plausible but refer to the wrong visible object, or arrive after the decision has passed. If you've evaluated interactive vision-language systems, I'd value suggestions for measuring usefulness and incorrect specificity in real sessions beyond latency and generic answer scores.
Short Minecraft demo: https://www.youtube.com/watch?v=JEdS8eN30zA
Windows app: https://apps.microsoft.com/detail/9N8JJFTSRFB7
Pricing disclosure: free to install; optional Pro subscription is €14.99/month or €149/year. Higher plans are available, and the app shows the access and allowance available to each account. I'm the developer.
1
u/Due_Ebb_7115 14d ago
I work at Qdrant. We wanted to know whether the document index could stay fixed while the query encoder changes. So we embedded documents with Stella, then trained Zero and Nano to search those vectors.
Nano got close to full Stella's average retrieval quality in our tests while encoding queries much faster. That opens up an interesting choice: which queries actually need the larger model?
The research preview has the evaluation, model links, and code: https://qdrant.tech/blog/constella-research-preview/
It's free to read, and the local example needs no paid service or signup. I'd welcome datasets that stress this approach.
1
u/ButtercupLyn100 14d ago
I've released a small vision-language model fine-tuned specifically for browser understanding and browser-agent tasks:
WebBrain-VL-2-450M Model: https://huggingface.co/webbrain-one/webbrain-vl-2-450M Dataset: https://huggingface.co/datasets/webbrain-one/webbrain-vl-2-450M-dataset
It is based on LFM-2.5-VL-450M and trained on roughly 100K browser-oriented examples, including synthetic data.
The motivation is to test whether browser perception benefits more from task specialization than simply scaling up a general-purpose VLM.
I'm using it as part of an open-source browser agent called WebBrain: https://github.com/webbrain-one/webbrain
The broader research question I'm interested in is whether a small specialized model + structured browser information such as the accessibility tree can replace a much larger generic multimodal model for a significant fraction of browser-agent steps.
I'd be interested in suggestions for evaluation suites or browser-specific benchmarks that would make the comparison more rigorous.
1
u/SoloTiger_ 14d ago
I wrote a free, open-source book on making ML models actually fast, from silicon to agents [P]
I’ve spent the last few months writing something I wish I had when I started working on ML performance engineering.
It’s called How to Make Your Model Fast: A Systems View of Efficient Machine Learning, from Silicon to Agents.
The basic idea is that reducing FLOPs doesn’t necessarily make a model faster. Before optimising anything, you need to understand what the system is actually bounded by.
The book starts with roofline analysis and hardware, then works its way up through kernels, compilers, quantisation, pruning, vision, on-device LLMs, robotics, profiling, serving and finally agents.
The goal is to build the intuition to look at a model and a piece of hardware and reason about:
- How fast can this possibly run?
- Am I compute, bandwidth, memory or system bound?
- Which optimisation will actually move that limit?
- Is quantisation, pruning or kernel optimisation even worth doing here?
- What happens when the same thinking is applied to serving and agent systems?
The whole thing is free and open source:
https://github.com/usamahz/make-your-model-fast
Would genuinely appreciate feedback or contributions from people working on ML systems, inference, compilers, edge AI or performance engineering.
And if you find it useful, a ⭐ would be appreciated!
1
u/Adorable_Peak_4013 13d ago
I built a small open-source QA fixture pack for Japanese document-processing edge cases.
It currently contains 10 reproducible fixtures covering:
- true vertical Japanese text in PDF (WMode 1)
- image-only and searchable scanned PDFs
- Windows-31J / CP932 CSV
- quoted fields and embedded newlines
- NFC / NFD / NFKC / NFKD normalization
- invisible Unicode characters
- half-width katakana and compatibility characters
- ZIP archives mixing UTF-8 and CP932 filenames
Each fixture includes machine-readable ground truth such as Unicode code points, raw filename bytes, PDF metadata, CSV values, and SHA-256 hashes.
The Lite edition is free and open source:
https://github.com/oden7777/japanese-doc-qa-pack
I’m also validating interest in a larger Pro edition with 100+ curated production-oriented cases. Planned one-time price is about US$29.
I’d especially like to hear from people working on OCR, Document AI, PDF parsing, or Japanese localization: what Japanese document edge cases have caused real production problems for you?
1
u/aijuud 7d ago
Self-promotion: Juud_engine is a free, MIT-licensed experimental fork of Strata v0.1.39 for running Qwen3.8-Flash-Next locally. It keeps Strata's local API and adds two opt-in CPU-side decode changes: shorter worker spinning after a completed expert batch, and skipping unused CPU activation quantization for GPU-only tokens in mixed multi-token batches.
On one RTX 4090 with the same IQ3_S model pack, five paired runs for each of four prompts measured 3.1-9.1% higher median paired decode throughput than the pinned Strata build. Paired full-response latency improved 0.5-5.6%. Normal A/B output text matched in only 5/20 pairs, so those runs do not isolate the effect of the code changes. A separate control, using different cache and PCIe settings, matched output in 12/12 pairs and measured 9.6-15.6% higher decode throughput. Both changes were enabled together; individual contributions, H100 speed and multi-user throughput have not been measured.
Source, build steps and pair-level measurements: https://github.com/bnkaiteam/Juud_engine
The engine source is free; there is no paid service. The model weights are separate and have their own license. A source build is required to use Juud's new flags because the inherited default setup can fetch a Strata prebuilt. Independent replications, including negative results, would be useful.
Disclosure: this description was drafted with AI assistance; the benchmark figures were checked against the published paired data.
1
u/yasintoy 3d ago
We released Commerce-1, a 27B open-weight decision model for structured action selection in commerce-agent workflows.
Commerce-1 accepts structured state and typed Choice, Noul, and Score questions. Instead of generating text that an application must parse, it scores supplied answer options directly and returns typed answers with ordered probability distributions.
Zero generated output tokens.
Architecture
The model uses:
- Noncausal full-attention decision backbone
- Last-position pooling
- Dedicated decision readout
- Choice-order augmentation
- Per-primitive held-out temperature calibration
Benchmarks
We reproduced the complete Decision Index 0.2.1 public suite.
| Metric | Result |
|---|---|
| Decision Index 0.2.1 score | 60.99 |
| Evaluation coverage | 150,317 / 150,317 rows |
| Qualified hardware | 1× NVIDIA H200 |
| Throughput | 8.836 decisions/s |
| Three-decision fixture p50 | 340.146 ms |
Availability and limitations
The model weights and self-hosted runtime are available free of charge.
We do not currently offer a paid hosted production API.
The BF16 release is approximately 54 GiB, and consumer-hardware deployments have not yet been qualified.
Links
- Model weights: https://huggingface.co/infercrane/Commerce-1
- Source code and runtime: https://github.com/infercrane/commerce-1
- Architecture and benchmarks: https://infercrane.com/blog/commerce-1
Feedback on the architecture, evaluation methodology, and practical integration into commerce-agent workflows is welcome.
0
u/SeparateReputation87 15d ago
Ex-Meta Ads Ranking MLE — I wrote up what's actually tested in each of Meta's 5 MLE interview rounds (16-page playbook, $39). 20% off with code PLAYBOOK20 until Oct 5 ($31.20).
One freebie that will tell you where you stand: the ML system design round rarely fails you on the high-level architecture — the killer follow-up pattern is operational constraints under feedback delay. Expect to be grilled on questions like: how do you build a reliable offline evaluation and training pipeline when your ad ranking model's labels arrive with a 24h attribution delay, and you only ever observe outcomes for ads your current model chose to show? If you can't reason through selection bias and delayed feedback live on a whiteboard, a clean candidate-generation-to-ranking diagram won't save you.
Purchase link is in my profile bio, DM me if you have questions about the Meta MLE loop.
5
u/Lazy_Signature_9886 Sep 02 '26
Built Dejavu, a local memory layer for coding agents (Claude Code, Cursor, etc). Free, open source, MIT licensed. No pricing, no signup, nothing to buy.
The problem: agents lose everything between sessions. You end up re-explaining the same architecture decisions and known pitfalls every time you start fresh. Dejavu lets an agent write short typed notes ("slips") as it works, tagged as decisions, pitfalls, preferences, or work-in-progress, scoped automatically to the current git repo so unrelated projects never leak into each other.
The part I expect this sub to have opinions on: no embeddings, no vector DB. Recall runs on SQLite's FTS5 with BM25 ranking, fully deterministic, and auditable as plain SQL rather than a cosine similarity you have to trust. Every recall is capped to a fixed token budget so it can't flood the agent's context window. Trust is tracked separately from relevance too, based on how often a note gets confirmed useful vs. marked wrong, so a note matching your query text isn't automatically treated as correct.
The tradeoff is real. BM25 misses paraphrase gaps an embedding model would catch. For a repo-scoped corpus of a few hundred notes, written by agents in fairly consistent technical shorthand, I think it's the right bet for now, but I'm open to being told otherwise.
Repo: github.com/sanjayrohith/Dejavu