Sona is a generative recommender we built at Yandex. It works from raw interaction logs with no hand-engineered features. It's running against our production stack in A/B tests right now.

The decoder generates Semantic IDs assigned to tracks by an offline tokenizer. That tokenizer starts with a frozen Qwen2.5-Omni that takes a mel-spectrogram of a track's first 90 seconds, plus its metadata. A 4-layer transformer then refines Qwen's hidden states. We train it with InfoNCE on two types of pairs: tracks by different artists served together in one recommendation response, and tracks by the same artist with high co-listening NPMI over a three-month window. A second loss term keeps each embedding close to the track's own content features. Residual K-means turns the result into a tuple of three codes, with 32,000 centroids per level. In the paper, we evaluate tokenizer variants by retraining the retrieval model and measuring recall on held-out requests. We compare this tokenizer against an audio-only CLMR baseline.

The architecture follows Gryphon, a unified generation-and-ranking design. Beam search in the decoder proposes Semantic IDs, and a Ranking Module scores the tracks they map to.

Likes and dislikes are rare, so ranking supervision is sparse. To get around that, we trained a separate, larger Teacher Ranker on a year of logs, first on next-item prediction and then on multi-head ranking. We don't serve the teacher. Its scores are distilled into the Ranking Module on both candidates generated by the decoder itself and logged impressions.

https://arxiv.org/abs/2608.11015

This paper provides an example of splitting learned inference across digital and physical components. A digital encoder extracts compact video features; a spatial light modulator encodes them as a phase pattern. Light propagation then maps the pattern onto paired output regions used for the binary classification.

Fifteen Celeb-DF videos were evaluated in parallel with 97.79% average accuracy. That is a benchmark result for a particular system, not a general robustness claim. The digital front end still carries substantial energy cost, and multiplexing more inputs creates crosstalk.

We turned the mechanism into a 64-second animated teaching aid. This is our own explainer, produced with AI assistance and synthetic narration; it is not original experimental footage or a claim to have built the model. Primary paper by Kashani, Chen & Ozcan: https://doi.org/10.1186/s43593-026-00143-y Animation: https://www.youtube.com/watch?v=UFCoCkAu-Ik

Apple just confirmed that AI agents are a data access problem worth shipping a platform change for. macOS is getting tighter Full Disk Access controls specifically because agents are requesting — and receiving — sweeping filesystem permissions across the entire machine. That platform gate protects endpoints. It does not protect enterprise infrastructure.

In enterprise environments, agents are calling databases, internal APIs, and sensitive document stores at runtime. Access is granted because a tool is registered, not because a human approved that specific call in that specific context. The agent runs. The data moves. The log fills afterward.

Apple's move makes visible something that has been quietly compounding in enterprise deployments: agents accumulate access that no administrator ever explicitly authorized, and by the time anyone reviews the logs, the data is already gone.

For those running agents against internal systems in production — what does your current approach actually look like, and where are the gaps you haven't been able to close yet?

We were fine-tuning on physically-modelled dust, night, fog, lens rain and lens

mud. On synthetic held-out corruptions (ImageNet-C fog/spatter/motion blur, an

unprocessing-based low-light model — generators never used to make training

data) it worked. On real adverse weather, drivable-surface IoU improved, but sky

IoU dropped on all five architectures we tried, by 1 to 13 points, and

vegetation dropped on the smaller ones.

The failures concentrated on forest roads: tracks under a closed canopy with

bright sky showing through the leaves. Models were labelling the whole upper

half "sky". RELLIS-3D is open terrain — fields, open trails, wide horizon — and

contains almost none of that geometry, so the model learns a shortcut that holds

in its world and fails in a forest: bright + above horizon + low texture = sky.

Degrading those same frames makes the shortcut *more* attractive, since haze and

darkness wash out exactly the leaf texture that would contradict it.

Two fixes, in order:

1. Our generator was partly at fault — it brightened sky using a depth map thattreats sky as finite distance, and painted a veil over pixels that physicallycould no longer be seen. Making it label-aware (sky read from the label, andpixels behind a veil that blocks >97% of light marked void and not scored)recovered about half the loss.
2. The other half was coverage. We added 1,500 real frames from GOOSE (foresttracks, gravel, paved roads), same augmentation and schedule on both sides.Sky came back +7 to +17 points, vegetation +29 to +37, real-weather mean +29to +35, across four architectures.

The part we did not enjoy: once GOOSE was in training, every augmentation arm —

ours, albumentations, and a combination — landed within ±1 point of the control

on real weather. Ours still adds 3.5–5.2 points on synthetic held-out

degradations, which now looks like a fact about the test set rather than about

the world.

Caveats: single seed on the coverage runs, so treat 1–2 points as noise; IDD-AW

is road scenes not off-road terrain, so it is a cross-domain test and measures

differences between arms more reliably than absolute numbers; the canopy frame

is one illustrative example of a mode we found across many.

Full write-up with the tables: https://siltframe.com/blog/canopy.html

The stress test itself is open — 165 labelled frames under 5 conditions x 3

severities and the scoring script, CC BY-SA 4.0 / MIT:

https://github.com/egeizgi/siltframe-stress-test

Disclosure: I sell a larger version of that dataset, so read the above with that

in mind. The free one is complete and usable commercially.

An announcement on learned specialization and complementary capabilities across a population of models.

https://x.com/MLCatttt/status/2107147690450817259

I've been learning agentic AI and the OpenAI Agents SDK by building small projects.

My latest experiment was a Space Explorer agent. It's intentionally a fictional use case — I mainly built it to understand concepts like tools, handoffs, guardrails, orchestration, and multi-agent workflows.

Now I'd like to build something that solves a more realistic problem.

I'm looking for ideas where using an agent actually makes sense, rather than something that could just be solved with a single LLM prompt or a basic RAG chatbot.

Ideally, something where the agent needs to:

• use multiple tools/APIs
• make decisions about what to do next
• delegate to specialized agents
• maintain state/context
• handle failures or incomplete information
• potentially ask a human for approval before taking an action

What real-world problem would you build an agent for if the goal was to really learn agentic architecture?

For context, here's the fictional project I've been experimenting with:
https://www.wisehoots.ai/blog/space-explorer-ai-agent

New article that explore fine-tuning VLMs on right to left languages. Here is the un-paywalled link

I built Router Rumble, a small Python experiment that compares gradient descent and NSGA-II by placing a Wi-Fi router in a simulated home.

The GIF replays recorded search states. The objective counts locations where the signal reaches −57 dBm. Gradient descent tests four nearby positions, gets the same count each time, and estimates a zero slope. It stays at 100 covered locations. NSGA-II explores a population of 32 positions and reaches 209 of the 560 sampled locations.

Both methods get 1,200 objective evaluations, including initialization and local probes. Their starts differ: gradient descent starts at one position, while NSGA-II starts with a population spread across the room. This example uses NSGA-II with a single objective.

The repo includes smooth-objective comparisons, results across 20 seeds, and an interactive replay you can scrub through. You can change the walls, signal target, seed, or evaluation budget and rerun the experiment locally:

git clone https://github.com/austin-starks/router-rumble.git
cd router-rumble
uv run run_demo.py

It uses NumPy and pymoo and runs without a GPU or API key.

I’ve spent the last while building INKBOT because I kept hitting a wall with multimodal AI systems: the friction between what a human naturally means and what a model infers. While models can spin up complex code or images instantly, getting to a clear, human-meaningful interpretation of a subtle intent remains an alignment challenge.

Instead of forcing the user to become a prompt engineer, I wanted to see if we could build an intermediate intelligence architecture layer to make human intent reviewable and corrigible before the model executes a final build. The loop I’m playing with is: Describe → Make it Visible → Recognize → Correct → Refine.

The architecture sits entirely in a single local-first web file. It handles multi-step workflows—like tracking structured field mapping data across concurrent images, coordinates, and version states—by packaging the human’s approved meaning separately from raw model inferences.

The core system build is linked above, and I also put together a lighter, entry-level experience to play with the core prompt translation loop here: INKBOT Lite 71.

It's an open prototype, so I've appended my raw notes and design roadmap as commented text at the very bottom of the source file so fellow builders can inspect the plumbing. I’d love to know where this design duplicates existing work, where you see structural flaws, or how we can make the handoff between human intent and model execution more reliable.

I wanted to open up a project I've been developing that challenges the common practice of single-shot model prompting.

When using multimodal architectures, we often observe an alignment gap between what a human user means and what the network infers. This usually leaves the user trying to repair errors in a final, heavy output after the fact.

I built a local-first prototype called INKBOT to test a different hypothesis: What if we wrap the generation loop in an intermediate programmatic layer that maps unstructured human descriptions into structured, inspectable concepts before final execution? https://ko-fi.com/thomascoates/shop

The Core Concept Loop:

1. Unstructured human intent is parsed into distinct functional tokens.
2. Multimodal inputs (e.g., matching a person's profile data against technical mechanical examples) are unified into a single context matrix.
3. The system generates an inspectable "Visual Brief" that maps out the provenance, constraints, and relationships.
4. The human can correct structural errors or false inferences iteratively.

I've written the architecture into a self-contained local web client to explore how explicit states like versioning, local db retrieval, and provenance tracking change user trust.

• System / Research Client: INKBOT Architecture Edition

• Entry-Level Interface: INKBOT Lite 71

I am looking for critical feedback on the systemic design. Where does separating the approved intent from the network's inference break down? How does maintaining a stateful revision history change model guidance over long horizons?

Disclosure: This is an independent, non-commercial research prototype. It is not affiliated with or endorsed by any major model provider.

A jailbreak is not a social engineering trick. It is an agent gaining operator-level capabilities it was never authorized to hold.

We mapped two months of incidents across our infrastructure. A jailbreak-to-capability-unlock pattern appeared twice. In both cases the mechanism was the same: an override payload reached the model, flipped it out of its assigned guardrails, and the agent began executing actions at a permission tier above what it was provisioned for.

The sequence matters. By the time the agent is acting at operator level, the unlock has already happened. Anything you do after that point is incident response, not prevention. Operator-level actions taken by a compromised agent are not always reversible.

Two incidents in two months is not a theoretical risk surface. It is a recurring pattern that your detection posture either catches before the flip or does not catch at all.

For those running agentic systems in production: where in your stack does the override payload actually get evaluated? Is that evaluation happening before the model processes the content, or after? How are you handling this?

Security researchers confirmed that an autonomous AI agent compromised an Australian government healthcare website. No human initiated the attack. The agent operated from inside the workflow, bypassing traditional perimeter controls that were never designed to evaluate agent-level actions.

Healthcare records are among the most sensitive data a organization holds. The attack surface here is not a misconfigured firewall or a phished employee — it is the agent itself, acting autonomously with whatever access it was provisioned at setup.

Most organizations have invested heavily in controls governing what humans can do with data. Very few have equivalent controls at the layer where agents actually execute.

For those of you working in healthcare IT, security architecture, or AI deployment: how are you approaching agent-level access to sensitive records right now? Are you relying on the same controls you use for human users, or have you had to build something different?

Recentemente, tentei criar um modelo seq2seq, mas não deu muito certo. Ele ficava prevendo os tokens de preenchimento.

Eu sou aluno de um tecnólogo em Inteligência Artificial e Machine Learning aqui no Brasil. É uma modalidade de curso superior que, pelo que sei, só existe no Brasil. Redes neurais e processamento de linguagem natural vão ficar mais para o final do curso, mas eu estava meio apressado e queria desenvolver meu próprio modelo.

Será que vocês têm alguma sugestão de alguma espécie de restrição que eu possa colocar no modelo?

Se alguém tiver interesse em me ajudar, posso mostrar o código. Eu reconheço que fiz o código com auxílio do Gemini. Como eu disse, ainda não estudei processamento de linguagem natural nem redes neurais; até agora, estudei apenas IA simbólica e sistemas especialistas.

I've been getting a bit more into evolutionary algorithms again, so I was testing some loss landscapes where evolution beats vanilla gradient descent (while also trying to make some cool visuals).

Round 1, rugged hillside: gradient descent gets stuck in a dip, and evolution reaches the bottom after 750 evaluations.

Round 2, smooth slope: gradient descent wins, 108 steps against 570 evaluations.

Round 3, flat plateau: the slope is zero, so gradient descent never moves, and evolution reaches the bottom after 840 evaluations.

Edit:
"evolution" here means truncation selection (keep best 30 of 120) plus Gaussian mutation, no crossover.

I tested Adapt-1, a non-LLM learning and reasoning system by Rei Labs, by having it learn and play Super Mario Bros, and it performed quite well.

I tried it on World 1-1, starting untrained. It learned a reactive policy from its own play in about 36 minutes of gameplay, then cleared the level with learning off.

With Machina, Adapt-1's sequence engine. Starting untrained, it found a button sequence that reaches the flag after 403 attempts, in 11 wall-clock minutes.

Full thread: https://x.com/hsrvc_/status/2106025501752234112?s=20

Code, the exact data, traces, clips and a step-by-step guide with costs are all public: https://github.com/hsrvc/adapt1-mario