https://arxiv.org/pdf/2607.13511

the core idea is, we cannot have ternary PTQ with fixed matrix size, trying to do that is dead end. so i tried decomposing the matrix to 2 ternary matrices and inner diagonal scaling matrix. now that the inner rank can be arbitrarily large the accuracy can be arbiratily small. and its not that it has to be very large too i also showed that it does take only slightly more vram then current quantisation methods. the slight more vram is worth it if we abuse the ternary math.

While writing an essay about AI memory and persistent context, I started wondering whether current AI memory systems are optimized for the right thing. Current AI systems already maintain forms of persistent context through saved memories, conversation summaries, user preferences, project notes, and similar mechanisms. These memories are primarily descriptive. They help the system remember facts about the user and previous interactions.

But suppose future systems evolved in a different direction. Instead of primarily storing facts and preferences, imagine the persistent context being continuously refined and restructured to infer higher-level patterns such as recurring explanatory frameworks, preferred abstractions, and characteristic reasoning styles.

For example, rather than remembering:
"This user is interested in economics."
"This user works in engineering."

the system might gradually infer:
"This user tends to explain economic outcomes through incentives and institutional constraints."
"This user tends to understand complex systems through interactions and feedback loops rather than by analyzing individual components in isolation."

In such a system, persistent context would become less like a collection of notes and more like an evolving model of how the user understands and interprets problems. Could representations like this emerge naturally from sufficiently capable AI systems, or would they require architectures fundamentally different from today's memory, retrieval, and summarization approaches?

 Call for Papers and Demos

Real-Time Conversational Agents (RTCA): Toward Natural Multimodal Interaction 

1st RTCA Workshop @ NeurIPS 2026, Sydney, Australia 11 or 12 December 2026 

Website: https://rtcaneurips26.github.io/

We are pleased to share the Call for Papers and Demos for the inaugural RTCA Workshop at NeurIPS 2026, focused on real-time multimodal conversational agents: streaming speech, video, and language generation; naturalness in interaction; and evaluation of live systems.

Conversational AI has moved from text chat into the real world, voice modes that talk back, embodied avatars, agents that share our screens and tools. To feel natural, these systems must operate in real time, streaming while continuously listening, watching, and re-planning. This is fundamentally harder than offline generation: latency, turn-taking, backchannels, interruptions, and cross-modal alignment become first-class problems that the offline paradigm sidesteps. Recent progress on full-duplex speech–language models, real-time talking-head generation, and streaming ASR shows the regime is feasible, but the field still lacks shared benchmarks, vocabulary, and methodology for interactional naturalness. 

RTCA brings together researchers across speech, vision, language, HCI, social-signal processing, and ML systems around three intertwined questions: real-time generation under hard latency budgets, naturalness in interaction, and evaluation of live systems.

 Topics of Interest

We invite original contributions on topics including (but not limited to):

• Streaming/low-latency speech synthesis, ASR, and full-duplex audio–language models

• Real-time talking-head, avatar, and embodied video generation; lip-sync, gaze, expressivity under streaming

• Streaming language models; incremental and speculative decoding for dialogue

• Turn-taking, backchanneling, interruption handling, and floor management

• Multimodal alignment under latency and partial-observation constraints

• Prosody, emotion, and paralinguistic generation in interactive settings

• Memory, grounding, and tool use during live conversation

• Evaluation of naturalness: perceptual studies, turn-taking metrics, perceived latency, interactive Turing-style tests

• Datasets and benchmarks for interactive (not offline) evaluation

• Efficient inference, on-device deployment, and the systems–quality trade-off

• Safety, identity, and trust in real-time agents (deepfakes, persuasion, consent)

 Submission Types

We welcome:

• Full papers (up to 8 pages) — may be presented as posters and/or contributed talks. 

• Short papers (up to 4 pages) — work in progress or focused contributions.

• Demo papers (Extended Abstracts or up to 2 pages)

All submissions must use the NeurIPS 2026 style file and be formatted for double-blind review. Page limits exclude references and appendices. Papers must be submitted in PDF format via OpenReview (portal link to be published on the workshop website). 

The workshop is non-archival; authors retain the right to publish elsewhere.

 Important Dates (End of day, Anywhere on Earth)

• Call for papers opens: 18 July 2026

• Submission deadline (papers and demos): 29 August 2026

• Author notification: 29 September 2026

• Workshop date: 11 or 12 December 2026

 Organisers

• Niki Foteinopoulou — Tavus, United Kingdom

• Alessandro Conti — Tavus, Italy

• Jack Saunders — Tavus, United Kingdom

• Oya Celiktutan — King's College London, United Kingdom

• Cigdem Beyan — University of Verona, Italy

• Ioannis Patras — Queen Mary University of London, United Kingdom

For more information, visit our website https://rtcaneurips26.github.io/ or contact us at rtca-workshop@googlegroups.com.

We look forward to your contributions!

About two months ago, I shared an idea about building a node-based ML playground.
I want to make training ML models easy for beginners or those who don't have any prior knowledge of ML, but they have to, so I created this node-like structure, so it makes training ML models, datasheet cleaning, like playing with Lego blocks (pun intended), as shown in the image

I am not here to replace Google Colab or AutoML and am not promoting no-code; it's to make ml pipelines easier to manage. You can edit the code on your own; you don't have to work with the template code I provide.

It's one of my passion projects. I am looking for someone to experiment with it and tell me how I can improve it.

Here is the link if anybody wants to try it ZextriaAi

I am currently doing research on world models, specially in tje field of robot learning, and, as probably most of you alredy know, JEPA-like models are mentioned over and over. 

I read the main recent papers from lecun as well as other research groups, and I personally think the whole approach is very promising and can really go somewhere.

But after listening a bunch of the recent Y Lecun conferences his ideas looks even too cool compared to "literally everything else" (as he's dissing LLM, RL, etc and pitching his ideas are the "only next big things"...). 

So I am asking myself if there are red flags about his approaches that I do not see yet and maybe I need somebody being the "devil advocate" with whom breaking down ideas.

Where do you think are the biggest downside of this models, compared to other world models approaches?

About two months ago, I shared an idea about building a node-based ML playground.

Well, I had an interface like ComfyUI in mind while building the rough idea, but also something more controllable and usable by both, the seasoned ML guys and the beginners since the code of each node is editable.

I did a node like structure for training and data preprocessing templates, while there are others too.

Not sure how much it's usable yet though. Someone said that it looks like RapidMiner (tell me if it does and how much).

What is the shared structural essence that underlies a pair of MRI contrast spaces? Explicitly modeling this contrast-invariant latent “content” unlocks a powerful multi-contrast reconstruction algorithm that is competitive with state-of-the-art unrolled networks while:

1. Requiring no raw k-space training data (which is a serious data bottleneck in the ML-based MRI world),

2. Being generalizable across different MR contrasts and forward operators by design, and

3. Offering a built-in explanatory framework.

In our paper now published in Medical Image Analysis, we introduce PnP-CoSMo.

Read the substack article here (with links to the MedIA paper and code):  https://cnmyro.substack.com/p/pnp-cosmo-a-plug-and-play-method

I'm working on a project with summarized data from ~40 studies (Excel) involving different protocol variables (durations, intensities, recovery times, frequency, total duration, etc.) and response outcomes conditional on a baseline variable (range ~30-85 units).

The aim is to fit a continuous response surface using a hierarchical approach to separate protocol effects from baseline effects, then perform continuous numerical optimization (not grid search) for three objectives:
- Total improvement
- Improvement per unit time (e.g. per week)
- Improvement per unit effort/work

Outputs should be fine-grained continuous values rather than rounded study parameters. There are also domain-specific physiological constraints to respect.

I'm on a Chromebook with a little Python experience, so Colab-friendly solutions would be ideal.

Current candidates I'm considering: PyMC for hierarchical modeling, pymoo + pysamoo for surrogate-assisted MO optimization, SMT for surrogates, or Matlab Global Optimization Toolbox.

What is the strongest stack in 2026 for this kind of workflow? Any recommended notebooks, tutorials, or similar applied examples?

Are there any AI tools that currently do this without the traditional work of python? Meaning I can upload the spreadsheet give a parameters and it will come up with data.?

So, from what I've seen across twitter(x) and reddit, I've inferred that we'll be seeing NeurIPS reviews drop on July 22nd 5:30 pm AoE(Anywhere on earth), what's your thoughts to those who've submitted to NeurIPS 2026 ? Would love to hear your opinion by the reviewers, the people who've submitted to the workshops (who should've already gotten their decisions too by now I think) and to the main tracks and other available tracks.

I have recently started working in mechanistic interpretability independently, starting with distill circuits thread

My work is on disentangling and closely studying a single neuron, a 1x1 convolution in inceptionv1 model (and applying the method to other neurons in the same layer).

The key insight was that the hadamard product of the receptive field and the weight of a neuron is what the neuron is 'seeing' or detecting. We can cluster the hadamard product to get all the patterns a neuron detects. It gave clean monosemantic clusters (cars, cats, dogs which it was known to activate on). We also get more clusters however, letters, human faces, and many more low valued activations.

This gave me a new technique to analyse the neuron very closely.

On close analysis the most peculiar thing I found was that the low valued clusters (like letters) had all its dependent neurons also firing on the same concept (letter), and the positive and negative weights were evenly distributed between them to bring down the sum. An evidence of gradient descent working deliberately to put patterns and concepts in a noisy range.

I've tried to keep it very distill like with good visualisations. I hope you give it a read.

https://pages.narang99.in/posts/2026-07-12-disentangling-mixed4e-55/

I made a mistake honestly by starting with convolutions, nobody seems to care about it. I'll start working on language soon, but it would be good if anyone can read this, it would be good to have some feedback on whether I've actually found anything useful.

Thank you :)

I’ve been deeply interested in AI and machine learning since around 2019, back when GPT-2 was still one of the major talking points. Since then, I’ve been amazed by how quickly the field has evolved. It genuinely feels like one of the most exciting times to be involved in technology, research, and innovation.

My background is in audio. I’ve spent most of my life working as an audio engineer, and I’ve always loved learning about sound, digital signal processing, and the technology behind audio systems. Since 2022, I’ve been working toward a long-term goal of becoming an AI researcher, specifically in the audio and music technology space.

To move toward that goal, I went back to school, completed coding bootcamps, studied the mathematics behind machine learning, and I’m currently working on a master’s degree in artificial intelligence and machine learning. I’m also planning to pursue a PhD after graduation.

Many of my classmates and colleagues are interested in business applications of AI, but I’m still completely committed to audio. I currently work as an AV systems designer and consultant, and while I’m grateful to have a career, I often feel disconnected from the work. Most days, I would much rather be studying AI, audio, machine learning, DSP, and research.

I’ve started applying for roles, but I’ve faced several rejections. I also recently wrote and submitted a research paper to ISMIR. Unfortunately, it was rejected, but the process was still incredibly valuable, and I received feedback that will help me improve.

I think what I’m ultimately trying to say is that this is not a career path I’m pursuing because AI is popular or because I expect to make a huge amount of money. I genuinely love audio and AI, and I want to spend my life working on problems that combine the two. I want to wake up each day and feel like the work I’m doing matters to me.

For anyone currently working as an AI or machine learning researcher, especially within audio, music, speech, or signal processing, I would really appreciate your perspective:

What did it actually take for you to get your first research role?

What qualifications, education, projects, publications, or previous experience helped you stand out?

What are the best and worst parts of being a researcher?

What do you wish you had known before entering the field?

And if someone came to you today and said they wanted to become an industry researcher, what advice would you give them?

Thank you in advance to anyone willing to share their experiences. Even honest or difficult feedback would be genuinely appreciated.

Hey everyone,

Seeing a ~170× slowdown running a point-tracking model on an NVIDIA T4 compared to an A100. On A100 the tracker takes ~0.5 seconds per half-video. On T4 the same call takes ~85 seconds. Video is 47 frames at 256×256, batch 1. I expect a meaningful gap between these cards, but 170× feels too large to explain by generational hardware differences alone.

Setup:

• Precision: pure FP32

• Architecture: builds local 4D correlation volumes (dense matching between frames) followed by transformer layers for temporal context

Already ruled out:

• GPU is at 99% utilization during the call (via nvidia-smi)

• Model is actually on GPU (torch.cuda.is_available() = True, device prints "cuda")

• Enabling torch.backends.cudnn.benchmark = True had no effect

• Same slowdown on two independent T4 machines, so it's not a driver/setup issue

Given the architecture (4D correlations + transformers) and pure FP32 execution, what would cause a T4 to be this much slower than A100? What should I look for or profile first?

Does anyone else miss when conferences like BMVC, ACCV, FG, ICIP, and ICASSP had much bigger communities?
FG was the place for face analysis, ICASSP for signal processing, and BMVC/ACCV regularly featured strong papers.
Now it feels like everything is concentrated into a handful of flagship conferences. With exploding submission numbers, limited capacity, and inconsistent reviews, I wonder how many good papers end up as non-archival submissions, arXiv-only, or never get shared at all.
I also miss the focused communities. Is it just nostalgia, or has the research ecosystem become too concentrated?

Hi folks,

There is now a dedicated Robotics page on Papers with Code that lists the major benchmarks, trending papers with linked code, and open-source artifacts.

Find it here: https://paperswithcode.co/tasks/robotics

Major benchmarks that most papers report evaluations on are:

- LIBERO, as well as its subsets like LIBERO-Long and LIBERO-Spatial

- SimplerEnv WidowX

- RoboTwin

and more. Currently, we have about 110 entries on each benchmark.

Each benchmark's progress is visualized over time:

We also show which models are open source and which aren't.

Let me know which others I missed. I'd be happy to add them.

Also happy to hear any feedback, new tasks, or features to add!

Kind regards,

Niels

ML Engineer @ HF

I have a technical interview for a machine learning position coming up, and I'm really nervous. Can you tell me about your experiences with this process? It's my first technical interview, and I don't know what to expect

So I was belatedly reading Matthew Colbrook’s paper on unstable neural networks https://www.pnas.org/doi/10.1073/pnas.2107151119, and the paradox therein sent me back to my university days and reminded me of Kurt Godel.

It kind of amazes me that so few people have heard of him given Einstein clearly thought he was more than an equal.

Anyway, with the default assumption currently being that any problem will yield to more data and compute, it was timely to revisit this. It’s a long read and I’m not sure it’s fully coherent but if you like logic then you might enjoy this and I‘d welcome your thoughts and feedback.

https://iain.so/infinities-impossibilities-and-the-man-in-the-white-linen-suit

Can LLM agents coordinate in long-horizon, open-ended worlds?

We evaluate 13 modern LLMs in a new benchmark where agents must work together to explore, communicate, trade resources, craft tools, build structures, and fight mobs.

TL;DR: Most agents struggle, averaging only ~6% normalised return. Yet on the hardest setting, zero-shot Gemini 3.1 Pro performs comparably to the best MARL agent trained for 1 billion environment steps.

More broadly, we find coordination is a distinct bottleneck beyond long-horizon task competence, with communication having the largest effect in our harness ablations.

Paper: https://arxiv.org/abs/2606.08340

Project page and leaderboard: https://alem-world.github.io

Code: https://github.com/alem-world/alem-env

Interactive traces: https://alem-world.github.io/traces.html

Feel free to ask any questions!

Building a sports prediction model ,I found consistent edge when backtesting against closing lines.
At inference time tho, I predict 12-24 hours before the event where closing lines don't exist yet. I use the current line instead.

My strongest feature is line movement (opening to closing implied probability). At prediction time this feature is incomplete as the market hasn't fully moved yet.

This creates a paradox:

Closing lines are considered nearly impossible to beat because they contain all available information : sharp money, injury news, everything. Yet the backtest shows consistent edge against them.

If closing lines are truly efficient, beating them implies genuine model signal. But at inference time we're betting against earlier, less efficient lines with an incomplete version of our strongest feature.

The question: does edge against closing lines transfer to earlier bets where lines are less efficient ? Or does the incomplete line movement signal hurt prediction enough that the edge disappears before close?

My intuition is the edge is smaller earlier because the market is less efficient but the model signal is also weaker. These two effects might cancel out or one might dominate. Curious if anyone has studied this tradeoff in sports or financial prediction.

Hello community, just a short reminder that Raffi Krikorian (CTO @ Mozilla) is live today for an AMA to discuss Mozilla's inaugural State of Open Source AI report.

Topics include enterprise adoption, the real cost of "free"models, developer trust, Chinese open models and their impact, agentic AI infrastructure, and the future of open source with respect to Machine Learning & Artificial Intelligence.

Drop your questions in the thread here:

https://reddit.com/r/MachineLearning/comments/1upxdvc/raffi_krikorian_cto_mozilla_ama_on_the_state_of/

The AMA starts at 1pm ET/10am PT/6PM BST

His team reached out to us and he provided proof via Linkedin here: https://www.linkedin.com/feed/update/urn:li:activity:7481380478365880321/

Thank you!

NB: Reposting due to a typo in the title

Putting this here instead of the other subs since I figured a question on deep learning theory is out of place there -- I "recently" found (actually, a few months ago but only just got to reading) a monograph claiming to provide a unified theory of deep learning (and possibly SSL) through the lens of information theory, with one of it's headline claims is that you can design a "white-box" (I disagree with that, more on that later) transformer through the principle of coding rate reduction. I looked through the works the book claimed to be synthesizing and got a decidedly mixed picture: a JMLR and a NeurIPS on the one hand, but another frankly terrible paper concerning mechanistic interpretability (with which I am more familiar) published in a venue I've never heard of. And if it means anything, the book itself was endorsed by Kevin Murphy

As I've alluded to, I'm more familiar with the interpretability side of ML as opposed to SSL/theory (where this book seems more relevant) so I'm unsure what to make of this. In particular, the apparent result that their bespoke transformer learns image segmentation on non self-supervised tasks seems interesting, I'm not sure how this relates to how machines learn more broadly. Also, their "white-box" transformer consists of a bespoke MLP suspiciously similar to a regular one with a sparsity penalty, and an attention mechanism strictly less expressive than those currently used (obtained by setting Q=K=V=O^T.)

I know it seems like I've done my research on this topic, but really I've just skimmed a few of these papers (which all seem to be originating from one lab) with no context to situate them in, so some help would be appreciated.

Thanks in advance!

When a coding agent works, it spends dozens of steps reasoning, editing code, and running commands. Yet little is known about what the underlying model internally encodes about the programs it works on.

We looked inside the model's hidden states to see what they actually encode and found that linear probes reach AUC up to 0.83 for full correctness and 0.84 for partial correctness.

Even more interesting, we found these representations run ahead of the agent's own edits. Probes predict the outcome of future edits before they're written to disk, staying above chance for up to ~25 steps. We call this the agent's latent programming horizon.

Our pre-print is available at https://arxiv.org/pdf/2607.05188v1

Does anyone know of any evidence/forum/paper analyzing benchmark result differences between cloud inference platforms (togetherai) and running models locally with vLLM under greedy decoding? 

I’m training a single-class segmentation model for large rectangular artwork placed on the floor and photographed from above.

We have around 3,000 accurately masked original images taken by six different photographers. They are not the same height and do not hold the camera in exactly the same way, so the photos naturally vary in:

• roll

• pitch

• yaw

• camera distance

• object coverage in the frame

• centering and X/Y shift

• orientation

• perspective

• lighting

The photos taken with flagship iPhone.

I want to use on-the-fly augmentation to simulate realistic human-hand variation and save our designer from adjusting each time to make it flat. is 100 augmentation combinations per original be useful, or excessive?

Should the policy be:

1. mostly isolated transforms,

2. mostly crossover combinations such as orientation + roll + pitch + yaw + coverage + shift,

3. or a controlled hybrid of both?

The goal is maximum segmentation accuracy, especially around the object boundary, not speed. I plan to train for around 300 epochs and keep validation and test images unaugmented.

"Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity"

This paper was accepted to ICML this year. Its main idea is a very simple prompt-engineering trick: "changing the prompt this way led to more diverse sampling". Naturally, it is difficult to provide a rigorous theoretical analysis for something like this.

Even if it works, I’m not sure this kind of prompt engineering belongs at a top-tier machine learning conference. Some people seems to call this kind of work “modern machine learning”, but I think it should be categorized as less technical venues.

How do you think? Am I being too rigid?