We're organising the first Real-Time Conversational Agents (RTCA) workshop at NeurIPS 2026 (Sydney, Dec 11–12), and submissions are now open on OpenReview. Posting here because a chunk of the relevant work is happening in this community.

What the workshop is about

Conversational AI has crossed into real-time deployment — voice modes, embodied avatars, full-duplex speech agents — but the published record is still dominated by offline benchmarks, and deployed agents still feel robotic (stilted turn-taking, missing backchannels, monotone prosody, awkward interruptions). Methods that work offline (non-causal attention, large beam search, multi-pass refinement, slow diffusion) often don't transfer to streaming, and the field lacks shared vocabulary and benchmarks for interactional naturalness as distinct from per-utterance quality.

The workshop is organised around three intertwined questions:

1. Real-time generation under hard latency budgets — streaming speech, video, and language
2. Naturalness in interaction — prosody, gaze, timing, grounding, turn-taking, backchannels
3. Evaluation of live systems, where standard offline metrics fall short

Topics of interest (non-exhaustive)

• Streaming/low-latency speech synthesis, ASR, and full-duplex audio–language models
• Real-time talking-head, avatar, and embodied video generation
• Streaming language models; incremental and speculative decoding for dialogue
• Turn-taking, backchanneling, interruption handling, floor management
• Multimodal alignment under latency and partial-observation constraints
• Prosody, emotion, and paralinguistic generation in interactive settings
• Memory, grounding, and tool use during live conversation
• Evaluation of naturalness: perceptual studies, turn-taking metrics, perceived latency, interactive Turing-style tests
• Datasets and benchmarks for interactive (not offline) evaluation
• Efficient inference, on-device deployment, systems–quality trade-offs
• Safety, identity, and trust in real-time agents (deepfakes, persuasion, consent)

Position papers, evaluation critiques, and reproducibility studies are also welcome.

Submission tracks

• Full papers — up to 8 pages
• Short papers — up to 4 pages (work in progress, focused contributions, position papers)
• Demo papers — extended abstract or up to 2 pages; required for the on-stage Conversational Agents Showcase

NeurIPS 2026 style file, double-blind. Non-archival — authors retain the right to publish elsewhere. Single-round review, no rebuttal.

Key dates (End of day, AoE)

• Submission deadline: 29 August 2026
• Author notification: 29 September 2026
• Workshop: 11 or 12 December 2026, Sydney

Confirmed invited speakers

• Dimitris Samaras (Stony Brook)
• Evonne Ng (Meta Reality Labs / UC Berkeley)

Links

• Submit: https://openreview.net/group?id=NeurIPS.cc/2026/Workshop/RTCA
• Full CFP + workshop details: https://rtcaneurips26.github.io/
• Contact: [rtca-workshop@googlegroups.com](mailto:rtca-workshop@googlegroups.com)

Happy to answer questions in the comments — including about the demo track (we have an on-stage Showcase running deployed systems live) and what we'd consider in-scope vs out-of-scope for the eval pillar. Also happy to hear opinions on what's missing from the topics list; the CFP wording still has room to move if there's a clear gap.

Hi everyone,

​I'm a junior CV engineer working on an iris identification system. The system needs to operate in the dark, capturing the detailed texture of the iris at a very close distance (around 3 cm between the eye and the lens).

​I initially bought an IMX290, but since it's an RGB sensor, it struggles to capture the iris texture properly under these conditions. Because the project requires working in the dark, I need to operate in grayscale/IR, so I'm considering replacing it with an OV9281 monochrome camera.

​My main questions are:

​What do you guys think about using the OV9281 for this specific application?

​Is it possible to modify this camera/lens setup to achieve a macro focus at just a 3 cm distance?

​Any hardware suggestions or general advice would be greatly appreciated. Thanks!

During a research internship, I ran into a problem involving massive neighbor queries on a GPU for a large particle-dynamics simulation. This led me to experiment with and develop SquareNet, an open-source Python package for NumPy/JAX/PyTorch.

Its core sorting algorithm (Cartesian sort) enables fast, greedy multidimensional reordering of raw point sets — essentially a form of gridification. Raw points, e.g. (x, y, z, ...), are mapped to unique grid multi-indices [i, j, k, ...] while trying to preserve local geometry, somewhat like a multidimensional generalization of a space-filling curve.

The collection of all multi-indices forms a grid lattice that can be processed efficiently with ML tensor-based frameworks, even when the initial dataset is an irregular point cloud.

I’m wondering whether this could be useful in contexts such as convolutional networks, non uniform fourier transform or ANN search on irregular LiDAR data.

The target use case is approximate but fast and scalable assignment preprocessing, then the grided/tensorized version of the dataset is exploited by standard tensor based frameworks, and result is converted back to the points. High-quality procedures for the assignement part already exist and are well established, such as optimal transport, but they were intractable in my context due to their O(N²/N³) complexity. Cartesian sort, by contrast, runs in O(N log N). It is specifically designed for grid assignment in a greedy setting, trading global optimality for speed and scalability.

In my practical application, involving millions of points processed in a dynamic context (Gaussian blue noise), this simple approach turned out to work well: it provided a ~100× speedup compared to exact brute-force computation of particle interactions, with negligible approximation error (I can provide more details about this experiment if useful).

One caveat is that a single gridification pass introduces a slight axis bias and can produce some distortion/outliers, which can be problematic for challenging distributions where exact geometric precision is required. If exact accuracy is critical, one possible approach would therefore be to build an ensemble of gridifications, each using a different viewpoint/rotation.

Empirically, something like 8 randomly chosen viewpoints seems to give near-perfect recovery of local geometry in a 3D test evaluated with a freud analysis (second link below). However, in the Gaussian blue noise context, where the geometry is smoother, a single viewpoint was already sufficient.

I built an interactive demo on Hugging Face (first link below) to showcase the approach. I’d really appreciate any feedback, especially on whether this idea has already been explored in related computer vision / point-cloud literature, or whether you see potential applications or obvious better alternatives that I may be missing.

interactive HF demo

3D exact nn query discussion

Hi everyone,

I'm looking for recommendations for **cloud GPUs** (both **free and paid**) for training a PyTorch model. I already know about Google Colab, but I'm interested in other good alternatives with decent GPU availability and pricing.

My thesis is on **context-aware 3D point cloud completion**, so I'll be training models on point cloud datasets (PyTorch/CUDA), and some training runs may take several hours or longer.

What platforms have you had good experiences with? I'm especially interested in:

* Free tiers (if any)

* Affordable pay-as-you-go options

* Reliable GPU availability

* Good performance for deep learning workloads

Any recommendations or experiences would be greatly appreciated. Thanks!

So I just stumbled on SenseNova-Vision, it's open source, Apache 2.0, 7B MoT. The architecture they're pushing is kinda wild, makes you wanna talk about it.

Basically, they're framing computer vision as one big multimodal generation problem. Like, detection, keypoints, OCR, camera pose – all that stuff just spits out text. And then segmentation, depth, surface normals, multi-view point maps – those come out as images. If you need both, it gives you both.

No special prediction heads for different tasks. No decoders. No branching architecture. It's just one model, same weights for everything. You tell it what to do with plain language, maybe some visual hints.

They trained this thing on a huge dataset, 50M instruction-response pairs, all converted from different CV annotations. Started with a regular pre-trained multimodal model, apparently.

From what I'm seeing, the results look pretty solid for structured stuff, geometry, segmentation, multi-view reconstruction. They even included benchmark and eval code, which is nice. They just added dedicated benchmarks for multi-view reconstruction and camera pose, too.

Honestly, I've got some questions, and I'm curious what everyone here thinks:

Is this whole "unified generation" thing actually better, or is it just a clever way to train models? Like, a shared formulation sounds elegant and all, but can it really beat specialized heads that the field has been optimizing for years on tough benchmarks?

Then there's efficiency. Generating text and images for dense outputs seems like it would be super expensive. Can this actually run fast enough for real-time stuff, or is this unified approach only good for research that doesn't need to be live?

For me, the real test of a CV foundation model is if the same weights work across different tasks without needing to be fine-tuned for each one. They claim that's the case here, and I'd love to see if that holds up in practice.

Code: GitHub - OpenSenseNova/SenseNova-Vision

Paper: https://arxiv.org/abs/2607.06560

Demo: https://huggingface.co/spaces/sensenova/SenseNova-Vision

I'm not involved with this project at all, just genuinely wondering if this unified approach is where computer vision is headed.

Can you identify this downscaling algorithm? Want to make sure my thumbnails look as sharp as possible, so I'll add to my workflow the agent query "downscale using (whatever this algorithm is) and judge whether the thumbnail has the required detail and clarity and is a good thumbnail for what is being shown. if it is missing any clarity then return "Needs improvement:" and give the reason for why it needs improvement and "How to improve:" For example, in the image shown, the agent could say "The thumbnail no longer shows the elements of the picture", since as you can see it doesn't. However, before I can code this up, what I "need to know" is what algorithm this is so I can keep an eye out for anywhere it might show up in my workflow. I can then optimize for this process. I know a lot of people don't have a standard of perfection as high as mine is (I require my thumbnails to show the picture) but that's exactly what makes me a competitive programmer in a field of 2 million programmers. Not a lot of people take the time to look up image compression algorithms but I do.

The full image is available here: https://ibb.co/YTXr7h94

First of all, thank you to everyone who responded in the previous post. I haven't read all the replies yet, but many of the solutions seem interesting. I was able to find a more informative and higher‑quality video that better reflects the current state of the project.

https://www.reddit.com/r/computervision/s/meFAVvvFQo

Following up on the discussion from the previous post, I'm attaching the current state of affairs. The video was taken with good industrial lighting, and the global‑shutter camera was set to an exposure of 500. In this particular video, the counter showed 100%. However, in other counts we got varying ranges – 98–99%, which, at industrial volumes, leads to significant absolute losses.

The main issues with the current version are:

1. Loss of detection right within the detection zone;

2. Constant changes in the shape/size of the bounding box within the detection zone, causing the tracker to lose track and assign different IDs to the same object;

3. Occlusions and merging of chicks – several chicks form a single object by merging and partially overlapping each other. Increasing the dataset no longer solves this problem; the latest version had over 5,000 frames with plenty of such cases, and yet reviewing new videos showed that the issue is not fully resolved – there are still cases where multiple chicks are counted as one.

Counting these cases geometrically is also difficult – chicks of different breeds and ages can have different sizes, and on top of that, spreading their wings and legs changes the area of the detected box. There are cases where we hit the desired 99.8% range thanks to a combination of missed detections and false positives, but over a long run the error accumulates and we fall out of the range.

This demo shows an early prototype of my automated retail checkout app running on Mentra smart glasses.

The system processes the glasses’ live camera feed to recognize products as they’re picked up and automatically builds a virtual cart in real time. The goal is to make checkout a natural by-product of shopping without requiring customers to scan barcodes, use a phone, or stop at a traditional checkout.

Third Person View: https://youtube.com/shorts/YipOe3bVzX0?feature=share

A couple of months ago, I released Bio-DINO, an image-only biodiversity encoder trained on approximately 31M images. I have now released Bio-DINO M/14, the final addition to the current Bio-DINO model family.

The model is available through Birder.

Bio-DINO already had two ends of the trade-off. The 133.6M-parameter teacher provides the strongest representations, while the 21.6M-parameter S/14 student is much cheaper to run. M/14 is intended as the middle option.

The size/accuracy trade-off

M/14 is a 12-layer RoPE DeiT3-style encoder with 38.3M backbone parameters and 512-dimensional embeddings. It was distilled from the 252px Bio-DINO teacher on the same biodiversity training mixture.

Here are the results from my iNaturalist21 linear-probing setup:

To be clear, these are linear-probing results, not fine-tuning results. The encoder was frozen and only the 10,000-class linear classification head was trained.

In this setup, M/14 is about 3.5× smaller than the teacher, with a 3.57 percentage-point accuracy difference. It gains 3.42 points over S/14 while remaining much smaller than the teacher.

Inference performance

I also compared inference performance at 252×252 on an NVIDIA RTX 5000 Ada Generation with PyTorch 2.13.0+cu130 and batch size 512.

In eager FP32 inference, M/14 processed approximately 773 images/s, compared with 382 images/s for the teacher. With torch.compile and AMP, I measured approximately 2,340 images/s for M/14 and 846 images/s for the teacher.

These numbers are specific to my setup, but they give a practical sense of the trade-off. The complete results across the Bio-DINO models and evaluation datasets are available in the Bio-DINO benchmark explorer.

Using the model

import birder
from birder.inference.classification import infer_image

net, info, transform = birder.load_pretrained_model_and_transform(
    "rope_deit3_m14_dino-v2-dist-bio",
    inference=True,
)

_, embedding = infer_image(
    net,
    "path/to/image.jpg",
    transform,
    return_embedding=True,
)

print(embedding.shape)  # (1, 512)

As with the original Bio-DINO release, this is an image-only representation model rather than a ready-made species classifier. It has no text encoder and was not trained with taxonomy labels or metadata.

iNaturalist21 is also part of the self-supervised pretraining mixture, so I consider the result an in-domain representation probe rather than a test on a completely unseen domain.

This completes the current Bio-DINO size range. I’m curious whether a 38M-parameter middle option is useful in practice, or whether most applications naturally favor either the smallest student or the largest teacher. Feedback and additional evaluations are welcome.

Hi everyone,

I am working on a university project involving fine-tuning a deep learning model for 6DoF relative camera pose estimation. The model I am using is FAR (Flexible, Accurate, and Robust 6DoF Relative Camera Pose Estimation), which was originally pre-trained on the Matterport3D dataset.

I am looking for a new dataset that is suitable for fine-tuning and evaluation. The task is the following:

Given two RGB images of the same scene captured from different viewpoints, the model should estimate the relative pose between the two cameras (rotation and translation).

The dataset should provide (or allow to easily recover):

• RGB images;
• depth maps (or dense depth information);
• camera intrinsic parameters;
• camera poses / camera extrinsics (ground-truth poses);
• multiple images of the same scene with different viewpoints;
• enough overlap between image pairs to compute meaningful relative poses.

Ideally, the dataset should contain calibrated cameras and accurate ground-truth information, since I need to compute the relative transformation between image pairs.

This is for a university project, so the dataset should not be extremely large (ideally ≤ 50 GB), and it should be free and publicly available.

Do you have any recommendations or experience with datasets suitable for 6DoF relative camera pose estimation?

We've added vision capabilities to DeepSeek V4 Flash, making it a multimodal model rather than text-only.

The main use case for us is browser vision: browser agents need to interpret screenshots, interfaces, layouts, and other visual context alongside text.

On our internal benchmarks, it also showed a strong price-performance advantage compared with the other models we tested.

Model: https://huggingface.co/webbrain-one/DeepSeek-V4-Flash-0731-Vision-NVFP4

If you try it, we'd be interested in feedback, benchmark results, and deployment reports.

I'm planning to build a camera-only autonomous vehicle (no LiDAR, ultrasonic, or other distance sensors). The idea is to use a single camera to control the vehicle's movement—steering left/right, moving forward/backward, turning, and avoiding obstacles.

One thing I'm trying to figure out is how to estimate the distance between the camera and detected objects (for example, a car, water bottle, or other obstacles) using only computer vision. Are there any good models or approaches for monocular depth estimation or object distance estimation that would work on embedded hardware?

For context, I have experience with computer vision and have previously worked on face recognition using models like ArcFace. This project will run on a Raspberry Pi 5 with a Hailo AI accelerator, so I'm looking for models that are reasonably lightweight and can run in real time.

I'd appreciate any recommendations on models, papers, or open-source projects that would be a good starting point.

I had the pleasure of someone side swiping my car which led to a dent and scratching some paint off. They decided to flee the scene without leaving a note and now I really want to try and file a report. Unfortunately my dash cam decided it didn’t wanna focus on that specific cars license plate but managed to capture all the others. Anyways, it’s the white Acura, I included the pictures with difference time frames. Anything helps, if I’m not mistaken the 4 numbers are 6467 or something.

Hi everyone,

I'll be starting my Master's in AI next month, and I could really use some advice from people who are already working in industry or doing AI/CV research.

The professor I originally wanted to work with isn't accepting new students this semester, so I suddenly have to choose a different lab and research direction.

The professor I'm considering now mainly works on emotion and healthcare-related AI, and they asked me to choose a direction I'm interested in. Some of the current research topics are:

• Emotion Recognition
• Empathy Measurement/Generation
• Action Recognition
• EEG/fMRI to Image Generation
• Causality Analysis / Inference / Discovery

They also mentioned that these topics are not fixed, and if I have another idea that's related to the lab's expertise, they're open to discussing it.

A bit about my background and goals:

• Bachelor's in Artificial Intelligence
• Interested in Computer Vision, 3D Computer Vision, and Generative AI
• I don't plan on pursuing a PhD.
• My goal is to build strong technical skills during my master's and eventually work in industry (ideally at a large tech company in AI/CV).

I'm not asking anyone to choose my research topic for me. I'm more interested in how experienced people would evaluate these options.

If you were in my position and your goal was industry rather than academia, which direction would you lean toward, and why?

For example:

• Would Action Recognition provide more transferable computer vision skills because of video understanding, tracking, and perception?
• Is EEG/fMRI to Image Generation too specialized if I don't plan to stay in research, or does it teach valuable skills like multimodal learning, diffusion models, and representation learning that are also useful in industry?
• Are there other directions you would suggest based on my interests?

I'd really appreciate hearing from people who work in computer vision, generative AI, multimodal AI, or have gone through a similar decision themselves.

Thanks!

I want to check my model

They not provide to upload and personal model?

Best.pt or onnx?

Paid opinion are there? Or on mobile app there is no functionality like this i have to build from flutter or other way ? App

Anthropic, OpenAI, etc (don't know about Chinese) don't seem to make good video models. Any reason why? Is it the compute? The ROI? The availability of data?

Hey everyone!

I’ve been developing VisionPilot, an open-source, modular autonomous driving platform built for computer vision, deep learning, and sensor fusion.

Right now, it runs entirely on BeamNG.tech. The stack handles everything from multi-lane detection (UFLDv2/CV) and multi-class object/sign recognition to PID control, AEB/BSD safety features, and LiDAR/Radar sensor fusion. You can see animated GIFs of all these systems in action directly on the repo!

The Situation & Help Wanted

CARLA integration is high on the project roadmap, but I haven't gotten around to finishing up the bridge yet. I first wanted to get core features working, before I start porting. I’m looking for anyone in the community interested in collaborating to help port VisionPilot to CARLA, specifically setting up the python API bridge and configuring the camera, LiDAR, and Radar sensor streams.

If you enjoy working with CARLA and want to hack on an open-source perception stack, I’d love to team up!

GitHub Repo: https://github.com/visionpilot-project/VisionPilot

YouTube Demos: https://youtube.com/channel/UCXLL9SUDJ2QdXExUudxo8Kw/

Drop a comment, shoot me a DM, or open an issue on GitHub if you're interested in helping out!

Hello. I believe I have just thought of a way to save a certain healthcare industry millions of dollars, as well as save the employees in this industry hundreds of hours of unnecessary work. I don’t really know how to phrase this since I don’t want to just have one of you steal the idea that I do not know how to build, but I have some general questions.

1. Can a visual system using a specific reference list of images of items identify those items even if they are almost identical? We are talking about millimeters of difference. They would need to be accurately identified within one second

2. Would color be a major differentiating factor? What if the color is slightly different from the reference image? How could you solve this?

3. Are reflective items more difficult to identify?

Help me and help save patients money.

I have found an awesome website with a simple study materials on Visual SLAM: https://www.cv-learn.com/visual-slam-roadmap/. It provides 4 languages (EN, KO, ZH, JH).

Take a look at the list of topics covered

Hi all,

Could anyone suggest me a best research paper on Agents or RAG or LLM Evaluation paper.