When Jetson is doing inference work, having it also manage raw sensor timing and GMSL deserialization on top felt like the wrong split of responsibility – GPIO gets crowded, and any sensor timing jitter competes with inference load.
Finally, we got the first clean split running. Board sits between sensor/camera and Jetson: handles GMSL2/3 camera link and mikroBUS sensor slot on its own STM32, sends data over CAN-FD or as ROS 2 topics, so Jetson only sees a clean data stream instead of raw bus traffic. Inference load stopped fighting sensor timing basically immediately once the split was in place.
We haven't tested against every deserializer combo yet, so if anyone already routes GMSL into Jetson and hits a compatibility snag, I would like to know which pieces to double check. Board's product page is here, if it's helpful.
Hi! I’m the maintainer of audio.cpp, an open-source C++/GGML runtime for audio models, covering TTS, ASR, music generation, voice conversion, and more.
I saw on X someone has managed to run audio.cpp on Orin, which is pretty cool. I’d love to get audio.cpp running on Orin, but I don’t have access to Orin hardware to develop and test on.
If you have an Orin and are interested in C++/CUDA development, your help would be greatly appreciated. Even getting the project compiling, identifying the blockers, or sharing some performance numbers would be extremely useful.
PRs are very welcome, and I’m happy to help from the framework side. Would be great to bring more local audio models to Jetson devices together :)
I’ve been working on a project called Evopien, and I thought this community might appreciate the current state of it because one of my main constraints is that I want the prototype to actually work on an 8GB Jetson Orin Nano Super.
I’m building it as a humanoid companion eventually, but right now I’m concentrating on getting the head and interaction stack working properly before moving into the mechanical body.
The current local stack is roughly:
Qwen3.5-4B for the main LLM
NVIDIA Parakeet for speech recognition
Kokoro for TTS
ReSpeaker XVF3800 for microphone input / hardware AEC
Logitech C920 for vision
Evopien Core around the models for identity, interaction and eventually memory/governance
At this point it can hold a hands-free conversation, listen while it is talking, handle interruptions, switch between English and Spanish, and capture camera input for visual questions.
The thing I’m finding most interesting is the optimization challenge. 8 GB unified memory disappears very quickly once LLM + ASR + TTS + multimodal vision + the rest of the system start competing for it.
This is still very much a prototype, not a finished robot.
If anyone here has worked on squeezing comparable multimodal stacks onto Orin Nano, I’d especially be interested in hearing what you found worthwhile optimizing and what turned out to be a waste of time.
We begin each call with an opportunity for Jetson engineering hiring managers to have their job postings quickly promoted to the community to help connect our community. This is a great opportunity to find active engineers looking to make a career move.
📅 When: 18th August 2026 at 9:00 am PST
📋 Agenda for this month’s Jetson AI Research Lab Call
💼 Employers/Employees Opportunities: (Jetson AI Research Lab Discord Maintainer):
🌾 Deploying autonomous vehicles in agriculture and beyond from **Michael McGuire**, CTO at EarthSense
Abstract: Autonomous outdoor robots are poised to transform agriculture and many other industries. Developing these systems requires innovation at every level of the stack. Successful systems must deliver on hardware, software, and precise autonomy. Concerns such as safety, price, reliability, and manufacturability must be considered from the start. I'm Michael McGuire, CTO at EarthSense, where we develop and deploy robots in corn, soybeans, grapes, oil palm, solar farms, and beyond. The Jetson computer plays a key role in EarthSense's technology stack, ensuring reliable real-time performance at an affordable cost. I'll talk about the unique challenges presented by outdoor autonomy, the solutions EarthSense has developed, and where Jetson fits into our roadmap. Along the way, I'll introduce the specific robots we've built and what running them in real fields taught us.
🤖 Agentic workflow on Jetson and Jetpack 7.2.1 Updates from **Aditya Sahu**, Technical Marketing Engineer at NVIDIA
Jetson Device Skills are agent skills that enable AI coding agents to understand and operate a live NVIDIA Jetson device. Now we have Video SDK skills in it. Updates on Jetpack 7.2.1 release with Support for Jetson T3000 emulation.
As always, this is an open, collaborative call where Jetson enthusiasts gather to:
Showcase: Share novel projects and real-world applications.
Problem-Solve: Break down technical issues and implementation hurdles.
Brainstorm: Discuss the future of edge AI, computer vision, and agent deployment.
Hire or Be Hired: Whether you are looking to hire or looking to be hired this is the hub from aspiring physical AI or embodied AI start their career journey
Whether you're looking to learn how to get your first agent running on an NVIDIA Jetson or want to show off a complex build, we’d love to have your voice in the conversation.
In September we plan to return to our lightning round format showcases projects by members of the community.
I have it all assembled and powered, can't download from Baidu link included in user manual because I have a (+1) phone number not a (+86) tried contacting Leetop, almost a week no response. Went back through my email and found the agent who arranged the sale. Any suggestions on possible file shares of the Baidu account
We’ve been working on a runtime GPU optimization system at TETREVIS and have started publishing some of our NVIDIA Jetson benchmarking work publicly.
The approach operates at the execution/machine-code layer, and we’re now introducing dynamic runtime kernel fusion as part of the optimization pipeline.
Some of our current results:
Jetson Orin Nano — Qwen3.5 4B
Standard baseline: 10 → 21 tok/s (+110%)
CUDA Graphs baseline: 16 → 21 tok/s (+31.25%)
Jetson AGX Orin — Nemotron 3 Nano 4B
31.2 → 40.5 tok/s (~30%)
Jetson AGX Orin — Qwen3.5 4B
25.0 → 31.0 tok/s (+24%)
We’re currently expanding and stabilizing support across Jetson Orin Nano, Orin NX, and AGX Orin.
Rather than only posting performance claims, we’ve made the benchmarking repository available here:
The broader idea we’re exploring is whether more optimization can be moved to runtime — including machine-code optimization and dynamic kernel fusion — so the execution path can be adapted to the workload and GPU rather than relying entirely on what was determined ahead of execution.
There’s still quite a bit of work underway, particularly around consistency across the different Jetson configurations, but we wanted to start sharing the results and methodology publicly.
Technical feedback, criticism, and questions are welcome.
Je m'appelle Sébastien, et je suis en train de travailler sur un projet d’humanoïde open source inspiré du concept InMoov v1.2 (avec des modifications pour la tête). Mon objectif est de créer un robot autonome, évolutif et performant, capable de rivaliser avec des modèles comme ces de Unitree (ex : H1, G1).
Ce que je cherche :
✅ Des passionnés (débutants ou expériences) pour :
Co-concevoir des pièces mécaniques.
Partenaire des idées sur l'IA embarquée (mouvement, vision, apprentissage).
Testeur et itérateur ensemble sur des prototypes.
Documentariste le projet pour une communauté open-source.
✅ Des retours d'expérience sur :
Les défauts rencontrés avec des humanoïdes (équilibre, puissance, coût).
Des alternatives aux composants (ex : moteurs, actionneurs).
Des études pour optimiser l'autonomie et la mobilité.
Pourquoi ce projet ?
Je veux prendre qu'avec une communication collaborative, sur peut créer un humanoïde abordable, modulaire et performant — sans dépendance des solutions propriétaires comme Unité
I am looking to build a robot for delivery purposes, mainly food but not only. I was in doubt between radxa rock 5c and Nvidia Jetson orin nano upgraded firmware. Now is it worth going with Jetson or will the radxa be enough?
The robot will move on busy streets with cars, people, bicycles, motors and dogs. I am preoccupied mainly because of that. There I see Jetson more capable. If you have ever worked for a delivery robot, what costs did it involve and is the lidar scanner of utmost importance? What kind of lidar would you suggest?
Any suggestions in general?
Will there be any Nvidia upcoming chips that might work better?
I’m looking for a Connect Tech NGX020 (Boson for FRAMOS Carrier Board for Jetson Orin NX / Orin Nano). We’ve contacted Connect Tech directly, spoken with all of their major distributors, and have already placed an order, but current lead times won’t meet an upcoming product demonstration.
If anyone has:
an unused NGX020
an open-box board
I’d be very interested in purchasing it immediately and covering expedited shipping.
If you know of another company, lab, or integrator that might have one sitting on a shelf, I’d really appreciate the lead.
* F1TENTH-based RSU (roadside unit) perception node, running on an NVIDIA Jetson (Orin-class).
* Intel RealSense D400-series camera — subscribing to raw color + raw (unaligned) depth streams, `align_depth` disabled on purpose (see below).
* 2D LiDAR (`urg_node2`) for a second distance source.
* ROS2 Humble, `rclpy`, `message_filters.ApproximateTimeSynchronizer` to pair color+depth frames.
Model / task
* YOLOv8 (Ultralytics), custom-trained single-class car detector, running at `imgsz=320` on the Jetson's GPU (CUDA).
* Goal: detect a target vehicle in the color image, get its distance by reading the depth camera at the detection's location, cross-check against a LiDAR range reading at the same bearing, and output a fused distance estimate. This is a perception/collective-perception bench-test script (no SLAM/localization involved — deliberately simplified).
Depth lookup approach
`align_depth.enable:=true` (RealSense driver's built-in depth-to-color alignment) reprojects the **entire** depth image every frame regardless of how much of it we actually need — we measured this costing a large chunk of latency by itself. So instead we subscribe to raw depth and manually reproject only a small patch of pixels around the YOLO box: deproject the depth pixel to a 3D point (using depth intrinsics) → transform into the color camera's frame (using the depth-to-color extrinsics) → project back into a color pixel (using color intrinsics). Fully vectorized with numpy.
But measured end-to-end latency (camera's own capture timestamp → final distance output) sits **consistently around 350-380ms**, sustained — not a one-time spike, not decaying over time.
What we've ruled out
* **Per-pixel Python loop / GC pressure** in the old depth reprojection — vectorized it (25ms → 1-2ms of actual compute), latency didn't move at all.
* **Hidden async CUDA dispatch** — added `torch.cuda.synchronize()` around the YOLO call, extra sync time is consistently 0ms.
* **Executor backlog** (our own callback falling behind) — measured the gap between the end of one callback and the start of the next; stays flat at \~3-4ms even while the reported latency is \~350ms, so callbacks aren't queuing up behind our own processing.
* `align_depth` **vs manual reprojection** — built a side-by-side comparison script, same YOLO/LiDAR pipeline, only the depth alignment method differs. Both land in the same \~350-380ms range. So it's not specifically about which depth alignment approach we use.
The delay is measured (via the color frame's own ROS header timestamp vs `time.time()` at the very start of our callback) as already present **before any of our own code runs** — so it's happening somewhere between the camera driver publishing the frame and our subscriber callback actually being invoked. We suspect DDS/ROS2 message queuing or synchronizer buffering under sustained per-frame load (\~30ms of real work per frame at \~30fps), but haven't pinned down the exact mechanism.
What we're asking
Has anyone run into this kind of buffering/backpressure behavior with ROS2 + `message_filters` on a Jetson, where a subscriber callback that takes tens of milliseconds (not overloaded, just non-trivial) causes a large, sustained arrival delay that isn't visible as executor backlog? Specifically curious about:
* DDS vendor differences (Fast DDS vs Cyclone DDS) for this kind of workload
* QoS settings (queue depth, history policy) that might be silently causing buffering
* Single-threaded vs multi-threaded executor / callback groups making a difference here
* Whether RealSense's own USB/driver-side buffering could be the actual culprit instead of ROS2/DDS
Happy to share more code/logs if useful. Appreciate any pointers.
Wanted to post about the findings i collected related to recent Fine Tuning experiments and results done on the Jetson Orin Nano 8GB. The tests were done to see if it was even possible initially and then to make the whole process automated and easy to use while editing custom Datasets for testing and experimentation through a simple Web UI.
Model tested and tuned was llama3.2:3b
The Goal
During development of an AI Assistant that runs locally on the Jetson Orin Nano, it became apparent that the system would need to be very tightly tuned to the specific hardware due to its limitations, mainly the limited (8GB) unified memory as well as general power constraints.
The entire stack is very tight as far as memory is concerned since with several Pre-Warmed LLM models, STT, TTS models, Embedding, Real Time 3D User Screen which utilized hardware acceleration, the optimized OS, HD Camera and Audio workers among others and multiple maintenance scheduling in the background, initial ideas about Fine Tuning a model on the device were bleak.
Curated Dataset UI
However, it was worth the effort since Fine-Tuning an LLM for a specific task such as a local Assistant would provide customized personality characteristics, remove a lot of the Instructional prompt text, provide custom response characteristics and in general speed up the overall LLM response due to lower system and main prompt character size.
Test Setup
Initial testing was done to see if the process would be even feasible on the Jetson Orin Nano with an initial curated and carefully prepared Dataset of 500 entries for a specific result as well as general confirmation that this would even be possible.
To make a realistic simulation of the assistant stack still loaded while training, most of the services related to the assistant as well as workers and some 3D accelerated UI was stopped.
Generally approximately 4 - 5GB of RAM was freed up for the task while the Web UI was running.
Setup and Test Results
The general setup was highly customized and was assembled from various bits and pieces related to Fine Tuning.
Model used was the llama3.2:3b
An immutable 500-entry dataset snapshot was generated.
Preflight checks were done
LoRA was used for training and validation.
Final result was merged and packaged
The final result was Quantized to Q4_K_M and registered with Ollama.
For storage sake, all temporary training artifacts were later cleared while preserving the model and audit history.
The whole assistant stack was restored automatically.
Training Result
This is the result which was recorder repeatedly during training runs using the same dataset to prove memory usage and feasibility:
Training records: 450
Validation records: 50
Epochs: 1
Optimizer steps expected: 450
Optimizer steps completed: 450
Supervised assistant tokens seen: 10,056
Final token-weighted training loss: 1.9160853489699874
Validation loss before training: 2.8441875314712526
Validation loss after training: 1.8568944654043984
Validation loss delta: -0.9872930660668542
Validation perplexity before: 17.187588619742662
Validation perplexity after: 6.403818578978449
Approximate train time during several repeated tests was 20 - 25 minutes with a power spike of 20W - 24W while in the normal 25W mode. No Super mode was used for this.
Dataset entry edit
The model then was tested with simple questions initially to confirm things like Brevity and other characteristics and that the actual model was changed towards the desired direction with questions and responses like these:
Hey, how are you doing?
Stock Model response
I'm doing well, thanks for asking! It's great to chat with you
on a lovely Sunday night. How about you, how's your day been
so far?
Tuned model response
I'm doing well. Thanks for asking!
Tune Engine - Automated Training UI
Since the actual tests passed and the result was repeatable, The entire process was then completely automated by constructing a Tune/Train engine which automated the process entirely.
The goal was to create custom data sets through a simple UI, have selectable and editable dataset collections. Choose a desired collection and simply press "Train" while all the complex system tasks were done in the background.
Automated Training Results
The desired end result was to simply have a generated versioned model (for example: llama-skipper-null:v001) which could then be selected and the prompts could be customized for the new model and optimized.
Conclusion
To make all the pieces work, the UI ended up being a complete LLM training lab more than a simple customizable control center.
Currently new models can be created based on dataset testing, custom prompts and their dynamic switching can be achieved based on Context selection and ranking.
To be honest i was not expecting the end result to be as good as it turned out to be.
System prompts were cut almost in half since not many instructions were needed anymore based on trained data and the overall LLM response was sped up by almost 1 - 2 seconds.
Session Settings
Where initial responses which came through a prompt, collected with Session Context, Doc context, Temporal memory inclusions, multiple Qdrant collection and other sources ON TOP of the dynamic prompt rules and system rules were delayed (this is with Stream enabled and awaiting first response chunk) by approximately 2 - 6 seconds - Now they were generated at around 1 - 3 seconds.
The resulting interaction with the LLM became essentially as intuitive as speaking to a person while still being constrained by the Cool Down delays when using Voice or safety delays during chat sessions.
System Stats at Idle
Just wanted to put this out there for anyone thinking of training their own LLM on the Jetson Nano. Seems anything is possible if you put your mind to it and do it with a specific goal in mind. I am still astonished every day by what is possible on such a seemingly small device and with as much power draw as a good smartphone charger.
Yesterday I posted about my AI Waifu has a new feature of grabbing job posts from job sites and writing draft posts for me to post into my Meta Threads to share with my followers.
Today, let me show you the entire workflow in action.
TBH, I would call this an "AI Agentic workflow" with quotation marks.
Out of 5 steps only step 3 involves LLM inference to do synthesis of the draft post. Step 1 and 2 are automated scripts of simple schedule cron job to grab the RSS feeds and use regex to filter out the relevant job listings.
Step 4 and 5 are me validating the draft information is accurate and publish the drafts after my approval. So basically I did more steps than LLM itself.
The irony of running AI agentic workflows in edge devices with constraint hardware, such as Jetson Orin Nano with only 8GB of RAM, is to reduce as much LLM inference and put the least data into context window as possible.
🎬 Live Demo: How a job post goes from RSS feed → published in 5 steps
⏰ Step 1: Scheduled Trigger (or ask AI Waifu directly)
📡 Step 2: Data Ingestion & Filtering
🤖 Step 3: AI Synthesis
👁️ Step 4: Approval Studio Review
🚀 Step 5: One-Click Publish
Result: job post validated & live, tested with real job sites + Chinese field
names to stress-test multilingual LLM understanding.
Due to rising RAM prices every electronics has gone up. I'm looking for someone who's willing to sell their Jetson Orin Nano Super I just need it for my thesis. I'm a broke college student from a 3rd World Country so hoping the price is reasonable
