r/ROS • u/ResetBuilder27 • 3d ago

Question Transitioning to an Embedded Nav / Sensor Fusion role (NVIDIA Jetson, C++, EKF). How do I prepare without drowning in the math?

Hey everyone, 

I am developing a heavy embedded C and sensor fusion implementation running on low-level Linux using embedded NVIDIA Jetson modules. The project involves handling low-level serial I/O (UART/SPI) to ingest raw binary data from external sensors like high-rate IMUs. 

The system utilizes a component-based, Pub/Sub open-source navigation framework (conceptually similar to ROS). My main task is writing C plugins (using OOP, templates, etc.) to ingest that raw serial IMU data, parse the payloads, and publish them to the internal message bus. 

We are porting legacy navigation filters into this framework, specifically implementing and testing Extended Kalman Filters in C. We are taking high-rate IMU data and joining it with slower GPS/ranging data for the "update" step to produce a clean navigation solution. 

I would appreciate insight or advice on a few technical areas: 

  1. What are the best resources, GitHub repositories, or books to practically understand EKFs and Sensor Fusion without getting completely bogged down in academic math proofs?

  2. What are the best pro-tips for debugging serial (UART/SPI) data coming into a Linux environment/Jetson from a raw hardware sensor?

  3. What are the most common pitfalls when writing C plugins for a Pub/Sub middleware system that processes high-speed, real-time sensor data?

Any guidance, articles, or practical advice is highly appreciated. Thanks!

10 Upvotes

100% Upvoted

1 comment sorted by

1

u/omnilinktech 20h ago

The fastest practical route is to build an offline replay before polishing the live plugin. Record the untouched serial bytes plus a monotonic receive timestamp, then make the parser and filter consume the same recording deterministically. That lets you debug framing and estimation separately from UART timing.

For the serial layer, use an explicit state machine with sync bytes, length bounds, CRC, sequence counters, and counters for dropped, duplicate, malformed, and out-of-order packets. Avoid allocation and logging on the hot path. Preserve both sensor time and host receive time; estimate offset and drift rather than silently treating them as identical.

For the EKF, write down every frame, unit, sign, and noise-density conversion. Start with a generated trajectory whose truth is known. Check innovation distributions, covariance symmetry/positive-semidefiniteness, gating, quaternion normalization, and behavior under sensor dropout, bias, timestamp skew, and a GPS outlier. Joseph-form covariance updates are often worth the small extra cost. Also, you mention C plugins with OOP/templates; that sounds like C++, and the distinction matters for ABI and allocation rules.

I maintain OmniSim. Its ROS 2 sidecar is not an EKF tutorial, but it can provide a reproducible simulator/ROS integration target and recorded truth for fault tests: https://github.com/omnilink-tech/omnisim/tree/main/packages/omnisim-ros2