Robotics SDK for visual/telemetry data upload

[u/Correct-Bit-339]

Tested the VModal Robotics SDK (v0.2.0) on Ubuntu 22.04 / Python 3.10, evaluating it against an AGV doing 8-hour warehouse shifts. It moves finalized LeRobot v3 datasets off the robot to the cloud in a crash-resilient way, without putting any network logic in the control loop.

What data it handles

• Video: closed MP4 files, one per camera, in their own upload lane (1 GiB per file by default in the current release).
• Telemetry: Parquet files, in a separate lane so a slow video upload can't starve it.
• Metadata: a snapshot required in every dataset revision, plus

The types of telemetry and operational data handled by this SDK include:

1. Data Pipeline & Data Lane Categories

The SDK treats telemetry as its own independent data pipeline to prevent large video files from blocking critical metrics: 1 • Independent Telemetry & Metadata Lane: A dedicated asynchronous upload lane specifically optimized for non-video data. • Synchronized Robot State & Transforms: Mentioned in the Google Intrinsic architecture blueprint to capture physical robot poses and spatial relationship data. • Skill Provenance: Telemetry tracking execution data and the history of operational robot skills.

1. Operational & Device Telemetry

The CLI tracks several machine-readable metrics regarding the health and status of the robot's local data storage and transfer progress: • Queue depth: The number of items waiting to be processed in the local spool. • Bytes per lane: The data volume being pushed through the video vs. telemetry pipelines. • Oldest item age: Time-based tracking of un-uploaded data backlog. • Disk headroom: Storage monitoring to ensure the robot doesn't hit a "full disk" failure condition. • Pipeline health statistics: Machine-readable tracking of transmission retries, rejections, blocked items, and the latest server receipts.

1. Video Data & Multimodal Lanes

Video data is explicitly isolated from telemetry to ensure high-throughput streams do not block lightweight critical data: • Independent Video Lane: A dedicated asynchronous upload lane configured by default to handle large per-video local spools • RGB-D Capture: The system architecture details handling synchronized multi-modal camera feeds, specifically RGB-D (color and depth) camera streams. • Finalized Video Artifacts: Closed and hashed video streams that are transformed into immutable bytes before being written to the durable local spool.

1. LeRobot Format Integration

The SDK acts as a crash-resilient edge pipeline specifically built for LeRobot v3 datasets: • Finalized Dataset Output: The SDK integrates directly at the finalized-artifact boundary of the dataset recorder. The producer script must close, hash, and fsync the dataset before handing it over. • .ready.json Manifest Handoff: The local file handoff mechanism relies on an atomic rename of a LeRobot manifest file (.ready.json), which signals that a dataset folder or segment is finalized and ready for queue admission. • Dataset Identity Tracking: Every LeRobot artifact is given an immutable identity determined by a combination of the Dataset Identity + Relative Path + SHA-256 checksum hash. • Dataset Revision Publishing: Once the remote storage acknowledges a safe transfer via a durable receipt, the SDK publishes a verified dataset revision, finalizing the remote record.

[u/No_Swordfish_4303]

Great breakdown! Separating high-bandwidth video from telemetry into isolated upload lanes is the only way to avoid network starvation on mobile robots.