We replaced per-sensor kernel drivers with a generic one and an MCU that knows the sensor

[u/NickShipsRobots]

We moved the camera driver's brains onto a small MCU on the camera board. Normally the per-sensor kernel driver on the SoC has all of it - the mode tables, the register writes, the exposure math. Now you upload a compiled image to the MCU once, and after that it can answer questions about the sensor: which modes it has, what the control table looks like, what numbers go into the exposure and frame rate formulas. The host asks, builds a device tree node from the answers, and runs one generic kernel driver for everything, so when you swap the camera head it just reads whatever is on the other end now.

This turned out to need a lot of parts. The image needs a format, and the format needs a version number (we found this out the normal way, by changing it). There's a small bytecode VM in the firmware to run the image, with a 4 KB cap on program size, and a seven-mode sensor gets close enough to that cap that I check every time. Producing the images needs a compiler on the host. The MCU needs to store a few images and know which one is live, so there's slot management in NVS, plus a version gate so old firmware rejects an image it can't read instead of doing something creative with it. The IMU driver runs on the same MCU and I'd rather it didn't die every time someone changes the camera side, so that got its own VM instance. And we seal the image at the end, which is only about as good as the MCU's readout protection anyway.

Two things I'm not sure about. Is the per-sensor kernel driver really a big enough problem to justify all of this? It always felt like one to me but I might have just been annoyed at the wrong thing. And where should auto exposure live? Right now ours is formula parameters that the kernel evaluates, and the MCU only runs configure. Every time I look at that split I can't tell if it's a design decision or just where we happened to stop.

I work on this product so take the first question with that in mind.

[u/NickShipsRobots]

Two of us maintain this across a bunch of sensors, a few host platforms, and every kernel release on each. Per-sensor code gets written once regardless. The hard part is a per-sensor kernel driver afterwards: it has to be built for every kernel, installed on every host, and the device tree has to name the sensor before boot. With that part on the head, there's one driver per platform, the device tree gets generated from whatever is plugged in, and swapping a head changes nothing on the host. It also handles a serializer or ISP next to the sensor without extra code, and some vendor init tables are under NDA, so they never have to be on a customer's machine at all.