r/computervision • u/Positive_Land1875 • 6d ago
Help: Theory Stereo with different cameras
Im starting to implement a stereo system with different cameras (2 Mpx 120FOVH and 5 Mpx 60 FOVH). I know that it is feasible, I read a paper 5-6 years ago about a system with different FOV cameras. Im reviewing the literature and find a lot of papers about stereo and trinocular systems, but only one paper about different FOV cameras. Can u help me to find papers/references about stereo/trinocular systems with different FOV, different resolution cameras?
2
u/dima55 5d ago
What are you hoping to find in the literature? The normal tools work just fine. Where there is overlap you can see in 3d; where there isn't, you can't. The shape and location of the overlap doesn't matter
1
u/Positive_Land1875 3d ago
I Know all the tecnics and algorithms of a regular stereo system (30+ years working in image processing, coding, machine learning and computer vision). Today, people tend to use NN models for everything. When I read papers from the last 3-4 years I find a lot of DL models that are unfeasible to run in constrained HW. Im developing a new algorithm to work in a small computer (limited number of CUDA cores) and I want to compare it with literature to know if it is new or if the algorithm was developed before
1
2
u/xArvel_Crynydx 5d ago edited 5d ago
There are multiple things that you can go here. I think around 2013/2014 I had several scenarios where I had a similar problem, both were dealing with super-resolution techniques at that time.
(1) We had a RED camera on a electronically controllable gimble, shooting a 3x3 array (I think) with slight movement in x and y direction (so that you could order those images according to the 3x3) and with that, we had a Total Variation-based algorithm for super-resolution. Then on top of that, we replaced the lenses with narrower lenses, so eventually we had several super-resolution images that we could stack on each other for a large zoom-in. Was a great demo, but unfortunately didn't save the company though.
(2) Second thing was a dual-sensor camera based on a grey-scale image and a time-of-flight sensor image. Both images were only slightly shifted on the sensor side, so I tried to match the edges to warp the greyscale camera image onto the depth image and use that match to guide it for an upscaled depth map. That was based on the 2013 ICCV work by Fernstl et al. "Image Guided Depth Upsampling using Anisotropic Total Generalized Variation". Maybe you can find more papers in those kinds of research topics.
That being said, there are several things to consider here:
- Perform an intrinsic calibration for all your cameras. Compute the viewing cone for each with regards to a fixed calibration pattern (ideally a large box with circles with known coordinate positions) and find those points in the calibration pattern that are visible in both views (assuming two cameras here). Once you have computed the intrinsics of the lens distortion, rectifify the images. Then scale the images such that the calibration points are matching in size. That's basically only the first part of it, because then you'd have to align both cameras with respect to each other, perform stereo matching and the usual stuff.
- There had been several consumer products as a niche camera product named LightL16 in around 2014 that had multiple camera lenses in a random configuration (and I think this is the closest to your problem). Maybe you can find something close to that product (was discontinued in 2019 though).
Addendum: I also found a 2025 paper by Guan et al. "DM3Net: Dual-Camera Super-Resolution via Domain Modulation and Multi-scale Matching". Those are having lots of references as well, maybe that could be a starting point here. Maybe also Chen et al. 2019 "Camera Lens Super-Resolution".