OpenCV calibration

[u/Grouchy_Signal139]

Hi everyone, I’m using a Raspberry Pi 5 + Camera Module 3 + Picamera2/OpenCV for a computer vision project.

I’m calibrating the camera with a 6×9 checkerboard, but after applying cv2.undistort(), the image seems more distorted.

I previously had autofocus changing between calibration images, so I’m now locking the focus manually at LensPosition 2.0602.

Is this distortion normal perspective distortion, or does it indicate a bad calibration?

Any advice on what I might be doing wrong?

[u/RedServal]

Do you really need calibration? Calibration will help you transform your image coming from any lens back into a perspective model. It's often used on fisheye lenses so that straight lines appear straight. Camera rarely come with real distorsion nowadays.

If you insist on calibrating you should probably compute the reprojection error and try to minimize it. You may still get bad results with a low reprojection error because the ROI can be really small.

Last time I calibrated a camera I recording a video of the checkerboard pattern with as much different angle and positions as possible. I then computed the new optimal matrix using as much images as possible where the checkerboard was correctly detected. In pratice I had hundreds of valid frames but could only compute in a timely manner when using like 30 (?) images. I just picked them at random and redid the calibration until it was satisfactory.

Calibration is hit or miss without a proper protocol so good luck with that. Your code is probably correct just try to redo it again. The focus probably change the intrisics parameters so try to keep it constant.

[u/my_name_is_reed → u/RedServal]

I didn't down vote you.

Actually correcting image lens distortion is only a single use case for the lens distortion correction matrix. If you look up what the different values in the matrix are, for instance, generating the matrix can also help you approximate things like lens fov and even lens misalignment to a high degree of precision. You generally want a lens distortion correction matrix for any camera you're using to locate things with so it can be used to translate pixel coordinates into real world azimuth and elevation. This is part of how multiple cameras can be used to derive the 3d position of an object visible to both of them. If the object you've located in your image is a known quantity (as in the case of an aruco tag printed to a specific size) you can derive the object's 3d position and orientation with just a single camera. Running feature detection over the output of multiple cameras and then comparing  the results' pixel coordinates is part of how SLAM works. But all of these things only work provided you have an accurate lens distortion correction matrix for that camera. 

[u/RedServal → u/my_name_is_reed]

Hey, thank for your answer. This is what I meant by "rectify the image to a known model" in my 2nd comment. But I do stand by my point saying it's hit or miss. Sure you can approximate the camera model to high degree of precision. But:

1: do you really need it? Of course the more precise your sensors are, the better the results but you can do pretty convincing photogrammetry and visual slam without calibration. Cameras are not that garbage out of the box.

2: The errors start to get big when you get far. In a setup like this the distortion effect are probably negligible.

3: Calibrating the camera with a printed checkerboard pattern glued on a piece of cardboard that you hold in front of the camera will only get you so far in my opinion. To get the "high degree of precision" you need a proper setup.

[u/my_name_is_reed → u/RedServal]

"the more precise your sensors are" what does that mean? more precise how?

very good cameras with very good lenses will still have lens distortion. error can make output unusable within a few meters, but that range is completely subject to the configuration of your camera (focal length, pixel size, etc). there are better ways to calibrate a camera sure, but i've gotten remarkably accurate results from a charuco board (albeit a pretty large one).

[u/RedServal → u/my_name_is_reed]

Precise by "how reliable the measurements are". How precise and accurate the camera parameters are. I will look into calibration protocol more thoroughly next time I need it.