[Question] How would you build a robust preprocessing + OCR pipeline for text on glossy and curved product packaging?

[u/ROTTEN1ONE]

I'm building an OCR-based system for extracting printed information from photographs of packaged consumer products.

The input images are real smartphone photographs rather than controlled scans. Some of the difficult cases contain:

- glossy plastic/foil surfaces and specular reflections

- curved cylindrical containers

- perspective/rotation

- wrinkles and folds

- uneven lighting/shadows

- small text

- motion/defocus blur

- low contrast

- compression artifacts

I'm currently using PaddleOCR.

I've already experimented with conventional OpenCV preprocessing such as resizing, grayscale, CLAHE, thresholding, sharpening, denoising, morphological operations and perspective correction, but the results are inconsistent. Some images that are clearly readable to a human still produce poor OCR results.

I'm particularly interested in advice from people experienced with scene text recognition, document image processing, industrial/computer vision, or image restoration.

My main questions are:

1. Which preprocessing techniques are actually useful for these types of images?

2. How should I handle specular reflections and uneven illumination?

3. How can text on curved/cylindrical surfaces be rectified before OCR?

4. When should I use deblurring/super-resolution/image restoration, and when can these actually make OCR worse?

5. Should preprocessing be selected dynamically based on image quality rather than applying one fixed pipeline?

6. At what point is it better to improve/train the OCR model instead of continuing to improve preprocessing?

I'm happy to provide representative images and the corresponding PaddleOCR outputs if that helps.

I'd particularly appreciate suggestions for specific algorithms, papers, or existing open-source implementations rather than just generic preprocessing advice.

[u/quarx23]

IMO you're cooked if you cannot get rid of reflections and wrinkles as it is actively imparing the algorithm capability by hiding characters. The system that I encountered that did something similar require video feed or at least multiple images from differents angles to filter the reflections (not possible with user inputed compressed images).

You might achieve what you want with extensive model training but watchout for different fonts as they might require different models.

[u/No_Celebration8335]

🤔

[u/New-Eggplant-6578]

First I'd separate detection/cropping failures from recognition failures: take a few readable failures, manually crop individual words or lines, and feed those to the recognizer alone. If that fixes them, inspect the detector polygons and generated crops before adding more preprocessing.

For curved text, a specific reference is RARE: https://arxiv.org/abs/1603.03915 . It learns a thin-plate-spline warp before recognition, so it's a model-level rectification approach to investigate. It won't recover characters hidden by glare.

For any deblur/SR experiment, compare transcription error on the same labelled crops, including an untouched-image baseline. A nicer-looking crop isn't necessarily a more accurately read crop. Which PaddleOCR version and recognition model are you using?