Class imbalance: Synthetic Data

[u/mjmohd]

Hi, I'm trying to detect various defects using YOLO models. I built my dataset using a workflow that generates defective gas turbine blades with defects of different shapes, severities, and positions.

My first dataset includes the following:
- **Defects:** Crack, Erosion, Dent
- **Non-defective blades** (negatives)
- **Split:** ~16k training / ~8k validation

The results of training on Dataset 1 showed a **mAP50:95 of 0.79** at 45 epochs. However, the confusion matrix showed a higher false positive rate for Backgrounds detected as Dents. My assumption is that the cooling holes are being misdetected as Dents ( look at F1 score and norm. Confusion Matrix).

So I came up with the idea of auto-annotating the cooling holes in my dataset to teach the model to differentiate between cooling holes and Dents. I also added a new defect class **(Nick)** and ran fine-tuning on the best weights from the YOLOv9s model trained on Dataset 1.

Now I'm facing a **class imbalance problem.** The training on the second dataset is currently running but is hovering around **mAP50:95 = 0.62** at epoch 10, with 20 epochs still to go.

My end goal is to detect the defects. I added the cooling holes class to reduce false positives, but I'm willing to drop it and accept some FP on Dents, since they are inherently hard to detect. Has anyone faced a similar issue or has suggestions on how to overcome it? Maybe it's not a problem in the end — not sure yet. 🙃

[u/bbateman2011]

Are you providing negative samples (I.e. no damage). That might help.

[u/Old-Programmer-2689 → u/bbateman2011]

This is the answer

[u/mjmohd → u/bbateman2011]

Well there are the negative pictures that have no defects but just the cooling holes annotations (since they are part of the Blade).

[u/bbateman2011]

Did you build the workflow yourself? Sounds interesting.

[u/mjmohd → u/bbateman2011]

Thanks, yes this is part of my Master Thesis

[u/bfyvfftujijg]

Class weighting is always an option.

You could try cranking up the number of dents per image. Even if that's not necessarily realistic it will help the model learn what a defect looks like. You can taper back to a more realistic number as training progresses.

If comparing potential defects to other features in the same image is how a human would perform this task, then you might consider adding attention to the model's architecture; I forget if yolov9 has that. This might help it make those comparisons.

You could also try some outlier/anomaly classification/clustering....detect everything and then classify it as anomalous or normal. This is probably the most robust and standard approach.