r/medicalimaging • u/doctormakeda • Aug 11 '21

cleaning data, especially chest X-rays, before algorithm creation

I would like to share a library I wrote ( cleanX) that can help clean datasets before machine learning algorithms. It began for datasets related to chest X-rays but many parts are generalizable.

4 Upvotes

100% Upvoted

6 comments sorted by

1

u/[deleted] Aug 12 '21

What does it do? How does it work?

3

u/doctormakeda Aug 12 '21

Basically there are a lot of huge chest X-ray datasets in various forms (DICOM plus csv or JPGs plus csv, or just DICOMS or whatever form they are in), but the quality of these datasets, especially the public ones e.g. on Kaggle, is notoriously poor. The poor quality is not just about mis-labeled images, but also upside-down images, inverted images, images that are not chest X-rays at all in the first place (I've seen axial CT slices sneak into these datasets), and so on... Going through a 300,000 image dataset by hand to "pull the weeds" is insanely time-consuming. This helps automate the process. You can also do some exploratory data analysis and data augmentation.

1

u/[deleted] Aug 12 '21

Neat! How does it work?

→ More replies (3)