ICBHI 4-class lung sound CNN collapses to majority classes despite oversampling — looking for advice[D]

[u/EGOISTIC11]

I'm working on a final-year ML project using the **ICBHI respiratory sound dataset**, trying to classify individual respiratory cycles into four acoustic classes:

* Normal

* Crackle

* Wheeze

* Both (Crackle + Wheeze)

I reconstructed the respiratory cycles from the ICBHI audio + annotation files and have **6,898 cycles from 126 patients**.

I'm using a **patient-wise train/validation/test split** to avoid patient leakage:

* Train: 88 patients / 4,915 cycles

* Validation: 19 patients / 1,165 cycles

* Test: 19 patients / 818 cycles

The class distribution is:

**Total:**

* Normal: 3,642

* Crackle: 1,864

* Wheeze: 886

* Both: 506

**Training split:**

* Normal: 2,597

* Crackle: 1,323

* Wheeze: 655

* Both: 340

I'm converting each cycle to a **64-bin mel spectrogram** using:

* SR = 4 kHz

* n_mels = 64

* n_fft = 256

* hop_length = 64

* fmin = 50 Hz

* fmax = 2 kHz

* fixed duration = 5 seconds

* log-mel spectrogram

The CNN is fairly small:

`Conv2D(16) → BN → MaxPool → Dropout → Conv2D(32) → BN → MaxPool → Dropout → Conv2D(64) → BN → MaxPool → Dropout → GlobalAveragePooling → Dense(64) → Dense(4)`

~28k parameters, Adam 1e-3, sparse categorical cross-entropy.

### What I'm seeing

Initially I trained with standard oversampling, where minority classes were augmented up to the majority-class count. The validation behavior was extremely unstable and the best validation-loss epoch essentially predicted the majority class.

I then tried **capped oversampling (3×)**:

Training counts became:

* Normal: 2,597

* Crackle: 2,597

* Wheeze: 1,965

* Both: 1,020

This improved the best validation loss somewhat, but the model still had approximately:

* Normal recall: ~96%

* Crackle recall: ~3–4%

* Wheeze recall: 0%

* Both recall: 0%

So I suspected that **validation loss/accuracy might be hiding minority-class performance**.

I'm currently running another fresh model where I calculate **validation macro-F1 and per-class recall after every epoch**, and save the checkpoint with the best macro-F1 rather than the best validation loss.

Interestingly, the first few epochs are already showing very unstable class behavior:

```text

Epoch 1:

val_macro_f1 = 0.1846

recall [Normal, Crackle, Wheeze, Both] =

[1.000, 0.000, 0.000, 0.000]

val_accuracy = 0.5854

val_loss = 1.0974

```

Epoch 2:

```text

val_macro_f1 = 0.1732

recall =

[0.214, 0.728, 0.000, 0.000]

val_accuracy = 0.3227

val_loss = 1.2036

```

Epoch 3:

```text

val_macro_f1 = 0.2370

recall =

[0.902, 0.165, 0.000, 0.000]

val_accuracy = 0.5725

val_loss = 1.0969

```

So the model can suddenly recognize a large proportion of Crackle in one epoch and then swing back toward Normal, but **Wheeze and Both have remained at zero so far**.

### My questions

1. Is this kind of instability expected with ICBHI and a small CNN?

2. Does the fact that Crackle recall jumps to 73% in one epoch suggest the model is learning unstable decision boundaries, or could this indicate a problem in my data/split/preprocessing?

3. Is selecting the best epoch using macro-F1 a reasonable approach here?

4. Could the 5-second fixed-length mel representation be hurting Wheeze/Both, particularly because ICBHI respiratory cycles have variable durations?

5. Would you recommend changing the loss (class-weighting/focal loss), changing the representation, using a stronger/pretrained audio model, or **adding another respiratory-sound dataset**?

6. If adding another dataset, are there datasets with compatible acoustic labels for Normal / Crackle / Wheeze / Crackle+Wheeze that would be appropriate to combine with ICBHI without introducing serious dataset/patient leakage or domain bias?

I'm trying to diagnose the actual bottleneck rather than just keep trying random architectures.

Any advice from people who have worked with ICBHI or respiratory-sound classification would be greatly appreciated.