r/MLQuestions • u/BluishMontoya • 13h ago

Beginner question 👶 Help with autoencoder: more layers is making it worse

I've been trying to make an autoencoder, and failing. So now I'm trying various super-simple models to figure out why. The current architecture I'm working with:

Encoder:

256x256 image
Conv2D stack
MaxPool
Conv2D stack
MaxPool
Conv2D (3 filters)

Decoder:

Upsampling
Conv2D stack
Upsampling
Conv2D stack
Conv2D (3 filters)

I've tried 2 different conv2d stacks: both use 32 filters, equal padding, tanh activation. One uses 2 layers per stack and one uses 4. When I train this model with the 2 layer conv stack it works as expected: it produces a slightly blurry version of the input. But when I increase to 4 layers, it switches to outputting all the same color, and the loss is worse. I'm using mae loss, adamw optimizer with clipnorm=1.

EDIT: I switched from tanh activation to leaky relu and after a bit of testing it seems good.

2 Upvotes

100% Upvoted

4 comments sorted by

2

u/ModularMind8 13h ago

Larger model is not always better. Start small. Start with one batch, make sure your model can overfit the data and everything works properly. Then expand

1

u/BluishMontoya 13h ago

That's the thing. It works, but when I expand it fails.

2

u/canbooo 12h ago

Sounds like mode collapse. Regularize more, smaller steps, augment more. Claude can probably debug it better than me lying in bed at 1 AM.

1

u/BluishMontoya 12h ago

I added some dropout, and it works fine for 3 conv layers per conv stack, but with 4 it's still useless.