Hmmm you’re right, ResNet50 only has a Linear(2048, num_classes) as the fully connected.

If I remember well, I was just looking for ways to increase the accuracy and I think I found this in an article somewhere. I started with just a Linear layer, then added the others and eventually that was the configuration that gave me the best results.

I used this in a “competition” on the Oxford dataset, where the original ResNet was supposed to get something like 92–93%, and I got it up to 99.90%. Winner? No, some people got over 99.95%.