I’m not smart enough to design a neural network architecture so I’m using designs from the experts. That is how the ResNet people did it… Pretty much I just changed that linear to get from 512 to 10 classes (of course, change 10 to your actual number of classes).