I’ve just finished the chapter on the basics of neural networks, coding simple feed-forward, recurrent (both regular and LSTM) and convolutional networks. Each example used essentially the same synthetic data (in either 2D or 3D form depending on the requirements of the network) and the same loss function and optimizer, so it was easy to compare what was required to create each one. Pretty straight-forward really.
I think I’ve got my head around some ambiguities. In RNNs, for example, we have both a hidden layer and a hidden state, which appear to have nothing to do with each other, except that the size of the hidden state is partly dependent on the size of the hidden layer. Channels in CNNs are equivalent to input features in other kinds of network. I’m probably misunderstanding something, but it seems that becoming familiar with the jargon is a large part of learning neural networks. Of course actually applying them to real problems is a different story.