> Stripped of anything else, neural networks are compositions of differentiable primitives
I’m a sucker for statements like this. It almost feels philosophical, and makes the whole subject so much more comprehensible in only a single sentence.
I think François Chollet says something similar in his book on deep learning: one shouldn’t fall into the trap of anthropomorphising and mysticising models based on the ‘neural’ name; deep learning is simply the application of sequences of operations that are nonlinear (and hence capable of encoding arbitrary complexity) but nonetheless differentiable and so efficiently optimisable.
And then you learn about binary or ternary networks where gradients don’t really exist anywhere, and you start to wonder about the importance of this differentiability.
Wow, just skimmed a bit, but this book looks amazing so far. Really understandable but with an intuitive presentation of the underlying maths that invites the reader to go deeper if they want to by giving them what they need to get started.
xanderlewis ·21 days ago
I’m a sucker for statements like this. It almost feels philosophical, and makes the whole subject so much more comprehensible in only a single sentence.
I think François Chollet says something similar in his book on deep learning: one shouldn’t fall into the trap of anthropomorphising and mysticising models based on the ‘neural’ name; deep learning is simply the application of sequences of operations that are nonlinear (and hence capable of encoding arbitrary complexity) but nonetheless differentiable and so efficiently optimisable.
Show replies
p1esk ·20 days ago
Show replies
seanhunter ·20 days ago
gfaure ·20 days ago
glonq ·21 days ago
Show replies