The most fundamental explanation of a Neural Network. History of Neural Networks and why we need to make them deep.
It has been shown that such a network can learn to approximate any mathematical model to any level of accuracy! Even though this is theoretically possible, in practice we need more layers to achieve good performance. More layers == "deep" and this is how we get deep learning!
A neural network uses a very flexible function which can fit to every data as long as it gets the correct parameters. In this fastai tutorial we get an intuitive feeling why this is true.
A very intuitive explanation of a Neural Network by using an Excel sheet. It shows that a Neural Network is basically just matrix multiplication. For more details how to implement this from scratch in Pytorch (including a recap of the basic ideas) see here. Pytorch uses Broadcasting here, which makes code very clean and simple. Doing everything from scratch as shown in the linked video above, helps to understand each step in a Neural Network. As seen here and here it makes sense to use a framework which handles best practices for you automatically.
What is a pretrained model and what does it mean to Fine-Tune a pretrained model? We look at different layers. Each layer learns a different feature. The first layers learn the basics, which are useful for multiple kinds of predictions. The last layers are very specific. We remove the last layer and fine tune it for our problem.
Some words about Transformers and comparison with ULMfit. Basic ideas are similar.