Neural Networks (fundamentals)
Neural networks without activation functions (pure linear combinations of inputs) can only solve linear problems. This is shown in this intuitive and simple example of a neural network incl. some math. → video or here
However, modern neural networks use activation functions specifically to introduce non-linearity, which allows them to approximate any continuous function (this is known as the Universal Approximation Theorem).