Regularization - Ridge (L2) Regression
Reducing Features with Lasso Regression
<aside> 💡 Ridge and Lasso Regularization are frequently combined to get the best of both worlds.
</aside>
Ridge works better when most of the variables are useful and Lasso works better when a lot of the variables are useless. A combination of both can be the best of both worlds. More details on that in StatQuest p.176.
https://www.youtube.com/watch?v=l9V5tlIWTvs&ab_channel=Udacity
Lasso in Simple Words:
Imagine you're shopping with a strict budget. Lasso is like a budget-conscious shopper who must decide which items are truly worth buying. When features (variables) aren't helping much to predict your outcome, Lasso completely eliminates them by setting their importance to zero. It's ruthless - if a feature isn't pulling its weight, it gets kicked out of the model entirely. This makes Lasso great when you have many features but suspect only some are truly important, resulting in simpler models that are easier to interpret.