Tweets about hyperparameter optimization
Find best hyperparameter for your model
<aside> 💡
Finding the right parameters only works through experiments. How to conduct and evaluate these experiments can be seen in the XGBoost book, pages 92-96. But keep in mind: First tune the architecture, don’t tune the parameters! If you want to train multiple architectures check this.
</aside>
When looking for the best hyperparameters you can spend a lot of compute. So much so, that you can also spend too much. It is a sutble thing, but if you're not careful you can become a victim to something that's known as "the optimisers curse". One advice would be to plot the effect of tuning one parameter agains your desired metric. This way you get a feeling if this parameter only adds noise or if it actually helps. → very general explanation incl. code in video.
<aside> 💡
Check out: AutoML & Experiment tracking
</aside>
You should systematically and automatically search for the optimal hyperparameters for your model.
Scikit-learn for Hyperparameter Tuning
For neural networks: