Scikit-learn is the go-to Python library for machine learning, hosting a wide array of predictive algorithms. Scikit-learn is also the go-to library to optimize the hyperparameters of its own algorithms.
Scikit-learn supports Grid Search and Random Search with cross-validation through the GridSearchCV and RandomizedSearchCV. Scikit-learn also offers support for multi-fidelity hyperparameter optimization through successive halving, which he have not covered in this tutorial.
The entire Scikit-learn package provides a range of machine learning-related modules that can be applied with minimum effort. It comes with built-in with tasks that allow you to carry out any activity —from preprocessing to using the sklearn.model_selection module to select the best hyperparameters.
Scikit-learn, and other similar packages for hyperparameter tuning, such as scikit-optimize and scikit-hyperband, have a consistent interface across many implemented classes, providing a massive benefit to data scientists. Built on top of NumPy and SciPy it also provides tools for data analysis and data mining, making it one of the engineers’ favorite packages.
skore is a Python open-source library designed to help data scientists apply recommended practices and avoid common methodological pitfalls in scikit-learn. It can be used for experiment tracking as shown here.