Unlike a Grid Search, which tries all combinations of hyperparameters, a Random Search runs the model through random hyperparameter combinations. In addition, unlike in Grid Search, in Random Search the data scientist defines distributions for each hyperparameter instead of specific values.
So for example, for the hyperparameter max_depth, we could define a uniform distribution constrained between 1 and 5. For the hyperparameter n_estimators we could define a uniform distribution between 10 and 1500. For a learning rate we could define a log-uniform distribution between 0 and 1. For categorical hyperparameters like the method to evaluate the decrease in impurity, we would still need to define each value, for example Gini or entropy.
In Random Search, hyperparameter values will be sampled at random from these distributions. Given that, very often, similar values of hyperparameters return similarly performing models, it is not necessary to assess every possible value. Examining values at random will be more than enough to find the area or specific range of values that return the best performing models.
This decreases the time and complexity it takes to find the best hyperparameters, as well as the total number of combinations on which the models are trained. As a result, it’s usually the preferred model when compared to Grid Search.
The Random Search method also ensures that we don’t end up with a biased model that relies on value sets chosen arbitrarily by users, as is the case with a manual search.
https://www.youtube.com/watch?v=1FMnKAcaVPk
Selecting Best Model Using Randomized Search