Hyperparameter
These parameters are set randomly before the model training (ex: the estimators of random forest).
Machine learning tunning is to find the optimal hyper parameter combination which have the minimum loss function or the maximum accuracies.
Manual Search
we can initial the hyper parameter based on our experience and calculate the loss or accuracy after the model training. Repete this process untill we can get a satisfied result.
Random Search
In Random Search, we create a grid of hyperparameters and train/test our model on just some random combination of these hyperparameters.
Using cross validation to split the data into train and validation folder. This guarentee that a good parameter should not only work in train folder but also in test (avoid the overfitting).
Grid Search
Including all possible combinations of hyper parameters.
Model Parameter
These parameters are learned during the model training.
