Hyperparameter Search with Optuna
- Hyperparameters (HPs) can have significant impact on the final model parameters and model performance.
- Studies show that the most significant HPs are learning rate and training epochs, while other parameters also matters. Depends on the applications such as CV, NLP, the HPs can be significantly.
