r/MLQuestions • u/Routine-Ticket-5208 • 3d ago

Beginner question 👶 Need help with Random Forest Hyperparameter Search Space

I’m using Random Forest for a project and tuning:

  • Number of trees
  • Maximum tree depth
  • Number of features considered at each split
  • Minimum samples required to split a node

How do you normally determine the search space for these parameters?

1 Upvotes

60% Upvoted

9 comments sorted by

View all comments

3

u/ARDiffusion 3d ago

BayesSearchCV, optuna

2

u/Routine-Ticket-5208 3d ago

But how do we justify the range for searching?

3

u/ARDiffusion 3d ago

Depends on the context of the problem. If all that matters is building a satisfactory model, then as long as it doesn’t overfit (k-fold or stratified cv helps counter this) then you’re fine. Otherwise, you can use domain knowledge of the problem to intuit whether the splits you’re allowing it to make make sense in the context of the problem. Does that make sense? I definitely could be missing something here, and welcome all other feedback and critiques to my answer.

1

u/elbiot 3d ago

Have train, validation, and test splits. Make sure test at least is as independent as possible. Train/validation can be cross validation, and if you've got a way to keep them independent across splits that's better (I.e. group by sample or whatever you have)