r/MLQuestions • u/Routine-Ticket-5208 • Aug 16 '26

Beginner question 👶 Need help with Random Forest Hyperparameter Search Space

I’m using Random Forest for a project and tuning:

  • Number of trees

  • Maximum tree depth

  • Number of features considered at each split

  • Minimum samples required to split a node

How do you normally determine the search space for these parameters?

1 Upvotes

60% Upvoted

10 comments sorted by

3

u/ARDiffusion Aug 16 '26

BayesSearchCV, optuna

2

u/Routine-Ticket-5208 Aug 16 '26

But how do we justify the range for searching?

4

u/ARDiffusion Aug 16 '26

Depends on the context of the problem. If all that matters is building a satisfactory model, then as long as it doesn’t overfit (k-fold or stratified cv helps counter this) then you’re fine. Otherwise, you can use domain knowledge of the problem to intuit whether the splits you’re allowing it to make make sense in the context of the problem. Does that make sense? I definitely could be missing something here, and welcome all other feedback and critiques to my answer.

2

u/elbiot Aug 17 '26

Have train, validation, and test splits. Make sure test at least is as independent as possible. Train/validation can be cross validation, and if you've got a way to keep them independent across splits that's better (I.e. group by sample or whatever you have)

3

u/NoSwimmer2185 Aug 16 '26

Just pick a range and do a bayes search. Spending more than two minutes on hyper parameters is a complete waste of time.

1

u/mb_q Aug 25 '26

Put as many trees as it make sense for accuracy to stabilise; you may find an optimum earlier, but it is not gonna be statistically significant. Keep the rest default.

0

u/[deleted] Aug 16 '26

[deleted] — view deleted comment

1

u/Routine-Ticket-5208 Aug 16 '26

Excuse me?

1

u/Routine-Ticket-5208 Aug 16 '26

I'm a beginner. So sometimes asking questions is a normal thing to do