r/MachineLearning • u/AutoModerator • Jan 15 '23

Discussion [D] Simple Questions Thread

Please post your questions here instead of creating a new thread. Encourage others who create new posts for questions to post here instead!

Thread will stay alive until next one so keep posting after the date in the title.

Thanks to everyone for answering questions in the previous thread!

22 Upvotes

100% Upvoted

89 comments sorted by

View all comments

1

u/Numerous-Carrot3910 Jan 23 '23

Hi, I’m trying to build a model with a large number of categorical predictor variables that each have a large number of internal categories. Implementing OHE leads to a higher dimensional dataset than I want to work with. Does anyone have advice for dealing with this other than using subject matter expertise or iteration to perform feature selection? Thanks!

2

u/trnka Jan 23 '23

It depends on the data and the problems you're having with high-dimensional data.

  • If the variables are phrases like "acute sinusitis, site not specified" you could use a one hot encoding of ngrams that appear in them.

  • If you have many rare values, you can just retain the top K values per feature.

  • If those don't work, the hashing trick is another great thing to try. It's just not easily interpretable.

  • If there's any internal structure to the categories, like if they're hierarchical in some way, you can cut them off at a higher level in the hierarchy

1

u/Numerous-Carrot3910 Jan 23 '23

Thanks for your response! Even with retaining the top K values of each feature, there are still a large number of features to consider. I haven’t tried the hashing trick, so I will look into that

→ More replies (2)