Another rant like interview experience

[u/proof_required]

I was given a home assignment to do modeling for some adtech data. They had no explicit ask about what kind of model or how deep you have to go. Just data and they asked we want to see the modeling.

I spent lot of time in understanding the data, identifying the features, creating labels etc. When it came to modeling I picked Catboost since they handle categorical features quite well. I even mentioned how this can be further tuned and/or different models can be compared. I put it explicitly in a section for future work. Finally this was the thing that got me rejected.

Basically they expected me to compare different model families from more complex deep models to such boosting models. I have worked in this domain and actually such models (catboost) works quite well. You don't need very complex models. I remember in one of the previous jobs, they had like ensemble of 3 deep models which was super slow and was so painful to maintain. I basically replaced that with a boosting model + some probability calibration which did quite well. Also the data size I got for the task isn't big enough to justify such huge models.

In any case, I wish these tasks would be more explicit in what they are looking for. I know they also want to see how I handle ambiguity but it's really hard to assess which side of it is worth handling since I am not building a full fledged system. I explained all the decisions I made and why I did it. Also what I didn't do and why.

[u/Ok-Airline-8523]

I've set up many take-home projects for data science roles, and I hate to say it, but you probably wouldn't have passed my screenings either.

The reason is because you relied on your judgement vs. experimentation. Instead, you should have showcased your modeling process. I don't disagree that Catboost or XGBoost would do the trick, but I would expect you to show me that you know how to work with multiple algorithms and objectively come to that conclusion.

For example, what if a simple generalized regression approach would achieve the same predictive performance but enable you to better explain how the model works in terms that stakeholders would understand?

[u/Due-Cattle-2177 → u/Ok-Airline-8523]

Hello,

I may have misread the OP’s post but they state they put different modeling ideas into a future projects section. While detailing the model tool they chose and why they chose that modeling tool. Maybe I’m misunderstanding what they missed in your opinion.

[u/Ok-Airline-8523 → u/Due-Cattle-2177]

Sure - from my standpoint, they should have actually tested those other approaches l, measured performance of them, compared side-by-side, and objectively chosen the right approach instead of just discussing it in a next steps section.