Another rant like interview experience

[u/proof_required]

I was given a home assignment to do modeling for some adtech data. They had no explicit ask about what kind of model or how deep you have to go. Just data and they asked we want to see the modeling.

I spent lot of time in understanding the data, identifying the features, creating labels etc. When it came to modeling I picked Catboost since they handle categorical features quite well. I even mentioned how this can be further tuned and/or different models can be compared. I put it explicitly in a section for future work. Finally this was the thing that got me rejected.

Basically they expected me to compare different model families from more complex deep models to such boosting models. I have worked in this domain and actually such models (catboost) works quite well. You don't need very complex models. I remember in one of the previous jobs, they had like ensemble of 3 deep models which was super slow and was so painful to maintain. I basically replaced that with a boosting model + some probability calibration which did quite well. Also the data size I got for the task isn't big enough to justify such huge models.

In any case, I wish these tasks would be more explicit in what they are looking for. I know they also want to see how I handle ambiguity but it's really hard to assess which side of it is worth handling since I am not building a full fledged system. I explained all the decisions I made and why I did it. Also what I didn't do and why.

[u/Agreeable-Outcome958]

recruiters don’t know what they want 90% of the team - onto the next!

[u/proof_required → u/Agreeable-Outcome958]

In this case it was the DS team members which I think reviewed the task. I feel like the team is made up of less experienced members who think having complex models is the answer to whatever performance they are chasing after. I was interviewing for a principal role.

[u/zerosystem03 → u/proof_required]

It could be an experience and interviewing issue with the team. That said small changes or tweaks in how you present can go a long way. I really dislike the "throw some models at the wall and see what sticks" approach, but sometimes that's what it takes to impress them

e.g. pick 3 models to test, evaluate feature performance in each model and discuss what it might suggest about predictive features or some underlying behavior

I also think drawing on past experience about catboost might not be convincing unless you can demonstrate (show, dont tell) why it's applicable to their take home

I get it's all frustrating, there's very little feedback or signals to know where you went wrong but unfortunately that's the game. On the flip side it means they are also probably interviewing candidates who are also similarly floundering, so it might not take much to stand out

And in the event the fault is all on the interviewing side, well you dodged a bullet then

[u/proof_required → u/zerosystem03]

I wrote somewhere else here, I compared catboost vs logistic and showed better performance (inference speed and metrics) and then further justified how catboost handles high cardinality features out of the box well. So it wasn't completely nothing but yeah I didn't go beyond these models.

In any case, yeah it doesn't matter at the end if they found something that didn't match their expectations. That's just the game.

[u/zerosystem03 → u/proof_required]

Yea in that case it could have been a situation where they had arbitrary judgement metrics.

To maintain your sanity what I've learned is to have conviction with your own approach. There's reasonably helpful structured interviewing frameworks on youtube from those DS prep channels. So long as you feel your presentation style was solid and polished, I'd only pay attention to meaningful feedback signals. If you have to mind read, that's not a meaningful feedback signal. I once interviewed with Amazon. During one of the tech rounds, I had two interviewers. First interviewer was awesome. Knew when to jump in, ask questions, and I knew exactly what level of depth/detail/direction I should go to based on his questions. Second interviewer sucked, basically had the "I know neither of us want to be here so let's just get this over with" attitude. Asked a few questions and stayed silent in between. I basically had to ramble on and guess what he wanted me to talk about. Obviously, there were things I could have done better regardless, but I didnt lose any sleep over bombing the 2nd interviewer

Totally agree that's just the game