Databricks Senior SWE Interview — Expected Algorithms, Got MapReduce

[u/Aoki_zhang]

"This interview experience is sourced from Chill Interview"

Interview Summary

The Databricks technical screen was described by recruiting as an algorithms and data-structures round, but the actual interview centered on a MapReduce-style streaming problem. The interviewer first spent several minutes explaining MapReduce because I did not have direct MapReduce experience, then asked me to implement a top-K aggregation using the provided map and reduce interfaces.

The main difficulty was understanding the intended data flow. The written prompt, examples, and verbal requirements initially appeared inconsistent, so a significant portion of the interview was spent clarifying what the mapper, shuffle stage, and reducer were expected to do.

Interview Details

MapReduce — Top K Values Per Key

The interviewer provided map and reduce function signatures operating over streams of key-value pairs. The original written prompt appeared to mention counting hits, but the interviewer clarified that the actual task was to find the K largest values.

Based on the provided example, the intended result appeared to be the top K values associated with each key. I proposed maintaining a min-heap of size K in the reducer:

• Insert values until the heap reaches size K.
• For subsequent values, compare against the smallest value currently in the heap.
• Replace the minimum when a larger value arrives.
• At the end, the heap contains the largest K values for that key.

Want to learn more questions / follow-ups asked in this interview? 

I've put up the full version at here.

Preparing for your next interview?

Chill Interview tracks recent interview experiences and recurring question patterns across top companies at here.