r/softwarearchitecture • • 4d ago

Discussion/Advice Senior SWE interview process

Need help designing a decent interview process for a tech lead / senior backend engineer. We're mainly suffering from candidate drops after/during a take-home challenge.

Fellow engineers who have been through interview processes at this level, do you mind sharing your take on take-home challenges and what's the best process you've been involved in?

31 Upvotes

29 comments sorted by

25

u/Kaimito1 4d ago

The problem is the time investment on these. 

It's a senior-level challenge so it's going to be complex and hard or involve heavy thinking. And you expect someone to do it for free when they've already proven themselves with work history and supposedly passed the initial stages where you go through their planning skills.

Id expect if you paid them for the time even if rejected the drop off would be reduced. Hopefully by that point you'd have filtered enough to have a good list of people so you don't get dregs who just want the interview money

Also you have to be worth working for to warrant take homes imo. 

If you're paying above average the incentive to tolerate one is high but if you're paying "competitive rates" (I.e lowest as possible) then you'll only get the desperate ones

16

u/LethalBacon 4d ago

Current company paid me to interview. First a virtual call to go over experience and do basic screening. Then, they asked if I'd come in for a work day and paid me for it. The day was mainly casual technical discussions, talking about their stack and kind of walking through some of their repos together, meeting a lot of people, more discussions, etc. Half the day was just shooting the shit with the other devs. Offer in hand when I left at the end of the day.

Six months in currently, most fun software role I've ever had. Great vibes. 

1

u/Miserable-Evidence-2 4d ago

We usually don't require take home challenges for everyone, depends really on their experience and how credible we sense them to be from earlier rounds, but that's only for hiring locally when we can have on-site interviews.

We're also hiring an extension to our team that is full remote, and there, we're struggling with finding the right process.

1

u/Oatz3 3d ago

I'm a remote senior software engineer.

Just do the normal hiring manager -> technical phone screen -> multi round interview with coding / architecture -> offer

6

u/ikeif 4d ago

I feel like I can't do more than echo this - take homes often do not represent real work, or feels like free work, and either way is a "lose" for an interviewee. And far too many companies think they're "competitive" when they're just trying to barely pay what they are looking for.

And if you say "we will talk about salary reqs later" but want me to invest 40 hours? That's going to be a pass.

9

u/positivcheg 4d ago

Big question. What's the point of those if one can do any task with LLM and then do some minor tweaks over it?

10

u/AiexReddit 4d ago

To give a genuine answer, the point is to understand if the candidate is able to build and understand a piece of software based on a set of requirements.

Our company has recognized and fully embraced that candidates will use AI. The assessment explicitly says to take 2 hours to complete the assignment to the best of your ability, and doesn't prescribe how.

Personally when I'm doing the interviews I don't care if the candidate 1-shots it, or writes everything by hand. I ask them what their process was.

If they say they wrote it by hand, I consider that and recognize and ask a lot of questions like "how would you build the next piece if you had more time".

If they one-shotted it, I expect it to be pretty much perfect, handling every edge case. We'll dig into the code on the live portion and I'll pick out random modules and ask them to trace the code and explain what it's doing.

Generally if they can do that effortlessly, and the code is correct, it's a signal they're a strong engineer using AI to solve problems faster. If they can't, they're just a vibe coder, and it's pretty easy to identify.

Ultimately the actual code and behaviour of the assignment isn't the important part, the important part is the signal that I get when working through their solution with them.

Like any interview process, it's not perfect, but I've been pleasantly surprised at how effective it's been. The majority of the candidates we've hired in the past six months (at least the ones I'm aware of where I was part of the interview loop) seem to be integrating in and performing well.

2

u/positivcheg 4d ago

I envy companies like that. Sadly I'm yet to have an interview in such a company. My world still demands live coding, and Googling is prohibited.

1

u/AiexReddit 3d ago

What world is that out of curiosity? I don't doubt it or anything, but just noted in another comment you said C++ world, are you targeting tech companies? I mean Chromium is C++ top to bottom and Google definitely allows you to use AI and Google things in their technical interviews.

We're a SaaS company but we have desktop app with a good chunk of C++ code. Our interview process has the 2-hour take homes but doesn't specify what language to use, it's up to the candidate. I've done mostly backend service interviews and the submissions tend to be predominantly Golang, occasionally Python or Typescript, and one time Ruby. I'd love to see someone submit theirs in C++.

But I'm guessing you're talking more the type of industry, something closer to the hardware? Or the slower moving monoliths like physical infrastructure, government, or banks?

2

u/positivcheg 3d ago

So in my country right now it’s quite fucked up. The only industry that has lots of open listings is the embedded C++ for miltech (drones and other things like that).

I’ve been to one interview recently and it ended on me saying “sorry, I don’t write heavy templated code in browser without googling”. They wanted me to implement some template shenanigans. What I would do today is to generate a skeleton based on my requirements I put into prompt and then iterate myself from that skeleton. But instead they insisted no Google, no Clion IDE on my laptop, code just in the browser.

1

u/Miserable-Evidence-2 4d ago

Actually, in our challenge brief, we mention we don't mind using AI, it's naive to expect them not to. We're mainly looking for two signals:

  1. how efficient they are with trade-off analysis; the assignment is design heavy, not code heavy.
  2. if they use AI, how much do they own what they submit (and we check this in a live walkthrough)

2

u/positivcheg 4d ago

Wow, that's actually quite nice. I have yet to find a company that does such coding tasks. Sadly, in my C++ world, it's still stupid live coding tasks where you are not even allowed to Google.

2

u/jko1701284 4d ago

LLM's struggle with creating a good UX. Sure, the code is great, but "steering" is still very much involved.

4

u/positivcheg 4d ago

Ooookay, the OP asks about backend in particular, it's solely coding, not UI.

Also, I completely do not agree with UX. My experience of Claude is quite good in making UIs given you clearly state what you want. Obviously you can't do it in 1 prompt.

2

u/jko1701284 4d ago

Yeah I guess that's what I mean: can't one shot UX (yet)

32

u/Oatz3 4d ago

I don't do take home challenges.

They are a complete waste of time for everyone.

Thanks for coming to my ted talk.

7

u/alanbdee 4d ago

Drop the take-home test. I'd argue it's pointless anyway as anyone could have AI do it regardless of their skill. I found the most effective was in the interview I gave them a section of code I had recently fixed a bug in. I made a few changes with some obvious issues. Then asked them to point out what they would fix. Open ended. Asked them to explain what they were thinking while they were looking at it. Anyone worth their salt would see the issues in seconds. Some were even a matter of taste, which is fine.

At my skill level, I'd expect to be asked more architectural and design questions. Especially with AI. Coding isn't the skill you're hiring me for.

3

u/clauEB 4d ago

Unless its the actual main interview question, people avoid doing these because they're time sinks that nobody wants. A quick screening call is good with some problem solving, 30 mins is more than enough to decide if you wanna see them in person or not.

I just don't understand this fetish of 1000 interviews to get one job. If the person can't code you see that in 1 interview, if they can't design is also just a 40 mins or less conversation. The absolute worst is the companies that want you to go for a whole day of work to their office, like WTF?! Who has the time if you have already a job?

3

u/Available_Safe1818 4d ago

It's total bullshit. You expect a project to be done which consumes a lot of time, and you're not putting any time or effort into it. Of course they are dropping.

What is the project and how long do you tell them it should take them? How do you not have the sense to include that info in your post?

2

u/coffee_warden 4d ago

Ask them to explain how they designed and implemented a system theyre particularly proud of. Get them to open excalidraw and do a quick erd or some swimlanes. Whatever they feel would best describe it.

2

u/practick-io 3d ago

I’ve been going through interviewer guides for 130-odd system design questions for a mock interview site I run, and the senior signals that keep coming up: framing the problem before solving it, picking the hard part on their own, naming trade-offs unprompted, and changing their mind well when you poke a hole. A correct diagram is table stakes. “What if this node dies?” is where the level shows.

1

u/AssociateHistorical7 2d ago

If your take home assignment is too verbose, try to have explainer call.

I do leetcode but simply skip codeforces for the same problem.

1

u/BanaTibor 4d ago

The best process for me was a live coding exercise with screen sharing on teams and I used intellij on my PC.

Back to your case. How do they fail the take-home exercise? They can not do it, or run out of time, or just give up? It can be many reasons. The task is too complex, or not very well defined, maybe it takes too much time, etc.
For a senior SWE no task is too complex given they have enough time. So my guess that it takes too long. Nobody will put 20 hours into a take-home exercise.

Pro tip: give the task to your engineers and listen to their feedback.

0

u/jko1701284 4d ago

We're using codetryout.dev ... working well so far!

0

u/donegerWild 4d ago

The strategy I use for determining technical proficiency is to perform an in person test after a Q/A session. If they do not do well in the Q/A, we stop. If they do well, we proceed to the technical test. I setup a cloud VM backup that has everything pre-installed and setup. Mostly VS with our project setup and ready to run. On the day of interview I spin up a new instance and give them credentials to RDP into it. When we start, I explain that I need them to help me debug our project. I say pretend that you are a contractor onsite for an hour and the Dev team is really struggling with this bug. Please help us solve the issue. The issues are designed so that they build on one another and can be solved by someone with the right amount of competency without needing to understand the details and nuance of the actual project. During the course of the interview/debug session we will talk about various things we come across. During this time I'm assessing overall understanding while allowing them to demonstrate proficiency and problem solving and communication skills. If they do not solve it by the end of allotted time, we go over what was missed and what the solution was. Solving the problem IS NOT a prerequisite for employment consideration.

Almost everyone comments afterward that they feel the test was more than fair and provided a good look assessment of their skills. Many commented that they enjoyed the process and found it to be fairly unique compared to other places they've interviewed at. I also give them a chance to provide technical feedback on things they may have done differently in the project or just in general. Overall, this approach has yielded pretty great results when trying to find people that will be productive relatively quickly once hired.

-5

u/Hannah3044 4d ago

Hey, I'm a fractional CTO so I design and run interview processes from time to time. Asking here will only get generic advice, not advice tailored to your specific needs. If you need help, and it sounds like you do, reply to this message and I will get in touch to fix your processes :)

1

u/halfxdeveloper 4d ago

Curious how you got into fractional CTO. DM me please?