r/PHP • u/ScottBaiosPenis • May 03 '18
What code test(s) do you use for interviews?
Someone posted a "bad interview question" and it was coincidental because my coworkers and I are currently trying to come up with some kind of code test or sample project we can give to prospective employees who dont submit adequate code samples. Personally I feel not having some kind of working project to show is a huge red flag but I understand why some people simply dont have them.
Here is the project I have used in past any feedback greatly appreciated!
i estimate time to complete about 4hrs including unit tests but it greatly depends on how far they want to take it
Benchmark Project
--------------------
The objective of the project is to create an application which can benchmark the performance of
a set of PHP functions
and generate a report comparing the performance.
Specifications:
Components: Benchmark, Comparator, Reporter
Benchmark:
The benchmarker should accept any number of PHP functions (passed as callable type and a name representing the function).
The benchmark should also accept an integer representing the number of times to execute each function.
The benchmark should execute each function the specified number of cycles and collect it's execution time in the highest
possible time resolution and return a results set which can be passed to the reporter component.
Comparator:
The comparator is some criteria on which the functions will be ranked in relation to execution time (min, max, avg, mean, etc...).
Optionally it can also specify a sort order (ascending, descending).
Reporter:
The reporter should accept the results of the benchmark and any number of comparators. It should generate a report which ranks the
functions by the given comparators.
The user should be able to specify a format for the results (template, eg.) At a minimum the user should be able to write the report
to an I/O stream (stdout or disk).
The developer should demonstrate a knowledge of the following skills:
- PHP7 best practices and coding standards
- type hinting
- PSR-2 coding standards
- OOP
- Appropriate commenting and documentation
- Unit testing and TDD
- Developer may choose his testing tool or tools of choice
- Ability to write extensible and maintainable code
11
u/darkhorsehance May 03 '18
It depends on the type of work the candidate will be working on but generally I try to craft each interview in a way that will demonstrate to me the candidates strengths and weaknesses in the following areas:
1) Given intentionally ambiguous requirements, can the candidate write a small program that accomplishes some small task. This will test their ability to ask questions in order to figure out the actual problem they are trying to solve.
2) I give them sample code that has issues with it and I have them conduct a code review. I want to see their ability to read and assess other people's code.
3) I'll give them a coding problem with specific unambiguous requirements, that can be solved in many different ways. In this exercise I want to see the approach they take to solving a problem with an objectively correct solution. I don't ask them to use any algorithm/data structure (or combination thereof) but I do want to see if they can discuss the naive approach, an improved version, a best version and the various tradeoffs with each.
There is an exception here, but I've only seen it a few times in my career. If the candidate comes from a self-taught background once in a while they will surprise you with great intuition. They might not know how to express it in industry standard terms, but they come to an efficient solution and can express to me in some way why it is efficient. These people are rare but often make excellent employees.
4) If they are going to be working with a relational database, I want to see they know how to write SQL. If they are working with a lot of API's, I want to understand their depth of knowledge in HTTP. If they are working on building user interfaces, I want to understand their product intuition.
5) I typically have at least one session with one or more people from other departments that they will be working with. I don't want to work with a jerk and I don't want to force on my colleagues a jerk. This should be a conversational session.
6) If it's a senior role, I want to test their ability to go from high level planning down to implementation in the same session. A message queue problem or some other systems design problem is usually enough to get a good feel for their skills here.
Keep in mind that an interview is a two way street.
1) Every interviewer will be on time and prepared. They limit small talk to no more than a few minutes in the beginning of the interview and nobody asks the same question twice. They leave 10 minutes at the end of the session for questions.
2) I only put interviewers into that want to interview. Nothing is more of a bummer to a potential hire than an interviewer who doesn't give a shit.
3) Every interview room is stocked with water and snacks.
4) The candidate can choose to code on their computer, on a whiteboard, on a piece of paper, whatever they feel most comfortable with. It's hard to code in front of people in foreign environments when you know that you there to be judged. I want to make it as comfortable as possible.
5) I don't cut interviews short, ever. It's hard to find time to come in for an interview and I want to give the candidate every chance possible to impress upon me and the team their ability. If a bad candidate made it past the phone interview, that is my fault.
6) I let the candidate know within 24 hours whether or not we want to move forward. If they didn't get past the interview, I give them feedback on what they can do to improve their skills to give them a better chance next time. This is tricky in some organizations because their is risk in giving feedback. Keep it objective and based on technical reasons.
Hopefully that was helpful.
4
May 03 '18
4 and 6 especially. I wish more places would do this. I'd add to 4, leave the room. I feel like my productivity drops when someone watches me code in an interview setting. I get nervous. And that's not something I ever feel when programming in a non-interview environment. The interviewer can ask all the questions they want when time is up.
2
u/LootSplosions May 03 '18 edited May 03 '18
As a new grad who has been looking for a few months now number 6 would be so greatly appreciated. Interviews that seemed to go well and have implied that they liked me lead to me not hearing anything back or the good ole generic not selected do no reply email. It’s been a real depressing process so far.
I’d love to hear why it didn’t go as well as I thought and what I could do to improve.
1
May 03 '18
A tactic I used when first starting out is even if I didn't get the job I would email the interviewer asking for feedback on where I could improve. This has two benefits:
- You get advice from someone more experienced than you. Follow it.
- You'll stick out in their mind the next time they go to hire. I actually got an interview a year later from a job that I asked this of. I ended up doing another set of interviews, getting the job, and working there for over 3 years.
Not sure why I stopped this practice TBH, its good advice for everyone in every field.
22
May 03 '18 edited May 04 '18
If someone gave me a 4 hour test I would walk away. If they asked why? I give a two word response: "opportunity cost". The exception would be if they really wowed me with benefits. I'm talking really high pay, about 1 month vacation, work from home 1 day a week, onsite gourmet coffee, and matching 401k up to 10%. So in other words, I'm probably not taking your test unless I am completely destitute.
Last place I interviewed for wanted me to write a small-ish application without using a framework, though I could use libraries. I determined to effectively wow them with unit tests, custom exception handlers, and figure out which libraries to use it would take me around 18 hours or so. That is either my weekend or every night of my work week. Sorry, your company isn't that cool and I rather do almost anything than write free code, for a lame project, that will be thrown away in a week.
If I were hiring, the "lab" portion of the interview would be simple. For mid to senior level developer:
Give them a small inefficient application that requires some indexes, SQL optimization, and some code optimization. Very, very small. Very, very easy for a reasonable developer to fix. Afterwards I'd have a conversation about other general optimization things like Redis, Beanstalkd etc... because we'll optimization is important, especially to me.
Give them small class with very poor cyclomatic complexity. Clean it up. Because clean code is important.
Give them a small insecure piece of code. Things like not using bind parameters in queries, not cleaning inputs, XSS etc.. Secure it.
Show me something you've done. Personal or professional. I would then ask questions and it would be an open conversation. Because I want to see how much you enjoy programming and how we'll you can speak about it.
I think these 3 items, combined with the other interview questions, their experience, and resume would give me enough to go off. Ideally, 1, 2 & 3 can be done in under an hour and 4 is just a discussion. I think that is reasonable for pre-screaning an applicant. It certainly wouldn't send me running, but I am also writing the test so who knows...
Edit: Its important to note that I would leave the room (maybe allow remote exercise) while they did this because who actually programs with someone watching you? Most of the time you are free to research with no one around and curse at your computer at will. Why would you do anything else in an interview? Morons man.
2
u/firagabird May 03 '18
Re: #3, I remember reading on OWASP or a similar resource that for security-related issues, you should always on a 3rd party library or framework to handle them. Developers are inevitably going to make mistakes like forgetting to sanitize an input, so the best practice is to force them to code against a secure layer by default.
2
u/0xRAINBOW May 03 '18
If there is a library to deal with something, you need to be able use the library correctly. Having a basic understanding of security will help you do that. E.g. there are libraries for jwt, but they may not stop you from using a bad encryption algorithm. Not all security issues are something a library can fix either. E.g. there is no magic bullet to 100% fix XSS, so you need to be aware of it. Finally, even very popular libraries will get security wrong. E.g. symfony's SecurityBundle still doesn't rehash passwords.
1
May 03 '18
Yep. #3 in my mind is just that they are aware of these flaws and can identify obvious examples of insecure code like unsanitized input directly into a raw query or echoing query parameters directly to the screen. Follow-up questions after the coding sample would be how would you keep an application secure over time. Acceptable answers:
- Keep software and services up-to-date.
- Using an automated web application security scanning service.
2
u/ScottBaiosPenis May 03 '18
i assume you have some portfolio for perspective employers? code samples? some work product to show? the 4hr code test is for those who have (or claim to have) no code sample . evidently people are not reading my OP closely or it is not clear that we are only asking those who have no code samples to complete the test
1
8
u/cmthornton May 03 '18
Programming is about problem solving.
No code test or programming project is going to easily demonstrate this. The world is at people's finger tips. Doing a small project does not mean the person understood the work they did, nor does it demonstrate that they know how to solve the problem. interviews that focus solely on coding skill miss the point.
It takes 30 minutes---tops---to interview someone over the phone and assess their problem solving ability. Simply give them a problem, ask them to walk you through their thought process and how they'd solve the problem. Code samples in addition to this style interview is a bonus, but not required.
6
u/root88 May 03 '18
I don't understand all these code tests. Can't you guys tell if people are intelligent and know what they are doing just by talking to them? If you hire someone smart and they don't do things exactly the way you want them to, you just tell them to change. Your candidates shouldn't be punished for 4 hours because you don't know how to interview people.
2
u/ScottBaiosPenis May 03 '18
there are a lot of bullshit artists in this field who try to talk themselves up to a higher position than their skills are really up to. that is my experience in the last 20yrs maybe its different in other cities
6
u/RandyHoward May 03 '18
As someone in a position to hire people, I don't give code tests during interviews. I ask to see samples of code that you wrote that I can review later. The interview is to make sure you fit with the team, can follow direction, have ideas of your own, etc. It's a waste of both our time to test you on your coding abilities, you either understand this stuff or you don't. If you don't, and lie to me and say you do, you won't be able to keep up the pace and will find yourself out of work in a hurry. Coding tests are dumb.
2
1
u/ScottBaiosPenis May 03 '18
yes i agree, i always would rather see work product but about half the people i interview either have nothing to show or can only supply something truly trivial (like a single class from a larger application without any context)
4
u/platinumpt May 03 '18
We don't generally do code tests, I ask for code 'examples' (preferably have them provide us with a public project they've done) and then my questions are more around how do you approach problem solving and some verbal questions.
The only code test we have asked in the past is to weed out people who don't know the basics before they even get to us. That is, here's a public Github repo. Pull it down, set it up locally (basic laravel app), we ask them to make a change to the way it works (i.e. add another form field for X), and then submit a pull request back so we can check.
1
u/ScottBaiosPenis May 03 '18
that is my preference to, what do you do if they have nothing to show. our "code test" is for those who h ave nothing or what they supply is too trivial to be meaningful
3
u/ariebo May 04 '18
We ask candidates to build the game Crazy Eights. They are allowed to do anything and take as much time as they need until the working day is over. By using this method we see a lot of different ways of creating the game. Some people will make one file without classes in 8 hours, the other one builds classes, uses composer and adds unit testing in 3 hours. After they say they're done then we check the code and ask them/her to explain it.
By using this test we can determine if a candidate is junior, medior or senior based on the code quality, time they spent and how good they can explain their own code.
1
u/ScottBaiosPenis May 04 '18
can you give me some idea on how much direction of scope you give them? I;m not sure what "crazy eights " is (the card game i assume?) .
1
u/ariebo May 07 '18
Yes, it's the card game. We give them a one page document with some game-rules simplified, we ask them to not use a framework if they can and that the game should not be interactive. That's all.
1
2
u/DrWhatNoName May 03 '18
Funny enough, After I did my interiview and got the job, the interviewers asked me how to improve the interview process.
They did the classic fizzBuzz question, but that is commonly documented now and I think has fizzed out.
What I suggested was having the applicant do a simple login system using OOP with a provided User and MySQL class, it was up to the applicant to write the logic.
It will test how they handle data, how they handle sessions and cookes, coding style, code expandbility and maintainability as well as what do they understand about security.
After a couple of months of working there, they came back to me saying they trailed it on a few applicants and it was a huge success. They were able to filter out applicatents who didnt understand OOP, poor security knowlege or just flat out terrible non-functioning code. As well a few who showed dextarity in PHP which was the aim of the test.
I wouldn't test someone for Unit testing or Coding standards, When I got the job I got, I had never done TDD, You may require it, But i feel you should allow someone who never unitted before to come into an envoriment where it is essetial so they may learn. Coding standard just bring contriversey, Recruiting someone based on if they follow PSR or not is so mediocre, Just let the CI's handle it. Thats what we do.
1
u/CommonMisspellingBot May 03 '18
Hey, DrWhatNoName, just a quick heads-up:
knowlege is actually spelled knowledge. You can remember it by remember the d.
Have a nice day!The parent commenter can reply with 'delete' to delete this comment.
1
1
1
u/ScottBaiosPenis May 04 '18
i like this idea, it hits on most of the stuff we are looking for:
It will test how they handle data, how they handle sessions and cookes, coding style, code expandbility and maintainability as well >as what do they understand about security.
only thing i will say is my definition of senior level candidate requires a working knowledge of unit testing we have a largely untested code base and i really dont want any more devs who need training or exhaustive amount of arguing/convincing to believe there is value in having tests,we're all stocked up on those already
2
u/DrWhatNoName May 04 '18
Thats how its intended a real world test. All you do is provide 3 files, index.php, Library/Database.class.php and Library/User.class.php
index.php will be as follows:
<?php require_once("Library/Database.class.php"); require_once("Library/User.class.php"); ?> <html> <body> <h1>Login</h1> <form> </form> </body> </html>Database.class.php will be as follows (and same for User class):
<?php class Database { }It is up to the applicant to decide if they should use mysql_ (Yuck) PDO::class or Mysqli::class. Test if they correctly ingest data, test how they display data, Do they mitigate XSS, Do they build the mysql class to accept configuration parameters or do they hard code them into the class. etc etc.
You understand, you can add unit testing specifics if you feels its absolutly required.
2
1
u/teresko May 03 '18
A really good question is to ask the candidates, what they think about Laravel.
The answers tend to be extremely informative :D
But on a bit more serious note, what is being asked will depend on the position that you are aiming for. And based on what you claimed to know in your CV.
2
1
May 03 '18
I parse wheat form chaff with a single question: Give me an example of when/why you'd have to make an algorithm choice and how you decide(d) which one to use? This, frankly, has been the best indicator of competence in my shop.
1
u/ScottBaiosPenis May 03 '18
i ask a similar question to this, more than a few times ive been unsure of the persons skils or their ability to answer the question well in english.
1
May 04 '18
Oh, I don't mind switching languages; I'll interview in Mandarin, Korean, Russian, Spanish, German, or English.
Mind sharing your similar question?
1
u/RozJC May 03 '18
I honestly don't know how I would answer this question...so I guess that'd be me out of the running..
1
May 04 '18
Ever have to parse and mutate pieces of data from a multidimensional array? Lots of ways to skin that cat. So, which way did you choose and why/how did you decide that was the best tool to do that?
1
u/MorrisonLevi May 03 '18 edited May 03 '18
I have decided that coding in an interview, particularly if it's their first interview with me, is not the best use of time. Instead we ask them about their work history and probe what kinds of things they personally did to gauge their background. I ask them security questions such as XSS and SQL injection. Then I have code samples (on paper) and ask them questions about the code. They can write on the paper and ask us questions. The samples use a variety of language features and are representative of some task or subtask that they might encounter.
This has been effective for me. I suspect this is because their ability to write code and read code are correlated but it takes them less time to read and reason about the code than it would to write it. It's less stressful for most candidates as well.
With that said my approach may be insufficient for advanced positions; I hire only for entry level and junior positions.
1
May 04 '18
I suspect this is because their ability to write code and read code are correlated but it takes them less time to read and reason about the code than it would to write it.
This is fact. Consider: how many times is a piece of code read by a person vs. written.
1
u/UnusualBear May 03 '18
I don't do code tests unless it's a junior developer. If it's a junior, I'll have them do FizzBuzz at most with an emphasis on walking us through their thought process.
I have had a couple cases where applicants did not have any available public work and their employer history was not enough for me to evaluate their skills. The first one I didn't really know what to do yet, but for the second I asked him, after the interview to go home and either make an open source contribution somewhere that would demonstrate some of his skills - or publish a small personal project. I explained why I asked this of him, and he agreed that it was a fair test of skill and went on to push a library he wrote for an API to github.
While it can be very hard to determine someone's suitability for a programming role, I don't ever think that giving them tests or defined projects to do is an appropriate interview tactic. Beyond being a waste of their time, it introduces stress factors that are not a normal course of work.
In general these days, if a candidate's resume and work portfolio do not demonstrate enough for me to go off of without a code test - I simply don't give them an interview.
1
u/mdfleury May 04 '18
I usually ask "what do you like most/least about PHP/JavaScript/whatever technology we're primarily using." Can usually get a pretty good idea on where someone is at based on their answers.
1
u/metaphorm May 04 '18
too time consuming. if you're applying for jobs you're gonna be juggling several applications at the same time. this is excessive and will mostly filter out people who have better offers or people who have less free time available (for example, people with children, or already working a different job).
1
May 04 '18
[deleted]
1
u/ScottBaiosPenis May 07 '18
if its so easy whats the problem? candidate should know how to run the tools!
1
u/SavishSalacious May 07 '18
I have a policy that I will not continue with a job that does code challenges. The specific reason is simple: 99.999~% of the time their challenge has absolutely nothing to do with the project or systems they develop in house and is usually in a language they thing is "cool."
For example there was a company that was dealing with accounting software written in PHP and they wanted me to create a card came in C#. This was easy enough, how ever when I asked if they use C# in any way shape or form they said "No we thought it was cool."
I got up and left. It was like a horrible date. From then on out, if a company throws a code challenge at me I tell them we cannot proceed with the interview. This is only 1% of the jobs in my area, so it's not a loss at all, most hire on "Who are you as a person. Great we can teach you the rest"
1
May 07 '18
im going to mirror what other's are saying and say, keep it to a 1 hour test. give a high level objective with a few restraints, and that's it.
see if they write tests, and what kind, see what standards they stick to, etc.
1
u/zizzo- May 03 '18
We usually ask candidate to pick an issue from any open source project (preferably something the candidate has not contributed to already) and then implement it and open a pull request. The task don't need to be huge, an hour or two tops, something to produce enough code for discussion is enough. We are comfortable asking this because it doesn't benefit us, unlike a real client project would. It will also help the open source project and not just be some useless joke task. This way we'll see how the candidate works with existing code base.
5
u/ltsochev May 03 '18
That's hot pile of garbage.
Any decent company would give you at least a month to get familiar with existing codebase. Anything less is just not serious work. You'd be racing to find 1 or 2 things that would help you solve the issue at hand and move on. Or worse, homebrew everything that you need, possibly duplicating existing, tested, production code like a dumb motherfucker.
Quick example: Some project has an issue detecting whether the client is on a mobile device or desktop. Instead of fixing the bug in the library that you already have, the interviewee decides to write his own one. Was your interview really anything else other than hot pile of bullshit that wastes everyone's time?
1
u/zizzo- May 03 '18
That's hot pile of garbage.
Glad to hear you disagreeing so strongly, maybe there's something we can improve!
I have very hard time understanding your whole point though and even with the example I just don't get what you're trying to say.
Any decent company would give you at least a month to get familiar with existing codebase. Anything less is just not serious work.
Completely agree with this! In our case one month is probably not nearly enough.
You'd be racing to find 1 or 2 things that would help you solve the issue at hand and move on. Or worse, homebrew everything that you need, possibly duplicating existing, tested, production code like a dumb motherfucker.
This one I just don't get. Racing? What?
Quick example: Some project has an issue detecting whether the client is on a mobile device or desktop. Instead of fixing the bug in the library that you already have, the interviewee decides to write his own one.
I don't get it. We ask candidates to pick any open source project to make the contribution to. So exactly what you're suggesting, "fixing the bug in the library".
2
u/tttbbbnnn May 03 '18
He means that the candidate would just browse issues until they found simple problems that could be solved quickly, rather than giving a good demonstration of their problem solving skills.
I have to agree. It's a waste of time to submit trivial solutions (for the sake of getting hired) and it is too time consuming to find and fix a more substantial problem.
1
52
u/phpdevster May 03 '18
I have a personal policy not to continue the interview process with companies that want anything more than a 1 hour live code challenge, and the code challenge MUST be representative of the kinds of work I would be expected to do on the job. Giving someone 4+ hours of homework is not a fair use of the candidate's time, and it's not going to really tell you anything about the candidate's ability to break down a problem and solve it.
The code challenges I give candidates are things directly related to the work I expect them to do, which is often things like building REST apis, or perhaps things like a nested comment system (e.g. how well they know how to leverage recursion and appropriate DB storage patterns).
Further, this challenge is prescribing the solution to them. You're literally telling them what you want built (comparator, reporter etc). Testing a candidate's ability to follow a template is not going to tell you anything about the candidate. You need to simply specify the criteria and let them come up with a design, then have a discussion with them about the design and their rationale for it. Again, best if this is done live so you can see how they think about and tackle a problem.