I’ve been in a fair few interviews in my time, that’s for sure. Some good, some bad, some horrific. Sometimes that horrific situation has been my fault, and sometimes it’s been the interviewing party.
Anyway, over the years, I’ve learned what I like and what I dislike in a technical interview, and I thought I’d share some these thoughts. As a jobbing engineer, I like to think I’m kind of everyman. I have some math – I understand trig and all the rest of it, but I don’t spend that much time thinking about Big O notation (for a good reason – I wait for the profiler to tell me I’m spending too much time in a function before I worry about it’s speed) or stuff like that.
This is a long post – a big wall of text -, but hopefully it’s worth it.
Obviously some of this stuff is wish list and in some situations isn’t possible – preparation, for example, can only happen if you are forewarned. But most people are, and still don’t do it and it just smacks of disrespect.
1) Prepare. Don’t walk into an interview without having looked at the resume of the person, and at least looked at the websites of places he’s worked. If there are things to see – a blog, a set of lecture notes, whatever, then look at them. From the interviewee position, there is nothing worse than the guy you need to impress rolling into the interview and desperately looking at a resume that it’s obvious he’s never even seen before.
Far too many interviews are basically just the interviewer reading a resume and groping for questions to ask about each of the jobs you’ve had. It smacks of disrespect, and that’s a killer in terms of the person trying to get the job actually wanting it.
2) Have some questions prepared. And I don’t mean of the technical variety, I mean of “Who are you?” variety. You are likely to be working in close proximity to this person – and may even be working with them directly – and as such, you need to understand if this person is someone you can work with. Whether they are likely to arrive stoned, or drunk. Whether they have flash points regarding C++ Code notation layout. Whether they are going to drone on and one about the band Gwar etc.
Basically, everyone need to get a handle on cultural fit for the individual, way beyond the root question of “Can they do the job?”
Some companies break interviews down into separating out different parts – person A does technical acumen, person B does cultural fit. This is not a good approach. What it does is concentrate all of the potential elements that go towards a “yes” answer in one persons hands. If one person doesn’t like that the interviewee doesn’t know dot product, he’s now in a position to completely torpedo that application, regardless of what everyone else thinks.
Now obviously not everyone can do technical acumen – you need to be an engineer to judge that – but everyone can ask some questions about what the candidate is into, what they like to do on their own time, what their favorite repository is, biggest mistake they’ve ever made and why they thought it was, and what they’d do about it now etc. Questions that talk to what that candidate is – or at least thinks they are – rather than just “Can you tell me the code for Dot Product?”
NOTE: Be careful that your personal questions aren’t too personal. Run them by other people (or HR) first, just to be sure you aren’t contravening a law when you ask about the candidates family (in actual fact, that’s a no no anyway).
3) Do some of your technical questioning before the person comes into the office. By the time they get to the office – and are taking up resources time to sit and talk to them – you should already be fairly confident they can do the job from a technical point of view. Make them do a test – and don’t ever do that test while they are in the office. That’s just wasting everyone’s time. The fact is, you’ll probably get one day to determine if this person is going to fit into your team. One day. So the last thing you want to do is waste it, putting that individual in a room by themselves, while they knaw on a pencil trying to remember the syntax for using STL.
It’s also worth pointing out that tests like this are entirely artificial. The fact is, if you employ them, they are not going to be working like this day in day out. They aren’t going to be standing and writing code on a grease board in front of three board mid level engineers. They’ll be looking stuff up on the internet, trying stuff, refactoring it and so on. The only thing that on site tests really tell you about a candidate is whether or not they do well in high stress situations where you’ve got a bunch of other engineers standing watching you and judging you, and where you can’t look up stuff. If that’s how your company operates as a norm, then you’ve probably got more problems than worrying about how your interviews go.
4) Your technical questions are not a competition, nor are they an excuse to hand out “Find me a pebble” questions.
I’ve had situations where I’ve walked into the interview and it’s become instantly obvious that the person on the other side of the table has their own self worth issues and needs to prove some degree of mastery over me instantly in order that they dominate in some way. I literally had one guy correct the very first thing I said – where I was being generic – and I just looked at him. In that instant I knew I wouldn’t take the job even if I was offered it, if it meant working with this guy. Any human who feels the need to establish a pecking order in the first sentence is not someone I want to work for/with.
Also, when it comes to technical questions, a lot of engineers either have their own pet questions to ask – where in reality what they want to know is “do you think like me?” They have a question that can have many answers and they have a favorite answer and if you don’t get it, well, you’ve failed. The reality is you haven’t read their mind or come to the same conclusion they would have, ergo you aren’t as good as them.
The bottom line is, I probably know as many really-hard-to-answer questions as you do, and I can ask them back, because my time is valuable too. If all you want to do is establish a pecking order, well, I can do that too, and make you look as stupid as you are set on making me look. Then what?
For many engineers – particular those with fragile egos – interviews are about proving they are better than the candidate (mainly because a lot have a worry that they are not, even though they’ve already got the job) – even more about discovering if the candidate is good in his own right.
The fact is, even if he gives you a different solution, that’s ok too. Explore that. Don’t get frustrated that he hasn’t come to the solution you would. Thinking differently is ok, and will almost certainly get you out of a jam some day, when you can’t see a solution and someone else can.
Also, be sure that your questions can be worked out by the candidate, and don’t require an either “you know it or you don’t” question. Examples of that are asking someone to write an STL vector prototype. If you don’t remember that off the top of your head, then you’ve failed – because there’s no way to work it out. This is why some of the questions from Microsoft in the 90′s were such a hit – asking “How many ping pong balls fit in a 747?” for example, or “How many gas stations are there in Seattle?” – these are all questions where it’s more about working out ways to get an answer than it is about what the answer actually is.
The more a question relies on some esoteric knowledge, the less valuable it is, because what it’s really testing is “Has this candidate come across this before” and not “Can this candidate work it out if he’s not come across this before”, which is what you should want to know.
And while we are on that subject, the more obscure the knowledge you ask for is, the more you as an interviewer look like a prick for asking it – this goes along with with point 1), where I’ve seen a lot of engineers, struggling to retain coder dominance in the interview, resort to more and more weird and specific edge case questions in a way of tripping up an interviewee. And lets not kid ourselves, this is exactly what it is – it’s not “exploring the depth of this candidates knowledge” – it’s a way of the interviewing coder feeling better about himself because he knows some obscure fact that the interviewee does not.
Also, asking for equations sucks. It just does. No one writes equations in code. If you need to know if someone knows the code for line intersection, don’t ask for the equation. Ask them to write the code.
Which leads me on to…
5) Ask for what you want. Don’t be clever and ask a leading question and hope to gently push the individual into what you really want. I had a graphics engineer once ask me for an early box discard algorithm – which is 4 if-then’s and that’s it; can’t get much quicker than that – but he wanted something else. In the end, what he wanted was a binary search algorithm. It took half an hour of me trying to figure out what it was he actually wanted. When I finally worked out what it was, I gave him his binary search algorithm, and then said “Look, if that’s what you wanted, then ask me for it. Don’t make me spend half an hour wondering what it is you want.”
It was done purely for the sake of the engineer in question doing it, and I have to say, it pissed me off no end.
Convoluted and obscure questions just waste everyone’s time and you may think you are clever for asking it; more than likely you are making the candidate roll his eyes internally.
6) Sometimes, knowing what the use you put something to is more important that knowing how it is generated. For example, Dot Product. If an engineer can describe what a dot product is (e.g., it’s the difference of angle between two vectors, described as from -1 to 1.) and give you four or five usage cases for it, that is way more important than being able to write the code from memory.
Some engineers put great stock by remembering formula by rote, and while there is some value to this, again, this is not normal day to day coding situations. If I don’t remember the code for a dot product, I go look it up. I spend some time comprehending what it is, write the code in my way, and forget about it until I need to do it again.
There is an argument that if you remember the source code, you ‘know’ the code, but that’s not actually real. Most coders who remember the source code couldn’t tell you what happens if you feed a dot product two vectors that are *not* normalized. If you can’t do that, then you don’t understand the root math.
Questions like this tend to show up a lot because they are easy to validate – either you know it or you don’t. Questions that really plumb how someone works are far harder – and require a lot more effort on the part of the interviewer – to set up.
The answer “I’d look that up” IS a valid response, even if you are mortally offended that someone doesn’t know the exact same things you do – or worse still, doesn’t associate the same value to it that you do.
7) Understand that your value system – of what’s important or not – is not going to be the same for everyone. Value systems are generated via experience, and by definition, everyone’s experiences are different.
Therefore, quite a lot of people will have very different reactions to what they see based on their experience – and sometimes the situation they are in too. Reacting to finding an incomplete API is very different when you have a deadline to get something done by, and find it’s blocked by this, than it is if you are just perusing the API docs and find something not quite right in it.
The problem with engineers is that they tend to be very black and white in what they believe. Way X is the “one true path” and everything else is just not. Any one who comes at it from a different point of view is therefore wrong, and to be argued with, in order that they see the error of their ways. Missing the point entirely that a) no one is likely to be swayed by your superior knowledge (just pissed off instead) and b) sometimes there is more than one way to skin a cat, and that’s – god forbid – ok.
8) Understand the guy in front of you is sweating, and wondering what it is that you want to hear, and will attempt to give it to you, regardless of the reality of their beliefs.
It’s an unfortunate statement that most people who are interviewing need the job, and therefore will contort themselves to be anything they can discern that you need. If you ask them leading questions, they’ll pick up on it and then try and mold the answers to whatever it is that they think you want.
E.g. “How do you feel about unit tests?” – that’s a leading question, and the right answer here is obviously “They are great, I use them all the time“. And that’s almost certainly the answer you will get, regardless of whether the candidate hates them with a passion or not.
You just aren’t going to get the truth by asking questions like this – that’s the reality here.
It’s taken me 25 years of interviewing to understand that I need to be myself in an interview, and not try and be what the interviewer necessarily wants. If I’m hired, then I’m going to end up being me anyway, and if that’s not what they want or can deal with, then I’m better off knowing that immediately rather than uprooting my family and moving across the world to take this job, and then find out that I’m really not a fit.
I don’t want to give anyone a false impression of me, so I have to be myself. If it’s not a fit then a) either I’m really not a fit or b) the people interviewing me really don’t have a good understanding of what they want. Either way, I’m probably not going to be happy there, and neither are they, so bullet dodged by both parties.
But most people aren’t like that, unfortunately (at least, *I* think it’s unfortunate). They want / need the job too much, so they’ll twist themselves into whatever pretzel they think the interviewer wants and then get the job and have a disastrous time because they are now going to revert to type. Life is too short for that I’m afraid – but most people haven’t gotten to the point of having it happen to them repeatedly, and haven’t learned that lesson.
9) encourage them to ask questions.
And what I mean by that is not “Do you do crunch here?” The point I am trying to make is that they have as much right to test your technical acumen as you have to test yours.
A lot of engineers don’t get that an interview is a two way street. They think “Hey, you are trying to get a job here. You have to impress me and not the other way around. I’m the one asking the questions.” Well, I’ve taken two jobs in my life where I’ve not done a technical due diligence on whether the people I am going to work with can actually get the job done, and in one case, it resulted in me getting stuck in Arizona for five years (and counting). Sure, I want to ask you about your technical abilities, what your development process is, how you QA stuff, what some of the biggest hurdles you’ve hit have been. If only to get an idea if some of the experience I have will be helpful.
Some engineers have their noses put out of joint when the candidate starts asking penetrating questions – this is an attitude that needs to be beaten out of people. They have as much right to know that you are going to work well with them as you do to know if they will work with you. The more that right is denied, the more you look bad as a potential employer.
Some thoughts on things that might be helpful.
I heard a story once about an engineer who, in his technical interview, sat down opposite the candidate, and then declared “I am a debugger. There is code that is crashing. It’s supposed to do X, but crashes when we do Y. What do you do?”
The interviewee then asked for more details, and the interviewer said “I’m a debugger. I can’t answer those questions. You can’t ask Visual Studio those questions and get answers – what would you do with Visual Studio?”
Eventually, the interviewee got the hang of it, and started asking things of the debugger that the debugger was able to answer – “What’s the stack trace?”, “What are the local variable values?”, “What does the source code look like?” and so on.
This is an excellent way of assessing how a candidate thinks, but while it’s great, it’s not without it’s problems too. The interviewer has to have all this stuff in memory and be able to reproduce it accurately, based on questions he cannot predict, and that’s difficult. One wrong answer and the candidate might not be able to solve the issue at all.
But, what it does tell you is exactly what the candidate would do – and it’s done in such a way that the candidate is more interested in what is going in in the interview than they are sweating out what they think the interviewer wants to hear.
One of the techniques I use now is to sit with a candidate, and describe a current problem I am having – what I know about the problem space, what the solution needs to provide and then say “Ok, what do we do about this? Lets work together on this.”
The idea is to mirror the experience he’ll actually have if the two of you collaborated on some design or implementation aspect, and get to see what he suggests. See if it’s off the wall, or what you were thinking or if he see’s issues you didn’t.
At root, you are attempting to see what happens in the real world of development at your studio, rather than the mostly artificial situation that an interview really is. How they work, how they think and how they collaborate.
Some great questions I’ve seen in my time include this gem – “Tell me, if I hired you, what would I most regret about that decision in six months time?” It’s a terribly leading question and really puts people on the spot because I guarantee you, they won’t have a prepared answer for that one.
The interesting aspect of that question is not what the actual answer is, it’s generally how long they take in answering it. Too long, and they are trying to find something that’s positive that they think you’ll accept “Oh, I’m too much of a perfectionist”. Or too little, and they are riddled with faults and are very quick to point that out.
Speaking of questions, the questions “Describe your greatest success” questions are interesting, but I’m not sure what they tell you about a candidate vis-a-viz the job in hand? “Describe your greatest failure” questions are more useful, particularly when followed up by the relevant “What would you do to avoid this in future” and “How would you fix that situation now?”
Something else I try and do is relax the candidate a bit. Most are pretty uptight – they need the job and want you to like / respect them. They aren’t at their best in this situation – almost no one is, so its your job to get them to relax.
At some point you need to say to the candidate “Look, there are no right or wrong answers here. There is no answer that is going to get you definitely hired or not hired. We are here to see how you tick and where your technical limits are. It’s not a problem that you have them – everyone does, and it’s all to do with experience. I’m just interested to know where they are. If you don’t know something, that’s ok – you aren’t being judged on that – I’m more interested in how you deal with something when you don’t understand it. It’s more a combination of everything. So relax, lets get some work done!”
Asking their opinion on something – asking why they did something, not in an accusing way but in way that asks them about their experience that brings out their conclusion – will grant you far greater insights into the candidate.
Also, ask them if they have questions and give them time to ask them.
Ask them if they want water or the bathroom. People do need to pee.
Last, remember, while you are giving this person a job and it means you’ll see them every day – which in itself is a big deal – for them it may well mean selling a house and moving a family across or continent, or even country. This is a big fucking deal for them. Please treat it as such.
The bottom line is, have some respect for the people who are sweating bullets in front of you, hoping to god they don’t say something you violently disagree with. Because they have no clue what you expect, what you want to hear or what is going to make you drop that black ball in the sack.