Hard Questions

We spoke a couple of weeks ago about examining your pet questions and putting them through a Talgo filter to see if they are really giving you the data you think they are.

There is one class of question that I want to draw special attention to: the “hard” interview question. This is a question that some—most?—candidates will struggle to answer well, and interviewers often think that it serves as a valuable gating mechanism. While it might help you filter some candidates out, it can carry some hidden risks.

Just so we’re on the same page, a hard question is one that puts the candidate “on the spot” in a way that may have been somewhat difficult to prepare for but is not a full problem-solving interview (those are typically 30-45 minutes and can be an excellent use of time provided you conduct them properly).

Here are some examples of what I’d consider to be popular flavors of hard questions:

• What has been your most profound “aha” moment at your last job?
• What’s something you believe that no one else does?
• Tell me of a time where you disagreed with your boss and changed their mind.
• Tell me of an time when you were wrong course and changed your direction.
• What’s your best example of leading a cross-functional team under a tight deadline with an especially difficult client.

The hope—with any of these questions—is that bad candidates will fail the question and good candidates will pass. If only it were that easy!

What happens instead is that candidates who are relatively quick on their feet, extraverted, and good at spinning stories on the fly will tend to over-perform on these hard questions. Candidates who are more introverted, or who need slightly longer to think, will tend to be more easily overwhelmed by such questions and will generally perform less well.

None of these questions might seem that difficult when you’re scanning the list from the comfortable vantage point of a blog reader, but things change when you’re in the vulnerable, high-pressure position of being a candidate during an interview.

The one thing all these questions have in common is that if you took a stopwatch and recorded candidate response times (we have), you’d notice that candidates take significantly longer to respond to these questions. (Compared to “What was your proudest accomplishment at Company XYZ?”). This is a strong indication that candidates do not have easy access to their episodic memory, and as a result, you are likely to hear a story that is either:

1. Less representative of who they really are and more designed to answer your “hard” question; or
2. A (partially) fabricated response to avoid the intense awkwardness of not being able to answer your question.

There are also certain interview personalities that like to “grill” or “pressure cook” candidates to see how they respond (aka “stress interviews”) and these will result in a similar candidate skew.

I’m not saying these questions are entirely without merit. But I do think the average interviewer:

1. Expects too much from these hard questions. They are attempting to convert a difficult decision (the hiring decision) into a much easier decision (how well did the candidate “perform” on this particular question; and
2. Doesn’t appreciate just how much these questions skew in favor of candidates who perform well, as opposed to candidates who have a demonstrated track record of proving the attributes in question.

The gold standard of CEO hiring—the interview format that top PE firms pay top dollar for—is basically the following for each chapter in someone’s career:

1. Who did you report to and what were their expectations of you?
2. What was your proudest accomplishment in that role? (Repeat 1-2x)
3. What was your biggest mistake in that role? (Repeat 1-2x).
4. Tell me about the team composition that you inherited. What changes did you make. When? What was the team like when you left?
5. What was it like working with [your Boss]. What would [your Boss] say were your strengths and weaknesses?
6. Why did you leave? Tell me about the departure.

When billions of dollars are on the line for an executive hire, these are the questions that the market has determined produce the best and most reliable data. Notice: none of these questions are hard. (It’s an astonishingly “simple” format in terms of question complexity.) These are short, simple, open-ended questions that you can deliver with fascination and curiosity and move the candidate along with great time management skills.

Interviewing and hiring at an elite level is hard. There is a mixture of IQ, EQ, curiosity, and business acumen that is required to do it extremely well. But while attaining that suite of skills may be hard, the questions themselves should not be.

Did this pique your interest on whether or not you’ve been asking some “too hard” questions? We’re hosting a free webinar on March 7th: “What Interview Questions Should I Ask?” and you can register here. Please feel free to share this invite with anyone who you feel might benefit from our approach to interviewing and hiring.