Anybody spending 6 hours per candidate on site can afford to test them on some real work. I'll typically do 2 hours for pair programming, and/or 1 hour for reviewing some existing code. 1-2 hours is also a good amount of time for a joint design session on some real problem.
You're definitely right that we're missing good people. I keep coming across people who didn't get the right job until they invested a bunch of time into learning to beat the Mensa-puzzle interview. So one way to make the numbers better is to find more good people.
I think if we're going to do the math, I also think we need to account for the cost of getting people who are not so good. If we test some thing that we hope correlates with doing the work, we're asking for large downstream costs. The more our interview tests what people actually need for long-term success, the better off we are.
How do you keep interviews consistent for candidate comparison with pair programming? I'm assuming you mean that y'all are pairing on actual work. That can vary. Yesterday was simple CRUD, "candidate nailed it." Today, there is an obscure concurrency bug, and the candidate would need more than 1-2 hours to understand the landscape of the complex code base we are asking them to pair in; "the candidate asked some ok questions I guess."
I fully agree with "the more our interview tests what people actually need for long-term success, the better off we are."
^^ This. As tokenadult always points out, a work sample test is the way to go.
I do it by pairing on a standard problem, one I'll use over and over. That can be real work in the domain or a toy problem. Both seem to work pretty well to me.
I'm pretty sure having candidates do unpaid work is illegal in California, which is another reason not to have them pair on actual work that you plan to ship.
Anybody spending 6 hours per candidate on site can afford to test them on some real work. I'll typically do 2 hours for pair programming, and/or 1 hour for reviewing some existing code. 1-2 hours is also a good amount of time for a joint design session on some real problem.
You're definitely right that we're missing good people. I keep coming across people who didn't get the right job until they invested a bunch of time into learning to beat the Mensa-puzzle interview. So one way to make the numbers better is to find more good people.
I think if we're going to do the math, I also think we need to account for the cost of getting people who are not so good. If we test some thing that we hope correlates with doing the work, we're asking for large downstream costs. The more our interview tests what people actually need for long-term success, the better off we are.