I don't understand why can't someone just come up with an interview where they propose an issue that is truly something that can be expected in the role, give the applicant a few hours to come up with an action plan and a solution architecture, put it down in writing, sketch some graphs, and then present it...
Why is that so groundbraking? wouldn't that immediately tell you a lot about the applicant, regardless of the final implementation being a 10 or not?
That's just assessing someone's ability to design a solution, while presumably what you are hiring for is the proven ability to deliver solutions, add value to the organization over time, etc.
Of course it would be a major red flag if someone couldn't come up with a good sounding solution to something they claimed to be capable of, but a having a plan is not enough. Ideas are really a dime a dozen, and the ability to deliver is the what you really need, which is why VC's tend to fund people, not ideas.
The one thing interviewers need to stay aware of though is that they will have immense insight into the problem whereas the candidate likely has never thought through the problem space before. And this can lead to unfair judgements. I know I've fallen into this trap before, where I'd run the same session with so many candidates that I had a thorough big picture view of every possible approach. I'd see a candidate stumbling a bit and think, "Why can't they see the obvious answer in front of them?" when in fact I wouldn't have seen it on my first go either. I just had seen dozens of candidates go through the same exercise to the point where it seemed like old hat.
Why is that so groundbraking? wouldn't that immediately tell you a lot about the applicant, regardless of the final implementation being a 10 or not?