Interviews Measure the Wrong Signal
Execution Insights
By: Brandy Brown
Custom CSS
Add your custom CSS code below
Interviewing is a real skill.
- Asking good questions.
- Pulling on threads.
- Uncovering what’s actually going on.
For individual contributors, that matters.
- Engineers do it when debugging.
- PMs do it when understanding a problem space.
- Everyone does it when trying to make sense of incomplete information.
But being interviewed is something else entirely.
That’s a different skill—one that’s valuable in the right context.
Public-facing roles, executives, people operating in media—they invest heavily in it. It’s trained, coached, and often prepared for in advance. There’s context. There’s alignment. In many cases, there’s even negotiation around what will be discussed.
This isn’t that.
There’s no shared context. No expectation-setting. No negotiation.
Just a series of questions—sometimes thoughtful, sometimes completely disconnected from the work—asked in an environment where one side is being evaluated and the other isn’t.
We don’t train ICs for that.
Because for individual contributor roles, that’s not how the work happens.
And yet, it’s what we optimize for.
For individual contributor roles, the work doesn’t happen in isolation, explaining your thinking.
It happens in context. With other people. Over time.
And yet, most hiring processes are built almost entirely around evaluating people in the opposite environment.
Interviews don’t lack signal—they give you a very specific one.
They give you a direct signal—of how someone performs in a very specific environment:
- individual
- time-boxed
- self-focused
That’s “ME” mode.
But most IC work happens in a different mode:
- collaborative
- iterative
- context-rich
- shared ownership
That’s “WE” mode.
And these aren’t just different environments.
They pull in opposite directions.
One is centered on the individual—explaining, framing, selling their own thinking.
The other is centered on the group—building, adjusting, and figuring things out together.
You can see how someone contributes individually by observing them in a team.
But you can’t understand how they’ll work in a team by evaluating them in isolation.
So what do we do?
We add more interviews.
More panels. More perspectives. More questions.
But we’re not increasing signal—we’re repeating the same one.
We’re getting more confident in the same thing.
So we add structure.
STAR frameworks. Personality assessments. Behavioral patterns.
Ways to make interviews feel more rigorous.
But they don’t fix the problem.
They just standardize the wrong signal.
And in doing so, we start removing the human from the process.
People get reduced to patterns—
structured answers, personality outputs, predefined signals.
Clean. Consistent. Easy to compare.
But disconnected from what actually matters.
Which personality box someone fits into doesn’t matter.
What matters is how—whoever they are—actually works with the team.
What does better look like?
Not better interviews.
Fewer of them.
Or none at all.
Instead of asking candidates how they work, we should work with them.
What this actually looks like
You still start the same way: resume, portfolio, maybe a short written response.
Just enough to narrow things down.
Not to decide—just to filter.
And then—no interviews.
Instead of interviews, you just work with the candidate for a few hours.
One at a time.
On something real the team is actually working on.
It could be a problem.
It could be something in progress.
It could be an idea you’re still figuring out.
What matters is that it’s real—and that the team hasn’t already solved it.
You give a bit of context:
“Here’s what’s been going on…”
Not full onboarding. Just enough to engage.
They spend some time looking through it. Not solving. Just getting familiar.
Then you regroup as a team.
“Alright—let’s talk through this and figure out where to start.”
A conversation happens.
Someone jumps in. Someone builds on it. Someone challenges it.
You pick a direction.
Then you work.
“Want to pair on this for a bit and see what we find?”
You dig in.
Try things. Build on ideas. Adjust as you go.
Sometimes it flows.
Sometimes it doesn’t.
Sometimes you hit a wall. Sometimes you don’t.
You might follow something through.
You might pivot halfway.
You might realize you were looking at it the wrong way entirely.
All of that is part of the work.
At some point, you take a break.
Grab lunch. Reset.
Then come back and close it out.
“Hey, can you show us what y’all ended up doing?”
This is the demo.
A core part of the job—preparing and walking through work.
Which is exactly why it’s the right signal.
They walk through what they explored.
What worked. What didn’t. What they’d do next.
What you just saw
Not answers.
Not performance.
But:
- how they enter a problem
- how they think in motion
- how they work with others
- how they adapt when things don’t go as planned
- how they communicate real work
No hypotheticals.
No puzzles.
No “tell me about a time.”
Just work.
What about technical ability?
You don’t need puzzles.
If you want to know how someone codes, work with them.
Pair on a real slice of a real problem.
You’ll know quickly.
What about evaluation?
It doesn’t matter if they got it “right.” They’re working with limited context, on an unfamiliar system, in a short amount of time.
You wouldn’t expect a new team member to be right on day one.
What matters is how they worked through it.
Evaluation starts independently. Each person captures what they observed:
- how the candidate approached the problem
- how they worked with others
- how they adapted
- how they communicated
Not impressions.
Evidence.
Then you compare.
Not to reach consensus—but to surface differences.
“What did you see?”
“What did I miss?”
Because this isn’t about who was right.
It’s about whether you saw how this person works.
What about cost?
This is where most people hesitate.
A few hours of real work—plus compensating candidates—looks expensive.
Because the cost is visible.
But the current process isn’t cheap.
It’s just distributed.
Across:
- multiple interviews
- multiple people
- weeks of fragmented time
Along with:
- context switching
- lost focus
- repeated, wrong-signal conversations that don’t just add noise—they compound it
We’re comfortable spending a lot of invisible time to avoid a small visible cost.
This isn’t about adding expense.
It’s about reallocating it.
Toward a higher-signal way of making one of the most important decisions a team makes.
There’s another effect
Most hiring processes don’t create anything.
Candidates perform. Teams evaluate. Everyone moves on.
A working model changes that.
Both sides:
- engage with a real problem
- see how the other thinks
- experience what it’s like to actually work together
Even if it doesn’t lead to a hire, something real happened.
Not just evaluation.
Interaction.
The shift
We’ve built hiring processes around asking people how they work.
But the most direct way to understand that is to work with them.
Because if the goal is to understand how someone will perform—
watching them do the work will always tell you more
than asking them to talk about it.
