Tim Wilson talks about this idea of inside information. You can imagine that information that you've experienced firsthand seems like it's more relevant to the decisions that you make. You could take an example like if you're trying to decide whether to invest money in a bakery, say, so you have Bakery A and Bakery B. You go in to Bakery A and you're talking to the employees; you're sampling the goods; you talk to the manager about their business plan and everything else, but you don't do that for Bakery B. Bakery B, you just read about in the newspaper or some report or something. Now when it comes to investing money in the two, you feel more confident with A than you would with B, but you probably, when you're actually tallying the results of doing that investment, you probably wouldn't be any better with A than B. If anything, that inside personal information that you have with A might actually hurt you in the end because it feels like it's something that you have privileged access to, so it's probably going to give you misinformation about that bakery and about the investment that you're about to make.
The same holds for job interviews, right?
Absolutely.
We think if we can spend ten minutes with a person, we would be able to predict exactly what kind of employee they will be, so, "Yes, she was very confident. I think she will be able to lead a team very well," or, "I'm not so sure about her. She was unsure of herself, and I don't think she's the best fit for this organization," but the data say that interviews are entirely non-predictive of the performance, of job performance.
That's right. In fact, there has been a—we keep talking about these giant analyses. There was one done by I think it was Frank Schmidt and Jack Hunter [sic], and they looked at I think it was 32,000 employees across every job that you can imagine, from farmers to musicians to sales people and so on. They actually tried to figure out—so they did the experiment to see how people would predict that people would do, the employees would do in a particular job, and how they actually did after the fact on the basis of these interviews. They found that pretty much these standard interviews were almost completely useless.
I mean, I think it accounted for what's called eight percent of the variance. To put that in lay terms, it means that—for example, if you had, if you placed your 100 employees on a scale from the best to the worst, and then you actually saw how they performed, and then ranked them again from the best to the worst, you would be right on about eight of those people, in putting them in the correct spot, out of 100 employees. That's not really good.
Now, remember, the point to this episode is that we don't really have much insight into our own behavior. These experiments in this topic mean that we're not very good at predicting other people's behavior as well. In fact, we're not much better, as Richard Nisbett said, we're not much better at predicting our own behavior compared to other people's behavior.
The reason that interviews are so bad, I think, is because of something called the confirmation bias. We see what we expect to see. When you're interviewing a job applicant; you've read their CV; you've read their resume, you have a pretty good idea of whether you like the person or not before they even enter the room. Then when they enter the room, you ask them questions. The thing you ask them is going to be consistent with what your expectations are. So the question might be, "Are you a strong leader?"
Exactly. So, "Are you a strong leader," or you only ask them about the things that will confirm your beliefs. So the very questions that you ask are only going to be ones that make them look better, that they might respond well to. That's why these standard interviews are so bad.
What's better is something called a structured interview. If you asked every single applicant that comes in the door exactly the same things, then it gets a little better in terms of predicting their behavior and in terms of future performance.
Better would be to have—this is work by a colleague of mine, Kevin Eva, and what he's
doing is interviewing medical school applicants, who are already exceptionally good because by virtue of applying for medical school, but what they found is that if they asked people in different rooms and different scenarios—you have different people that are asking these same people different things completely independent of one another— then that is even better than a structured interview. The same person asking this—you have different people asking very structured sorts of questions, and that even gets you a little bit further. The sort of standard interview that we're used to, that most of us have had throughout our careers, is virtually useless, I think.
I think it's also true that the best predictor of future behavior could be the past behavior, so, yes, you could do a structured interview, but even better might be to get a sort of standard measures of people's performance in the past. Whether you're trying to select a job applicant or even a roommate, how—in the past, have they paid their rent on time?
In the past, have they had good performance evaluations and good outcomes?—things that it's difficult to fake or control—it seems that those things, that those standard measures over a long period of time, are much better at predicting job performance or behaviour in the future.
Absolutely, and it's not just—so it's sort of these long-term predictors, as you've said, so, for example, if in university, it would be GPA or even high school, right?
Yes.
So these grades that you've accumulated over a period of four years, it's hard to fake that, right?
On the other hand, if you have, say, a final exam—one high-stakes testing is what they call it—so if you have one exam, one critical exam, like the Graduate Record Examination or your LSATs or MCATs or, you know, all of these different sort of standardized exams, those really aren't good predictors because you could be sick that day; you didn't have much sleep; you had a bunch of things working against you.
This is the idea of what's called multiple independent error factors. At any given moment you have things working for you and things working against you, so, yes, "I missed my alarm," "I missed the bus," "I didn't eat breakfast," all of these kind of things that just happen randomly work against you, and it produces the worst test-taking. Your neighbors were up the night before, and they kept you up, and you just studied all the wrong things, it seems like.
It works in the other way as well. I mean, in some cases everything is going to work for you. So you had just the best sleep. You had the best meal. You studied just the right things. But, again, in every circumstance, when you take long-term behavior, then it's very unlikely that everything is going to work for you or work against you all at once because there are many times for these things to rear their heads, but on a one-trial task like a single test, then, yes, you're at the whim of all sorts of things, which is why it's not a very good predictor of future performance.
What do you think the upshot is then if we don't have much of an insight into our own behavior or the behavior of others?
Well, the title of this episode is called "Know Thyself," and I think that's fairly apt.
One of the other titles that we've covered is the title of Tim Wilson's book, called "Strangers to Ourselves," and I think you're going to see this theme playing out throughout the entire course. We'll see in the next episode, for example, that we have to resolve this.
If we can't, if we have no insight into our own behavior, into why we do the things that we do—and we saw in the last episode that the way that the world works may not be exactly as it seems, so seeing, hearing, remembering all involve considerable knowledge and so on and we're being swayed by any sort of factors whatsoever, whether media reports and everything else—that's a problem. What are we going to do about that? I mean, if we don't even know when it's happening, and these things are actually operating, then what? I think we can get there, but it's going to take a little bit of work.
The same holds for job interviews, right?
Absolutely.
We think if we can spend ten minutes with a person, we would be able to predict exactly what kind of employee they will be, so, "Yes, she was very confident. I think she will be able to lead a team very well," or, "I'm not so sure about her. She was unsure of herself, and I don't think she's the best fit for this organization," but the data say that interviews are entirely non-predictive of the performance, of job performance.
That's right. In fact, there has been a—we keep talking about these giant analyses. There was one done by I think it was Frank Schmidt and Jack Hunter [sic], and they looked at I think it was 32,000 employees across every job that you can imagine, from farmers to musicians to sales people and so on. They actually tried to figure out—so they did the experiment to see how people would predict that people would do, the employees would do in a particular job, and how they actually did after the fact on the basis of these interviews. They found that pretty much these standard interviews were almost completely useless.
I mean, I think it accounted for what's called eight percent of the variance. To put that in lay terms, it means that—for example, if you had, if you placed your 100 employees on a scale from the best to the worst, and then you actually saw how they performed, and then ranked them again from the best to the worst, you would be right on about eight of those people, in putting them in the correct spot, out of 100 employees. That's not really good.
Now, remember, the point to this episode is that we don't really have much insight into our own behavior. These experiments in this topic mean that we're not very good at predicting other people's behavior as well. In fact, we're not much better, as Richard Nisbett said, we're not much better at predicting our own behavior compared to other people's behavior.
The reason that interviews are so bad, I think, is because of something called the confirmation bias. We see what we expect to see. When you're interviewing a job applicant; you've read their CV; you've read their resume, you have a pretty good idea of whether you like the person or not before they even enter the room. Then when they enter the room, you ask them questions. The thing you ask them is going to be consistent with what your expectations are. So the question might be, "Are you a strong leader?"
Exactly. So, "Are you a strong leader," or you only ask them about the things that will confirm your beliefs. So the very questions that you ask are only going to be ones that make them look better, that they might respond well to. That's why these standard interviews are so bad.
What's better is something called a structured interview. If you asked every single applicant that comes in the door exactly the same things, then it gets a little better in terms of predicting their behavior and in terms of future performance.
Better would be to have—this is work by a colleague of mine, Kevin Eva, and what he's
doing is interviewing medical school applicants, who are already exceptionally good because by virtue of applying for medical school, but what they found is that if they asked people in different rooms and different scenarios—you have different people that are asking these same people different things completely independent of one another— then that is even better than a structured interview. The same person asking this—you have different people asking very structured sorts of questions, and that even gets you a little bit further. The sort of standard interview that we're used to, that most of us have had throughout our careers, is virtually useless, I think.
I think it's also true that the best predictor of future behavior could be the past behavior, so, yes, you could do a structured interview, but even better might be to get a sort of standard measures of people's performance in the past. Whether you're trying to select a job applicant or even a roommate, how—in the past, have they paid their rent on time?
In the past, have they had good performance evaluations and good outcomes?—things that it's difficult to fake or control—it seems that those things, that those standard measures over a long period of time, are much better at predicting job performance or behaviour in the future.
Absolutely, and it's not just—so it's sort of these long-term predictors, as you've said, so, for example, if in university, it would be GPA or even high school, right?
Yes.
So these grades that you've accumulated over a period of four years, it's hard to fake that, right?
On the other hand, if you have, say, a final exam—one high-stakes testing is what they call it—so if you have one exam, one critical exam, like the Graduate Record Examination or your LSATs or MCATs or, you know, all of these different sort of standardized exams, those really aren't good predictors because you could be sick that day; you didn't have much sleep; you had a bunch of things working against you.
This is the idea of what's called multiple independent error factors. At any given moment you have things working for you and things working against you, so, yes, "I missed my alarm," "I missed the bus," "I didn't eat breakfast," all of these kind of things that just happen randomly work against you, and it produces the worst test-taking. Your neighbors were up the night before, and they kept you up, and you just studied all the wrong things, it seems like.
It works in the other way as well. I mean, in some cases everything is going to work for you. So you had just the best sleep. You had the best meal. You studied just the right things. But, again, in every circumstance, when you take long-term behavior, then it's very unlikely that everything is going to work for you or work against you all at once because there are many times for these things to rear their heads, but on a one-trial task like a single test, then, yes, you're at the whim of all sorts of things, which is why it's not a very good predictor of future performance.
What do you think the upshot is then if we don't have much of an insight into our own behavior or the behavior of others?
Well, the title of this episode is called "Know Thyself," and I think that's fairly apt.
One of the other titles that we've covered is the title of Tim Wilson's book, called "Strangers to Ourselves," and I think you're going to see this theme playing out throughout the entire course. We'll see in the next episode, for example, that we have to resolve this.
If we can't, if we have no insight into our own behavior, into why we do the things that we do—and we saw in the last episode that the way that the world works may not be exactly as it seems, so seeing, hearing, remembering all involve considerable knowledge and so on and we're being swayed by any sort of factors whatsoever, whether media reports and everything else—that's a problem. What are we going to do about that? I mean, if we don't even know when it's happening, and these things are actually operating, then what? I think we can get there, but it's going to take a little bit of work.
Комментариев нет:
Отправить комментарий