Hi, I'm Michael Corayer and this is Psych Exam Review. In the previous video we talked about
assessing intelligence and in this video we're going to look at how we assess those assessments.
So, in other words, we've created some assessment for intelligence and then we want to know,
is this a valid assessment? Is this really assessing intelligence or not? How do we
determine that? So when we think about something like intelligence, remember it's a property. It's
sort of a hypothetical idea. It's something we think exists and then the reason that we think it
exists is because there's certain behaviors that we think are associated with it. So the reason we
think that there's something like intelligence that varies amongst people is that we see that
people vary in their performance on certain types of tasks and then we create an assessment
and hopefully that assessment is related to this hypothetical property of intelligence that we're
thinking about. So if we look at these three parts here; we have some property, in this case that
would be intelligence, and then we have associated behaviors. And in this case that might be things
like grades in school or job performance, right? These are things that make us think intelligence
exists. And then we have assessment, which would be things like IQ scores on some sort
of intelligence test. So the idea here is that we think there's some relationship between these
behaviors and this property that we've created. We say "ok, people do, you know, some people do well
in school, some people don't do as well maybe that means there's this thing called intelligence". And
then we say ok, well we want to assess that, so we create some sort of test that we think is related
to this idea of intelligence and then we also want to see that there's a relationship between the
behaviors that we associate with intelligence and the IQ results, the assessment results that we get
from this test that we've created. And so we have this little triangle here of different possible
relationships and what we're doing when we assess validity is we're sort of testing the strength of
these relationships. And so we have multiple ways to think about these relationships that means
we're going to have multiple types of validity. Now I'm going to give you a bunch of types,
it might seem like sort of a laundry list of validity types, but don't feel overwhelmed by
it. The important point is that we're assessing different relationships and each different name
for a type of validity gives you a more specific way to talk about it. That way when you're talking
about the validity you don't just say this test isn't valid. I say "well why exactly isn't it
valid? Is it that it's not really associated with the property, or does it not allow us to learn
things about the associated behaviors?". And so each type of validity will tell you exactly why
it is or is not valid. So let's take a look at some of these different types of validity. The
first one is one that I talked about in the research methods unit and this is construct
validity. This is sort of a fundamental type of validity. If you don't have this then there's
really no point to doing your assessment. So what does construct validity refer to? Well this
is the idea that there's a relationship between the property and the assessment. So we want to
have a clear relationship between our property and how we've measured it, right? So if we look
at our diagram here, construct validity is sort of this line right here saying "Ok, is there actually
some clear relationship between intelligence and assessment results?". So again, you know,
you could pick some silly example and say "ok, I want to know about intelligence and my assessment
is to ask you your favorite color" or something and we might say "I don't think there's really
a clear relationship between someone's favorite color and their intelligence". So in that case,
we say probably has low construct validity. Or if I wanted to know about your intelligence and
I say "well, intelligent people maybe earn more money and therefore they have more cars and so
I'm going to count how many cars that you have and that's going to tell me your intelligence".
Again we look at that and say there's not a clear relationship between those two things,
therefore we would criticize it for having low construct validity. Now another way to think
about an assessment is to look at the assessment and say "does it appear to measure what it says
it's measuring?". This is called face validity and it's a surface level assessment. It's not a
detailed analysis. So I design some intelligence test and if you look at it and there's a bunch
of sort of different types of problem-solving questions you might look at and say "yeah, that
looks like it could be an intelligence test". So it has this surface level validity where I think
intelligence is associated with problem-solving ability and this is a test of problem-solving
ability therefore it appears to be a good way to do it. One way you could think about this is if I
wanted to assess an artist's technical ability. I might give them a test of brushstrokes, ok,
there's different types of brush techniques I want to see if you can do them; here's a test of
all these different brush techniques. Now, on the surface that looks like a good test of artistic
technique, right? It sort of has the general appearance that it's a test that would tell me
something about an artist's ability, so it's the surface level assessment and here's, my pen is not
working, let's see, here we go, so a surface level assessment and the idea is that it appears to be
related but doesn't mean that it's fully related. It doesn't mean that it's the best assessment of
something like artistic talent. So to continue with this sort of analogy I'm going to get to the
next type of validity which is content validity and what content validity asks is not just does
it appear to be related but is it comprehensive? In other words, does it cover all the things that
are associated with that property? So if I took the results from my brush stroke test and then
I said this test tells you everything you need to know about your artistic talent then it would be
criticized for content validity. You'd say "ok, it has face validity. It appears to be related,
but in terms of content there's a lot more to being an artist than just brush stroke technique."
And therefore we'd say this doesn't have content validity. It's not comprehensive enough to really
tell you about artistic talent. It doesn't cover all the aspects associated with that. So we're
going to need to have other types of things in our assessment if we really want to be assessing
artistic talent rather than just brush stroke technique. Now this idea of content validity is
important for thinking about intelligence because as we saw in the video on defining intelligence,
researchers disagree on what should be considered to be all of the aspects of intelligence. So a
common criticism would be content validity. If I look at an IQ test and it has all sorts of tests
of problem-solving but it maybe doesn't assess creativity very much, so somebody might say well
it appears to be related to intelligence but it's not comprehensive enough because it doesn't test
divergent thinking. Or maybe if we if we think that there's some sort of thing like emotional
intelligence than we could criticize the test for not assessing it and therefore not having content
validity because it doesn't cover all the aspects that could be associated with intelligence. Ok,
so that's the difference between face validity and content validity. Now the next two types
of validity are related they're both called criterion-related validity and the reason
for this is that they are both seeing if the assessment is actually related to some other
criteria. So the idea is we want to know if there's a relationship with other outcomes and
there's two ways that we could assess that. The first way is what's called concurrent validity.
In concurrent validity we ask "does it relate to other outcomes at the same time?". So if
you remember that concurrent means occurring simultaneously then all we're really saying is
that we're looking at things at the same time. So in the case of an IQ test you might say ok,
let's say I give an IQ test to a bunch of children then I might ask "does the IQ test that they've
just taken now tell me about their performance in school right now as well?". So if we go back
to our diagram here, you can think of it as being this line right here. It's saying "do
these assessment results have a relationship with other associated behaviors like grades in school
or current job performance?". And for concurrent validity it's asking is there a relationship right
now. You take the IQ test today and based on that maybe I can predict something about your current
grades in school. And the idea might be if grades in school are associated with intelligence and my
IQ test is associated with intelligence then there should be some relationship. Doesn't mean it'll be
a perfect relationship, we can imagine you know the sort of lazy student in school who doesn't
do his homework but actually has a very high IQ score, or we can imagine the opposite the student
who has a lower IQ score but works really hard and manages to get good grades in school so it's not
going to be a perfect relationship. We want to see some relationship between the behaviors and
the assessment right now. Now you can probably guess that predictive validity is similar. It's
also a criterion-related validity but it's looking at other outcomes in the future. So can our test
predict future outcomes? So in this case I might wonder "ok, the IQ test you take when you're 10
years old, does that, do your results on that IQ test allow me to make predictions about your SAT
score or about your college performance?". And that would be assessing predictive validity. I'd
say okay there's some behaviors associated with intelligence like high SAT scores,
let's say or with, you know, high level college performance and can I predict those if you take
an IQ test when you're 10 years old? And so that would be an assessment of the predictive validity
of my IQ test. Ok, so those are the different types of validity and in the next video we'll
start talking a little more about reliability of tests. I hope you found this helpful, if so,
please like the video and subscribe to the channel for more. Thanks for watching!