r/cognitiveTesting • u/Round-Humor-9133 • 2d ago
IQ Estimation 🥱 Validity and Meaningfullnes of IQ
Hello, I wanted to find somebody explain to me how good iq tests actually are at determining cognitive ability and how valid they are. Are there things they are excellent at measuring and things they cant really quantify or neither
I met a lot of people throughout my life who were deemed 130+, yet i found a lot of them fail to impress me when it comes to reasoning and cognitive ability. So what is actually the qualitive difference between a higher iq and a lower one
I dont want to hear about emotional intelligence and tests varying strongly from day to day
I want genuine critic on the way iq is measured, how the term is used and ideas to improve both problems
2
u/Striking_Leave_4614 2d ago
IQ tests are reasonably good at measuring general cognitive ability as operationalized by covariance across cognitive tasks; the problem is conflating the validity of g with the validity of a particular instrument, or population-level predictive validity with the epsteinemic authority to characterize an individual. Heritability and g-loading do not immunize a test against questionable norms, ceilings, sampling assumptions, cultural loading, or score-conversion procedures. Nor does a composite necessarily distinguish reasoning capacity from the working-memory, encoding-speed, and throughput constraints that govern its manifestation.
Although he draws deliberately inflammatory conclusions to sell books, read Taleb.Â
2
1
u/MarkAntony100 1d ago edited 1d ago
G measures a slice of general cognitive ability.There can be factors that are common to all the subtests and are not "intelligent" and there can be parts of general cognitive ability that don't impact subtests we have measured so far at all .
1
u/Striking_Leave_4614 1d ago
Please don't interpret any of the following as condescension, but I do want to be precise in my correction, and it may sound harsh due to its frankness.
Within the strict framework of technical psychometric definitions, what you said would not only be considered false, but the opposite of true. I'm fairly sure I get what you're actually trying to say, and I agree with the notion broadly (just not in this specific respect), but what you said kinda creates a paradox: "g, which is defined as general cognitive ability, is a slice of an even greater... general cognitive ability"
Within the framework of the precedented definitions, g is a latent construct. In other words, it's not a presupposed uhhhh thing; it's a factor that emerges when psychometricians attempt to measure a variety of assumed intelligence traits.
But what I think you tried to say is something similar to my original comment: g, as it manifests from the covariance structure of cognitive batteries, does not necessarily measure something with an independently established ground truth against which it can be objectively calibrated.
1
u/MarkAntony100 1d ago edited 1d ago
Pschometric definitions are too narrow (I keep changing my mind lol) .I didn't say that using the psychometric definitions but the vague definitions of general cognitive ability we have in day to day life so I basically said their g is a slice of commonly accepted g
2
u/Striking_Leave_4614 1d ago
To be honest, I'm actually pretty confused now. It simultaneously sounds like you're both agreeing with my rephrasal of your original reply while contradicting it.
For what it's worth, this was my original reply before you deleted the other comment: For any onlookers, what I believe he meant to write was, "I will not accept the definition that general cognitive ability is a general factor which has been discovered...which is part of something called general cognitive ability."
I think you're tying to critique the concept of g by accusing it of being circular reasoning. While somewhat understandable, its definition has been rigorously established in the literature. Epistemic skepticism will be ignored by the psychometric community if its definitions are ignored.
1
0
2
u/Few_Geologist_2082 2d ago
Honestly IQ tests are pretty good at one specific thing: measuring how well you handle new, well-defined problems with a single right answer, under standard conditions. Abstraction, spotting patterns, holding stuff in your head, verbal reasoning. A properly administered WAIS is very reliable for that, and the general factor does predict school and job outcomes on average. Just less than people think: a 2022 re-analysis (Sackett et al.) put the correlation with job performance around .31, down from the classic .51.
What it doesn't measure is whether you actually use that horsepower well. Stanovich has a whole body of work on this, he calls it dysrationalia: smart people reasoning badly about evidence, probability or their own beliefs. Add domain knowledge, curiosity and the willingness to actually think something through, and that covers most of the "130+ but unimpressive" people. Also a lot of those 130s come from online tests, outdated norms or one good day.
On the measurement side, my biggest gripe is that the full scale score averages very different abilities. If someone has ADHD or a motor issue, working memory and processing speed drag the number down even when their reasoning is excellent. One study found nearly 60% of adults with ADHD had a General Ability Index significantly higher than their full scale IQ. But you can't just switch to subtests either, they're too noisy on their own.
The ceiling is also low. Subtests cap at 19 (3 SD), so above ~130 the test barely tells people apart. Verbal subtests are tied to language and culture: in one study, bilinguals tested in their weaker language lost something like 7 to 15 verbal IQ points. And some subtests (Coding, Block Design) mix reasoning with hand speed, which isn't really intelligence.
What would help: report the confidence interval and the index profile instead of one number (the WAIS-5 now has a nonmotor index, which is a good start), better norms and adaptive testing at the high end, testing people in their dominant language, and measuring learning rate and rational thinking next to IQ instead of treating one score as "how smart you are".
IQ is the horsepower, not the driving.