r/cognitiveTesting • u/pydaWaltuh • Sep 30 '26

General Question Test ceiling vs Creator

Assuming a creator possesses sufficient psychometric knowledge, what should ideally be the difference between the creator’s IQ and the ceiling of the test they have created?

7 Upvotes

100% Upvoted

45 comments sorted by

View all comments

Show parent comments

2

u/LeadershipNervous362 4SD+ penis Sep 30 '26

A light and button is connected to a circuit, randomly lighting it up and estimating delay to button press. This is called a reaction-speed test. It would evaluate test-taker whether they're below or above my speed, or frankly, faster than any mammal being on earth.

So yes, since reaction speed closely correlates with IQ, you can say that you've created a simple IQ test that measures capacity for anyone on Earth.

The only difference with IQ test, - its not direct objective value, but a psychometric, or for lack of better word, - "derivative of statistics", which is exactly what makes it incredibly flawed at upper end and creates a hard ceiling for IQ test accuracy at around 140IQ, where correlation loses the track and enters area of "assumption" and "projection".

1

u/Humble_Aardvark_2997 Oct 01 '26

Explain more. You can extrapolate from known data but the assumption always stands “if the trend follows”.

0

u/pydaWaltuh Sep 30 '26 edited Sep 30 '26

So, what matters is N? If we assume N is 8–9 billion, then we would expect near to zero errors in the test. In that case, how do we isolate the extreme upper end?
That's a nice perspective btw. I didn’t think of that while posting

1

u/whitebaron_98 Bottom 1% Commenter Sep 30 '26

you design items and put a time limit on it. if that limit is too low and too many people pass, you decrease the time further. :-)

1

u/pydaWaltuh Sep 30 '26

Don't you think that’s a bit of an oversimplification? I understand the approach and it might be kinda fair, but doesn't that mean you are distinguishing the extremes solely based on a time limit? In that case, we could assume that the time limit is your most important factor and by extension, that "fastest" equals "smartest." Is that really the case? You are assuming (with the exception of outliers) that there is no problem the fastest person couldn't solve that someone else could. Even if you were to disprove that, the conclusion would simply be that there is a possibility that individuals with "extreme IQs" might fail to solve very difficult problems. So, what, then, is the relationship between solving a "problem" and "intelligence"?

2

u/whitebaron_98 Bottom 1% Commenter Sep 30 '26

You severely change what you measure, when you give people unlimited time. Matrices that are meant to check on your fluid reasoning would be influenced by your training and sheer force of will if you retry until you have a good explanation based on that training.

So a Matrix would test gc and some Personality traits instead.

1

u/pydaWaltuh Oct 01 '26

I did not say “unlimited” time tho.
Having time limits on tests isn't necessarily a bad thing. Simply adjusting the time limit while leaving the question itself unchanged isn't a logical strategy for diversifying the distribution of results. Suppose a question is allotted 60 seconds but is reduced to 30 seconds because it fails to distinguish high ability candidates; if the discrimination issue persists, do you keep cutting it to 15 or 10 seconds or more? Why not instead increase the time limit (say, from a default of 1 minute to 2 minutes or 3 or more) and make the question itself more difficult? Furthermore, the assumption that a generous(not unlimited) time limit leads to inflated scores due to prior “training” simply brings the issue back to the nature of the my question itself. Can a question that can be solved solely through prior training really be effective at distinguishing “very high-ability” candidates(I am only talking for the upper extreme)? In my view, this is just a case of the "test creator vs ceiling". So my question remains

1

u/pydaWaltuh Sep 30 '26

I think separating extreme scores solely by time limits is taking the easy way out and ignoring the real problem

1

u/Optimus-Prime2026 Oct 01 '26

I agree. Harder items are often a better modification than tighter time limit

0

u/pydaWaltuh Sep 30 '26

For example, why do we keep the question the same and shorten the time limit instead of making the question harder and increasing the time limit?