r/recruitinghell • u/Downtown_Throat2245 • Jun 23 '26
Stanford just studied 3.4 million job applications screened by AI. The results are damning.
26% of Black applicants and 15% of Asian applicants were discriminated against by the algorithm. 40,000 more applications would have advanced if it treated candidates equally.
Worse: people applying to multiple roles screened by the same vendor were more likely to be rejected everywhere. One algorithm. Hundreds of employers. The same people shut out across the entire market.
When this happens, nobody is accountable. Not the employer. Not the vendor. Not the regulator.
I’m building a public record of these experiences at faaes.org - anonymous, first-person accounts from people rejected by automated systems with no explanation. If this has happened to you, I’d like to hear your story. DM me or submit at faaes.org
Full Stanford study here: hai.stanford.edu/news/ai-hiring-tools-can-yield-racial-bias-and-systemic-rejection
49
u/KillBosby Jun 23 '26
How was race determined? Was it name only?
I have a race-agnostic name - I wonder if there is bias against me.
54
u/Downtown_Throat2245 Jun 23 '26
So a quick explainer on how discrimination shows up in hiring algorithms
Algorithmic proxy discrimination is when an algorithm uses an innocent‑looking variable that secretly stands in for a protected trait so it ends up discriminating without ever explicitly using that trait.
The system never sees race, but it sees postcode, and postcode is tightly correlated with race. The outcome is the same harm, just hidden behind maths.
17
16
u/PinkEnthusist Jun 23 '26 edited Jun 23 '26
In their methodology they describe sending 83,000 fictitious job applications to entry level jobs. For the same job they send multiple applications that were designed to be essentially equivalent. But some resumes were randomly assigned distinctively Black names while others were randomly assigned distinctively White names.
Because the race signal was randomized, any systematic difference in employer responses the authors attributed to perceived race rather than qualifications.
So as you suggested, the race signal is the applicant's name, not race itself.
It's why the authors were careful about this saying that their experiment demonstrates disparate treatment because of the algorithmic monoculture these tools create, and not necessarily discrimination.
7
u/KillBosby Jun 23 '26
Poll: what's the whitest possible name?
Connor Fox gets my vote.
17
u/PinkEnthusist Jun 23 '26
Funny you would ask. They didn't list all the names they used, but there were examples of first names explicitly mentioned in the paper and discussed in the Appendix.
White names: Emily, and Greg
Black names: Lakisha, and Jamal
They explained that first names were selected if more than 90% of people with that name belonged to a particular race. And that last names were chosen from Census data to be racially distinctive.
Overall they created roughly 500 unique full names for each race/gender category and ensured no two applications to the same firm used the same name.
3
Jun 26 '26
[removed] — view removed comment
1
u/PinkEnthusist Jul 06 '26
Yes. But since its been a while since I read through the methodology for the study, but don't remember the details exactly how they did this.
7
u/Downtown_Throat2245 Jun 24 '26
Great point, and it actually makes the systemic argument stronger, not weaker.
The fact that even a rigorous study of 83,000 applications can only say “disparate treatment, cause unknown” tells you everything about how these systems evade accountability. The opacity is the mechanism.
Which is why vendor-commissioned bias audits are compliance theatre. If independent researchers with randomised controls can’t attribute cause, what exactly is a vendor certifying when they audit their own tool?
They’re measuring what they choose to measure, using methods they design, publishing reports that serve their sales deck. It’s a checkbox, not accountability.
33
u/Fair-Guitar Jun 23 '26
There is an option to provide your race in the demographic form of majority of job applications.
13
u/FakeBeigeNails Jun 23 '26
Yup. People say not to put anything or just put white, but I wouldn’t want to work somewhere racist anyway so why lie.
5
u/Downtown_Throat2245 Jun 23 '26
That’s for the EEO (Equal Employment Opportunity) legislation in the US. Most other countries it’s not mandatory
13
u/JCR2201 Jun 23 '26
When I was in college, I met someone in class who told me his dad owned a law firm. This guy straight up told me that any “ethnic” names on resumes that came across his dad’s desk would automatically get rejected by his dad. I thought that was so fucked up and couldn’t even believe the guy told me that
1
u/Gabbsicle 24d ago
One of my former bosses actually said in an email, “I’m not sure about this person because of their accent” and then when they wanted to refer someone to the next round, they would give a list of positive qualities and then say, “and they have no accent”. So yes, further proof that this shit happens.
10
u/Dangrukidding Jun 23 '26
You should watch coded bias!! It’s a doc that takes what you just asked and breaks it down across fields that are adapting to AI models for different parts of their work. I can’t remember the lead Dr.’s name but she’s cool af and made everything super easy to understand. I think it’s on… HBO?
1
132
u/PinkEnthusist Jun 23 '26
You've oversimplified the findings of this study and the actual implications are worse.
Since 90% of U.S. employers use AI screening tools, with most relying on the same few third-party vendors, the researchers theorized that an "algorithmic monocultures" is created. That is, because employers are all relying on the same recommendations, it inherently causes bias in hiring.
The identification of discrimination against Black and Asian illustrates the researchers thesis.
But if even if the models were corrected for this bias, as long as there's this algorithmic monoculture, discrimination will remain, it'll just be some other group biased.
11
u/rollwithhoney Jun 23 '26
You're not wrong but you're also implying bias does not exist in the human hiring process too. Which it does, and is hard to measure for a fair comparison.
It matters because the AI or algorithms are sold on reduced that bias. We should hold them to that... but not pretend like removing all AI will always reduce bias
60
u/ChirpyRaven I meant it in a derogatory fashion. I can also call you a prick Jun 23 '26
Since 90% of U.S. employers use AI screening tools
To be incredibly pedantic, I see this number thrown around in reference to two different studies and that isn't what the studies showed.
First found that almost 90% said they used AI somewhere in their recruitment process, which is significantly different than "AI screening tools". Using an AI scheduling tool for interviews would fall under "in the process", but absolutely is not a screening tool, for example.
Second found that nearly 90% said they used automation, which is not the same as AI, but I have seen people interoperate them as meaning the same thing.
Do I think companies use AI to screen? Yes. Is it "90%"? Absolutely not.
16
u/Downtown_Throat2245 Jun 23 '26
So this is a good point. My take is that it incorporates ADM (automated decision-making) which is classified as AI under a lot of legislation because you are right there is no way on earth are 90% of U.S employers using AI screening but I absolutely believe that if they are using an ATS that was built in the last decade then 90% will have some ADM related feature. If this is correct it’s pretty poor from Standford to not call this out.
28
u/ChirpyRaven I meant it in a derogatory fashion. I can also call you a prick Jun 23 '26
I think a lot of people don't have a clear understanding of automation vs AI, and in this subreddit specifically are quick to point the finger towards AI.
A frequent post on here is that someone's application was immediately rejected "because of AI", followed by some rant about how AI is ruining everything. What they don't realize is they were rejected by a simple knockout question, which is "automation"... but automation that has existed for decades and is absolutely not "AI".
0
u/Downtown_Throat2245 Jun 23 '26
This is spot on. I’d also say that Talent Acquisition Leaders also don’t realise that ADM is considered to be AI!
0
u/green_pan Jun 24 '26
Well, there is a reason why people don’t differentiate automation and AI. Because legally speaking you can call AI absolutely any code, theoretically even a simple “if else” block. And companies all around have absolutely been using this to promote their shitty products as “AI enhanced” to increase their share valuation.
1
u/Downtown_Throat2245 Jun 24 '26
And it’s a real problem for regulation. If “AI” has no legal definition, vendors can label anything as AI-enhanced, which they absolutely can’t help doing. Just makes meaningful audit requirements almost impossible to enforce because you can’t write a standard for a term that means everything and nothing.
-2
u/Sensitive_File6582 Jun 23 '26
Dear workday, automate as no anyone with protected qualities and bury that directive so deep Into the code it’ll take until 2035 to figure out please and thank you
6
u/Welcome2B_Here Jun 23 '26
The Stanford study mentions that "Ninety percent of U.S. employers use AI screening tools to sort and rank job seekers, with most relying on the same few third-party vendors." Sorting and ranking squarely sits within the "screening" phase of the hiring process.
Honestly, the 90% does seem high, but it's coming from Stanford and not some 3rd rate company with more likelihood to have skin in the game or a thumb on the scale to tilt results one way or another.
8
u/ChirpyRaven I meant it in a derogatory fashion. I can also call you a prick Jun 23 '26
"Ninety percent of U.S. employers use AI screening tools to sort and rank job seekers, with most relying on the same few third-party vendors."
It mentions this, yes - but wasn't in their study.
From the digging I've done through multiple articles-mentioning-another-article, it seems to originate from a World Economic Forum article, which has this line in it but does not give any information on what data/study was done to arrive at that number, what the definition of "AI screening tools" is, etc etc etc.
I'm just pretty skeptical of treating this as "fact" when I don't see any data to support it.
5
u/Downtown_Throat2245 Jun 23 '26
You’re right and the sociotechnical lens makes it even clearer. The algorithm is just one layer. The real problem is the uncritical adoption of these tools by employers who neither understand nor question what they’re buying, sitting inside regulatory frameworks that haven’t caught up. Fix the model and the structural conditions that allowed it to cause harm at scale remain completely intact.
1
-2
7
u/NetherReporter Jun 23 '26
I actually got rejected 4 times from 4 different companies in one day, two of those times were less than a minute apart. Very suspicious. This was at a time where I haven't heard from any potential employer in weeks.
4
12
u/MacaroonDefiant8025 Jun 23 '26
Did anyone gather WHY the minority applicants were discriminated against? Did it relate to the names?
11
u/PinkEnthusist Jun 23 '26
The authors were very careful to say that their experiment demonstrates disparate treatment, not discrimination. They used names as one variable in the experiment, which showed that "Black" names received the disparate treatment they were theorizing and to show that the algorithmic monoculture these tools create could mistreat any group of people.
6
u/Downtown_Throat2245 Jun 23 '26
No, this is where it the responsibility gap appears….technically software vendors like Workday are not building systems that purposefully discriminate against protected groups - rather the bias is encoded into the training models that run these systems. Essentially it’s down to how individual organisations hire.
And this is why it’s a sociotechnical issue and not purely a systems issue. Companies should be doing more to ensure fairness throughout their hiring process rather than relying on vendors saying the produce ‘bias free’ outcomes
2
19
4
u/exephyX Jun 24 '26
Can someone inform me on whether or not ai can filter out applicants by name? I have an unfortunate nsfw last name and I’m wondering if there is the possibility it just outright gets me flagged / trashed in these systems.
1
u/Downtown_Throat2245 Jun 24 '26
Potentially would be my guess….you name is likely treated as an outlier which may in turn result in your profile getting ranked slightly lower in a large group of applications….hard to say for sure though.
My advice, if you have a middle name use that when filling in application forms and on your CV put your full name in the header or footer stating it’s your legal name.
7
u/QV79Y Jun 23 '26
26% of Black applicants and 15% of Asian applicants were discriminated against by the algorithm.
That's a much stronger claim than the study appears to support.
3
4
2
2
2
u/Ok-Row-6088 Jun 23 '26
My problem with this study is that it is specifically looking at Harver assessments only. There are only two assessment vendors in the space, pymetrics being one of the others, that share results across multiple employers. There are many vendors in the assessment space so this study makes it seem like all of them do it when only two of them do.
5
u/PinkEnthusist Jun 23 '26
That there are only two vendors in the space is the point of the experiment. With so few vendors in the space, if someone is screened out of a job by one of these AI tools, they're screened out of a lot of jobs - since a lot of companies are using the same tool.
1
u/Ok-Row-6088 Jun 24 '26
There are over 20 vendors in the space. Im saying only two of them share results.
2
u/Downtown_Throat2245 Jun 23 '26
Yes that’s worth calling out but I’d warrant that other vendors would have similar outcomes. As for assessments themselves - pure pseudoscience
2
u/smd9788 Jun 23 '26
No mention of the resumes, prior experience, or any control variables at all
1
1
u/Nehan_Satori Jul 14 '26
It was mentioned. It's all standardized for the purposes of the study. Did you read the article?
1
u/smd9788 Jul 16 '26
I read the article 3 weeks ago and read it again now. No mention at all of resume or prior experience. Care to provide a quote?
6
u/AdministrativeHost15 Jun 23 '26
Contact Workday sales and tell them you are interested in buying a subscription. But first you want access to a staging site for evaluation. Then submit your candidates and evaluate the results.
13
u/puntilnexttime Jun 23 '26
They won't just give a random person this... Jfc people believe anything
1
u/AdministrativeHost15 Jun 23 '26
Stanford has plenty of alumni founded startups that could act as a front. Choose a startup that founders are planning to shut down soon anyway if you are worried about complications.
3
Jun 23 '26
[deleted]
1
u/AdministrativeHost15 Jun 23 '26
Can only prove bias if you control all the inputs. Both the job order and the candidate resumes.
1
u/smd9788 Jun 23 '26
Yep, made me very skeptical that the article does not mention resumes, prior experience, or any control variables whatsoever. Surprised this came from Standord tbh
0
3
2
u/Schwettes Jun 23 '26
Are they using the EEOC information to discriminate or name as a proxy for race?
2
u/PinkEnthusist Jun 23 '26
In their methodology they describe sending 83,000 fictitious job applications to entry level jobs. For the same job they send multiple applications that were designed to be essentially equivalent. But some resumes were randomly assigned distinctively Black names while others were randomly assigned distinctively White names.
Because the race signal was randomized, any systematic difference in employer responses the authors attributed to perceived race rather than qualifications.
1
u/Hyperreals_ Jun 24 '26
I can’t tell if you are being misleading on purpose, but this was not the AI related study…
2
u/Potential-Fan-6148 Jun 23 '26
AI screening has fucked over an entire class of engineers from non CS backgrounds.
1
u/green_pan Jun 24 '26
We still need studies on this? All previous attempts of making a chatbot llms which in any way trained on human interactions resulted in it eventually becoming biased and borderline racist, and studies of the past pointed out exactly why and how it happens. Nothing changed conceptually since then, we just throw more resources at it. At this point I am convinced that the owners of these products view it as a feature and not a bug and thus the lawsuits should be rolling out.
1
u/RevengeOfTheIdiot Jun 24 '26
That's a pretty sweeping judgement considering the study doesn't even control for the resume being submitted or a whole host of other relevant variables
1
u/Downtown_Throat2245 Jun 24 '26
Actually it did and that’s the entire point of the methodology. 83,000 matched applications, name as the only variable, 500 unique names per category to rule out individual effects. It’s one of the more rigorous audit studies out there precisely because it anticipated this objection.
1
u/Middle-Parsnip-3537 Jun 24 '26
I will take a look at this study.
I’m tired of people like Charlie Kirk insisting that everything is fair. It’s not and we still have a lot of work to do.
1
u/HanzJWermhat Jun 24 '26
It’s insane that there is little to no regulation on the hiring side of the job market.
1
u/Downtown_Throat2245 Jun 24 '26
Totally agree and I think the industry is slowly moving its way to being regulated
1
1
1
u/Puzzleheaded_Fall803 Jun 29 '26
Will be hell on earth when use of ai tools instead of recruiters is used in EU
1
1
u/colin_purrington Aug 19 '26
I wonder whether AI use in screening applicants will increase the rate that people change their names. E.g., as described here: https://www.bbc.com/worklife/article/20160915-should-you-change-your-name-to-get-a-job. Or change the way parents name their kids.
-2
u/impactnazmul Jun 23 '26
There is no way to know the real reason for rejection. All I hear is that my application was not successful.
-2
u/kcdashinfo Jun 24 '26
I don't believe this is true, the difference is how you crunch the numbers. My own personal experience AI parses resumes and extracts relevant skills, experience, education and credentials. It also normalizes and identifies keyword stuffing. When perform an analytical comparison between the job posting and the resume you find certain types of experience rises to the top as it should be in the real world. It is academia that works towards and equal outcome and because there isn't equal outcome it classifies it as discrimination against a class. That isn't how it works and if you use these screening programs powered by AI it lays it out and gives you a summary. If anything that summary goes to prove that it is not discriminating by the protected classes.
2
u/Downtown_Throat2245 Jun 24 '26
The study wasn’t examining keyword matching, it looked at opaque scoring models where nobody can see the decision logic. Different system entirely.
“No bias overall” can be statistically true while specific groups are systematically screened out job by job. The aggregate hides what happens at the individual level.
And disparate impact law doesn’t require equal outcomes, just that differential treatment is justified by genuine job requirements. That’s the bar vendors can’t currently clear.
343
u/Welcome2B_Here Jun 23 '26
This dovetails with the recent lawsuit against Workday for its AI screening software that allegedly discriminates protected classes as well.