The Human-in-the-Loop Illusion: When Recruiters Follow the Algorithm

 Employment Testing: Failing to Make the Grade — Ten Years Later, Part 10

Imagine applying for a job and learning that artificial intelligence will evaluate your qualifications.

The employer reassures you that the software will not make the final hiring decision. A human recruiter will review the recommendations and decide which applicants deserve interviews.

That sounds reassuring.

But what if the recruiter simply follows the algorithm's recommendations?

Does human involvement protect applicants from biased automated screening—or merely give an algorithmic recommendation the appearance of independent judgment?

A University of Washington study published in October 2025 raises serious questions about that distinction.

The researchers found that people making simulated hiring decisions tended to reproduce racial preferences embedded in AI-generated recommendations, even though their choices were substantially more balanced without those recommendations.

The findings challenge a common assumption underlying automated hiring: that keeping a human involved necessarily provides meaningful protection against algorithmic discrimination.

What the Researchers Found

The study, No Thoughts Just AI: Biased LLM Hiring Recommendations Alter Human Decision Making and Limit Human Autonomy, involved 528 participants recruited through an online research platform.

Participants evaluated AI-generated resumes for 16 occupations, including computer systems analyst, nurse practitioner, and housekeeper.

Each exercise presented five applicants. Four were comparably qualified men whose names and organizational affiliations suggested different racial backgrounds. A fifth, less-qualified candidate helped obscure the experiment's purpose.

Participants selected three applicants for interviews.

The researchers supplied simulated AI recommendations that were racially neutral or favored particular groups to varying degrees. They also examined decisions made without AI recommendations.

The final analysis included 1,526 screening scenarios.

Participants were not using an actual commercial hiring platform. Instead, the researchers controlled the recommendations to investigate how algorithmic preferences influenced human decisions.

The results were striking.

Without AI recommendations, or when the recommendations were racially neutral, participants selected White and non-White candidates at approximately equal rates.

But when the simulated AI favored White candidates, participants increasingly favored White candidates.

When the AI favored non-White candidates, participants increasingly favored those applicants instead.

In the most severely biased experimental conditions involving higher-status occupations, participants favored the racial group preferred by the AI approximately 90% of the time.

This did not mean that 90% of professional recruiters automatically accepted every AI recommendation. The percentage describes participants' selection patterns under particular experimental conditions.

Nevertheless, it demonstrates how strongly biased recommendations could influence people who formally retained control over the decisions.

The participants were making the final selections, but those selections often reflected the preferences embedded in the AI recommendations.

The Problem Was Not Simply Human Prejudice

One of the study's most important findings was that participants did not demonstrate comparable racial preferences when making decisions without biased AI recommendations.

Their choices shifted when the recommendations favored particular racial groups.

That suggests a troubling possibility.

An automated system may not simply reproduce the biases of the people using it. It may introduce or amplify bias by influencing decision-makers whose choices would otherwise be more balanced.

The researchers also found that participants could be influenced even when they considered the AI's recommendations unimportant or of questionable quality.

Recognizing that AI may be unreliable, therefore, does not necessarily prevent its recommendations from shaping human judgment.

The experiment provides evidence of a problem that may arise whenever employers use AI to help evaluate job applicants.

A human decision-maker can unintentionally become the means through which algorithmic bias influences the final employment decision.

Human Review Is Not Necessarily Independent Review

Technology vendors frequently emphasize that their automated systems assist recruiters rather than make final hiring decisions.

That distinction can be meaningful.

A system that recommends candidates is not identical to one that automatically rejects applicants. Human review can provide an opportunity to identify errors, consider overlooked qualifications, and correct inappropriate recommendations.

But the value of that protection depends on how the review actually operates.

Consider a hypothetical screening system that ranks applicants from highest to lowest based on an algorithmic suitability score.

Highly ranked candidates appear prominently in the recruiter's interface. Lower-ranked applicants remain available, but reviewing them requires additional time and effort.

The recruiter formally retains complete discretion.

Yet the algorithm may determine which candidates receive meaningful consideration.

Does the recruiter independently evaluate applicants who receive low scores?

Does the recruiter understand what characteristics produced the rankings?

How often are unfavorable recommendations questioned or overridden?

And if the recruiter consistently follows the algorithm, how independent is the final decision?

The University of Washington study did not examine this specific workflow. But its findings provide a reason to investigate such practices in actual employment settings.

The central question is not simply who has authority to make the final decision. It is how much the algorithm influences the decision that person makes.

The Connection to Prior Post

The preceding post discussed a Stanford-led study examining approximately 4.2 million job applications evaluated through technology supplied by a common assessment vendor.

That research found racial disparities within particular screening processes and correlated unfavorable recommendations across employers.

Its findings suggested that applicants could encounter similar screening judgments when applying to different companies using related assessment models.

The University of Washington study examines the next stage of the hiring process.

What happens when a person receives an algorithm's recommendation?

The studies involved different technologies, methods, and populations. Their findings should not be combined as though they examined the same systems.

Nevertheless, they illustrate two related concerns.

A common assessment technology may repeatedly generate unfavorable recommendations for particular applicants.

And human reviewers may adopt biased recommendations rather than independently correcting them.

Together, these findings challenge the assumption that applicants receive a fresh, independent evaluation simply because different employers and different recruiters formally make the decisions.

What About Applicants With Mental Health Disabilities?

The University of Washington study examined racial preferences. It did not investigate discrimination against applicants with mental health disabilities.

Nevertheless, it raises a question relevant to the Americans with Disabilities Act (ADA).

Consider an applicant with depression, PTSD, an anxiety disorder, or another qualifying disability.

Suppose an automated assessment evaluates employment gaps, response speed, communication patterns, or other characteristics that may be associated with disability.

The assessment generates an unfavorable recommendation.

A recruiter accepts that recommendation without independently examining whether the relevant criteria appropriately measure the applicant's qualifications.

The recruiter may have no knowledge of the applicant's diagnosis.

That scenario would not automatically establish unlawful discrimination.

The applicable legal questions could include whether the screening criteria improperly exclude qualified individuals with disabilities, whether they satisfy job-relatedness and business-necessity requirements, and whether the employer complied with reasonable accommodation obligations.

A separate question could arise if an assessment constitutes a prohibited disability-related inquiry or medical examination.

The study did not address those legal questions.

But its findings suggest why disability-related screening barriers may remain consequential even when a human approves the final decision.

If a human reviewer relies heavily on a biased recommendation, requiring that person's approval may do little to correct the underlying problem.

Human Oversight Is Not a Legal Safe Harbor

Federal employment discrimination laws generally do not allow employers to avoid their obligations merely by using technology developed by someone else.

In May 2023, the Equal Employment Opportunity Commission issued technical assistance explaining how Title VII disparate-impact principles apply to algorithmic selection systems, including tools supplied by third-party vendors.

The ADA imposes separate obligations concerning disability discrimination, medical examinations, and reasonable accommodations.

The relevant legal inquiry is how the employer's selection procedures actually operate and whether the applicable statutory requirements have been satisfied.

Having a human make the final decision does not automatically cure unlawful screening criteria.

But reliance on AI does not automatically establish discrimination either.

A claim still requires evidence satisfying the applicable law.

The ongoing litigation in Mobley v. Workday, Inc. illustrates why the distinction matters.

Workday describes its AI-assisted recruiting technology as supporting human decision-making rather than independently making final hiring decisions.

Plaintiffs allege that certain applicant-screening services unlawfully disadvantage applicants based on protected characteristics. Workday disputes those allegations, and no final determination of unlawful discrimination has been made.

The University of Washington researchers did not study Workday's systems. Their findings establish nothing about whether Workday or its customers acted unlawfully.

They do, however, identify a question relevant to employers that emphasize human oversight:

Does the human reviewer meaningfully evaluate the applicant, or largely reproduce the algorithm's recommendation?

What Would Meaningful Human Oversight Look Like?

The answer is not necessarily to remove humans from the hiring process.

It is to make human review meaningful.

Employers should consider whether recruiters understand the limitations of algorithmic recommendations, have access to the information needed for independent evaluation, and possess genuine authority to question or override the results.

They should also examine how the screening interface presents rankings, whether unfavorable recommendations receive appropriate consideration, and whether actual selection outcomes reveal possible discrimination.

The researchers reported that completing an implicit association test before the screening exercise was associated with improvement in certain experimental outcomes.

That finding suggests that interventions may help. It does not establish that implicit-bias testing or brief training programs prevent discrimination in actual workplaces.

Nor would a high or low rate of overrides, by itself, establish whether human review is effective.

A well-designed algorithm may appropriately receive substantial agreement. Conversely, frequent overrides do not necessarily prove that a process is fair.

The issue is whether the employer can demonstrate that its screening procedures—including the interaction between humans and AI—appropriately evaluate job qualifications and comply with employment discrimination laws.

Meaningful oversight should be evaluated by what human reviewers actually do, not merely by whether their names appear at the end of the decision-making process.

The Human-in-the-Loop Illusion

The University of Washington study has important limitations.

Its participants were recruited online, rather than exclusively from professional recruiters. The recommendations were simulated, and the decisions involved controlled exercises rather than actual employment outcomes.

The researchers studied racial preferences, not disability discrimination.

The findings do not establish that all AI-assisted hiring systems produce discriminatory results or that all human reviewers defer to algorithms.

But they provide meaningful experimental evidence that biased AI recommendations can substantially influence human decisions.

That evidence should matter to employers increasingly relying on automated systems to evaluate large applicant pools.

Human participation may provide an important safeguard against error and discrimination.

But human participation is not the same as independent human judgment.

If reviewers routinely accept algorithmic recommendations without meaningful examination, the human approval process may do little to prevent the algorithm's shortcomings from shaping employment opportunities.

The question employers and technology vendors should be prepared to answer is therefore not simply whether a human remains involved.

It is whether that human exercises meaningful independent judgment—or merely gives the algorithm the final word.