When One Hiring Algorithm Rejects the Same People Everywhere

 Employment Testing: Failing to Make the Grade — Ten Years Later, Part 9

Imagine applying for ten jobs at ten different companies.

Each employer has its own managers, hiring requirements, and recruiting procedures. Each application appears to offer another opportunity to demonstrate your qualifications.

But what if several of those employers rely on automated assessments developed by the same technology vendor?

You might receive ten unfavorable screening recommendations without receiving ten genuinely independent evaluations of your qualifications.

A major study published in May 2026 provides evidence that this concern is more than theoretical.

Researchers from Stanford University and other institutions analyzed more than four million actual job applications and identified racial disparities and repeated unfavorable recommendations associated with a common automated-screening vendor.

They examined a phenomenon known as algorithmic monoculture: different decision-makers relying on identical or similar algorithms and consequently producing correlated outcomes.

Their findings raise important questions about employment discrimination, assessment validity, vendor responsibility, and whether applicants receive genuinely independent opportunities when they apply to different employers.

Four Million Applications, One Screening Vendor

The study, Algorithmic Monocultures in Hiring, examined approximately 4.2 million applications submitted by 3.37 million people to 1,746 positions across more than 150 employers.

The applications were submitted between 2018 and 2022 and evaluated using technology supplied by pymetrics, a developer of game-based employment assessments.

Applicants completed computerized exercises designed to measure characteristics such as attention, risk tolerance, and other behavioral tendencies.

Machine-learning models analyzed the results and generated recommendations about whether applicants should advance for further consideration.

The researchers identified 42 screening models in the data.

These models did not necessarily perform identically, but multiple employers relied on models developed within the same vendor's assessment system.

Importantly, the study examined algorithmic recommendations, not final hiring decisions. An unfavorable recommendation did not necessarily mean that an employer rejected the applicant.

Nevertheless, recommendations can significantly influence which candidates receive interviews or further consideration.

The researchers identified two particularly important patterns.

Racial Disparities Hidden by the Averages

When the researchers combined screening outcomes across all positions, the overall results did not show adverse impact under the four-fifths measure they applied.

But evaluating individual positions revealed a different picture.

Approximately 25.9% of applications submitted by Black applicants and 14.7% of applications submitted by Asian applicants were directed to positions where the screening recommendations met the study's criteria for adverse impact against those racial groups.

The researchers estimated that approximately 40,000 additional applications from Black and Asian candidates would have received favorable recommendations if the relevant groups had been recommended at the rates of the most-favored groups.

Those are application-level findings, not percentages of all Black or Asian individuals who suffered unlawful discrimination.

The distinction between aggregate results and position-specific outcomes is critical.

An assessment might recommend Black applicants at relatively high rates for certain jobs while recommending them at much lower rates for other positions.

Combining all positions can conceal those differences.

A screening system can appear relatively fair when outcomes are averaged across thousands of jobs while producing concerning disparities in particular hiring processes.

That finding has important implications for employers and vendors that rely on broad assessments of algorithmic fairness.

The relevant question is not merely whether a product performs acceptably across an entire customer base. It is whether the selection procedures used for particular jobs satisfy applicable legal requirements.

The Four-Fifths Rule Is a Warning, Not a Verdict

The researchers examined adverse impact using the four-fifths rule associated with federal employment-selection guidelines.

Under that measure, a selection rate for one group below 80% of the rate for the most-favored group generally signals possible adverse impact.

But the rule is not a definitive legal test.

A selection-rate ratio below 80% does not automatically establish unlawful discrimination. Nor does a ratio above 80% necessarily demonstrate compliance.

Statistical significance, sample sizes, job requirements, validation evidence, business necessity, and other considerations may affect the legal analysis.

The study therefore did not establish that pymetrics or its employer customers violated Title VII.

Its more important finding is methodological.

Evaluating a common screening technology only through aggregate outcomes may conceal disparities that become apparent when particular jobs and selection procedures are examined.

That is a significant warning for employers relying on vendor-generated fairness reports.

When the Same Applicant Is Repeatedly Screened Out

The second major finding may have even broader implications.

The researchers examined applicants who submitted multiple applications to positions using the vendor's assessment technology.

Among applicants who applied to four positions, approximately 10% received unfavorable recommendations for all four.

Among those who applied to ten positions, approximately 4% received unfavorable recommendations for all ten.

These percentages describe applicants who submitted the specified numbers of applications, not the entire applicant population.

More importantly, the repeated unfavorable outcomes occurred more frequently than would be expected if the recommendations for different positions were statistically independent.

The researchers concluded that shared screening models could produce correlated outcomes across employers.

In practical terms, an applicant who receives an unfavorable recommendation for one position may face an elevated risk of receiving unfavorable recommendations from other employers using related models.

This does not mean every company uses identical hiring criteria or that all unsuccessful applicants would have been qualified for every position.

But it challenges a fundamental assumption about job searching.

Applying to another employer may not provide a genuinely fresh assessment when both employers rely on closely related screening technology.

The First Domino—Thirteen Years Later

In August 2013, this blog published When the First Domino Falls: Consequences to Employers of Embracing Workforce Assessment Solutions.

The concern was that widespread reliance on commercially developed employment assessments could create consequences extending across numerous employers.

If common screening criteria improperly disadvantaged qualified applicants, their effects could be reproduced wherever the assessments were used.

Thirteen years later, the Stanford-led study provides empirical evidence supporting an important part of that concern.

It shows that a common assessment vendor can generate correlated unfavorable recommendations across employers, including repeated negative outcomes for the same individuals.

It also shows that broad assessments of fairness can overlook disparities within particular hiring processes.

The research does not establish that the vendor engaged in unlawful discrimination or that every employer using its assessments violated employment laws.

But it identifies a mechanism through which potential problems in employment screening can extend well beyond an individual company.

The first domino may not fall within one employer's hiring process. Similar screening outcomes may occur across numerous employers using related assessment technologies.

What About Applicants With Mental Health Disabilities?

Although the research focused on racial disparities and repeated screening outcomes, it raises another important question.

Could common behavioral assessments repeatedly disadvantage applicants with mental health disabilities?

Consider an applicant with depression, PTSD, an anxiety disorder, or another qualifying disability.

Suppose a computerized assessment evaluates response speed, sustained attention, risk preferences, or patterns of social interaction.

Depending on how those characteristics are measured and used, the assessment might disadvantage some applicants because of disability-related characteristics rather than an inability to perform essential job functions.

If multiple employers use related assessment models, an affected applicant could encounter similar barriers repeatedly.

But the distinction between a legitimate job-related measurement and an unlawful screening criterion is essential.

Some behavioral characteristics may be relevant to particular jobs. An assessment does not violate the Americans with Disabilities Act (ADA) simply because it measures attention, reaction time, or behavioral preferences.

The legal questions include whether the criteria unlawfully screen out qualified individuals with disabilities, whether they satisfy applicable job-relatedness and business-necessity requirements, and whether reasonable accommodation obligations have been met.

A separate question arises if an assessment constitutes a prohibited pre-offer medical examination or disability-related inquiry.

These are different ADA theories, each with its own requirements.

The Stanford study did not establish discrimination against applicants with mental health disabilities, nor did it determine that the assessments constituted prohibited medical examinations.

Nevertheless, its methodology suggests how researchers might investigate whether disability-related characteristics produce similar patterns of repeated unfavorable recommendations.

That investigation would require relevant disability information, appropriate privacy safeguards, and careful examination of how the assessments operate.

The possibility deserves serious attention.

Who Is Responsible for a Shared Screening System?

The findings also raise questions about legal responsibility.

Employers generally retain obligations under federal employment discrimination laws when they use third-party screening systems.

The fact that a vendor developed an assessment does not automatically establish that the employer's use of the assessment is lawful.

Likewise, a vendor's assertion that its product has been validated does not necessarily demonstrate that it is appropriate for every job or employer.

The relevant evidence may include the assessment's actual criteria, validation studies, model versions, applicant characteristics, selection rates, accommodations, and the employer's use of the resulting recommendations.

A separate question concerns whether the vendor itself may face liability.

The ongoing lawsuit Mobley v. Workday, Inc. is testing whether an automated hiring vendor can face liability under particular employment discrimination theories, including when it acts as an employer's agent in performing delegated hiring functions.

Workday disputes the allegations, and no final determination of unlawful discrimination has been made.

The Stanford study and Mobley concern different vendors, technologies, evidence, and legal questions.

Their significance should not be conflated.

But together they demonstrate why the legal analysis cannot always stop with the employer whose name appears on the job advertisement.

When technology vendors play substantial roles in evaluating applicants, understanding those roles becomes increasingly important.

Why Independent Research Matters

One of the most important features of the Stanford study is its use of actual applicant-screening records.

Much of the public discussion of AI employment assessments relies on product descriptions, laboratory simulations, vendor validation studies, or theoretical models.

This research examined millions of recommendations generated in operational hiring systems.

That makes its findings particularly valuable.

At the same time, the limitations matter.

The study analyzed historical data from one vendor using particular game-based assessments. It did not establish that all AI hiring systems produce similar disparities.

The researchers also could not establish all the underlying causes of the observed racial differences.

And unfavorable recommendations were not necessarily equivalent to actual employment denials.

These qualifications do not diminish the study's importance.

They help identify the additional evidence needed to evaluate whether screening systems are reliable, appropriately validated, and lawful.

For employers, the lesson is that claims about fairness should be tested against actual outcomes for the jobs and populations involved.

For vendors, it is that product-wide averages may not adequately describe what happens within individual customer hiring processes.

For regulators and researchers, the study demonstrates the value of obtaining data that permit both position-specific analysis and examination of applicants' experiences across employers.

When the Same Door Closes Everywhere

Artificial intelligence may enable employers to evaluate large applicant pools more efficiently.

But efficiency and consistency are not the same as fairness.

A system that consistently identifies genuinely qualified applicants may provide important benefits.

A system that repeatedly disadvantages qualified applicants for reasons unrelated to their ability to perform the job presents a different concern.

The Stanford researchers found evidence of racial disparities in actual automated screening recommendations and correlated unfavorable outcomes across different employers.

They did not establish that those outcomes necessarily constituted unlawful discrimination.

But their findings challenge the assumption that applicants receive genuinely independent opportunities each time they apply to another company.

When many employers rely on the same or closely related assessments, an unsuccessful applicant may not be starting over with each new application. The applicant may be encountering substantially the same screening judgment again and again.

Thirteen years after this blog warned about the systemic consequences of commercially developed workforce assessments, the 2026 research gives that warning a new empirical foundation.

The question is no longer simply whether one employment assessment can produce discriminatory effects. It is whether widespread reliance on common assessment technologies can reproduce those effects across employers—and systematically limit opportunities throughout the labor market.

Next: The Human-in-the-Loop Illusion—When recruiters rely on AI recommendations, how independent are their decisions?