When Retention Becomes the Test

 

How AI Can Mistake Workplace Barriers for Employee Characteristics

Suppose an employer wants to reduce employee turnover.

It hires a technology company to develop an AI-powered assessment identifying applicants most likely to remain employed for several years.

The company analyzes employment records, comparing personality assessments, work histories, attendance patterns, and other characteristics with how long employees stayed.

It discovers statistically significant relationships and develops an assessment that appears capable of predicting retention.

But there is a fundamental question.

Did employees leave because of their personal characteristics—or because of the conditions under which they were expected to work?

The answer matters because predicting who will remain employed is not necessarily the same as identifying who can perform the job successfully.

When Leaving a Job Is Not a Measure of Ability

Consider two employees hired for the same position.

Both possess the necessary skills and perform their duties effectively.

One receives appropriate training, supportive supervision, and the resources needed to succeed.

The other encounters scheduling conflicts, inadequate support, and workplace practices that make the job unnecessarily difficult.

After three years, the first employee remains. The second has left.

The employer's records show different employment outcomes, but they do not explain why.

The employee who left may have been equally capable. Different working conditions might have produced a different result.

Retention measures how long someone remained employed. It does not necessarily measure how well that person could perform the job.

Employees leave for many reasons, including compensation, management practices, family responsibilities, advancement opportunities, and competing job offers.

Some factors may legitimately help predict retention. Others may reflect conditions within the employer's control.

When retention becomes the measure of success, those differences matter.

The Disability Connection

Consider an employee with an anxiety disorder who can perform the essential functions of a job but needs a modified schedule or another reasonable accommodation.

Suppose the employee requests an accommodation that would be reasonable and would not impose an undue hardship, but the employer fails to provide it.

The employee struggles with the resulting conditions and eventually resigns.

The employer's records show that the employee left after eight months. They may not show that the departure followed an unresolved accommodation request.

Now imagine thousands of employment records containing similar gaps.

Some employees may have left because disabilities were not appropriately accommodated. Others may have encountered inaccessible technology, inflexible schedules, or discriminatory treatment.

Not every departure involving an employee with a disability reflects discrimination. But when workplace barriers contribute to turnover, historical retention data may partly reflect the employer's own practices.

An AI assessment trained on those records may learn which applicants are most likely to remain under existing workplace conditions—not which applicants are capable of doing the work.

The algorithm does not need information explicitly identifying disability.

It may discover relationships involving employment gaps, attendance histories, or scheduling preferences that, in some circumstances, correlate with disability-related experiences.

Those characteristics do not establish that any particular applicant has a disability. Nor are such relationships necessarily improper.

The concern arises when relying on them disadvantages qualified applicants because of disability, particularly when the underlying turnover resulted from barriers that reasonable accommodations could have addressed.

When Statistical Accuracy Conceals the Problem

Suppose the technology company discovers that applicants with certain characteristics are more likely to remain employed for three years.

It develops an assessment favoring those characteristics and tests it against historical retention records.

The results demonstrate predictive accuracy.

But what has been established?

Perhaps the assessment identifies characteristics genuinely associated with longer employment.

Perhaps it identifies characteristics associated with an employee's ability or willingness to tolerate particular workplace conditions.

Or perhaps it measures both.

In employment assessment research, the outcome used to evaluate a test is often called the criterion.

If retention is the criterion, an assessment that predicts retention may be statistically useful for that purpose.

But the relationship does not establish why employees stayed or left.

The employer may have influenced the outcome through its compensation practices, management decisions, workplace policies, or treatment of employees with disabilities.

A model can accurately predict who is likely to remain employed without identifying everyone who could perform the job successfully.

The problem is not necessarily that the prediction is inaccurate.

It is that the employer may use the prediction to make hiring decisions without understanding the conditions that produced the historical pattern.

When Workplace Barriers Become Hiring Standards

Now suppose the employer uses the assessment to screen applicants.

Candidates resembling employees who remained for several years receive higher scores.

Candidates resembling employees who left receive lower scores.

Some qualified applicants with disabilities may be disadvantaged because the model associates certain characteristics with shorter employment.

Those applicants may never have an opportunity to demonstrate that they could perform the job successfully with appropriate accommodations.

The process can reinforce itself.

Applicants predicted to have shorter tenure are screened out. Because they are never hired, their potential performance and retention remain unknown.

Future assessments may continue relying on the same historical patterns.

What began as a workplace problem may become a hiring standard that excludes people who could succeed if those workplace barriers were addressed.

The employer may believe it is improving retention when it is partly reproducing the consequences of its own employment practices.

The ADA Does Not End at the Hiring Decision

The Americans with Disabilities Act generally requires employers to provide reasonable accommodations to qualified individuals with disabilities, absent undue hardship.

Depending on the circumstances, accommodations may include modifications to schedules, equipment, or workplace practices.

The ADA does not require employers to eliminate essential job functions, excuse every performance problem, or retain employees regardless of performance.

Nor does turnover involving an employee with a disability automatically establish discrimination.

But employers cannot assume that historical retention patterns justify excluding applicants with similar characteristics.

When an assessment screens out or tends to screen out qualified individuals because of disability, the employer may need to establish that the challenged criterion is job-related and consistent with business necessity. Separate reasonable-accommodation obligations may also apply.

Retention may be a legitimate business objective. The legal concern is whether the particular screening criteria are justified and whether their use unlawfully disadvantages qualified applicants with disabilities.

Predictive accuracy does not, by itself, establish that an employment assessment complies with the ADA.

What Should Employers Actually Measure?

Before using retention to validate an AI assessment, employers should examine what their records actually reveal.

Were employees who left performing their jobs successfully?

Did departures result from performance problems, voluntary career changes, inadequate compensation, workplace barriers, or failures to provide reasonable accommodations?

Did employees have comparable access to training, advancement, supportive supervision, and necessary resources?

Would some employees who left have remained under different working conditions?

These questions may not have complete answers. But they help distinguish employee characteristics associated with retention from conditions that contributed to turnover.

Employers should also determine whether retention is an appropriate outcome for the positions involved and whether using it to screen applicants creates unjustified disability-related barriers.

An employer concerned about turnover may sometimes achieve better results by improving working conditions rather than attempting to identify applicants willing or able to tolerate existing ones.

When the Employer Becomes Part of the Prediction

The previous articles in this series examined how AI assessments can learn from historically selected workforces, biased performance evaluations, and incomplete information about rejected applicants.

Using retention as the measure of success introduces another problem.

The employer may have helped create the very outcome its algorithm is trained to predict.

An employee who leaves after one year may have been poorly suited to the position.

But that employee may also have been highly capable and unable—or unwilling—to continue working under conditions that could have been improved.

An assessment trained on retention records may not distinguish between those explanations.

Predicting who will stay is not the same as identifying who can succeed.

And when workplace barriers contribute to the departure of qualified employees with disabilities, using those departures to develop future hiring standards risks turning earlier barriers into continuing exclusion.

The fundamental question is not simply which applicants are most likely to remain employed.

It is whether the employer is measuring applicants' characteristics—or predicting the consequences of its own workplace practices.

---

Sources and Further Reading

• U.S. Equal Employment Opportunity Commission, Applying Performance and Conduct Standards to Employees with Disabilities.

• U.S. Equal Employment Opportunity Commission, Enforcement Guidance on Reasonable Accommodation and Undue Hardship under the Americans with Disabilities Act.

• U.S. Department of Justice, Algorithms, Artificial Intelligence, and Disability Discrimination in Hiring (May 12, 2022).

• Society for Industrial and Organizational Psychology, Principles for the Validation and Use of Personnel Selection Procedures, Fifth Edition (2018).

Next: When an AI Assessment Is Valid for One Job but Not Another—why evidence that a hiring test works in one setting may not establish that it is appropriate for other jobs or employers.

The Applicants Who Disappear from the Data

 Why an AI Hiring Test Cannot Measure the Performance of People It Never Hired

Imagine an employer that hires a technology company to develop an AI assessment predicting which applicants will become successful employees.

The company examines personality assessments, employment records, and performance evaluations. It identifies characteristics associated with stronger performance and develops a test favoring applicants with those traits.

The results appear promising. Assessment scores show a statistically significant relationship with job performance.

But there is a fundamental problem.

The company knows how its hired employees performed. It does not know how the rejected applicants would have performed.

And some of those rejected applicants might have become its best employees.

The Performance Records That Do Not Exist

Consider an employer that receives 10,000 applications for 500 positions.

Every applicant completes an assessment measuring personality characteristics, communication styles, and other traits the employer considers important.

The employer uses those scores, along with other information, to select 500 applicants.

Over the next two years, it evaluates their performance.

The technology company now has two sets of information for those employees: their original assessment scores and subsequent job performance.

It can examine whether higher scores predicted better performance among those hired.

But what about the other 9,500 applicants?

The company knows their assessment scores and that they were rejected. It does not know how they would have performed if hired.

Some might have struggled. Others might have performed equally well—or better.

The employer has performance data for the people it selected, not for those its selection process excluded.

That distinction matters when those records become the foundation for future hiring standards.

When the Sample Has Already Been Filtered

This problem involves two related statistical concerns: missing performance outcomes and range restriction.

Missing outcomes arise because rejected applicants never perform the job. Their potential performance cannot be directly observed.

Range restriction occurs when employees included in a study represent a narrower range of characteristics or assessment scores than the broader applicant population.

Suppose an employer historically favored applicants who scored highly on extraversion and assertiveness.

Its workforce may contain relatively few employees with lower scores.

A study comparing personality scores with job performance will therefore provide limited direct evidence about how less extroverted or assertive applicants might perform.

Researchers have statistical methods for addressing range restriction, but those techniques depend on assumptions about the selection process and available data.

The problem becomes more complicated when hiring decisions involve interviews, recommendations, and multiple assessments.

A relationship observed among selected employees does not automatically establish the same relationship among all qualified applicants.

That does not make an assessment invalid. It means researchers must examine whether the evidence supports applying their findings to a broader population.

The Disability Connection

These limitations are particularly important when historical hiring practices may have disadvantaged applicants with disabilities.

Consider an employer that traditionally favored candidates who appeared outgoing, enthusiastic, emotionally expressive, and comfortable with conventional workplace interactions.

Qualified applicants with autism, depression, anxiety disorders, or other disabilities may have been disadvantaged by those preferences.

Some may have been rejected because their communication styles or emotional expressions differed from what interviewers expected.

An AI assessment developed from the performance records of previously hired employees may therefore contain limited information about the potential performance of excluded applicants.

Yet the algorithm may learn that applicants resembling existing successful employees are more likely to succeed.

The absence of evidence that excluded applicants would succeed is not evidence that they would fail.

Under the Americans with Disabilities Act, an assessment that screens out or tends to screen out qualified individuals because of disability raises important legal questions.

The employer may need to establish that the challenged criterion is job-related and consistent with business necessity. Reasonable-accommodation obligations may also apply.

Statistical limitations do not automatically establish discrimination. But favorable validation results do not automatically establish ADA compliance, either.

What Does a Validation Study Actually Prove?

Employment assessments are often evaluated by examining whether test scores predict subsequent job performance.

This approach, known as criterion-related validation, is an established method of evaluating employee-selection procedures.

Properly conducted, these studies can provide valuable evidence about whether an assessment predicts relevant aspects of performance.

But their conclusions depend on the data, research design, and population studied.

Suppose an assessment accurately predicts performance among the 500 employees hired in our example.

That finding may be meaningful.

What it does not necessarily establish is that the other 9,500 applicants lacked the ability to perform the job—or that the employer selected the best 500 candidates.

Those conclusions require additional evidence.

The federal Uniform Guidelines on Employee Selection Procedures recognize the importance of appropriate validation methods, including consideration of restricted score ranges and the populations to which findings will be applied.

An assessment can predict performance among selected employees without proving that rejected applicants would have performed poorly.

The question is not whether every rejected applicant's performance must be observed. That is generally impossible.

It is whether the study provides a sound basis for drawing conclusions about applicants whose performance was never measured.

Why More Data May Not Solve the Problem

Artificial intelligence can analyze enormous amounts of employment information.

A vendor may possess records from hundreds of employers and millions of applicants.

But database size does not necessarily resolve its limitations.

Suppose a database contains assessment scores for one million applicants, but performance outcomes for only the 50,000 who were hired.

The system has extensive information about those employees.

It still lacks direct evidence of how the other 950,000 applicants would have performed.

If participating employers used similar hiring criteria or assumptions about desirable personality characteristics, the missing information may be especially important.

A larger database may provide more precise estimates of relationships among the employees studied without establishing whether those relationships apply equally to excluded applicants.

More data cannot, by itself, supply performance outcomes that were never observed.

AI can estimate those missing outcomes, but its predictions depend on assumptions and patterns learned from existing information.

Predictions are not the same as observed performance.

What Responsible Validation Requires

None of this means employment assessments are inherently unreliable or that existing-employee data have no legitimate value.

Researchers have long recognized the problems created by restricted samples and missing performance information.

They have developed methods for addressing some of these limitations, including statistical corrections, independent validation samples, and research examining different applicant populations.

But those methods require sound assumptions and careful interpretation.

Employers and vendors should examine how employees in validation studies were selected, whether important applicant groups are underrepresented, and whether findings can reasonably be applied to applicants with different characteristics.

They should also determine whether an assessment measures genuine job requirements and whether qualified applicants with disabilities may be improperly excluded.

The objective is not to guarantee that every rejected applicant would have succeeded.

It is to establish meaningful evidence of job-related ability without unlawfully screening out qualified individuals.

The People an Algorithm Cannot See

The previous articles in this series examined two related problems.

When the Workforce Becomes the Test explained how historical hiring decisions can produce a workforce that does not represent everyone capable of performing a job.

When Success Becomes the Test examined how biased performance evaluations can cause AI assessments to learn supervisors' preferences rather than genuine measures of ability.

There is a third problem.

Even when performance evaluations are accurate, they exist only for people who were given the opportunity to perform.

An AI assessment may identify genuine relationships among employees who were hired. It may even outperform traditional hiring methods.

But those findings do not establish that applicants receiving lower scores would necessarily have performed poorly.

Some may possess abilities the assessment fails to recognize. Their potential contributions remain unobserved.

An algorithm cannot learn directly from the job performance of people who were never given the opportunity to perform the job.

And that raises a question every employer relying on AI hiring assessments should consider:

How many capable applicants are missing from the data used to determine who is capable?

---

Sources and Further Reading

• U.S. Equal Employment Opportunity Commission and other federal agencies, Uniform Guidelines on Employee Selection Procedures, 29 C.F.R. Part 1607, particularly §§ 1607.5, 1607.14, and 1607.15.

• Society for Industrial and Organizational Psychology, Principles for the Validation and Use of Personnel Selection Procedures, Fifth Edition (2018).

• U.S. Department of Justice, Algorithms, Artificial Intelligence, and Disability Discrimination in Hiring (May 12, 2022).

Next: When Retention Becomes the Test—how employment data can cause AI to mistake workplace barriers for employee characteristics.

When Success Becomes the Test

 How Biased Performance Evaluations Can Shape AI Hiring Decisions

Suppose an employer wants to identify the personality characteristics of its most successful employees.

It hires a technology company to develop an AI-powered assessment that will identify applicants most likely to succeed.

The company examines personality questionnaires, performance evaluations, and employment records. It identifies characteristics shared by employees the employer considers its strongest performers.

The assessment may produce reliable scores and demonstrate statistically significant relationships between personality characteristics and performance ratings.

But there is a fundamental question:

How did the employer decide which employees were successful in the first place?

The answer matters because the assessment may predict something quite different from actual job performance.

When Favorable Evaluations Define Success

Consider two software developers.

Both consistently meet deadlines, produce reliable code, and solve difficult technical problems.

One is outgoing, speaks frequently in meetings, and enthusiastically promotes new ideas.

The other is quieter, communicates more deliberately, and demonstrates initiative through careful analysis and high-quality work.

Their measurable performance is comparable.

Yet their supervisors may evaluate them differently.

The outgoing employee might receive higher ratings for leadership, teamwork, communication, and attitude. The quieter employee might receive equally strong technical ratings but lower scores for initiative or cultural fit.

Those differences could reflect legitimate job requirements. But they might also reflect a supervisor's preference for one personality or communication style.

If those evaluations become the definition of success, an AI assessment may learn the supervisor's preferences rather than the requirements of the job.

The Disability Connection

This problem is particularly important for employees with psychiatric disabilities, autism, and other conditions that may affect communication, interaction, or emotional expression.

An autistic employee may communicate differently without being less capable of performing the job.

Someone experiencing depression may not consistently display the enthusiasm a supervisor associates with a positive attitude.

An employee with an anxiety disorder may find certain workplace interactions challenging while remaining highly capable of performing essential job duties.

People with these conditions do not share a single personality profile, and employers may legitimately evaluate job-related performance differences.

The concern is that supervisors may mistake differences in communication, emotional expression, or social behavior for differences in capability.

When these judgments influence performance ratings, they can become part of the data used to train employment assessments.

When Statistical Validation Validates the Wrong Thing

Suppose an AI developer discovers that employees with the highest supervisor ratings tend to score highly on measures of extraversion, assertiveness, and emotional expressiveness.

The developer builds an assessment favoring those characteristics and tests it against historical performance ratings.

The results show a statistically significant relationship.

But what has been demonstrated?

Perhaps the assessment identifies characteristics that genuinely contribute to performance. Perhaps it identifies characteristics supervisors prefer. Or perhaps it measures some combination of the two.

This illustrates a problem called criterion contamination.

When researchers validate an employment assessment, they compare its results against a measure of success, called the criterion.

If supervisor ratings are influenced by stereotypes or preferences unrelated to job performance, the criterion may be contaminated by bias. A related problem arises when ratings fail to capture important aspects of successful performance.

Even apparently objective measures, such as promotion histories or sales results, may reflect unequal access to assignments, resources, and opportunities.

An assessment can accurately predict who receives favorable evaluations without accurately predicting who can perform the job successfully.

Statistical sophistication cannot overcome a flawed definition of success.

How AI Can Turn Preferences into Hiring Standards

These concerns are not unique to AI. Traditional assessments and human hiring decisions can also reflect biased judgments.

But AI can analyze enormous amounts of data, identify subtle relationships, and apply those patterns to thousands of applicants.

It does not need to be instructed to discriminate.

If historical performance ratings reward particular communication styles, emotional expressions, or personality characteristics, an algorithm may incorporate those preferences into its scoring system.

Applicants with different characteristics may receive lower rankings even when they possess the skills needed for the job.

And because the assessment has been validated against historical performance ratings, its results may appear scientifically objective.

The algorithm may not eliminate subjective judgment. It may convert that judgment into an apparently objective hiring standard.

The ADA Does Not Disappear Behind an Algorithm

Under the Americans with Disabilities Act, employment assessments must not unlawfully exclude qualified applicants because of disability.

When a challenged criterion screens out or tends to screen out qualified individuals because of disability, the employer may need to establish that it is job-related and consistent with business necessity. Separate reasonable-accommodation requirements may also apply.

An employer therefore cannot assume that an assessment is lawful merely because its scores correlate with favorable evaluations of existing employees.

The employer must examine whether the challenged criteria are genuinely related to job performance and whether their use unlawfully disadvantages qualified applicants with disabilities.

Assessment vendors should likewise examine whether the performance measures used to validate their products accurately reflect job requirements and may contain bias.

A vendor's validation study does not automatically relieve an employer of its legal obligations.

The Question Behind the Numbers

Supervisor evaluations are not inherently unreliable. Nor are personality characteristics necessarily irrelevant to job performance.

The problem arises when employers confuse characteristics that contribute to successful performance with characteristics that merely influence how employees are perceived.

An employer developing an AI-powered assessment should examine how success is measured, whether different approaches to performing the job are recognized, and whether employees with disabilities are evaluated fairly.

A model that predicts biased evaluations may be statistically accurate while perpetuating the bias those evaluations contain.

AI can identify patterns in historical employment data with remarkable sophistication. But it cannot establish, without appropriate evidence, whether those patterns reflect genuine job requirements or the preferences of the people who produced the data.

If an employer's definition of success is biased, teaching an algorithm to recognize success may simply teach it to recognize that bias.

---

Sources and Further Reading

• U.S. Department of Justice, Algorithms, Artificial Intelligence, and Disability Discrimination in Hiring (May 12, 2022).

• Society for Industrial and Organizational Psychology, Considerations and Recommendations for the Validation and Use of AI-Based Assessments for Employee Selection (2023).

• U.S. Equal Employment Opportunity Commission, Applying Performance and Conduct Standards to Employees with Disabilities.

When the Workforce Becomes the Test

 How AI Can Turn Past Discrimination into Future Hiring Standards

Imagine an employer that has spent decades favoring applicants who fit a particular personality profile: outgoing, confident, enthusiastic, and comfortable with conventional workplace expectations.

Over the years, qualified applicants with depression, anxiety disorders, autism, or other disabilities may have been overlooked or excluded. Some may have experienced outright discrimination. Others may have been disadvantaged by assumptions about their communication styles, emotional responses, or ability to fit into the workplace.

Now suppose the employer hires a technology company to develop an AI-powered personality assessment.

The company studies the employer's most successful employees, identifies characteristics they have in common, and develops a test designed to find applicants with similar traits.

The resulting assessment may be sophisticated. It may produce consistent scores and demonstrate statistically significant relationships between certain personality characteristics and employee performance.

But there is a fundamental problem.

The assessment is learning from a workforce that may already reflect decades of discriminatory employment decisions.

The People Missing from the Data

People who were previously excluded are largely missing from the employer's data. Their abilities were never measured in that workplace. Their potential contributions were never observed.

The employer cannot determine from its existing workforce alone whether those excluded applicants might have performed just as well—or better.

This is a statistical problem known as selection bias.

The assessment identifies characteristics associated with success among people who were hired. It does not necessarily identify every combination of characteristics that could produce successful job performance.

Consider an employer whose highest-performing employees tend to be outgoing and socially assertive.

That finding does not establish that someone who is more reserved, communicates differently, or experiences anxiety cannot perform the same job successfully.

A different employee might achieve equally strong results through preparation, attentive listening, technical expertise, or a different approach to workplace relationships.

There may be many ways to succeed at a job, but an assessment trained on a historically narrow workforce may recognize only some of them.

This distinction is especially important for applicants with disabilities.

People with depression, anxiety disorders, autism, and other disabilities do not share a single personality profile. Nor does a diagnosis establish someone's ability—or inability—to perform a particular job.

Yet some assessments may assign lower scores based on emotional responses, communication patterns, or other characteristics associated with disability, even when those characteristics have little relationship to successful job performance.

The Department of Justice Recognized This Problem

In May 2022, the U.S. Department of Justice issued guidance titled Algorithms, Artificial Intelligence, and Disability Discrimination in Hiring.

The guidance specifically warned about hiring technologies that compare applicants with successful current employees.

DOJ explained that because people with disabilities have historically been excluded from many jobs, they may be missing from the workforce used to develop these technologies. As a result, the systems may discriminate against qualified applicants with disabilities.

That warning goes directly to the problem of using historical employment data to determine future hiring standards.

Under the Americans with Disabilities Act, an assessment that screens out or tends to screen out qualified individuals because of disability raises important legal questions. The employer may need to establish that the challenged selection criterion is job-related and consistent with business necessity. Separate reasonable-accommodation obligations may also apply.

A scientifically reliable personality assessment is not necessarily a legally permissible employment screening tool.

And an employer cannot assume that an assessment is lawful simply because no one intended it to discriminate.

How AI Can Reinforce Historical Exclusion

These problems are not unique to artificial intelligence. Traditional personality assessments and human hiring decisions can produce similar results.

But AI can identify subtle patterns, examine many variables, and apply its conclusions to thousands of applicants.

An algorithm does not need to be instructed to discriminate.

If the historical employment data reflect earlier discriminatory practices, the algorithm may learn those patterns and incorporate them into future hiring decisions.

The resulting assessment may favor applicants who resemble existing employees while disadvantaging others who could perform the same work equally well.

And the process can reinforce itself.

Applicants whose characteristics differ from the preferred profile may be screened out. Because they are never hired, their potential performance remains unknown. Future assessments may continue relying on data from employees selected under the same narrow criteria.

The employer may believe it is improving its hiring process through increasingly sophisticated analysis.

In reality, it may be making earlier selection preferences more systematic and harder to challenge.

Identifying Similarity Is Not Identifying Ability

None of this means that personality assessments are inherently discriminatory or that existing-employee data have no legitimate value.

But employers and assessment vendors must examine how the workforce was assembled, whether the assessment measures genuine job requirements, and whether qualified applicants with different characteristics are being improperly excluded.

The central question is not simply whether successful employees share certain personality traits.

It is whether those traits meaningfully predict job performance—and whether people with different characteristics might perform just as well.

Knowing what successful employees have in common does not tell us what capable applicants are missing.

That is the danger of turning the existing workforce into the test.

An algorithm may become very good at identifying people who resemble employees an organization previously chose to hire.

But identifying similarity is not the same as identifying ability.

And when historical employment practices have excluded qualified people with disabilities, using those practices as the foundation for future hiring decisions risks reproducing the very discrimination that employment laws were designed to prevent.

---

Source: U.S. Department of Justice, Algorithms, Artificial Intelligence, and Disability Discrimination in Hiring (May 12, 2022).

Next: When Success Becomes the Test—how biased employee evaluations can shape AI hiring decisions even after someone has been hired.

The Human-in-the-Loop Illusion: When Recruiters Follow the Algorithm

 Employment Testing: Failing to Make the Grade — Ten Years Later, Part 10

Imagine applying for a job and learning that artificial intelligence will evaluate your qualifications.

The employer reassures you that the software will not make the final hiring decision. A human recruiter will review the recommendations and decide which applicants deserve interviews.

That sounds reassuring.

But what if the recruiter simply follows the algorithm's recommendations?

Does human involvement protect applicants from biased automated screening—or merely give an algorithmic recommendation the appearance of independent judgment?

A University of Washington study published in October 2025 raises serious questions about that distinction.

The researchers found that people making simulated hiring decisions tended to reproduce racial preferences embedded in AI-generated recommendations, even though their choices were substantially more balanced without those recommendations.

The findings challenge a common assumption underlying automated hiring: that keeping a human involved necessarily provides meaningful protection against algorithmic discrimination.

What the Researchers Found

The study, No Thoughts Just AI: Biased LLM Hiring Recommendations Alter Human Decision Making and Limit Human Autonomy, involved 528 participants recruited through an online research platform.

Participants evaluated AI-generated resumes for 16 occupations, including computer systems analyst, nurse practitioner, and housekeeper.

Each exercise presented five applicants. Four were comparably qualified men whose names and organizational affiliations suggested different racial backgrounds. A fifth, less-qualified candidate helped obscure the experiment's purpose.

Participants selected three applicants for interviews.

The researchers supplied simulated AI recommendations that were racially neutral or favored particular groups to varying degrees. They also examined decisions made without AI recommendations.

The final analysis included 1,526 screening scenarios.

Participants were not using an actual commercial hiring platform. Instead, the researchers controlled the recommendations to investigate how algorithmic preferences influenced human decisions.

The results were striking.

Without AI recommendations, or when the recommendations were racially neutral, participants selected White and non-White candidates at approximately equal rates.

But when the simulated AI favored White candidates, participants increasingly favored White candidates.

When the AI favored non-White candidates, participants increasingly favored those applicants instead.

In the most severely biased experimental conditions involving higher-status occupations, participants favored the racial group preferred by the AI approximately 90% of the time.

This did not mean that 90% of professional recruiters automatically accepted every AI recommendation. The percentage describes participants' selection patterns under particular experimental conditions.

Nevertheless, it demonstrates how strongly biased recommendations could influence people who formally retained control over the decisions.

The participants were making the final selections, but those selections often reflected the preferences embedded in the AI recommendations.

The Problem Was Not Simply Human Prejudice

One of the study's most important findings was that participants did not demonstrate comparable racial preferences when making decisions without biased AI recommendations.

Their choices shifted when the recommendations favored particular racial groups.

That suggests a troubling possibility.

An automated system may not simply reproduce the biases of the people using it. It may introduce or amplify bias by influencing decision-makers whose choices would otherwise be more balanced.

The researchers also found that participants could be influenced even when they considered the AI's recommendations unimportant or of questionable quality.

Recognizing that AI may be unreliable, therefore, does not necessarily prevent its recommendations from shaping human judgment.

The experiment provides evidence of a problem that may arise whenever employers use AI to help evaluate job applicants.

A human decision-maker can unintentionally become the means through which algorithmic bias influences the final employment decision.

Human Review Is Not Necessarily Independent Review

Technology vendors frequently emphasize that their automated systems assist recruiters rather than make final hiring decisions.

That distinction can be meaningful.

A system that recommends candidates is not identical to one that automatically rejects applicants. Human review can provide an opportunity to identify errors, consider overlooked qualifications, and correct inappropriate recommendations.

But the value of that protection depends on how the review actually operates.

Consider a hypothetical screening system that ranks applicants from highest to lowest based on an algorithmic suitability score.

Highly ranked candidates appear prominently in the recruiter's interface. Lower-ranked applicants remain available, but reviewing them requires additional time and effort.

The recruiter formally retains complete discretion.

Yet the algorithm may determine which candidates receive meaningful consideration.

Does the recruiter independently evaluate applicants who receive low scores?

Does the recruiter understand what characteristics produced the rankings?

How often are unfavorable recommendations questioned or overridden?

And if the recruiter consistently follows the algorithm, how independent is the final decision?

The University of Washington study did not examine this specific workflow. But its findings provide a reason to investigate such practices in actual employment settings.

The central question is not simply who has authority to make the final decision. It is how much the algorithm influences the decision that person makes.

The Connection to Prior Post

The preceding post discussed a Stanford-led study examining approximately 4.2 million job applications evaluated through technology supplied by a common assessment vendor.

That research found racial disparities within particular screening processes and correlated unfavorable recommendations across employers.

Its findings suggested that applicants could encounter similar screening judgments when applying to different companies using related assessment models.

The University of Washington study examines the next stage of the hiring process.

What happens when a person receives an algorithm's recommendation?

The studies involved different technologies, methods, and populations. Their findings should not be combined as though they examined the same systems.

Nevertheless, they illustrate two related concerns.

A common assessment technology may repeatedly generate unfavorable recommendations for particular applicants.

And human reviewers may adopt biased recommendations rather than independently correcting them.

Together, these findings challenge the assumption that applicants receive a fresh, independent evaluation simply because different employers and different recruiters formally make the decisions.

What About Applicants With Mental Health Disabilities?

The University of Washington study examined racial preferences. It did not investigate discrimination against applicants with mental health disabilities.

Nevertheless, it raises a question relevant to the Americans with Disabilities Act (ADA).

Consider an applicant with depression, PTSD, an anxiety disorder, or another qualifying disability.

Suppose an automated assessment evaluates employment gaps, response speed, communication patterns, or other characteristics that may be associated with disability.

The assessment generates an unfavorable recommendation.

A recruiter accepts that recommendation without independently examining whether the relevant criteria appropriately measure the applicant's qualifications.

The recruiter may have no knowledge of the applicant's diagnosis.

That scenario would not automatically establish unlawful discrimination.

The applicable legal questions could include whether the screening criteria improperly exclude qualified individuals with disabilities, whether they satisfy job-relatedness and business-necessity requirements, and whether the employer complied with reasonable accommodation obligations.

A separate question could arise if an assessment constitutes a prohibited disability-related inquiry or medical examination.

The study did not address those legal questions.

But its findings suggest why disability-related screening barriers may remain consequential even when a human approves the final decision.

If a human reviewer relies heavily on a biased recommendation, requiring that person's approval may do little to correct the underlying problem.

Human Oversight Is Not a Legal Safe Harbor

Federal employment discrimination laws generally do not allow employers to avoid their obligations merely by using technology developed by someone else.

In May 2023, the Equal Employment Opportunity Commission issued technical assistance explaining how Title VII disparate-impact principles apply to algorithmic selection systems, including tools supplied by third-party vendors.

The ADA imposes separate obligations concerning disability discrimination, medical examinations, and reasonable accommodations.

The relevant legal inquiry is how the employer's selection procedures actually operate and whether the applicable statutory requirements have been satisfied.

Having a human make the final decision does not automatically cure unlawful screening criteria.

But reliance on AI does not automatically establish discrimination either.

A claim still requires evidence satisfying the applicable law.

The ongoing litigation in Mobley v. Workday, Inc. illustrates why the distinction matters.

Workday describes its AI-assisted recruiting technology as supporting human decision-making rather than independently making final hiring decisions.

Plaintiffs allege that certain applicant-screening services unlawfully disadvantage applicants based on protected characteristics. Workday disputes those allegations, and no final determination of unlawful discrimination has been made.

The University of Washington researchers did not study Workday's systems. Their findings establish nothing about whether Workday or its customers acted unlawfully.

They do, however, identify a question relevant to employers that emphasize human oversight:

Does the human reviewer meaningfully evaluate the applicant, or largely reproduce the algorithm's recommendation?

What Would Meaningful Human Oversight Look Like?

The answer is not necessarily to remove humans from the hiring process.

It is to make human review meaningful.

Employers should consider whether recruiters understand the limitations of algorithmic recommendations, have access to the information needed for independent evaluation, and possess genuine authority to question or override the results.

They should also examine how the screening interface presents rankings, whether unfavorable recommendations receive appropriate consideration, and whether actual selection outcomes reveal possible discrimination.

The researchers reported that completing an implicit association test before the screening exercise was associated with improvement in certain experimental outcomes.

That finding suggests that interventions may help. It does not establish that implicit-bias testing or brief training programs prevent discrimination in actual workplaces.

Nor would a high or low rate of overrides, by itself, establish whether human review is effective.

A well-designed algorithm may appropriately receive substantial agreement. Conversely, frequent overrides do not necessarily prove that a process is fair.

The issue is whether the employer can demonstrate that its screening procedures—including the interaction between humans and AI—appropriately evaluate job qualifications and comply with employment discrimination laws.

Meaningful oversight should be evaluated by what human reviewers actually do, not merely by whether their names appear at the end of the decision-making process.

The Human-in-the-Loop Illusion

The University of Washington study has important limitations.

Its participants were recruited online, rather than exclusively from professional recruiters. The recommendations were simulated, and the decisions involved controlled exercises rather than actual employment outcomes.

The researchers studied racial preferences, not disability discrimination.

The findings do not establish that all AI-assisted hiring systems produce discriminatory results or that all human reviewers defer to algorithms.

But they provide meaningful experimental evidence that biased AI recommendations can substantially influence human decisions.

That evidence should matter to employers increasingly relying on automated systems to evaluate large applicant pools.

Human participation may provide an important safeguard against error and discrimination.

But human participation is not the same as independent human judgment.

If reviewers routinely accept algorithmic recommendations without meaningful examination, the human approval process may do little to prevent the algorithm's shortcomings from shaping employment opportunities.

The question employers and technology vendors should be prepared to answer is therefore not simply whether a human remains involved.

It is whether that human exercises meaningful independent judgment—or merely gives the algorithm the final word.

California's New AI Employment Rules: The Return of the Illegal Psychological Test?

Employment Testing: Failing to Make the Grade — Ten Years Later, Part 11

In 2005, a federal appeals court confronted a question that remains surprisingly relevant today.

Could an employer violate the Americans with Disabilities Act (ADA) by requiring employees seeking promotion to complete what the employer described as a personality assessment?

In Karraker v. Rent-A-Center, 411 F.3d 831 (7th Cir. 2005), the court concluded that the assessment included a prohibited medical examination because it incorporated questions designed, at least in part, to reveal mental illness.

The employer maintained that the assessment measured personality characteristics relevant to management positions rather than psychiatric disorders.

The court rejected that distinction.

Calling a psychological examination a personality test did not remove it from the ADA's restrictions.

Twenty-one years later, the same question is returning in a different form.

Instead of lengthy psychological questionnaires, employers may use computerized games, timed exercises, behavioral assessments, video interviews, and algorithms that analyze applicants' responses.

These systems can measure attention, reaction time, risk preferences, communication patterns, and other characteristics that may overlap with manifestations of mental health conditions.

And California has now expressly addressed how employment discrimination laws apply to these technologies.

What Changed on October 1, 2025?

California's regulations governing the use of automated-decision systems in employment took effect on October 1, 2025.

Adopted by the California Civil Rights Council, the regulations clarify how the state's Fair Employment and Housing Act (FEHA) applies to artificial intelligence, algorithms, and other technologies that make employment decisions or assist humans in making them.

The regulations are not limited to sophisticated AI.

They encompass qualifying computerized processes used to screen resumes, rank applicants, administer employment assessments, or analyze information obtained during interviews.

They also address proxies: characteristics that may serve as substitutes for legally protected traits.

California did not invent a new prohibition against discrimination by computers. Employers were already subject to civil rights laws when using automated assessments.

What changed is that California expressly identified modern screening technologies and clarified how existing legal protections apply to their use.

Three distinct legal issues deserve attention: prohibited medical examinations, discriminatory selection criteria, and reasonable accommodations.

When Does a Computer Game Become a Medical Examination?

Perhaps the most significant provision appears in California Code of Regulations, title 2, § 11071(e).

It expressly recognizes that medical or psychological examinations and disability-related inquiries may be administered through automated-decision systems.

The provision identifies assessments involving tests, questions, puzzles, games, and other challenges likely to elicit information about a disability.

Consider a job applicant completing a computerized game that measures reaction time, attention, emotional responses, or patterns of decision-making.

The employer describes the exercise as a measure of judgment, resilience, adaptability, or workplace compatibility.

Those characteristics may be relevant to particular jobs.

But what if the assessment is designed to reveal psychiatric conditions or mental impairments?

What if the system draws inferences about an applicant's mental health from behavioral responses without asking directly whether the applicant has a disability?

These questions require examination of the assessment's purpose, design, operation, and the information it seeks to obtain.

Not every measurement of attention, reaction time, or personality constitutes a medical examination. Ordinary employment assessments may lawfully measure job-related abilities and characteristics.

But California has made an important principle explicit:

An employer cannot avoid restrictions on disability-related examinations simply by embedding the examination in a game or algorithm.

The Lesson of Karraker

The parallels with Karraker deserve attention.

Rent-A-Center required employees seeking management positions to complete a battery of tests that included 502 questions from the Minnesota Multiphasic Personality Inventory (MMPI).

The MMPI contained scales associated with psychiatric conditions and characteristics, including depression, paranoia, and mania.

Rent-A-Center argued that it used vocational scoring methods to assess personality traits relevant to management positions.

The Seventh Circuit disagreed that this distinction removed the examination from the ADA's protections.

The court emphasized that the MMPI was designed, at least in part, to reveal mental illness.

It also recognized that employees with psychiatric disorders might receive unfavorable scores affecting their opportunities for promotion.

Although the plaintiffs were existing employees, the parties treated the assessment as pre-offer for purposes of the litigation. The court did not independently resolve whether that classification was appropriate.

The decision does not establish that every psychological or personality assessment is a medical examination.

But its central lesson remains important:

What matters is not merely how an assessment is marketed or scored, but what the assessment is designed to reveal.

California's regulations bring that longstanding principle directly into the era of automated employment testing.

Discrimination Without Asking About a Diagnosis

A second legal concern is different from the prohibition against improper medical examinations.

An assessment may unlawfully disadvantage applicants with disabilities even if it never seeks medical information.

Consider an applicant with depression, PTSD, an anxiety disorder, or another qualifying condition.

An automated assessment might evaluate response speed, sustained attention, employment gaps, or communication patterns.

The system may never identify the applicant's diagnosis.

Nevertheless, particular selection criteria could disadvantage qualified applicants because of disability-related characteristics.

California's revised § 11072 addresses employment tests and selection criteria, including those administered through automated-decision systems, that unlawfully screen out or adversely affect individuals with disabilities.

The applicable analysis may involve job-relatedness, business necessity, and less discriminatory alternatives.

The regulations also recognize that statistical comparisons between applicants with and without disabilities are not necessarily required to establish that a particular individual was screened out because of disability.

That distinction matters.

A qualified applicant should not necessarily have to demonstrate a statistical disparity across an entire workforce to challenge an assessment that unlawfully excludes that individual.

An unfavorable score, however, does not automatically establish discrimination. The applicant's qualifications, the screening criterion, and the employer's justification remain relevant.

An algorithm can raise disability-discrimination concerns without ever asking an applicant to disclose a disability.

That is a separate legal question from whether the assessment constitutes a prohibited medical examination.

When a Lawful Assessment Still Requires Accommodation

A third issue receives less attention but may be particularly important.

Even an otherwise lawful employment assessment may create barriers for qualified applicants who need reasonable accommodations.

A timed computerized exercise might disadvantage someone whose disability affects processing speed.

A video assessment could create barriers for an applicant with a hearing, speech, or communication disability.

A game requiring particular physical responses might be inaccessible to someone with a motor impairment.

Depending on the circumstances, the employer may need to provide a reasonable accommodation or an appropriate alternative method for demonstrating qualifications.

Employers are not required to eliminate essential job functions or disregard legitimate qualifications.

But they must comply with applicable accommodation requirements.

The distinction is straightforward:

A test can be lawful in its design yet unlawfully administered when an employer fails to provide a required accommodation.

This issue becomes increasingly important as employers replace conventional interviews and written assessments with standardized digital experiences.

Can the Vendor Be Responsible?

California's regulations also address the role of third parties performing functions traditionally exercised by employers.

The definition of an employer's agent encompasses certain entities acting on an employer's behalf, potentially including those performing applicant recruitment or screening through automated systems.

This raises questions similar to those being litigated in Mobley v. Workday, Inc.

In that case, plaintiffs allege that Workday's automated applicant-screening services unlawfully disadvantage applicants based on protected characteristics.

Workday disputes the allegations, and no final determination of discrimination has been made.

The federal court has permitted certain claims to proceed under theories involving Workday's potential role as an employer's agent.

California's regulations do not establish Workday's liability, and supplying software does not automatically make a vendor legally responsible for every decision made by its customers.

But the broader question deserves attention:

When a vendor performs consequential parts of the employment-screening process, where does the employer's responsibility end—and where might the vendor's begin?

The answer depends on the parties' actual functions, their relationship, and the applicable statutes.

What Evidence Supports the Assessment?

California's regulations also recognize that anti-bias testing and related efforts may be relevant when evaluating allegations of discrimination.

The scope, quality, timing, results, and effectiveness of such testing may matter, as may the steps taken in response to identified problems.

The regulations do not create a universal requirement that every employer commission a particular independent AI audit.

Nor does the absence of an audit automatically establish unlawful discrimination.

But they reinforce the importance of evidence.

Suppose an employer purchases a commercial behavioral assessment that promises to identify candidates likely to perform well.

Has the assessment been validated for the particular jobs involved?

What characteristics does it measure?

Could those measurements reflect disabilities rather than relevant qualifications?

Were reasonable accommodations considered?

And what happened when testing or applicant complaints identified possible problems?

A vendor's general assertion that its product is scientifically validated may not adequately answer these questions.

Commercial confidentiality may complicate access to technical information, but it does not automatically excuse an employer's legal obligations.

Four Years of Records: What the New Rules Actually Do

California's regulations also address preservation of records involving automated employment decisions.

Covered entities generally must preserve relevant employment records, including applicable automated-decision-system data, for at least four years.

An important distinction deserves emphasis: the four-year retention period was already established by California legislation enacted in 2021.

The 2025 regulations clarify its application to relevant automated-decision-system information.

The final regulations also limit the preservation requirement to applicable records that the covered entity created or received, rather than requiring the employer to preserve every piece of information maintained independently by a software vendor.

That limitation matters, particularly when proprietary models and detailed assessment data are held by outside providers.

Nevertheless, the recordkeeping requirement could have substantial practical consequences.

Investigating a disputed assessment may require identifying the criteria used, the results generated, the relevant assessment version, and how recommendations influenced employment decisions.

Without that evidence, applicants and regulators may have difficulty determining what occurred.

The rules do not automatically require public disclosure of proprietary algorithms or source code.

But they reinforce a basic principle:

Meaningful enforcement of employment discrimination laws increasingly depends on preserving evidence of how automated assessments actually operated.

Why California Matters Beyond California

California's regulations do not automatically govern every employer in the United States.

Their application depends on California law and the circumstances of the employment relationship.

Nevertheless, national employers and screening vendors operating in California may need to account for these requirements when developing and administering their assessment systems.

More importantly, the regulations illustrate a broader legal development.

Questions once associated primarily with conventional psychological tests are now arising in computerized games, behavioral analytics, and AI-assisted hiring.

The underlying legal protections are not new.

What has changed is the sophistication and scale of the technologies to which those protections must be applied.

The Old Psychological Test in a New Package

When Karraker was decided in 2005, the challenged assessment contained questions from a conventional psychological inventory.

Today, a hiring system may measure behavioral traits through computerized games, infer characteristics from patterns of responses, and rank applicants using factors that correlate with disability.

Some of these technologies may offer legitimate benefits.

Others may create barriers that are difficult for applicants to recognize, explain, or challenge.

California's 2025 regulations make explicit that tests, puzzles, games, and other automated challenges may implicate restrictions on disability-related examinations, discriminatory screening, and reasonable accommodations.

Twenty-one years after Karraker, the technology has changed dramatically. The fundamental legal questions have not.

What does the assessment actually measure? Is that measurement appropriate for the job? Does it seek information that an employer is prohibited from obtaining? Does it unnecessarily exclude qualified applicants with disabilities?

Employers and assessment vendors should be prepared to answer these questions with evidence.

Calling an assessment artificial intelligence does not make it any more lawful than calling a psychological examination a personality test.

When One Hiring Algorithm Rejects the Same People Everywhere

 Employment Testing: Failing to Make the Grade — Ten Years Later, Part 9

Imagine applying for ten jobs at ten different companies.

Each employer has its own managers, hiring requirements, and recruiting procedures. Each application appears to offer another opportunity to demonstrate your qualifications.

But what if several of those employers rely on automated assessments developed by the same technology vendor?

You might receive ten unfavorable screening recommendations without receiving ten genuinely independent evaluations of your qualifications.

A major study published in May 2026 provides evidence that this concern is more than theoretical.

Researchers from Stanford University and other institutions analyzed more than four million actual job applications and identified racial disparities and repeated unfavorable recommendations associated with a common automated-screening vendor.

They examined a phenomenon known as algorithmic monoculture: different decision-makers relying on identical or similar algorithms and consequently producing correlated outcomes.

Their findings raise important questions about employment discrimination, assessment validity, vendor responsibility, and whether applicants receive genuinely independent opportunities when they apply to different employers.

Four Million Applications, One Screening Vendor

The study, Algorithmic Monocultures in Hiring, examined approximately 4.2 million applications submitted by 3.37 million people to 1,746 positions across more than 150 employers.

The applications were submitted between 2018 and 2022 and evaluated using technology supplied by pymetrics, a developer of game-based employment assessments.

Applicants completed computerized exercises designed to measure characteristics such as attention, risk tolerance, and other behavioral tendencies.

Machine-learning models analyzed the results and generated recommendations about whether applicants should advance for further consideration.

The researchers identified 42 screening models in the data.

These models did not necessarily perform identically, but multiple employers relied on models developed within the same vendor's assessment system.

Importantly, the study examined algorithmic recommendations, not final hiring decisions. An unfavorable recommendation did not necessarily mean that an employer rejected the applicant.

Nevertheless, recommendations can significantly influence which candidates receive interviews or further consideration.

The researchers identified two particularly important patterns.

Racial Disparities Hidden by the Averages

When the researchers combined screening outcomes across all positions, the overall results did not show adverse impact under the four-fifths measure they applied.

But evaluating individual positions revealed a different picture.

Approximately 25.9% of applications submitted by Black applicants and 14.7% of applications submitted by Asian applicants were directed to positions where the screening recommendations met the study's criteria for adverse impact against those racial groups.

The researchers estimated that approximately 40,000 additional applications from Black and Asian candidates would have received favorable recommendations if the relevant groups had been recommended at the rates of the most-favored groups.

Those are application-level findings, not percentages of all Black or Asian individuals who suffered unlawful discrimination.

The distinction between aggregate results and position-specific outcomes is critical.

An assessment might recommend Black applicants at relatively high rates for certain jobs while recommending them at much lower rates for other positions.

Combining all positions can conceal those differences.

A screening system can appear relatively fair when outcomes are averaged across thousands of jobs while producing concerning disparities in particular hiring processes.

That finding has important implications for employers and vendors that rely on broad assessments of algorithmic fairness.

The relevant question is not merely whether a product performs acceptably across an entire customer base. It is whether the selection procedures used for particular jobs satisfy applicable legal requirements.

The Four-Fifths Rule Is a Warning, Not a Verdict

The researchers examined adverse impact using the four-fifths rule associated with federal employment-selection guidelines.

Under that measure, a selection rate for one group below 80% of the rate for the most-favored group generally signals possible adverse impact.

But the rule is not a definitive legal test.

A selection-rate ratio below 80% does not automatically establish unlawful discrimination. Nor does a ratio above 80% necessarily demonstrate compliance.

Statistical significance, sample sizes, job requirements, validation evidence, business necessity, and other considerations may affect the legal analysis.

The study therefore did not establish that pymetrics or its employer customers violated Title VII.

Its more important finding is methodological.

Evaluating a common screening technology only through aggregate outcomes may conceal disparities that become apparent when particular jobs and selection procedures are examined.

That is a significant warning for employers relying on vendor-generated fairness reports.

When the Same Applicant Is Repeatedly Screened Out

The second major finding may have even broader implications.

The researchers examined applicants who submitted multiple applications to positions using the vendor's assessment technology.

Among applicants who applied to four positions, approximately 10% received unfavorable recommendations for all four.

Among those who applied to ten positions, approximately 4% received unfavorable recommendations for all ten.

These percentages describe applicants who submitted the specified numbers of applications, not the entire applicant population.

More importantly, the repeated unfavorable outcomes occurred more frequently than would be expected if the recommendations for different positions were statistically independent.

The researchers concluded that shared screening models could produce correlated outcomes across employers.

In practical terms, an applicant who receives an unfavorable recommendation for one position may face an elevated risk of receiving unfavorable recommendations from other employers using related models.

This does not mean every company uses identical hiring criteria or that all unsuccessful applicants would have been qualified for every position.

But it challenges a fundamental assumption about job searching.

Applying to another employer may not provide a genuinely fresh assessment when both employers rely on closely related screening technology.

The First Domino—Thirteen Years Later

In August 2013, this blog published When the First Domino Falls: Consequences to Employers of Embracing Workforce Assessment Solutions.

The concern was that widespread reliance on commercially developed employment assessments could create consequences extending across numerous employers.

If common screening criteria improperly disadvantaged qualified applicants, their effects could be reproduced wherever the assessments were used.

Thirteen years later, the Stanford-led study provides empirical evidence supporting an important part of that concern.

It shows that a common assessment vendor can generate correlated unfavorable recommendations across employers, including repeated negative outcomes for the same individuals.

It also shows that broad assessments of fairness can overlook disparities within particular hiring processes.

The research does not establish that the vendor engaged in unlawful discrimination or that every employer using its assessments violated employment laws.

But it identifies a mechanism through which potential problems in employment screening can extend well beyond an individual company.

The first domino may not fall within one employer's hiring process. Similar screening outcomes may occur across numerous employers using related assessment technologies.

What About Applicants With Mental Health Disabilities?

Although the research focused on racial disparities and repeated screening outcomes, it raises another important question.

Could common behavioral assessments repeatedly disadvantage applicants with mental health disabilities?

Consider an applicant with depression, PTSD, an anxiety disorder, or another qualifying disability.

Suppose a computerized assessment evaluates response speed, sustained attention, risk preferences, or patterns of social interaction.

Depending on how those characteristics are measured and used, the assessment might disadvantage some applicants because of disability-related characteristics rather than an inability to perform essential job functions.

If multiple employers use related assessment models, an affected applicant could encounter similar barriers repeatedly.

But the distinction between a legitimate job-related measurement and an unlawful screening criterion is essential.

Some behavioral characteristics may be relevant to particular jobs. An assessment does not violate the Americans with Disabilities Act (ADA) simply because it measures attention, reaction time, or behavioral preferences.

The legal questions include whether the criteria unlawfully screen out qualified individuals with disabilities, whether they satisfy applicable job-relatedness and business-necessity requirements, and whether reasonable accommodation obligations have been met.

A separate question arises if an assessment constitutes a prohibited pre-offer medical examination or disability-related inquiry.

These are different ADA theories, each with its own requirements.

The Stanford study did not establish discrimination against applicants with mental health disabilities, nor did it determine that the assessments constituted prohibited medical examinations.

Nevertheless, its methodology suggests how researchers might investigate whether disability-related characteristics produce similar patterns of repeated unfavorable recommendations.

That investigation would require relevant disability information, appropriate privacy safeguards, and careful examination of how the assessments operate.

The possibility deserves serious attention.

Who Is Responsible for a Shared Screening System?

The findings also raise questions about legal responsibility.

Employers generally retain obligations under federal employment discrimination laws when they use third-party screening systems.

The fact that a vendor developed an assessment does not automatically establish that the employer's use of the assessment is lawful.

Likewise, a vendor's assertion that its product has been validated does not necessarily demonstrate that it is appropriate for every job or employer.

The relevant evidence may include the assessment's actual criteria, validation studies, model versions, applicant characteristics, selection rates, accommodations, and the employer's use of the resulting recommendations.

A separate question concerns whether the vendor itself may face liability.

The ongoing lawsuit Mobley v. Workday, Inc. is testing whether an automated hiring vendor can face liability under particular employment discrimination theories, including when it acts as an employer's agent in performing delegated hiring functions.

Workday disputes the allegations, and no final determination of unlawful discrimination has been made.

The Stanford study and Mobley concern different vendors, technologies, evidence, and legal questions.

Their significance should not be conflated.

But together they demonstrate why the legal analysis cannot always stop with the employer whose name appears on the job advertisement.

When technology vendors play substantial roles in evaluating applicants, understanding those roles becomes increasingly important.

Why Independent Research Matters

One of the most important features of the Stanford study is its use of actual applicant-screening records.

Much of the public discussion of AI employment assessments relies on product descriptions, laboratory simulations, vendor validation studies, or theoretical models.

This research examined millions of recommendations generated in operational hiring systems.

That makes its findings particularly valuable.

At the same time, the limitations matter.

The study analyzed historical data from one vendor using particular game-based assessments. It did not establish that all AI hiring systems produce similar disparities.

The researchers also could not establish all the underlying causes of the observed racial differences.

And unfavorable recommendations were not necessarily equivalent to actual employment denials.

These qualifications do not diminish the study's importance.

They help identify the additional evidence needed to evaluate whether screening systems are reliable, appropriately validated, and lawful.

For employers, the lesson is that claims about fairness should be tested against actual outcomes for the jobs and populations involved.

For vendors, it is that product-wide averages may not adequately describe what happens within individual customer hiring processes.

For regulators and researchers, the study demonstrates the value of obtaining data that permit both position-specific analysis and examination of applicants' experiences across employers.

When the Same Door Closes Everywhere

Artificial intelligence may enable employers to evaluate large applicant pools more efficiently.

But efficiency and consistency are not the same as fairness.

A system that consistently identifies genuinely qualified applicants may provide important benefits.

A system that repeatedly disadvantages qualified applicants for reasons unrelated to their ability to perform the job presents a different concern.

The Stanford researchers found evidence of racial disparities in actual automated screening recommendations and correlated unfavorable outcomes across different employers.

They did not establish that those outcomes necessarily constituted unlawful discrimination.

But their findings challenge the assumption that applicants receive genuinely independent opportunities each time they apply to another company.

When many employers rely on the same or closely related assessments, an unsuccessful applicant may not be starting over with each new application. The applicant may be encountering substantially the same screening judgment again and again.

Thirteen years after this blog warned about the systemic consequences of commercially developed workforce assessments, the 2026 research gives that warning a new empirical foundation.

The question is no longer simply whether one employment assessment can produce discriminatory effects. It is whether widespread reliance on common assessment technologies can reproduce those effects across employers—and systematically limit opportunities throughout the labor market.

Next: The Human-in-the-Loop Illusion—When recruiters rely on AI recommendations, how independent are their decisions?