When Retention Becomes the Test

 

How AI Can Mistake Workplace Barriers for Employee Characteristics

Suppose an employer wants to reduce employee turnover.

It hires a technology company to develop an AI-powered assessment identifying applicants most likely to remain employed for several years.

The company analyzes employment records, comparing personality assessments, work histories, attendance patterns, and other characteristics with how long employees stayed.

It discovers statistically significant relationships and develops an assessment that appears capable of predicting retention.

But there is a fundamental question.

Did employees leave because of their personal characteristics—or because of the conditions under which they were expected to work?

The answer matters because predicting who will remain employed is not necessarily the same as identifying who can perform the job successfully.

When Leaving a Job Is Not a Measure of Ability

Consider two employees hired for the same position.

Both possess the necessary skills and perform their duties effectively.

One receives appropriate training, supportive supervision, and the resources needed to succeed.

The other encounters scheduling conflicts, inadequate support, and workplace practices that make the job unnecessarily difficult.

After three years, the first employee remains. The second has left.

The employer's records show different employment outcomes, but they do not explain why.

The employee who left may have been equally capable. Different working conditions might have produced a different result.

Retention measures how long someone remained employed. It does not necessarily measure how well that person could perform the job.

Employees leave for many reasons, including compensation, management practices, family responsibilities, advancement opportunities, and competing job offers.

Some factors may legitimately help predict retention. Others may reflect conditions within the employer's control.

When retention becomes the measure of success, those differences matter.

The Disability Connection

Consider an employee with an anxiety disorder who can perform the essential functions of a job but needs a modified schedule or another reasonable accommodation.

Suppose the employee requests an accommodation that would be reasonable and would not impose an undue hardship, but the employer fails to provide it.

The employee struggles with the resulting conditions and eventually resigns.

The employer's records show that the employee left after eight months. They may not show that the departure followed an unresolved accommodation request.

Now imagine thousands of employment records containing similar gaps.

Some employees may have left because disabilities were not appropriately accommodated. Others may have encountered inaccessible technology, inflexible schedules, or discriminatory treatment.

Not every departure involving an employee with a disability reflects discrimination. But when workplace barriers contribute to turnover, historical retention data may partly reflect the employer's own practices.

An AI assessment trained on those records may learn which applicants are most likely to remain under existing workplace conditions—not which applicants are capable of doing the work.

The algorithm does not need information explicitly identifying disability.

It may discover relationships involving employment gaps, attendance histories, or scheduling preferences that, in some circumstances, correlate with disability-related experiences.

Those characteristics do not establish that any particular applicant has a disability. Nor are such relationships necessarily improper.

The concern arises when relying on them disadvantages qualified applicants because of disability, particularly when the underlying turnover resulted from barriers that reasonable accommodations could have addressed.

When Statistical Accuracy Conceals the Problem

Suppose the technology company discovers that applicants with certain characteristics are more likely to remain employed for three years.

It develops an assessment favoring those characteristics and tests it against historical retention records.

The results demonstrate predictive accuracy.

But what has been established?

Perhaps the assessment identifies characteristics genuinely associated with longer employment.

Perhaps it identifies characteristics associated with an employee's ability or willingness to tolerate particular workplace conditions.

Or perhaps it measures both.

In employment assessment research, the outcome used to evaluate a test is often called the criterion.

If retention is the criterion, an assessment that predicts retention may be statistically useful for that purpose.

But the relationship does not establish why employees stayed or left.

The employer may have influenced the outcome through its compensation practices, management decisions, workplace policies, or treatment of employees with disabilities.

A model can accurately predict who is likely to remain employed without identifying everyone who could perform the job successfully.

The problem is not necessarily that the prediction is inaccurate.

It is that the employer may use the prediction to make hiring decisions without understanding the conditions that produced the historical pattern.

When Workplace Barriers Become Hiring Standards

Now suppose the employer uses the assessment to screen applicants.

Candidates resembling employees who remained for several years receive higher scores.

Candidates resembling employees who left receive lower scores.

Some qualified applicants with disabilities may be disadvantaged because the model associates certain characteristics with shorter employment.

Those applicants may never have an opportunity to demonstrate that they could perform the job successfully with appropriate accommodations.

The process can reinforce itself.

Applicants predicted to have shorter tenure are screened out. Because they are never hired, their potential performance and retention remain unknown.

Future assessments may continue relying on the same historical patterns.

What began as a workplace problem may become a hiring standard that excludes people who could succeed if those workplace barriers were addressed.

The employer may believe it is improving retention when it is partly reproducing the consequences of its own employment practices.

The ADA Does Not End at the Hiring Decision

The Americans with Disabilities Act generally requires employers to provide reasonable accommodations to qualified individuals with disabilities, absent undue hardship.

Depending on the circumstances, accommodations may include modifications to schedules, equipment, or workplace practices.

The ADA does not require employers to eliminate essential job functions, excuse every performance problem, or retain employees regardless of performance.

Nor does turnover involving an employee with a disability automatically establish discrimination.

But employers cannot assume that historical retention patterns justify excluding applicants with similar characteristics.

When an assessment screens out or tends to screen out qualified individuals because of disability, the employer may need to establish that the challenged criterion is job-related and consistent with business necessity. Separate reasonable-accommodation obligations may also apply.

Retention may be a legitimate business objective. The legal concern is whether the particular screening criteria are justified and whether their use unlawfully disadvantages qualified applicants with disabilities.

Predictive accuracy does not, by itself, establish that an employment assessment complies with the ADA.

What Should Employers Actually Measure?

Before using retention to validate an AI assessment, employers should examine what their records actually reveal.

Were employees who left performing their jobs successfully?

Did departures result from performance problems, voluntary career changes, inadequate compensation, workplace barriers, or failures to provide reasonable accommodations?

Did employees have comparable access to training, advancement, supportive supervision, and necessary resources?

Would some employees who left have remained under different working conditions?

These questions may not have complete answers. But they help distinguish employee characteristics associated with retention from conditions that contributed to turnover.

Employers should also determine whether retention is an appropriate outcome for the positions involved and whether using it to screen applicants creates unjustified disability-related barriers.

An employer concerned about turnover may sometimes achieve better results by improving working conditions rather than attempting to identify applicants willing or able to tolerate existing ones.

When the Employer Becomes Part of the Prediction

The previous articles in this series examined how AI assessments can learn from historically selected workforces, biased performance evaluations, and incomplete information about rejected applicants.

Using retention as the measure of success introduces another problem.

The employer may have helped create the very outcome its algorithm is trained to predict.

An employee who leaves after one year may have been poorly suited to the position.

But that employee may also have been highly capable and unable—or unwilling—to continue working under conditions that could have been improved.

An assessment trained on retention records may not distinguish between those explanations.

Predicting who will stay is not the same as identifying who can succeed.

And when workplace barriers contribute to the departure of qualified employees with disabilities, using those departures to develop future hiring standards risks turning earlier barriers into continuing exclusion.

The fundamental question is not simply which applicants are most likely to remain employed.

It is whether the employer is measuring applicants' characteristics—or predicting the consequences of its own workplace practices.

---

Sources and Further Reading

• U.S. Equal Employment Opportunity Commission, Applying Performance and Conduct Standards to Employees with Disabilities.

• U.S. Equal Employment Opportunity Commission, Enforcement Guidance on Reasonable Accommodation and Undue Hardship under the Americans with Disabilities Act.

• U.S. Department of Justice, Algorithms, Artificial Intelligence, and Disability Discrimination in Hiring (May 12, 2022).

• Society for Industrial and Organizational Psychology, Principles for the Validation and Use of Personnel Selection Procedures, Fifth Edition (2018).

Next: When an AI Assessment Is Valid for One Job but Not Another—why evidence that a hiring test works in one setting may not establish that it is appropriate for other jobs or employers.