When the Workforce Becomes the Test

 How AI Can Turn Past Discrimination into Future Hiring Standards

Imagine an employer that has spent decades favoring applicants who fit a particular personality profile: outgoing, confident, enthusiastic, and comfortable with conventional workplace expectations.

Over the years, qualified applicants with depression, anxiety disorders, autism, or other disabilities may have been overlooked or excluded. Some may have experienced outright discrimination. Others may have been disadvantaged by assumptions about their communication styles, emotional responses, or ability to fit into the workplace.

Now suppose the employer hires a technology company to develop an AI-powered personality assessment.

The company studies the employer's most successful employees, identifies characteristics they have in common, and develops a test designed to find applicants with similar traits.

The resulting assessment may be sophisticated. It may produce consistent scores and demonstrate statistically significant relationships between certain personality characteristics and employee performance.

But there is a fundamental problem.

The assessment is learning from a workforce that may already reflect decades of discriminatory employment decisions.

The People Missing from the Data

People who were previously excluded are largely missing from the employer's data. Their abilities were never measured in that workplace. Their potential contributions were never observed.

The employer cannot determine from its existing workforce alone whether those excluded applicants might have performed just as well—or better.

This is a statistical problem known as selection bias.

The assessment identifies characteristics associated with success among people who were hired. It does not necessarily identify every combination of characteristics that could produce successful job performance.

Consider an employer whose highest-performing employees tend to be outgoing and socially assertive.

That finding does not establish that someone who is more reserved, communicates differently, or experiences anxiety cannot perform the same job successfully.

A different employee might achieve equally strong results through preparation, attentive listening, technical expertise, or a different approach to workplace relationships.

There may be many ways to succeed at a job, but an assessment trained on a historically narrow workforce may recognize only some of them.

This distinction is especially important for applicants with disabilities.

People with depression, anxiety disorders, autism, and other disabilities do not share a single personality profile. Nor does a diagnosis establish someone's ability—or inability—to perform a particular job.

Yet some assessments may assign lower scores based on emotional responses, communication patterns, or other characteristics associated with disability, even when those characteristics have little relationship to successful job performance.

The Department of Justice Recognized This Problem

In May 2022, the U.S. Department of Justice issued guidance titled Algorithms, Artificial Intelligence, and Disability Discrimination in Hiring.

The guidance specifically warned about hiring technologies that compare applicants with successful current employees.

DOJ explained that because people with disabilities have historically been excluded from many jobs, they may be missing from the workforce used to develop these technologies. As a result, the systems may discriminate against qualified applicants with disabilities.

That warning goes directly to the problem of using historical employment data to determine future hiring standards.

Under the Americans with Disabilities Act, an assessment that screens out or tends to screen out qualified individuals because of disability raises important legal questions. The employer may need to establish that the challenged selection criterion is job-related and consistent with business necessity. Separate reasonable-accommodation obligations may also apply.

A scientifically reliable personality assessment is not necessarily a legally permissible employment screening tool.

And an employer cannot assume that an assessment is lawful simply because no one intended it to discriminate.

How AI Can Reinforce Historical Exclusion

These problems are not unique to artificial intelligence. Traditional personality assessments and human hiring decisions can produce similar results.

But AI can identify subtle patterns, examine many variables, and apply its conclusions to thousands of applicants.

An algorithm does not need to be instructed to discriminate.

If the historical employment data reflect earlier discriminatory practices, the algorithm may learn those patterns and incorporate them into future hiring decisions.

The resulting assessment may favor applicants who resemble existing employees while disadvantaging others who could perform the same work equally well.

And the process can reinforce itself.

Applicants whose characteristics differ from the preferred profile may be screened out. Because they are never hired, their potential performance remains unknown. Future assessments may continue relying on data from employees selected under the same narrow criteria.

The employer may believe it is improving its hiring process through increasingly sophisticated analysis.

In reality, it may be making earlier selection preferences more systematic and harder to challenge.

Identifying Similarity Is Not Identifying Ability

None of this means that personality assessments are inherently discriminatory or that existing-employee data have no legitimate value.

But employers and assessment vendors must examine how the workforce was assembled, whether the assessment measures genuine job requirements, and whether qualified applicants with different characteristics are being improperly excluded.

The central question is not simply whether successful employees share certain personality traits.

It is whether those traits meaningfully predict job performance—and whether people with different characteristics might perform just as well.

Knowing what successful employees have in common does not tell us what capable applicants are missing.

That is the danger of turning the existing workforce into the test.

An algorithm may become very good at identifying people who resemble employees an organization previously chose to hire.

But identifying similarity is not the same as identifying ability.

And when historical employment practices have excluded qualified people with disabilities, using those practices as the foundation for future hiring decisions risks reproducing the very discrimination that employment laws were designed to prevent.

---

Source: U.S. Department of Justice, Algorithms, Artificial Intelligence, and Disability Discrimination in Hiring (May 12, 2022).

Next: When Success Becomes the Test—how biased employee evaluations can shape AI hiring decisions even after someone has been hired.