How Biased Performance Evaluations Can Shape AI Hiring Decisions
Suppose an employer wants to identify the personality characteristics of its most successful employees.
It hires a technology company to develop an AI-powered assessment that will identify applicants most likely to succeed.
The company examines personality questionnaires, performance evaluations, and employment records. It identifies characteristics shared by employees the employer considers its strongest performers.
The assessment may produce reliable scores and demonstrate statistically significant relationships between personality characteristics and performance ratings.
But there is a fundamental question:
How did the employer decide which employees were successful in the first place?
The answer matters because the assessment may predict something quite different from actual job performance.
When Favorable Evaluations Define Success
Consider two software developers.
Both consistently meet deadlines, produce reliable code, and solve difficult technical problems.
One is outgoing, speaks frequently in meetings, and enthusiastically promotes new ideas.
The other is quieter, communicates more deliberately, and demonstrates initiative through careful analysis and high-quality work.
Their measurable performance is comparable.
Yet their supervisors may evaluate them differently.
The outgoing employee might receive higher ratings for leadership, teamwork, communication, and attitude. The quieter employee might receive equally strong technical ratings but lower scores for initiative or cultural fit.
Those differences could reflect legitimate job requirements. But they might also reflect a supervisor's preference for one personality or communication style.
If those evaluations become the definition of success, an AI assessment may learn the supervisor's preferences rather than the requirements of the job.
The Disability Connection
This problem is particularly important for employees with psychiatric disabilities, autism, and other conditions that may affect communication, interaction, or emotional expression.
An autistic employee may communicate differently without being less capable of performing the job.
Someone experiencing depression may not consistently display the enthusiasm a supervisor associates with a positive attitude.
An employee with an anxiety disorder may find certain workplace interactions challenging while remaining highly capable of performing essential job duties.
People with these conditions do not share a single personality profile, and employers may legitimately evaluate job-related performance differences.
The concern is that supervisors may mistake differences in communication, emotional expression, or social behavior for differences in capability.
When these judgments influence performance ratings, they can become part of the data used to train employment assessments.
When Statistical Validation Validates the Wrong Thing
Suppose an AI developer discovers that employees with the highest supervisor ratings tend to score highly on measures of extraversion, assertiveness, and emotional expressiveness.
The developer builds an assessment favoring those characteristics and tests it against historical performance ratings.
The results show a statistically significant relationship.
But what has been demonstrated?
Perhaps the assessment identifies characteristics that genuinely contribute to performance. Perhaps it identifies characteristics supervisors prefer. Or perhaps it measures some combination of the two.
This illustrates a problem called criterion contamination.
When researchers validate an employment assessment, they compare its results against a measure of success, called the criterion.
If supervisor ratings are influenced by stereotypes or preferences unrelated to job performance, the criterion may be contaminated by bias. A related problem arises when ratings fail to capture important aspects of successful performance.
Even apparently objective measures, such as promotion histories or sales results, may reflect unequal access to assignments, resources, and opportunities.
An assessment can accurately predict who receives favorable evaluations without accurately predicting who can perform the job successfully.
Statistical sophistication cannot overcome a flawed definition of success.
How AI Can Turn Preferences into Hiring Standards
These concerns are not unique to AI. Traditional assessments and human hiring decisions can also reflect biased judgments.
But AI can analyze enormous amounts of data, identify subtle relationships, and apply those patterns to thousands of applicants.
It does not need to be instructed to discriminate.
If historical performance ratings reward particular communication styles, emotional expressions, or personality characteristics, an algorithm may incorporate those preferences into its scoring system.
Applicants with different characteristics may receive lower rankings even when they possess the skills needed for the job.
And because the assessment has been validated against historical performance ratings, its results may appear scientifically objective.
The algorithm may not eliminate subjective judgment. It may convert that judgment into an apparently objective hiring standard.
The ADA Does Not Disappear Behind an Algorithm
Under the Americans with Disabilities Act, employment assessments must not unlawfully exclude qualified applicants because of disability.
When a challenged criterion screens out or tends to screen out qualified individuals because of disability, the employer may need to establish that it is job-related and consistent with business necessity. Separate reasonable-accommodation requirements may also apply.
An employer therefore cannot assume that an assessment is lawful merely because its scores correlate with favorable evaluations of existing employees.
The employer must examine whether the challenged criteria are genuinely related to job performance and whether their use unlawfully disadvantages qualified applicants with disabilities.
Assessment vendors should likewise examine whether the performance measures used to validate their products accurately reflect job requirements and may contain bias.
A vendor's validation study does not automatically relieve an employer of its legal obligations.
The Question Behind the Numbers
Supervisor evaluations are not inherently unreliable. Nor are personality characteristics necessarily irrelevant to job performance.
The problem arises when employers confuse characteristics that contribute to successful performance with characteristics that merely influence how employees are perceived.
An employer developing an AI-powered assessment should examine how success is measured, whether different approaches to performing the job are recognized, and whether employees with disabilities are evaluated fairly.
A model that predicts biased evaluations may be statistically accurate while perpetuating the bias those evaluations contain.
AI can identify patterns in historical employment data with remarkable sophistication. But it cannot establish, without appropriate evidence, whether those patterns reflect genuine job requirements or the preferences of the people who produced the data.
If an employer's definition of success is biased, teaching an algorithm to recognize success may simply teach it to recognize that bias.
Sources and Further Reading
U.S. Department of Justice, Algorithms, Artificial Intelligence, and Disability Discrimination in Hiring (May 12, 2022).
Society for Industrial and Organizational Psychology, Considerations and Recommendations for the Validation and Use of AI-Based Assessments for Employee Selection (2023).
U.S. Equal Employment Opportunity Commission, Applying Performance and Conduct Standards to Employees with Disabilities.
No comments:
Post a Comment
Because I value your thoughtful opinions, I encourage you to add a comment to this discussion. Don't be offended if I edit your comments for clarity or to keep out questionable matters, however, and I may even delete off-topic comments.