Sound and Fury, Signifying Nothing

Incorporating elements of gamification, big data, machine learning, and predictive human analytics, Knack is a veritable buzzword oasis. According to Knack, their games are designed to test cognitive skills that employers might want, drawing on some of the latest scientific research. These range from pattern recognition to emotional intelligence, risk appetite and adaptability to changing situations.

John Funge, Knack's CTO, states that "we have used our games to infer cognitive ability, conscientiousness, leadership potential, creativity as well as predict how people would perform as surgeons, management consultants, and innovators." In an Economist article, Chris Chabris, a Knack executive, states that games have huge advantages over traditional recruitment tools, such as personality tests, which can easily be outwitted by an astute candidate. Many more things can be tested quickly and performance can't be faked on Knack's games, he says.

Gary Halfteck, Knack's founder and CEO, says playing a video game can be a better representation of who you are and your skill sets than an employer might get in a one-on-one conversation. "As people, we make many decisions that are biased, whether it's consciously or subconsciously, and we have no good tools to assess and evaluate, let alone predict, what one's potential is," he says.

If Knack's CEO admits that people make many decisions that are biased, what prevents the people at Knack from being biased in the creation, development and implementation of their games? Further, what prevents employers using Knack from being held liable for the biases of those games? The answer to both questions: Nothing.

Algorithmic Illusion

While many companies foster an illusion that scoring/classification is an area of absolute algorithmic rule—that decisions are neutral, organic, and even automatically rendered without human intervention—reality is a far messier mix of technical and human curating. Both the datasets and the algorithms used to analyze the data reflect choices, among others, about connections, inferences, and interpretation.

The recent White House report, “Big Data: Seizing Opportunities, Preserving Values," found that, "while big data can be used for great social good, it can also be used in ways that perpetrate social harms or render outcomes that have inequitable impacts, even when discrimination is not intended."

The fact sheet accompanying the White House report warns:

As more decisions about our commercial and personal lives are determined by algorithms and automated processes, we must pay careful attention that big data does not systematically disadvantage certain groups, whether inadvertently or intentionally. We must prevent new modes of discrimination that some uses of big data may enable, particularly with regard to longstanding civil rights protections in housing, employment, and credit.

Some of the most profound challenges revealed by the White House Report concern how data analytics may lead to disparate inequitable treatment, particularly of disadvantaged groups, or create such an opaque decision-making environment that individual autonomy is lost in an impenetrable set of algorithms. Please see Knack Testing Illegal Under ADA?

Systemic Risk

Workforce assessment systems like Knack's games, designed in part to mitigate risks for employers, are becoming sources of material risk, both to job applicants and employers. The systems create the perception of stability through probabilistic reasoning and the experience of accuracy, reliability, and comprehensiveness through automation and presentation. But in so doing, technology systems draw  attention away from uncertainty and partiality.

While Knack's approach may help reduce an employer's hiring costs and may reduce the impact of overtly biased or discriminatory behavior, the inclusion of one or more potentially "defective components" in the assessments means that employers face the risk that a finding of bias or discrimination of a Knack assessment used by one employer will put all employers that use the assessment at risk. Please see When the First Domino Falls: Consequences to Employers of Embracing Workforce Assessment Solutions.

These "defective components" in assessments may be either design defects (i.e., the adoption and use of certain personality models) or manufacturing defects (i.e., coding errors in the assessment software). The latter is analogous to the coding error at 23andMe that resulted in notices going out to some customers informing them that they had a chronic and life-shortening condition when they did not. Please see On Not Dying Young: Fatal Illness or Flawed Algorithm?

Each day an employer continues to use the Knack assessment, there are more potential plaintiffs with claims against that employer.  Labor and employment laws like Title VII and the ADA, permit an employer to use a third party like Knack to undertake the assessment of job applicants. The use of a third party, however, does not insulate an employer from any claims arising from the assessment usage. Under those laws, an employer is responsible (and liable) for any failures on the part of an assessment or assessment provider to comply with the provisions of those laws.

No Silver Bullet

Just as concerns about scoring systems are heightened, their human element is diminishing. Although software engineers initially identify the correlations and inferences programmed into algorithms, machine learning, predictive analytics, and big data promises to eliminate the human “middleman” at some point in the process.

As Hector J. Levesque, a professor at the University of Toronto and a founding member of the American Association of Artificial Intelligence, wrote:

"As a field, I believe that we tend to suffer from what might be called serial silver bulletism, defined as follows:

the tendency to believe in a silver bullet for AI, coupled with the belief that previous beliefs about silver bullets were hopelessly naıve. 

We see this in the fads and fashions of AI research over the years: first, automated theorem proving is going to solve it all; then, the methods appear too weak, and we favour expert systems; then the programs are not situated enough, and we move to behaviour-based robotics; then we come to believe that learning from big data is the answer; and on it goes."

Similarly, employment assessment companies like Knack market the benefits of science, precision and data over the past fifteen years under the guise of neural networks, artificial intelligence, big data and deep learning, yet what has changed? Employee engagement levels have hardly budged and employee turnover remains a continuing and expensive challenge for employers. Please see Gut Check: How Intelligent is Artificial Intelligence?

Systemic Risks for School Systems Employing TeacherInsight

According to Gallup Inc., its TeacherInsight (TI) assessment is an automated online interview used by many school districts to help those schools identify the best potential teachers. School districts across the country use the TI assessment as part of their teacher application and selection process.

Need to Independently Review Assessment.

In The (Non)Predictive Ability of the Gallup TeacherInsight Assessment, there is discussion of a study of the TI assessment undertaken by Michael T. Novotny. The Novotny study includes the following recommendations: 

School districts that are using, or considering the use of, the TI should require the Gallup Organization to show them the research in support of the instrument. The lack of independent research on the TI and Gallup’s unwillingness to publish their own research does not help support the credibility of the TI. If Gallup was to publish their own studies, then independent researchers could attempt to replicate those studies in order to either confirm or contradict their findings regarding the validity of the TI. Most large school districts have a research department with the capability of conducting such research.

As discussed under the headings "Ignoring EEOC Guidance" and "Disregarding Industry Standards" in Risks to Kroger Shareholders, a key element in guidance from both the EEOC and EEAC (an employer association) is the responsibility of the employer (e.g., local school district) to independently review the hiring assessment(s) it uses and to avoid reliance on the representations of the assessment provider (e.g., Gallup).

Accordingly, as Novotny recommends, school districts that are using, or considering the use of, the TI assesment should require Gallup to show them the research in support of the instrument. The Gallup-provided research should include information on the ability of the TI assessment to predict teacher performance, as well as validity studies undertaken by or on behalf of Gallup and evidence that the assessment does not discriminate against individuals who are members of classes protected by laws like Title VII and the ADA. 

What's Missing?

According to Gallup, a TI interview development study, originally completed in January 2002, demonstrated content, construct, and criterion-related validity as well as fairness across classifications of race, gender, and age. There are a number of other "protected classes," including persons with disabilities (including physical, developmental and mental disabilities), national origin, and religion, that are missing from Gallup's list.

As an example, Dr. Melanie Schneider has identified three primary concerns relevant to nonnative and bilingual speakers of English (raising issues of discrimination on the basis of national origin): 

• the timed nature of the TI assesment, 
• possible inequalities associated with limited access to computers or the Internet, and
• little perceived consideration for cultural and linguistic.

Dr. Schneider writes:

As mentioned earlier, parts of the TI contain timed questions. Unlike other standardized tests, there is no accommodation for applicants for whom English is a second or additional language or others who may have a disability that prevents a rapid-fire response. Waiting too long to respond to a question results in a missing response, which counts against an applicant’s total score. Although the timed nature of some types of questions affects all applicants who take the TI, nonnativespeakers of English are especially penalized when speed of response is required. 

Linked to the timed format of the TI, which may disadvantage some nonnative speakers of English, are possible inequalities due to the digital divide between low-income and middle-income students. ... English language learners in the United States are more likely to come from lower income families than their native-English-speaking peers. Inequalities between low-income and middle-income children in the use of and access to computers and the Internet have been well documented. ... Even among undergraduate students, values, attitudes, and beliefs about access to and facility with certain technologies may disadvantage certain groups of students, such as those from low-income immigrant families. 

Finally, a third concern for bilingual speakers of English is the belief that a singular view of talents characterizes successful teachers. Assuming that there is a single, preferred set of values, attitudes, beliefs, and behaviors associated with teacher success in the classroom ignores the role of culture in teaching and assessment

In Albemarle Paper Company v. Moody, the Supreme Court addressed a case in which an employer implemented a test (Wonderlic) on the theory that a certain verbal intelligence was called for by the increasing sophistication of the plant's operations. The company made no attempt to validate the test for job-relatedness, and simply adopted the national "norm" score as a cut-off point for new job applicants.

The Supreme Court cited the Standards of the American Psychological Association and pointed out that a test should be validated on people as similar as possible to those to whom it will be administered. The Court further stated that differential studies should be conducted on minority groups wherever feasible.

Ongoing Investigations of Assessments for ADA Compliance by EEOC

For more than six years, the EEOC has been investigating Kroger and Kronos, Kroger's assessment provider.  The investigation focuses on whether the Kronos assessment illegally screens out persons with mental illness. Please see Kroger and Kronos: Chaos and Disorder.

The case arose from a charge filed by Vicky Sands with the EEOC in 2007. The EEOC has converted Ms. Sands' investigation into a systemic investigation and has since started at least two additional systemic investigations involving companies using the Kronos assessment.

The EEOC investigations involve claims that the Kronos assessment is an illegal pre-employment medical examination, that the data collected by the assessments is confidential medical information, and that the assessment screens out or tends to screen out persons with mental illness. 

Systemic Risks

For many school districts, employment assessments like TI offer a standardized experience for all applicants. While this "one size fits all" approach helps to reduce a school district's costs and may reduce the impact of overtly biased or discriminatory behavior, the inclusion of one or more potentially "defective components" in the assessments means that school districts face the risk that a finding of bias or discrimination of an assessment used in one school district will put all school districts that use the assessment at risk. Please see When the First Domino Falls: Consequences to Employers of Embracing Workforce Assessment Solutions.

These "defective components" in assessments may be either design defects (i.e., the adoption and use of certain personality models) or manufacturing defects (i.e., coding errors in the assessment software). The latter is analogous to the coding error at 23andMe that resulted in notices going out to some customers informing them that they had a chronic and life-shortening condition when they did not. Please see On Not Dying Young: Fatal Illness or Flawed Algorithm?

Each day a school district continues to use the TI assessment, there are more potential plaintiffs with claims against that school district.  Labor and employment laws like Title VII and the ADA, permit a school district to use a third party like Gallup to undertake the assessment of job applicants. The use of a third party, however, does not insulate a school district from any claims arising from the assessment usage. Under those laws, a school district is responsible (and liable) for any failures on the part of an assessment or assessment provider to comply with the provisions of those laws.

Punitive Damages for Illegal Medical Examinations under the ADA

In a recent decision, a Pennsylvania federal court held that an acquiring company conducted over 300 unlawful pre-offer medical exams in violation of the ADA.  Cambria Care Center (“CCC”) had purchased the former Cambria County owned nursing home and engaged Grane Healthcare (a separate, but related entity to CCC) to interview and hire employees for the new facility, which was set to open later in the year.

All 300 of the former county nursing home employees were invited to apply for new employment.More than 300 employees from the acquired company applied for positions with  Grane [the acquiring company]. Grane ultimately hired roughly 225 of the applicants. Every applicant was required to undergo a medical examination. Several unsuccessful applicants for employment filed charges of discrimination with the Equal Employment Opportunity Commission (EEOC), alleging that Grane had violated the Americans with Disabilities Act of 1990 (ADA) by conducting pre-offer medical examinations of prospective employees and declining to hire some of them because of actual or perceived disabilities.

The record demonstrates that the employees seeking employment with Grane were subjected to a formalized process consisting of unlawful pre-offer medical examinations and illicit solicitations of detailed medical information. The individuals who were subjected to these illegal examinations and inquiries, including those who were ultimately rejected, were deprived of the prophylactic protection from discrimination that § 12112(d) was designed to create.

Section 12112(d) Claims

The provisions of the ADA pertaining to medical examinations and inquiries are codified at 42 U.S.C. § 12112(d). This statutory framework is designed to shield information about an applicant’s medical condition from his or her prospective employer until after an offer of employment is made. Before an offer of employment is extended, an employer may not ask a job applicant to undergo a medical examination or inquire as to whether he or she “is an individual with a disability.”

Unlike § 12112(a), which aims to protect a discrete class of “disabled” persons from discrimination, § 12112(d) contains no language limiting the category of applicants and employees entitled to statutory protection. Consequently, an individual who is subjected to an unlawful medical examination or inquiry can successfully assert a claim under § 12112(d) without establishing the existence of a statutory “disability.” In this vein, an applicant who is rejected by an employer based on information gleaned from an illegal pre-offer medical examination or inquiry may seek redress under § 12112(d)(2)(A) even if that information does not reveal a “disabling” medical condition. A § 12112(d) violation occurs as soon as “an employer conducts an improper medical examination or asks an improper disability-related question, regardless of the results or response.”

Current Charges with the EEOC

None of the seven charges currently being investigated by the EEOC require the employer to have knowledge that the applicant has a disability.  The claim that an employer used an unlawful pre-offer medical exam does not turn on the employer’s knowledge of the applicant's disability -- and in fact does not require that the applicant have a disability.

For those employers utilizing Five Factor Model-based personality tests as an element of their screening and hiring process, the § 12112(d) violation occurs each time the applicant completes the test and the results are submitted to the employer or assessment company.

The ADA’s prohibition against pre-offer medical examinations and inquiries is prophylactic in nature. Congress was concerned that medical information gleaned from such examinations and inquiries could be used to exclude disabled applicants from further consideration for employment. In order to discriminate against an applicant “on the basis of disability,” an employer must know that the applicant is disabled. By denying employers access to medical information until after offers of employment are made, § 12112(d) aims to ensure that such information does not infect the employee-selection process. In this way, § 12112(d) deters covert discrimination against disabled applicants by forcing employers to make hiring decisions before procuring the information upon which discriminatory decisions could be based.

Reducing Employer Risk Exposure

The timing or sequencing of the testing has a significant impact. Employers could reduce risk exposure by administering the test after having provided the applicant with a conditional offer of employment. The employer would still face the risks associated with tests that discriminate against persons with disabilities, but would not face the potential for claims from all applicants who took the test.

Employers sequence the testing first not because they want to include those who are the best fit for the company culture, but because they need a quick, low-cost method of excluding a significant number of applicants. As one employer (Xco) stated in its position statement,

“The [assessment] enables [Xco]  to assess a large volume of applicants cost-effectively and provides consistency in that assessment. Without the [assessment] reviewing and assessing all individual applications would be extremely costly, both in terms of the labor resources needed to conduct such screening and the costs associated with training managers in making the predictive assessments yielded by the [assessment]. It also would result in greater variability in the reliability of the assessments made by individual managers.

Available Alternatives

Xco, and other employers act as if there are only two alternatives – use assessments (quick, low-cost) and don’t use assessments (time-consuming, costly). There are a variety of alternatives, including tests that do not constitute pre-offer medical exams and do not screen out persons with mental illness. Both Starbucks and CVS are prospering without using illegal pre-offer medical examinations and they do not appear to have put themselves at a competitive disadvantage. 

For example, contrast the stock price performance of CVS (no assessment) and Walgreens (assessment) since CVS stopped using assessments following its settlement with the ACLU in Rhode Island. Please see "The CVS Example" in When the First Domino Falls: Consequences to Employers of Embracing Workforce Assessment Solutions

Systemic Risk 

Due to the lack of variability in the online assessments, any discriminatory element that makes its way into the test algorithm – like the screening out of persons with mental disabilities due to the use of the Five Factor Model - is propagated across the entire universe of job applicants.

Congress sought “to provide clear, strong, consistent, enforceable standards addressing discrimination against individuals with disabilities.” 42 U.S.C. § 12101(b)(2). Covered employers cannot erode those standards by procuring detailed medical information about applicants for employment and contending, at the end of the day, that such information has never been used to the detriment of those applicants. If it were otherwise, disabled individuals would be vulnerable to several forms of discrimination that the ADA was designed to prevent. Since most employment decisions involve elements of discretion, it is relatively easy for an employer to “concoct a plausible reason for not hiring” a particular individual. A decision declaring the admitted § 12112(d) violations to be “harmless” would seriously undermine the ADA’s policy of prophylactic deterrence.

Compensatory Damages 

Before 1991, plaintiffs proceeding under Title VII could only seek “equitable” remedies. Backpay was the primary form of monetary relief available to aggrieved individuals. Section 102 of the Civil Rights Act of 1991 added compensatory and punitive damages to the remedies otherwise available under Title VII and the ADA. The relevant statutory language, permits a “complaining party” to “recover compensatory and punitive damages” from a covered entity responsible for violating § 102 of the ADA. The provisions governing medical examinations and inquiries are included within § 102.

The compensatory damages available to individuals aggrieved by violations of Title I include remuneration for “future pecuniary losses, emotional pain, suffering, inconvenience, mental anguish, loss of enjoyment of life, and other nonpecuniary losses.” The total amount of compensatory and punitive damages available to “each complaining party” is capped at anywhere from $50,000 to $300,000, depending on the number of individuals employed by the covered entity.

Punitive Damages

The Civil Rights Act of 1991 permits any party to “demand a trial by jury” “[i]f a complaining party seeks compensatory or punitive damages” under Title I. The disjunctive wording of this language suggests that a plaintiff can seek an award of punitive damages without seeking an award of compensatory damages. Several Courts of Appeals have concluded that punitive damages may be assessed under the relevant statutory provisions even if no compensatory damages are awarded.

Employers will likely argue that most applicants should receive little, if any, in the way of compensatory damages like backpay. The argument is based on the limited number of positions available as compared with the large number of applicants; only a small percentage of the applicants could have been hired to fill the positions. The expansive scope of compensatory damages – including nonpecuniary losses – plus the availability of punitive damages is designed to provide relief to applicants whose rights have been violated.

As noted below, precluding awards of punitive damages in cases involving no easily quantifiable physical and monetary harm would quell the deterrence that Congress intended to provide when it enacted § 12112(d). The statutory caps on damages ensure against limitless awards in cases of insubstantial harm.

The damages cap for each of the seven companies subject to the EEOC charges is $300,000 per applicant.

A plaintiff seeking punitive damages under Title I must demonstrate that the offending employer “engaged in a discriminatory practice or discriminatory practices with malice or with reckless indifference to [his or her] federally protected rights.” In Kolstad v. American Dental Association, the Supreme Court construed this language to mean that a covered employer must “discriminate in the face of a perceived risk that its actions will violate federal law” in order to be liable for punitive damages.

Reckless Indifference

Examples of reckless indifference by the seven companies subject to the EEOC charges may include:

• Using an assessment is based on the Five Factor Model of personality, which serves as a basis for categorizing and diagnosing personality disorders in the Diagnostic and Statistical Manual of Mental Disorders.
• Failing to perform any adverse impact or validation studies on persons with disabilities. As Kronos stated in one of its filings in the EEOC litigation, “No adverse impact or validation studies have been performed by Kronos … with respect to potential adverse impact on individuals with disabilities.”
• Continuing to use the assessment for years after having been put on notice that the assessment may be an illegal pre-offer test. Contrast Kroger with CVS.
• Asking questions are neither “directly relevant” to the job nor “plainly job-related.” The questions, designed to reveal information about individuals’ “openness,” “conscientiousness,” “extraversion,” “agreeableness,” and “neuroticism,” based on the Five Factor Model, do not seek information about the ability of an applicant to perform the day-to-day functions of a job.
•  Ignoring the almost 40-year old mandate of the Supreme Court in Albemarle Paper Company v. Moody, 422 US 405 (1975) that a test should be validated on people as similar as possible to those to whom it will be administered (i.e., persons with disabilities). The Court further stated that differential studies should be conducted on minority groups – like persons with mental illnesses - wherever feasible. 
• Relying solely on the statements of the assessment company, without any independent review or verification by the employer. The EEOC fact sheet sets out selection and administration guidelines, including  that an employer: (i) ensure that tests and selection procedures are not adopted casually by managers who know little about these processes; (ii) ensure that the tests are valid, independent of the test vendor’s documentation; and, (iii) determine whether there is an equally effective alternative selection procedure that has less adverse impact and adopt that alternative procedure if a selection procedure screens out a protected group.

Punitive Damages Do Not Require Showing of Tangible Harm

The statutory language does not condition an award of punitive damages on a showing of tangible harm. It is worth noting that the term discriminatory practice” is defined more broadly in relation to Title I than it is in relation to Title VII. Although the term includes only “discrimination” in the Title VII context, it is broad enough to encompass “violations” of the ADA that might not constitute “discrimination.”

The breadth of this language, which specifically defines a term appearing in the portion of the statute governing awards of punitive damages, suggests that Congress intended to provide for assessments of punitive damages against employers responsible for intentionally violating Title I’s prophylactic provisions. Precluding awards of punitive damages in cases involving no “easily quantifiable physical and monetary harm would quell the deterrence that Congress intended” to provide when it enacted § 12112(d). The statutory caps on damages “ensure against limitless awards in cases of insubstantial harm.”

Accordingly, the EEOC can seek punitive damages on behalf of the applicants who were unlawfully subjected to pre-offer medical examinations and inquiries even if those applicants are not otherwise entitled to compensatory damages.

Since the examinations in this case were illegally conducted at the pre-offer stage, the EEOC can seek backpay and compensatory damages on behalf of any applicant rejected based on unlawfully procured medical information, regardless of whether his or her injuries are attributable to “discrimination based on a disability.” Moreover, recovery may be sought for less tangible injuries caused by the examinations and inquiries, irrespective of whether those injuries manifested themselves in the form of personnel decisions (suggesting that an “emotional” injury could constitute “actual damage”).

What Do Car Platforms and Employment Assessments Have In Common? Systemic Risk

A major manufacturing trend in the automotive industry has been the focus on global platform-based vehicle design and manufacturing. One of the trade publications predicts that by 2017 VW will build over 40 models on its MQB platform (Audi A3, VW Golf, etc.) totaling over 4 million vehicles worldwide. All of the global car makers will be above 2 million vehicles per year with their major platforms.

The savings from faster vehicle development time, lower tooling costs and more advantageous supply contracts (by leveraging higher volumes) could arguably add up to billions of dollars in savings. Suppliers also benefit as they service larger, more stable supply contracts that support broad-based utilization of their global manufacturing facilities.

And the risks? The same global scale that can yield big savings benefits could drive huge costs and reputation damage if a product defect or manufacturing quality issue surfaces. Instead of the defect being confined to a single plant, single model or even a single vehicle segment, the potential exists for the defect to be multiplied across thousands of vehicles around the world in a very short amount of time.

Employers and assessment companies face analogous risks. For many applicants, especially those applying for entry-level positions in retail and food service, employment assessments offer a standardized experience for job applicant. While this "one size fits all" approach helps to reduce an employer's costs and may reduce the impact of overtly biased or discriminatory behavior on the part of one or more recruiters, the inclusion of one or more potentially "defective components" in the assessments means that employers face the risk that a finding of bias or discrimination in one of the assessments will put all tests at risk. Please see When the First Domino Falls: Consequences to Employers of Embracing Workforce Assessment Solutions.

These "defective components" in assessments may be either design defects (i.e., the adoption and use of the five-factor model of personality) or manufacturing defects (i.e., coding errors in the assessment software). The latter is analogous to the coding error at 23andMe that resulted in notices going out to some customers informing them that they had a chronic and life-shortening condition - limb-girdle muscular dystrophy - when they did not. Please see On Not Dying Young: Fatal Illness or Flawed Algorithm?

Market Success ≠ Product Effectiveness

Pre-employment personality tests are marketed by hundreds of assessment companies and used by thousands of employers, including many of the largest employers in the U.S. There has been significant growth in the usage of personality tests over the past 10-15 years and their widespread adoption evidences market success, but does it evidence product effectiveness?

One might argue that product effectiveness is demonstrated by market success, that one could not have the latter without the former. That argument fails in light of the many instances in which market success did not correlate with product effectiveness.

Vioxx, DePuy, Polybutylene ...

Vioxx, a non-steroidal anti-inflammatory drug developed by Merck, received FDA approval on May 20, 1999. The drug gained widespread acceptance among physicians treating patients with arthritis and other conditions causing chronic or acute pain. Merck recorded more than $11 billion in Vioxx sales during the drug's years on the market from mid-1999 to September 2004. The drug was withdrawn from the market after a study showed it increased the risk for heart attacks and strokes. At the time of the withdrawal, Vioxx was Merck’s second-best selling drug, generating $2.3 billion in sales the previous year. Since the withdrawal, the company has paid nearly $6 billion in litigation settlements, not including legal-defense costs and possible payments from pending litigation. That pending litigation includes a derivative class action lawsuit on behalf of all shareholders who lost money on Merck common stock or options trades between May 1999 and September 2004.

In 2005, DePuy, a Johnson & Johnson company, started selling its Articular Surface Replacement, or A.S.R., hip for use in standard hip replacements in the United States. Close to 40,000 patients in the U.S. received a DePuy ASR hip implant from its introduction to the market in August 2005 through August 2010, when DePuy issued a recall for the device. On November 19, 2013, Johnson & Johnson announced its agreement to pay at least $2.5 billion to resolve thousands of defective DePuy ASR hip implant lawsuits. In addition, Johnson & Johnson may pay as much as $1 billion to Medicare and private health insurers who covered the medical costs of removing its recalled hip implants

Beginning in the late 1970s, polybutylene plastic plumbing systems—touted as being cheaper and more durable than copper pipe systems—were installed in new homes nationwide. Over the years, several million homes were built with polybutylene plumbing systems. Before long, the plumbing systems began to experience failures of the fittings and of the pipe itself. Consumers nationwide attributed the failures to various causes, including inadequate design, defective manufacturing, improper installation, and degradation of the materials from chemicals in the drinking water. More than ten years of litigation, and bankruptcy for one company, would follow, and hundreds of millions of dollars would be spent before reaching a final class action resolution.

Market Success = Greater Risk Exposure

Market success does not equate with product effectiveness, but market success may multiply the unintended consequences of an inherently flawed design due to path dependencies and systemic effects.

The success of assessment companies in marketing pre-employment personality tests over the past 10-15 years has created systemic risk for their employer customers. If one employer has violated the law and subjected itself to significant liability as a consequence of its use of an assessment provided by an assessment company, then all customers of that company are similarly at risk.

Similarly, the lack of diversity in the psychological model underlying many of the personality tests offered by assessment companies (the five-factor model of personality or Big Five) means that if one assessment company's personality tests that use the Big Five is found to screen out persons with mental illness or to be an illegal medical examination under the ADA, the personality tests marketed by other assessment companies that use the Big Five (and, more importantly, their employer customers) are similarly at risk.

As set out in When the First Domino Falls: Consequences to Employers of Embracing Workforce Assessment Solution, the potential risks to employers are substantial and include:

• Damages and injunctive relief for violating the ADA and Rehabilitation Act of 1973;
• Brand damage and lost revenues; and
• Illusory indemnification by the assessment companies.

Additional risks include those referenced in the Vioxx, DePuy and polybutylene examples discussed above, risks like shareholder derivative class actions, defense costs and claims by state and federal governments for costs incurred as a consequence of the illegal acts (i.e., unemployment insurance, income support payments - SSDI and SSI, Medicare and Medicaid costs).

Better Get While the Gettin's Good

On December 11, 2013, Reuters reported that the two private equity companies that took human resources management software firm Kronos Inc. private in 2007 are looking to sell the company. Hellman & Friedman LLC and JMI Equity are exploring a sale of Kronos, which could be valued at more than $4 billion. Interested purchasers are reported to include TPG, KKR and Bain.

The Reuters article states that Hellman & Friedman and JMI have taken advantage of Kronos' strong cash flow to draw more than $1.5 billion in dividends from Kronos, and so have already earned twice the $752.9 million they committed as equity when they agreed to acquire the company in 2007. In November 2013, the two companies had Kronos borrow to pay themselves a $490 million dividend.

Who should be interested in a potential sale of Kronos by Hellman & Friedman and JMI, other than the sellers, potential buyers and Kronos employees? The hundreds of employers that are customers of the Kronos talent acquisition and employee assessment services.

Why should those employers be interested? The risks to those employers from the ongoing systemic investigation by the Equal Employment Opportunity Commission (EEOC) of several Kronos customers, an investigation focused on whether the Kronos assessment services violate the Americans with Disabilities Act (ADA) by illegally screening out persons with disabilities.

What are EEOC systemic investigations? Systemic investigations involves pattern or practice, policy, and/or class cases where the alleged discrimination has a broad impact on an industry, profession, company, or geographic area. In connection with systemic investigations, the EEOC’s enforcement tools include issuing broad information requests and subpoenas on employers that are named as respondents in EEOC charges, particularly when the EEOC suspects systemic discrimination, and filing pattern or practice class lawsuits in federal court.

What are the risks to employers? Systemic investigations by the EEOC and class action claims by job applicants for damages and injunctive relief. For some employers, the potential class size can be measured in the millions of plaintiffs. Employers have primary liability under the ADA, but Kronos has indemnified many of its employer customers. If Kronos does not have the financial resources, however, the indemnification is illusory.

What is Kronos?

Kronos is a U.S.-based workforce management software and services company. According to the company, tens of thousands of organizations in more than 100 countries - including more than half of the Fortune 1000 - use Kronos.

In August 2006, Kronos acquired Unicru, Inc., a company specializing in software used to assess and hire hourly workers. At the time of the acquisition by Kronos, Unicru had as customers for its assessment (the Unicru assessment) more than 140 leading companies and brands, including SuperValu, Kroger, Toys "R" Us, Best Buy, CVS, Borders, Lowe's, Caribou Coffee, and Marquis Healthcare.

The Unicru assessment consists of a number of statements, to which an applicant must answer “strongly disagree,” “disagree,” “agree,” or “strongly agree.” It includes statements such as: “You have confidence in yourself”; "You try to sense what others are thinking and feeling”; “You always say whatever is on your mind”; and “It is easy for you to feel what others are feeling.”

The systemic investigation of Kronos assessment customers, including Kroger, arose from a charge filed with the EEOC more than six years ago by a Kroger job applicant.  The charge led to an investigation that has been ongoing for more than six years and has generated a number of district court and appellate court decisions as Kronos has unsuccessfully sought to avoid disclosing information about the Unicru assessment and its impact on persons protected by the ADA.

Cloning Employees and Institutionalizing Biased Hiring Practices

According to Kronos, the Unicru assessment is an artificial intelligence test that uses neural networks to “learn” the characteristics of a customer’s “best” employees.  As stated by Kronos’ Chief Scientist and the developer of the Unicru assessment, Dr. David Scarborough, in chillingly Orwellian terms, "[o]ur system allows you to clone your best, most reliable people."

First used for engineering and industrial applications during the mid-1980s, neural networks evolved from early artificial intelligence research. Modeled on the function of the human brain, a neural network attempts to imitate human reasoning. Large amounts of data are fed into the network, which looks for relationships and reaches conclusions.


"There are a couple of dangers," states Jai Shekhawat, CEO of Chicago-based Fieldglass Inc., which develops software for managing workers. "Is something a correlation--a predictor--or merely a coincidence? At best, [these methods] are complementary to human judgment, not a substitute for it."

Notwithstanding such dangers, Kronos customers like Kroger are substituting this “coincidence” for human judgment. Based on the prospective employee's answers on the application, the Unicru assessment categorizes the applicant as red, green or yellow. In most cases, red is usually an automatic discard, or, as Dr. Scarborough stated “[m]anagers are strongly discouraged from hiring first quartile (“red”) applicants …”

There is no evidence that the Unicru assessment determines whether an employer’s hiring practices are biased or discriminatory. For example, if the Unicru assessment had been utilized fifty years ago, many companies’ “best” employees would have the personality traits of white males – persons of color, women and those with disabilities need not have applied.

The Unicru assessment embeds and industrializes existing stigma, bias and discrimination in the hiring process. As stated by Cynthia Dwork and Deirdre K. Mulligan in a recent Stanford Law Review article:

While automated decisionmaking systems “may reduce the impact of biased individuals, they may also normalize the far more massive impacts of system-level biases and blind spots.” Rooting out biases and blind spots in big data depends on our ability to constrain, understand, and test the systems that use such data to shape information, experiences, and opportunities.

As a “blind” tool that “learns” from the employer, the Unicru assessment replicates the existing bias of the employer and applies it on a massive scale. All applicants have their test responses fed through a discriminatory filter that is the Unicru assessment (a filter that is biased both on its own and in conjunction with its “learned” behavior). 

Illegal Medical Examination

The ADA prohibits the use of pre-employment medical examinations. At the pre-offer stage, an employer is only entitled to ask about an applicant's ability to perform the essential functions of the job. The ADA's prohibition against pre-employment examinations seeks to ensure that the applicant's disability is not considered prior to the assessment of the applicant's qualifications.

EEOC guidance provides a seven-factor test for analyzing whether a test or procedure qualifies as a “medical examination,” including:

• whether the test is designed to reveal an impairment of physical or mental health such as those listed in the Diagnostic and Statistical Manual of Mental Disorders (“DSM”); and
• whether the test is interpreted by a health care professional.

According to the guidance, the presence of any one of the seven factors is enough to support a finding that the test is a medical examination and the Unicru assessment meets the two factors listed above. 

Since the Unicru assessment is based on the five-factor model (FFM) of personality it meets the first factor listed above. As set out in previous posts -  ADA, FFM and DSM and Employment Assessments are Designed to Reveal an Impairment - assessments based on the FFM are designed to reveal an impairment of mental health, such as those listed in the DSM.

As to the second factor, whether the test is interpreted by a health care professional, the individuals who developed the Unicru assessment are psychologists, most of whom are members of the APA. In developing the Assessments, the psychologists establish the rules by which the assessments are to be interpreted (i.e., how the responses to the questions are to be scored, including whether the applicant receives a green, yellow or red rating).

According to the APA Model Act for State Licensure of Psychologists, “[t]he practice of psychology includes … (a) psychological testing and the evaluation or assessment of personal characteristics, such as intelligence; personality; cognitive, physical, and/or emotional abilities; … [and] (f) provision of direct services to … groups for the purpose of enhancing … organizational effectiveness, using psychological principles, methods, and/or procedures … for making decisions about the individual, such as selection …”

EEOC guidance states that psychologists are among the “variety of health professionals [that] may provide documentation regarding psychiatric disabilities” for ADA purposes. Accordingly, the psychologists who developed the Unicru assessment are "health care providers" for purposes of the ADA.

The CVS Example

In July 2011, CVS and the Rhode Island Civil Liberties Union (ACLU) entered into a voluntary settlement addressing the ACLU’s complaint challenging CVS’s use of a pre-hire questionnaire that the ACLU claimed could have a discriminatory impact on people with certain mental impairments or disorders. 

The CVS questionnaire contained statements to which applicants were required to respond, including: “You change from happy to sad without any reason,” “You get angry more often than nervous,” “Your moods are steady from day to day,” and “There’s no use having close friends; they always let you down.”

Responding to a complaint filed by the ACLU, the Rhode Island Commission for Human Rights had issued a finding in February 2011 that there was "probable cause" to believe that the questionnaire used by CVS violated state anti-discrimination laws that bar employers from eliciting information that pertain to job applicants' mental or physical disabilities.

Although employers may legally ask questions designed to help determine an applicant’s personality or aptitude for a job, the ACLU’s complaint argued that questions found in the CVS pre-offer assessment “could have the effect of discriminating against applicants with certain mental impairments or disorders, and go beyond merely measuring general personality traits.” 

Pursuant to the settlement agreement, CVS agreed to permanently remove the questions at issue from its online application.

Systemic Risk to Employers

The success of workforce science companies in developing employment personality and assessment tests over the past twenty years has created "systemic risk" for their employer customers. If one employer has violated the law and subjected itself to significant liability as a consequence of its use of an assessment provided by a workforce science company, then all customers of that company are similarly at risk. Workforce science companies provide their services to thousands of employers, including many of the largest employers in the U.S. 

The lack of diversity in the psychological model underlying many of the personality tests offered by workforce science companies (the five-factor model of personality or Big Five) also means that if one workforce science company's personality tests that use the Big Five is found to be an illegal medical examination under the Americans with Disabilities Act (ADA), all workforce science companies that use the Big Five (and, more importantly, their customers) are similarly at risk. 

There are multiple risks to employers arising from the use of personality tests and workforce assessments, including: 

1. Claims under the ADA and the Rehabilitation Act of 1973 that the personality tests are illegal medical examinations or that they illegally screen out persons with mental illness (as set out above); 
2. Claims under the ADA and the Rehabilitation Act of 1973 that the employer fails to select and administer the assessment in the most effective manner to ensure that the assessment results accurately reflect the skills, aptitude or whatever other factor that the assessment purports to measure, rather than reflecting an applicant’s impairment; 
3. Claims that employers and workforce assessment companies fail to properly safeguard confidential medical information obtained from the personality tests and illegally use that confidential medical information in violation of the ADA; and
4. Claims under Title VII of the Civil Rights Act that the workforce analytics cause there to be a disparate impact on the hiring of blacks and Hispanics.

As to the potential size of the plaintiff classes for the claims listed above, they range from a percentage of all applicants (in the case of claims that the tests illegally screen out persons with mental illness and claims of disparate impact under Title VII) to all applicants over the past 12 months (in the case of claims that the personality test is an illegal medical examination) to all applicants, employees and ex-employees over a longer period of time (in the case of claims that employers and workforce assessment companies failed to safeguard confidential medical information).

For some employers, the potential class size can be measured in the millions of plaintiffs. Consistent with the 2011 Supreme Court decision in Wal-Mart Stores, Inc. v. Dukes, plaintiffs in a class action suit predicated on the use of personality tests and workforce analytics will be challenging a uniform, company-wide practice. The uniform use of testing by an employer demonstrates that "there are questions of law or fact common to the class," or commonality, as required by the rules governing class actions.

Illusory Indemnification?

A key element in continuing to use Kronos assessment services is Kronos' ability to indemnify its employer customers. As noted above, the success of workforce assessment companies in marketing personality tests and workforce analytics over the past twenty years has created "systemic risk" for its customers. If one employer has violated the law and subjected itself to significant liability as a consequence of its use of a solution provided by a workforce assessment company, then all customers of that workforce assessment company are similarly at risk.

Even assuming workforce assessment companies are willing to provide indemnification to all customers, those employers need to independently assess whether the workforce assessment companies and their insurers have adequate resources to indemnify all customers. 

As Kenexa, an employment assessment company, consistently noted in its annual 10-K risk factor disclosures prior to its December 2012 acquisition by IBM:

The failure of our solutions to comply with employment laws may require us to indemnify our customers, which may harm our business. Some of our customer contracts contain indemnification provisions that require us to indemnify our customers against claims of non-compliance with employment laws related to hiring. To the extent these claims are successful and exceed our insurance coverages, these obligations would have a negative impact on our cash flow, results of operation and financial condition.

Similarly, customers of Kronos might be concerned about Kronos' ability to fulfill its indemnification obligations. As noted above, Kronos' current owners have paid themselves significant dividends during their ownership tenure, including causing the company to borrow to pay a $490 million dividend earlier this year.

The current owners of Kronos are also delaying substantive interaction with the EEOC in connection with its systemic investigation of Kronos customers, including the more than five years of litigation over the EEOC's information requests, while at the same time looking to sell Kronos. It may be possible that Hellman & Friedman LLC and JMI Equity end up with more than $5 billion from a $752 million investment, while leaving the new owner with the contingent indemnification liabilities. Kronos, under the new owner, may not have sufficient resources to cover the indemnification claims of its employer customers.

* * * * *

"Better Get While the Gettin's Good," the title of this post, is a lyric from Credence Clearwater Revival's song Up Around the Bend. The song's first verse reads:

There's a place up ahead and I'm goin'
Just as fast as my feet can fly
Come away, come away if you're goin',
Leave the sinkin' ship behind.

The question is whether the owners of Kronos Inc. are trying to get while the gettin's good by selling the company and leaving that sinking ship behind?

Damages and Indemnification Challenges for Employers

Claims by Individuals

An employer using a personality assessment as part of its recruiting and hiring process may be subject to a variety of legal claims and charges by applicants and employees. These include claims based on alleged violations of Title 1 of the Americans with Disabilities Act, 42 U.S.C. § 12112 (ADA), including, without limitation:

• The assessment is a pre-offer medical examination that is prohibited by 42 U.S.C. §12112 (d)(2);
• The assessment screens out or tends to screen out an individual with a disability or a class of individuals with disabilities and the Assessment is neither job-related nor consistent with business necessity, violating 42 U.S.C. § 12112(b)(6);
• The employer fails to select and administer the assessment in the most effective manner to ensure that the assessment results accurately reflect the skills, aptitude or whatever other factor that the assessment purports to measure, rather than reflecting an applicant’s impairment, and, as a result, violates 42 U.S.C. § 12112(b)(7); and,
• The employer fails (i) to maintain the information collected from the assessment regarding the medical condition or history of the applicant on separate forms and in separate medical files and (ii) to treat such information as a confidential medical record, violating 29 C.F.R. § 1630.14(b)(1).

Damages available under Title I of the ADA include claims for back pay, front pay, compensatory damages and injunctive relief. Punitive damages may also be available if the employer has engaged in intentional discrimination against a protected class and has done so with reckless indifference to the federally protected rights of persons in that class (Please see the PunitiveDamages post).

If the assessment is determined to be a medical examination, each applicant (not just those who have a mental disability) who took the assessment has a claim against the employer under Title I of the ADA. Courts have repeatedly held that claims of improper disability-related inquiries or medical examinations, improper disclosure of confidential medical information, or retaliation may be brought by any applicant or employee, not just individuals with disabilities. As noted in the WhatAre the Issues post, some companies have more than one million applicants each year.

Indemnification Challenges

Any employer that continues to use pre-employment personality tests should seek indemnification from the testing company whose product it uses. If the employer is currently indemnified by the testing company, the employer should review the limits, restrictions and conditions of that indemnity.

The success of the testing companies in marketing employment personality tests over the past twenty years has created "systemic risk" for its customers. If one employer has violated the law and subjected itself to significant liability as a consequence of its use of an assessment provided by a testing company, then all customers of that testing company are similarly at risk.

Even if testing companies were willing to provide indemnification to all customers, those customers will have to determine whether the testing companies and their insurers have adequate resources to indemnify all customers. As Kenexa, an testing company now owned by IBM), consistently noted in its annual 10-K risk factor disclosures:

The failure of our solutions to comply with employment laws may require us to indemnify our customers, which may harm our business. Some of our customer contracts contain indemnification provisions that require us to indemnify our customers against claims of non-compliance with employment laws related to hiring. To the extent these claims are successful and exceed our insurance coverages, these obligations would have a negative impact on our cash flow, results of operation and financial condition.