Knack Testing Illegal Under ADA?

Wasabi Waiter looks a lot like hundreds of other simple online games. Players acting as sushi servers track the moods of their customers, deliver them dishes that correspond to those emotions, and clear plates while tending to incoming patrons. Unlike most games, though, Wasabi Waiter purportedly analyzes every millisecond of player behavior, measuring conscientiousness, emotion recognition, and other attributes that academic studies show correlate with job performance. The game, designed by startup Knack.it, then scores each player’s likelihood of becoming an outstanding employee.

Knack's assessments are based on games developed by the company that may be "played" on computers and mobile devices. Interesting, but how do persons with disabilities play these games? How would a blind person play these game? How would a persons with limb paralysis play these games? How would a person with diminished mental capacity play these games? How well would a person who may not be computer literate, an older person for example, play these games? What advantage, if any, does a gaming environment provide for one class of persons (young male online gamer ) versus another (mature female non-gamer)?

Screening Out Applicants

Tests that screen out or tend to screen out an individual with a disability or a class of individuals with disabilities are illegal under the Americans with Disabilities Act (ADA) unless the tests are job-related and consistent with business necessity.

Knack testing relies on gamification. Applicants "play" Wasabi Waiter, Balloon Brigade, and other video games to generate the data used by Knack to identify promising applicants. As noted above, however, the reliance on video games screens out persons with disabilities, whether physical disabilities like blindness and limb paralysis or mental disabilities like diminished mental capacity.

Phrased differently, how would physicist Stephen Hawking, clearly an innovator and high performer, fare in taking Knack's Balloon Brigade? Hawking has a motor neurone disease related to amyotrophic lateral sclerosis, a condition that has progressed over the years. He is almost entirely paralysed and communicates through a speech generating device.

From a practical standpoint, legal claims that an individual with a disability has been screened out do not require a statistical showing of disparate impact, or other comparative evidence showing that a group of disabled persons are adversely affected. The plain language of the law – “screen out or tend to screen out” and “an individual with a disability or a class of individuals with disabilities” – confirm that a claim may be supported by evidence that the challenged practice screens out an individual on the basis of their disability.  “In the ADA context, a plaintiff may satisfy the second prong of his prima facie case [impact upon persons with protected characteristic] by demonstrating an adverse impact on himself rather than on an entire group.” Gonzalez v. City of New Braunfels.

Illegal Medical Examination

The ADA prohibits employers, whether directly or via third parties like Knack, from administering pre-employment medical examinations. Guidance by the Equal Employment Opportunity Commission defines medical examination under the ADA by reference to seven factors, any one of which may be sufficient to determine that a test is a medical examination.

Physiological Responses

One of those factors is whether the test measures an applicant's physiological responses to performing a task. EEOC guidance on this issue states:

[I]f an employer measures an applicant's physiological or biological responses to performance, the test would be medical.

According to Knack, its test:

leverages cutting-edge behavioral and cognitive neuroscience, data science, and computer science to build games which produce thousands of data points describing how a player perceives, responds, plans, reacts, thinks, problem-solves, adapts, learns, persists, and performs in a multitude of situations.

Types of physiological responses include a reaction or response - a bodily process occurring due to the effect of some antecedent stimulus or agent. As noted in the prior paragraph, Knack tests create data points that track how an applicant perceives, responds, reacts, adapts, learns and persists. The Knack test, therefore, is an illegal medical examination under the ADA.

Five Factor Model of Personality

Justin Fox, executive editor of the Harvard Business Review Group, took two of the Knack assessments and received information in the following report:

As can be seen by the report, among the factors measured by Knack are conscientiousness, openness and stability. These are elements found in the Five Factor Model of Personality, a model that is currently being challenged in at least seven charges filed with the EEOC. Please see ADA, FFM and DSM.

The ADA prohibits pre-employment medical exams but allows employers to “make pre-employment inquiries into the ability of an applicant to perform job-related functions.” The Knack gaming measurements do not seek job-related information and are not consistent with business necessity. The measurements, designed to reveal information about individuals’ openness, conscientiousness, stability (also referred to as neuroticism), and other factors do not seek information about the ability of an applicant to perform the day-to-day functions of a job.

Knowledge of Disability Not Required

Neither the medical examination claim nor the "screen out" claim under the ADA require that an employer have knowledge that an applicant has a disability, a consistent holding from a number of jurisdictions, including the 7th, 9th, 10th, and 11th Federal Circuit Courts of Appeal.

ADA guidance states, in relevant part:

A covered entity shall not require a medical examination and shall not make inquiries of an employee as to whether such employee is an individual with a disability or as to the nature and severity of the disability, unless such examination or inquiry is shown to be job-related and consistent with business necessity.

According to guidance issued by the EEOC, "This statutory language makes clear that the ADA’s restrictions on inquiries and examinations apply to all employees, not just those with disabilities.”

Punitive Damages for Illegal Medical Examinations under the ADA

In a recent decision, a Pennsylvania federal court held that an acquiring company conducted over 300 unlawful pre-offer medical exams in violation of the ADA.  Cambria Care Center (“CCC”) had purchased the former Cambria County owned nursing home and engaged Grane Healthcare (a separate, but related entity to CCC) to interview and hire employees for the new facility, which was set to open later in the year.

All 300 of the former county nursing home employees were invited to apply for new employment.More than 300 employees from the acquired company applied for positions with  Grane [the acquiring company]. Grane ultimately hired roughly 225 of the applicants. Every applicant was required to undergo a medical examination. Several unsuccessful applicants for employment filed charges of discrimination with the Equal Employment Opportunity Commission (EEOC), alleging that Grane had violated the Americans with Disabilities Act of 1990 (ADA) by conducting pre-offer medical examinations of prospective employees and declining to hire some of them because of actual or perceived disabilities.

The record demonstrates that the employees seeking employment with Grane were subjected to a formalized process consisting of unlawful pre-offer medical examinations and illicit solicitations of detailed medical information. The individuals who were subjected to these illegal examinations and inquiries, including those who were ultimately rejected, were deprived of the prophylactic protection from discrimination that § 12112(d) was designed to create.

Section 12112(d) Claims

The provisions of the ADA pertaining to medical examinations and inquiries are codified at 42 U.S.C. § 12112(d). This statutory framework is designed to shield information about an applicant’s medical condition from his or her prospective employer until after an offer of employment is made. Before an offer of employment is extended, an employer may not ask a job applicant to undergo a medical examination or inquire as to whether he or she “is an individual with a disability.”

Unlike § 12112(a), which aims to protect a discrete class of “disabled” persons from discrimination, § 12112(d) contains no language limiting the category of applicants and employees entitled to statutory protection. Consequently, an individual who is subjected to an unlawful medical examination or inquiry can successfully assert a claim under § 12112(d) without establishing the existence of a statutory “disability.” In this vein, an applicant who is rejected by an employer based on information gleaned from an illegal pre-offer medical examination or inquiry may seek redress under § 12112(d)(2)(A) even if that information does not reveal a “disabling” medical condition. A § 12112(d) violation occurs as soon as “an employer conducts an improper medical examination or asks an improper disability-related question, regardless of the results or response.”

Current Charges with the EEOC

None of the seven charges currently being investigated by the EEOC require the employer to have knowledge that the applicant has a disability.  The claim that an employer used an unlawful pre-offer medical exam does not turn on the employer’s knowledge of the applicant's disability -- and in fact does not require that the applicant have a disability.

For those employers utilizing Five Factor Model-based personality tests as an element of their screening and hiring process, the § 12112(d) violation occurs each time the applicant completes the test and the results are submitted to the employer or assessment company.

The ADA’s prohibition against pre-offer medical examinations and inquiries is prophylactic in nature. Congress was concerned that medical information gleaned from such examinations and inquiries could be used to exclude disabled applicants from further consideration for employment. In order to discriminate against an applicant “on the basis of disability,” an employer must know that the applicant is disabled. By denying employers access to medical information until after offers of employment are made, § 12112(d) aims to ensure that such information does not infect the employee-selection process. In this way, § 12112(d) deters covert discrimination against disabled applicants by forcing employers to make hiring decisions before procuring the information upon which discriminatory decisions could be based.

Reducing Employer Risk Exposure

The timing or sequencing of the testing has a significant impact. Employers could reduce risk exposure by administering the test after having provided the applicant with a conditional offer of employment. The employer would still face the risks associated with tests that discriminate against persons with disabilities, but would not face the potential for claims from all applicants who took the test.

Employers sequence the testing first not because they want to include those who are the best fit for the company culture, but because they need a quick, low-cost method of excluding a significant number of applicants. As one employer (Xco) stated in its position statement,

“The [assessment] enables [Xco]  to assess a large volume of applicants cost-effectively and provides consistency in that assessment. Without the [assessment] reviewing and assessing all individual applications would be extremely costly, both in terms of the labor resources needed to conduct such screening and the costs associated with training managers in making the predictive assessments yielded by the [assessment]. It also would result in greater variability in the reliability of the assessments made by individual managers.

Available Alternatives

Xco, and other employers act as if there are only two alternatives – use assessments (quick, low-cost) and don’t use assessments (time-consuming, costly). There are a variety of alternatives, including tests that do not constitute pre-offer medical exams and do not screen out persons with mental illness. Both Starbucks and CVS are prospering without using illegal pre-offer medical examinations and they do not appear to have put themselves at a competitive disadvantage. 

For example, contrast the stock price performance of CVS (no assessment) and Walgreens (assessment) since CVS stopped using assessments following its settlement with the ACLU in Rhode Island. Please see "The CVS Example" in When the First Domino Falls: Consequences to Employers of Embracing Workforce Assessment Solutions

Systemic Risk 

Due to the lack of variability in the online assessments, any discriminatory element that makes its way into the test algorithm – like the screening out of persons with mental disabilities due to the use of the Five Factor Model - is propagated across the entire universe of job applicants.

Congress sought “to provide clear, strong, consistent, enforceable standards addressing discrimination against individuals with disabilities.” 42 U.S.C. § 12101(b)(2). Covered employers cannot erode those standards by procuring detailed medical information about applicants for employment and contending, at the end of the day, that such information has never been used to the detriment of those applicants. If it were otherwise, disabled individuals would be vulnerable to several forms of discrimination that the ADA was designed to prevent. Since most employment decisions involve elements of discretion, it is relatively easy for an employer to “concoct a plausible reason for not hiring” a particular individual. A decision declaring the admitted § 12112(d) violations to be “harmless” would seriously undermine the ADA’s policy of prophylactic deterrence.

Compensatory Damages 

Before 1991, plaintiffs proceeding under Title VII could only seek “equitable” remedies. Backpay was the primary form of monetary relief available to aggrieved individuals. Section 102 of the Civil Rights Act of 1991 added compensatory and punitive damages to the remedies otherwise available under Title VII and the ADA. The relevant statutory language, permits a “complaining party” to “recover compensatory and punitive damages” from a covered entity responsible for violating § 102 of the ADA. The provisions governing medical examinations and inquiries are included within § 102.

The compensatory damages available to individuals aggrieved by violations of Title I include remuneration for “future pecuniary losses, emotional pain, suffering, inconvenience, mental anguish, loss of enjoyment of life, and other nonpecuniary losses.” The total amount of compensatory and punitive damages available to “each complaining party” is capped at anywhere from $50,000 to $300,000, depending on the number of individuals employed by the covered entity.

Punitive Damages

The Civil Rights Act of 1991 permits any party to “demand a trial by jury” “[i]f a complaining party seeks compensatory or punitive damages” under Title I. The disjunctive wording of this language suggests that a plaintiff can seek an award of punitive damages without seeking an award of compensatory damages. Several Courts of Appeals have concluded that punitive damages may be assessed under the relevant statutory provisions even if no compensatory damages are awarded.

Employers will likely argue that most applicants should receive little, if any, in the way of compensatory damages like backpay. The argument is based on the limited number of positions available as compared with the large number of applicants; only a small percentage of the applicants could have been hired to fill the positions. The expansive scope of compensatory damages – including nonpecuniary losses – plus the availability of punitive damages is designed to provide relief to applicants whose rights have been violated.

As noted below, precluding awards of punitive damages in cases involving no easily quantifiable physical and monetary harm would quell the deterrence that Congress intended to provide when it enacted § 12112(d). The statutory caps on damages ensure against limitless awards in cases of insubstantial harm.

The damages cap for each of the seven companies subject to the EEOC charges is $300,000 per applicant.

A plaintiff seeking punitive damages under Title I must demonstrate that the offending employer “engaged in a discriminatory practice or discriminatory practices with malice or with reckless indifference to [his or her] federally protected rights.” In Kolstad v. American Dental Association, the Supreme Court construed this language to mean that a covered employer must “discriminate in the face of a perceived risk that its actions will violate federal law” in order to be liable for punitive damages.

Reckless Indifference

Examples of reckless indifference by the seven companies subject to the EEOC charges may include:

• Using an assessment is based on the Five Factor Model of personality, which serves as a basis for categorizing and diagnosing personality disorders in the Diagnostic and Statistical Manual of Mental Disorders.
• Failing to perform any adverse impact or validation studies on persons with disabilities. As Kronos stated in one of its filings in the EEOC litigation, “No adverse impact or validation studies have been performed by Kronos … with respect to potential adverse impact on individuals with disabilities.”
• Continuing to use the assessment for years after having been put on notice that the assessment may be an illegal pre-offer test. Contrast Kroger with CVS.
• Asking questions are neither “directly relevant” to the job nor “plainly job-related.” The questions, designed to reveal information about individuals’ “openness,” “conscientiousness,” “extraversion,” “agreeableness,” and “neuroticism,” based on the Five Factor Model, do not seek information about the ability of an applicant to perform the day-to-day functions of a job.
•  Ignoring the almost 40-year old mandate of the Supreme Court in Albemarle Paper Company v. Moody, 422 US 405 (1975) that a test should be validated on people as similar as possible to those to whom it will be administered (i.e., persons with disabilities). The Court further stated that differential studies should be conducted on minority groups – like persons with mental illnesses - wherever feasible. 
• Relying solely on the statements of the assessment company, without any independent review or verification by the employer. The EEOC fact sheet sets out selection and administration guidelines, including  that an employer: (i) ensure that tests and selection procedures are not adopted casually by managers who know little about these processes; (ii) ensure that the tests are valid, independent of the test vendor’s documentation; and, (iii) determine whether there is an equally effective alternative selection procedure that has less adverse impact and adopt that alternative procedure if a selection procedure screens out a protected group.

Punitive Damages Do Not Require Showing of Tangible Harm

The statutory language does not condition an award of punitive damages on a showing of tangible harm. It is worth noting that the term discriminatory practice” is defined more broadly in relation to Title I than it is in relation to Title VII. Although the term includes only “discrimination” in the Title VII context, it is broad enough to encompass “violations” of the ADA that might not constitute “discrimination.”

The breadth of this language, which specifically defines a term appearing in the portion of the statute governing awards of punitive damages, suggests that Congress intended to provide for assessments of punitive damages against employers responsible for intentionally violating Title I’s prophylactic provisions. Precluding awards of punitive damages in cases involving no “easily quantifiable physical and monetary harm would quell the deterrence that Congress intended” to provide when it enacted § 12112(d). The statutory caps on damages “ensure against limitless awards in cases of insubstantial harm.”

Accordingly, the EEOC can seek punitive damages on behalf of the applicants who were unlawfully subjected to pre-offer medical examinations and inquiries even if those applicants are not otherwise entitled to compensatory damages.

Since the examinations in this case were illegally conducted at the pre-offer stage, the EEOC can seek backpay and compensatory damages on behalf of any applicant rejected based on unlawfully procured medical information, regardless of whether his or her injuries are attributable to “discrimination based on a disability.” Moreover, recovery may be sought for less tangible injuries caused by the examinations and inquiries, irrespective of whether those injuries manifested themselves in the form of personnel decisions (suggesting that an “emotional” injury could constitute “actual damage”).

Better Get While the Gettin's Good

On December 11, 2013, Reuters reported that the two private equity companies that took human resources management software firm Kronos Inc. private in 2007 are looking to sell the company. Hellman & Friedman LLC and JMI Equity are exploring a sale of Kronos, which could be valued at more than $4 billion. Interested purchasers are reported to include TPG, KKR and Bain.

The Reuters article states that Hellman & Friedman and JMI have taken advantage of Kronos' strong cash flow to draw more than $1.5 billion in dividends from Kronos, and so have already earned twice the $752.9 million they committed as equity when they agreed to acquire the company in 2007. In November 2013, the two companies had Kronos borrow to pay themselves a $490 million dividend.

Who should be interested in a potential sale of Kronos by Hellman & Friedman and JMI, other than the sellers, potential buyers and Kronos employees? The hundreds of employers that are customers of the Kronos talent acquisition and employee assessment services.

Why should those employers be interested? The risks to those employers from the ongoing systemic investigation by the Equal Employment Opportunity Commission (EEOC) of several Kronos customers, an investigation focused on whether the Kronos assessment services violate the Americans with Disabilities Act (ADA) by illegally screening out persons with disabilities.

What are EEOC systemic investigations? Systemic investigations involves pattern or practice, policy, and/or class cases where the alleged discrimination has a broad impact on an industry, profession, company, or geographic area. In connection with systemic investigations, the EEOC’s enforcement tools include issuing broad information requests and subpoenas on employers that are named as respondents in EEOC charges, particularly when the EEOC suspects systemic discrimination, and filing pattern or practice class lawsuits in federal court.

What are the risks to employers? Systemic investigations by the EEOC and class action claims by job applicants for damages and injunctive relief. For some employers, the potential class size can be measured in the millions of plaintiffs. Employers have primary liability under the ADA, but Kronos has indemnified many of its employer customers. If Kronos does not have the financial resources, however, the indemnification is illusory.

What is Kronos?

Kronos is a U.S.-based workforce management software and services company. According to the company, tens of thousands of organizations in more than 100 countries - including more than half of the Fortune 1000 - use Kronos.

In August 2006, Kronos acquired Unicru, Inc., a company specializing in software used to assess and hire hourly workers. At the time of the acquisition by Kronos, Unicru had as customers for its assessment (the Unicru assessment) more than 140 leading companies and brands, including SuperValu, Kroger, Toys "R" Us, Best Buy, CVS, Borders, Lowe's, Caribou Coffee, and Marquis Healthcare.

The Unicru assessment consists of a number of statements, to which an applicant must answer “strongly disagree,” “disagree,” “agree,” or “strongly agree.” It includes statements such as: “You have confidence in yourself”; "You try to sense what others are thinking and feeling”; “You always say whatever is on your mind”; and “It is easy for you to feel what others are feeling.”

The systemic investigation of Kronos assessment customers, including Kroger, arose from a charge filed with the EEOC more than six years ago by a Kroger job applicant.  The charge led to an investigation that has been ongoing for more than six years and has generated a number of district court and appellate court decisions as Kronos has unsuccessfully sought to avoid disclosing information about the Unicru assessment and its impact on persons protected by the ADA.

Cloning Employees and Institutionalizing Biased Hiring Practices

According to Kronos, the Unicru assessment is an artificial intelligence test that uses neural networks to “learn” the characteristics of a customer’s “best” employees.  As stated by Kronos’ Chief Scientist and the developer of the Unicru assessment, Dr. David Scarborough, in chillingly Orwellian terms, "[o]ur system allows you to clone your best, most reliable people."

First used for engineering and industrial applications during the mid-1980s, neural networks evolved from early artificial intelligence research. Modeled on the function of the human brain, a neural network attempts to imitate human reasoning. Large amounts of data are fed into the network, which looks for relationships and reaches conclusions.


"There are a couple of dangers," states Jai Shekhawat, CEO of Chicago-based Fieldglass Inc., which develops software for managing workers. "Is something a correlation--a predictor--or merely a coincidence? At best, [these methods] are complementary to human judgment, not a substitute for it."

Notwithstanding such dangers, Kronos customers like Kroger are substituting this “coincidence” for human judgment. Based on the prospective employee's answers on the application, the Unicru assessment categorizes the applicant as red, green or yellow. In most cases, red is usually an automatic discard, or, as Dr. Scarborough stated “[m]anagers are strongly discouraged from hiring first quartile (“red”) applicants …”

There is no evidence that the Unicru assessment determines whether an employer’s hiring practices are biased or discriminatory. For example, if the Unicru assessment had been utilized fifty years ago, many companies’ “best” employees would have the personality traits of white males – persons of color, women and those with disabilities need not have applied.

The Unicru assessment embeds and industrializes existing stigma, bias and discrimination in the hiring process. As stated by Cynthia Dwork and Deirdre K. Mulligan in a recent Stanford Law Review article:

While automated decisionmaking systems “may reduce the impact of biased individuals, they may also normalize the far more massive impacts of system-level biases and blind spots.” Rooting out biases and blind spots in big data depends on our ability to constrain, understand, and test the systems that use such data to shape information, experiences, and opportunities.

As a “blind” tool that “learns” from the employer, the Unicru assessment replicates the existing bias of the employer and applies it on a massive scale. All applicants have their test responses fed through a discriminatory filter that is the Unicru assessment (a filter that is biased both on its own and in conjunction with its “learned” behavior). 

Illegal Medical Examination

The ADA prohibits the use of pre-employment medical examinations. At the pre-offer stage, an employer is only entitled to ask about an applicant's ability to perform the essential functions of the job. The ADA's prohibition against pre-employment examinations seeks to ensure that the applicant's disability is not considered prior to the assessment of the applicant's qualifications.

EEOC guidance provides a seven-factor test for analyzing whether a test or procedure qualifies as a “medical examination,” including:

• whether the test is designed to reveal an impairment of physical or mental health such as those listed in the Diagnostic and Statistical Manual of Mental Disorders (“DSM”); and
• whether the test is interpreted by a health care professional.

According to the guidance, the presence of any one of the seven factors is enough to support a finding that the test is a medical examination and the Unicru assessment meets the two factors listed above. 

Since the Unicru assessment is based on the five-factor model (FFM) of personality it meets the first factor listed above. As set out in previous posts -  ADA, FFM and DSM and Employment Assessments are Designed to Reveal an Impairment - assessments based on the FFM are designed to reveal an impairment of mental health, such as those listed in the DSM.

As to the second factor, whether the test is interpreted by a health care professional, the individuals who developed the Unicru assessment are psychologists, most of whom are members of the APA. In developing the Assessments, the psychologists establish the rules by which the assessments are to be interpreted (i.e., how the responses to the questions are to be scored, including whether the applicant receives a green, yellow or red rating).

According to the APA Model Act for State Licensure of Psychologists, “[t]he practice of psychology includes … (a) psychological testing and the evaluation or assessment of personal characteristics, such as intelligence; personality; cognitive, physical, and/or emotional abilities; … [and] (f) provision of direct services to … groups for the purpose of enhancing … organizational effectiveness, using psychological principles, methods, and/or procedures … for making decisions about the individual, such as selection …”

EEOC guidance states that psychologists are among the “variety of health professionals [that] may provide documentation regarding psychiatric disabilities” for ADA purposes. Accordingly, the psychologists who developed the Unicru assessment are "health care providers" for purposes of the ADA.

The CVS Example

In July 2011, CVS and the Rhode Island Civil Liberties Union (ACLU) entered into a voluntary settlement addressing the ACLU’s complaint challenging CVS’s use of a pre-hire questionnaire that the ACLU claimed could have a discriminatory impact on people with certain mental impairments or disorders. 

The CVS questionnaire contained statements to which applicants were required to respond, including: “You change from happy to sad without any reason,” “You get angry more often than nervous,” “Your moods are steady from day to day,” and “There’s no use having close friends; they always let you down.”

Responding to a complaint filed by the ACLU, the Rhode Island Commission for Human Rights had issued a finding in February 2011 that there was "probable cause" to believe that the questionnaire used by CVS violated state anti-discrimination laws that bar employers from eliciting information that pertain to job applicants' mental or physical disabilities.

Although employers may legally ask questions designed to help determine an applicant’s personality or aptitude for a job, the ACLU’s complaint argued that questions found in the CVS pre-offer assessment “could have the effect of discriminating against applicants with certain mental impairments or disorders, and go beyond merely measuring general personality traits.” 

Pursuant to the settlement agreement, CVS agreed to permanently remove the questions at issue from its online application.

Systemic Risk to Employers

The success of workforce science companies in developing employment personality and assessment tests over the past twenty years has created "systemic risk" for their employer customers. If one employer has violated the law and subjected itself to significant liability as a consequence of its use of an assessment provided by a workforce science company, then all customers of that company are similarly at risk. Workforce science companies provide their services to thousands of employers, including many of the largest employers in the U.S. 

The lack of diversity in the psychological model underlying many of the personality tests offered by workforce science companies (the five-factor model of personality or Big Five) also means that if one workforce science company's personality tests that use the Big Five is found to be an illegal medical examination under the Americans with Disabilities Act (ADA), all workforce science companies that use the Big Five (and, more importantly, their customers) are similarly at risk. 

There are multiple risks to employers arising from the use of personality tests and workforce assessments, including: 

1. Claims under the ADA and the Rehabilitation Act of 1973 that the personality tests are illegal medical examinations or that they illegally screen out persons with mental illness (as set out above); 
2. Claims under the ADA and the Rehabilitation Act of 1973 that the employer fails to select and administer the assessment in the most effective manner to ensure that the assessment results accurately reflect the skills, aptitude or whatever other factor that the assessment purports to measure, rather than reflecting an applicant’s impairment; 
3. Claims that employers and workforce assessment companies fail to properly safeguard confidential medical information obtained from the personality tests and illegally use that confidential medical information in violation of the ADA; and
4. Claims under Title VII of the Civil Rights Act that the workforce analytics cause there to be a disparate impact on the hiring of blacks and Hispanics.

As to the potential size of the plaintiff classes for the claims listed above, they range from a percentage of all applicants (in the case of claims that the tests illegally screen out persons with mental illness and claims of disparate impact under Title VII) to all applicants over the past 12 months (in the case of claims that the personality test is an illegal medical examination) to all applicants, employees and ex-employees over a longer period of time (in the case of claims that employers and workforce assessment companies failed to safeguard confidential medical information).

For some employers, the potential class size can be measured in the millions of plaintiffs. Consistent with the 2011 Supreme Court decision in Wal-Mart Stores, Inc. v. Dukes, plaintiffs in a class action suit predicated on the use of personality tests and workforce analytics will be challenging a uniform, company-wide practice. The uniform use of testing by an employer demonstrates that "there are questions of law or fact common to the class," or commonality, as required by the rules governing class actions.

Illusory Indemnification?

A key element in continuing to use Kronos assessment services is Kronos' ability to indemnify its employer customers. As noted above, the success of workforce assessment companies in marketing personality tests and workforce analytics over the past twenty years has created "systemic risk" for its customers. If one employer has violated the law and subjected itself to significant liability as a consequence of its use of a solution provided by a workforce assessment company, then all customers of that workforce assessment company are similarly at risk.

Even assuming workforce assessment companies are willing to provide indemnification to all customers, those employers need to independently assess whether the workforce assessment companies and their insurers have adequate resources to indemnify all customers. 

As Kenexa, an employment assessment company, consistently noted in its annual 10-K risk factor disclosures prior to its December 2012 acquisition by IBM:

The failure of our solutions to comply with employment laws may require us to indemnify our customers, which may harm our business. Some of our customer contracts contain indemnification provisions that require us to indemnify our customers against claims of non-compliance with employment laws related to hiring. To the extent these claims are successful and exceed our insurance coverages, these obligations would have a negative impact on our cash flow, results of operation and financial condition.

Similarly, customers of Kronos might be concerned about Kronos' ability to fulfill its indemnification obligations. As noted above, Kronos' current owners have paid themselves significant dividends during their ownership tenure, including causing the company to borrow to pay a $490 million dividend earlier this year.

The current owners of Kronos are also delaying substantive interaction with the EEOC in connection with its systemic investigation of Kronos customers, including the more than five years of litigation over the EEOC's information requests, while at the same time looking to sell Kronos. It may be possible that Hellman & Friedman LLC and JMI Equity end up with more than $5 billion from a $752 million investment, while leaving the new owner with the contingent indemnification liabilities. Kronos, under the new owner, may not have sufficient resources to cover the indemnification claims of its employer customers.

* * * * *

"Better Get While the Gettin's Good," the title of this post, is a lyric from Credence Clearwater Revival's song Up Around the Bend. The song's first verse reads:

There's a place up ahead and I'm goin'
Just as fast as my feet can fly
Come away, come away if you're goin',
Leave the sinkin' ship behind.

The question is whether the owners of Kronos Inc. are trying to get while the gettin's good by selling the company and leaving that sinking ship behind?

Kroger and Kronos: Chaos and Disorder

In classic Greek mythology, Kronos (also known as Cronus) was the leader of the first generation of Titans. Cronus was usually depicted with a sickle or scythe and the Greeks considered Cronus a cruel and tempestuous force of chaos and disorder.

Kronos, the company, is a U.S.-based workforce management software and services company. According to the company, tens of thousands of organizations in more than 100 countries - including more than half of the Fortune 1000 - use Kronos to control labor costs, minimize compliance risk, and improve workforce productivity.

In August 2006, Kronos acquired Unicru, Inc., a company specializing in software used to assess and hire hourly workers. At the time of the acquisition by Kronos, Unicru had as customers more than 140 leading companies and brands, including SuperValu, Kroger, Toys "R" Us, Best Buy, CVS, Borders, Lowe's, Caribou Coffee, and Marquis Healthcare.

The Unicru assessment consists of a number of statements, to which an applicant must answer “strongly disagree,” “disagree,” “agree,” or “strongly agree.” It includes statements such as the following: “You have confidence in yourself”; You are always cheerful”; “You try to sense what others are thinking and feeling”; “You always say whatever is on your mind”; and “It is easy for you to feel what others are feeling.”

Kroger, Kronos and the Unicru assessment are being investigated by the Equal Employment Opportunity Commission (EEOC) for compliance with labor and employment laws, including the Americans with Disabilities Act (ADA). The investigation has been ongoing for more than five years and has generated a number of district court and appellate court decisions as Kronos has sought to avoid disclosing information about the Unicru assessment and its impact on persons protected by the ADA.

Cloning Employees and Institutionalizing Biased Hiring Practices

According to Kronos, the Unicru assessment is an artificial intelligence test that uses neural networks to “learn” the characteristics of a customer’s “best” employees.  As stated by Kronos’ Chief Scientist and the developer of the Unicru assessment, Dr. David Scarborough, in chillingly Orwellian terms, "[o]ur system allows you to clone your best, most reliable people."

First used for engineering and industrial applications during the mid-1980s, neural networks evolved from early artificial intelligence research. Modeled on the function of the human brain, a neural network attempts to imitate human reasoning. Large amounts of data are fed into the network, which looks for relationships and reaches conclusions.

"There are a couple of dangers," states Jai Shekhawat, CEO of Chicago-based Fieldglass Inc., which develops software for managing workers. "Is something a correlation--a predictor--or merely a coincidence? At best, [these methods] are complementary to human judgment, not a substitute for it."

Notwithstanding such dangers, Kronos customers like Kroger are substituting this “coincidence” for human judgment. Based on the prospective employee's answers on the application, the Unicru assessment categorizes the applicant as red, green or yellow. In most cases, red is usually an automatic discard, or, as Dr. Scarborough stated “[m]anagers are strongly discouraged from hiring first quartile (“red”) applicants …”

There is no evidence that the Unicru assessment determines whether an employer’s hiring practices are biased or discriminatory. For example, if the Unicru assessment had been utilized fifty years ago, many companies’ “best” employees would have the personality traits of white males – persons of color, women and those with disabilities need not have applied.

The Unicru assessment embeds and industrializes existing stigma, bias and discrimination in the hiring process. As stated by Cynthia Dwork and Deirdre K. Mulligan in a recent Stanford Law Review article:

While automated decisionmaking systems “may reduce the impact of biased individuals, they may also normalize the far more massive impacts of system-level biases and blind spots.” Rooting out biases and blind spots in big data depends on our ability to constrain, understand, and test the systems that use such data to shape information, experiences, and opportunities.

As a “blind” tool that “learns” from the employer, the Unicru assessment replicates the existing bias of the employer and applies it on a massive scale. All applicants have their test responses fed through a discriminatory filter that is the Unicru assessment (a filter that is biased both on its own and in conjunction with its “learned” behavior). Hiring decisions are being made by Kroger and other Kronos customers based on this deeply flawed process.

Illegal Medical Examination

The ADA prohibits the use of pre-employment medical examinations. At the pre-offer stage, an employer, like Kroger, is only entitled to ask about an applicant's ability to perform the essential functions of the job. The ADA's prohibition against pre-employment examinations seeks to ensure that the applicant's disability is not considered prior to the assessment of the applicant's qualifications.

EEOC guidance provides a seven-factor test for analyzing whether a test or procedure qualifies as a “medical examination,” including:

• whether the test is designed to reveal an impairment of physical or mental health such as those listed in the Diagnostic and Statistical Manual of Mental Disorders (“DSM”); and
• whether the test is interpreted by a health care professional.

According to the guidance, the presence of any one of the seven factors is enough to support a finding that the test is a medical examination and the Unicru assessment meets the two factors listed above. 

Since the Unicru assessment is based on the five-factor model (FFM) of personality it meets the first factor listed above. As set out in previous posts -  ADA, FFM and DSM and Employment Assessments are Designed to Reveal an Impairment - assessments based on the FFM are designed to reveal an impairment of mental health, such as those listed in the DSM.

As to the second factor, whether the test is interpreted by a health care professional, the individuals who developed the Unicru assessment are psychologists, most of whom are members of the APA. In developing the Assessments, the psychologists establish the rules by which the assessments are to be interpreted (i.e., how the responses to the questions are to be scored, including whether the applicant receives a green, yellow or red rating).

According to the APA Model Act for State Licensure of Psychologists, “[t]he practice of psychology includes … (a) psychological testing and the evaluation or assessment of personal characteristics, such as intelligence; personality; cognitive, physical, and/or emotional abilities; … [and] (f) provision of direct services to … groups for the purpose of enhancing … organizational effectiveness, using psychological principles, methods, and/or procedures … for making decisions about the individual, such as selection …”

EEOC guidance states that psychologists are among the “variety of health professionals [that] may provide documentation regarding psychiatric disabilities” for ADA purposes. Accordingly, the psychologists who developed the Unicru assessment are "health care providers" for purposes of the ADA.

(Not) Walking the Talk

Kroger's Policy on Business Ethics states:

We are committed to a policy of equal opportunity for all associates without regard to race, color, religion, gender, national origin, age, disability or sexual orientation.

Kroger has six core values: Honesty; Integrity, Respect; Diversity; Safety; and, Inclusion. In a June 13, 2011 press release announcing the appointment of Kroger's chief diversity officer, Kroger's CEO is quoted as saying:

“Diversity is a core value at Kroger. We take our commitment to diversity seriously, both because it is right and because it makes us better at our business. When our decision-making is inclusive and reflects the diversity of our customers, we make better decisions.”

For job applicants with mental illness, there is no respect, no inclusion, no diversity, no honesty and no integrity. In the more than twenty years since passage of the ADA, there has been little positive movement in de-stigmatizing mental illness in the workplace (please see Mental Illness and Issues of Employment). People with mental illnesses identify employment discrimination as one of their most frequent stigma experiences. In its use of the Unicru assessment, Kroger, wittingly or not, continues the disturbing pattern of employment discrimination against citizens of the United States with mental illness.

Failing Customers and Shareowners

Kroger's Policy on Business Ethics also states:

As a retailer providing millions of Americans with their daily food and as a publicly owned company, The Kroger Co. has a special obligation to comply with the law and deal ethically with customers, suppliers, associates, and shareowners. 

Psychiatric medications are among the most widely prescribed and biggest-selling class of drugs in the U.S. In 2011, Americans spent $18.2 billion on antipsychotics to treat depression, bipolar disorder and schizophrenia, $11.0 billion on antidepressants and $7.9 billion on treatment for ADHD, according to IMS Health, which tracks prescription-drug sales. These three categories of prescription drug sales accounted for approximately 11.6% of all prescription drug sales in the U.S. for 2011

Kroger is the fifth-largest pharmacy operator in the United States, operating retail pharmacies in over 1,948 stores. During fiscal 2011, Kroger pharmacists filled over 146 million prescriptions at a retail value of approximately $7.3 billion. Assuming 11.6% of Kroger prescription drug sales were for antipsychotics, antidepressants and ADHD medications, prescription drugs for persons with mental illness accounted for approximately $847 million of Kroger prescription drug sales in 2011, some two-thirds of the amount of Kroger’s operating profit for that year.

Persons who have their prescriptions filled at Kroger, their family members and other loved ones also shop at Kroger for other products and services. Those persons, their family members and other loved ones provide a material percentage of Kroger’s overall revenue each year. How does Kroger repay this customer loyalty? By utilizing an unlawful pre-employment assessment to eliminate from consideration for employment persons with mental illness.

Why should persons with mental illness, their family members and other loved ones continue to shop at Kroger? Good question. 

Kroger's continuing use of the Unicru assessments calls into question Kroger's "special obligation to comply with the law and deal ethically" with its shareowners, As previously noted, Kroger and Kronos have been engaged in litigation with the EEOC for more than five years over legality of the Unicru assessment. To be precise, the five years of litigation have primarily addressed the unwillingness of Kroger and Kronos to provide information requested by the EEOC in order to conduct its investigation into the Unicru assessment. Two appellate courts, the latest in September 2012, have ruled decisively in favor of the EEOC and its right to investigate a broad set of nationwide and historical data from Kroger and Kronos.

At anytime over the past five years, Kroger could have ceased using the Unicru assessment, if only as a risk mitigation strategy for its shareowners. As noted in the Challenges to Pre-Employment Assessments posting, in July 2011, CVS and the Rhode Island Civil Liberties Union (ACLU) entered into a voluntary settlement addressing the ACLU’s complaint challenging CVS’s use of a pre-hire questionnaire that the ACLU claimed could have a discriminatory impact on people with certain mental impairments or disorders. Pursuant to the settlement agreement, CVS agreed to permanently remove the questions at issue from its online application.

Each day Kroger continues to use the Unicru assessment, there are thousands more potential plaintiffs with claims against Kroger. By now, the aggregate number of potential plaintiffs numbers in the millions - with each job applicant over the past 5+ years having a number of claims against Kroger.

Under the ADA, Kroger may use a third party like Kronos to undertake the assessment of Kroger job applicants. The use of a third party, however, does not insulate Kroger from any claims arising from the assessment usage. Under the ADA, Kroger is responsible (and liable) for any failures on the part of Kronos and the Unicru assessment to comply with the provisions of the ADA.

Any comfort Kroger or its shareowners take in the indemnification provided by Kronos should be tempered by the recognition that such indemnification may prove illusory. Kronos and its insurers may not have the capital necessary to indemnify Kroger and its shareowners for all claims arising from Kroger's continuing use of the Unicru assessment. Please see Damages and Indemnification Challenges for Employers.

Workforce Science: A Critical Look at Big Data and the Selection and Management of Employees

This post takes a critical look at the application of "Big Data" to company human resources; specifically, the selection and management of employees.

The post focuses on one company, Evolv, that describes itself as "a workforce science software company that harnesses big data, predictive analytics and cloud computing to help businesses improve workplace productivity and profitability."  Evolv was selected not because it is sui generis; rather, it is emblematic of numerous companies, from start-ups to well-established companies that market "workforce science" to employers.

According to Evolv, workforce science:

[I]dentifies the characteristics of the most qualified, productive employees within an hourly workforce throughout the employee lifecycle. By using objective, data-driven methodologies and machine learning, Evolv enables operational and financial executives to make better business decisions that result in millions of dollars in savings each year. 

The Evolv graphic below is intended to illustrate the process of workforce science.

The steps listed in this flowchart serve as the titles of the sub-headings below.

New and Existing Data:

Companies capture and store workforce data.

Questions arise concerning the nature of this data, its accuracy and usefulness. Most companies have vast amounts of HR data (employee demographics, performance ratings, talent mobility data, training completed, age, academic history, etc.) but they are in no position to use it. According to Bersin by Deloitte, an HR research and consultancy organization, only 20% of the companies believe that the data they capture now (let alone historically) is highly credible and reliable for decision-making in their own organization.

The complexity of working with myriad data types and myriad, often incompatible, systems was underscored by Dat Tran of the U.S. Department of Veterans Affairs at the 2013 MIT Chief Data Officer and Information Quality Symposium. "The VA does not have an integrated data environment; we have myriad systems and databases, and enterprise data standards do not exist. There is no 360-degree view of the customer," Tran said in a discussion of the obstacles facing an agency dealing with 11 petabytes of data and 6.3 million patients.  

"Bad data" is data that has not been collected accurately or consistently or the data has been defined differently from person to person, group to group and company to company. In the recruitment and hiring context, unproctored online tests allow an applicant to take the test anywhere and anytime. That freedom creates conditions ripe for obtaining "bad' data. As stated by Jim Beaty, PhD, and Chief Science Officer at testing company Previsor:

Applicants who want to cheat on the test can employ a number of strategies to beat the test, including logging in for the test multiple times to practice or get the answers, colluding with another persons while completing the test, or hiring a test proxy to take the test.

And what about the accuracy of tests responses from those who are hired? Analyzing a sample of over 31,000 employees, Evolv found that employees who said they were most likely to follow the rules left the job on average 10% earlier, were 3% less likely to close a sale and were actually not particularly good at following rules.

Decision-makers increasingly face computer-generated information and analyses that could be collected and analyzed in no other way. Precisely for that reason, going behind that output is out of the question, even if one has good cause to be suspicious. In short, the computer analysis becomes a credible reference point although based on poor data.

Supplement

Psychometric tools gather predictive data on the workforce throughout the employment lifecycle

Psychometrics is the field of study concerned with the theory and technique of psychological measurement, which includes the measurement of knowledge, abilities, attitudes, personality traits, and educational measurement. The field is primarily concerned with the construction and validation of measurement instruments such as questionnaires, tests, and personality assessments.

To what extent are psychometric tools accurate predictors of behavior or performance?

According to a 2012 study by Oracle and Development Dimensions International (DDI), a global human resources consulting firm whose expertise includes designing and implementing selection systems, more than 250 staffing directors and over 2,000 new hires from 28 countries provided the following perspectives on their organization’s selection processes (the following are excerpts from the study):

• [O]nly 41 percent of staffing directors report that their pre-employment assessments are able to predict better hires.
• Only half of staffing directors rate their systems as effective, and even fewer view them as aligned, objective, flexible, efficient, or integrated. 
• [T]he actual process for making a hiring decision is less effective than a coin toss.

In a 2007 article titled, “Reconsidering the Use of Personality Tests in Employment Contexts”, co-authored by six current or former editors of academic psychological journals, Dr. Kevin Murphy, Professor of Psychology at Pennsylvania State University and Editor of the Journal of Applied Psychology (1996-2002), states:

The problem with personality tests is … that the validity of personality measures as predictors of job performance is often disappointingly low. A couple of years ago, I heard a SIOP talk by Murray Barrick … He said, “If you took all the … [factors], measured well, you corrected for everything using the most optimistic corrections you could possibly get, you could account for about 15% of the variance in performance [between projected and actual performance].” … You are saying that if you take normal personality tests, putting everything together in an optimal fashion and being as optimistic as possible, you’ll leave 85% of the variance unaccounted for. The argument for using personality tests to predict performance does not strike me as convincing in the first place.

Cleanse and Upload

Structured and Unstructured Data Is Aggregated

Data isn't something that's abstract and value-neutral. Data only exists when it's collected, and collecting data is a human activity. And in turn, the act of collecting and analyzing data changes (one could even say "interprets") us. 

Workforce science requires enormous amounts of historic or legacy data. This data has to be consolidated from a number of disparate source systems within each company, each with their specific data environment and particular brand of business logic. That data consolidation then must be replicated across hundreds or thousands of companies.

Structured data refers to data that is identifiable because it is organized in a structure. The most common form of structured data is a database where specific information is stored based on a methodology of columns and rows (i.e., Excel). Structured data is understood by computers and is also efficiently organized for human readers.  

Unstructured data refers to information that either does not have a pre-defined data model or is not organized in a pre-defined manner. This results in irregularities and ambiguities that make it difficult to understand using traditional computer programs as compared to structured data.

Unstructured data consists of two basic categories; textual objects (based on written or printed language, such as emails or Word documents); and bitmap objects (non-language based, such as image, video or audio files).

There are many types of techniques that need to be put together in a complex data processing flow utilizing unstructured data. These techniques include: 

• information extraction (to produce structured records from text or semi-structured data)
• cleansing and normalization (to be able to even compare string values of the same type, such as a dollar amount or a job title)
• entity resolution (to link records that correspond to the same real-world entity or that are related via some other type of semantic relationship)
• mapping (to bring the extracted and linked records to a uniform schematic representation)
• data fusion (to merge all the related facts into one integrated, clean object)

Assumptions are embedded in a data model upon its creation. Data sources are shaped through ‘washing’, integration, and algorithmic calculations in order to be commensurate to an acceptable level that allows a data set to be created. By the time the data are ready to be used, they are already ‘at several degrees of remove from the world.’

Data is never raw; it’s always structured according to somebody’s predispositions and values. The end result looks disinterested, but, in reality, there are value choices all the way through, from construction to interpretation.

Analyze and Predict

Data Analyzed Using Machine Learning and Predictive Algorithms

The theory of big data is to have no theory, at least about human nature. One just gathers huge amounts of information, observes the patterns and estimates probabilities about how people will act in the future. One does not address causality.

In linear systems, cause and effect is much easier to pinpoint. However, the world around us is considered a complex system where there are often multiple variables pushing an outcome to occur. Nigel Goldenfeld, a professor of physics at University Illinois, sums it up best: “For every event that occurs, there are a multitude of possible causes, and the extent to which each contributes to the event is not clear.”

Algorithms and big data are powerful tools. Wisely used, they can help match the right people with the right jobs. But they must be designed and used by humans, or they can go very wrong. As David Brooks wrote in the New York Times:

Data creates bigger haystacks. This is a point Nassim Taleb, the author of “Antifragile,” has made. As we acquire more data, we have the ability to find many, many more statistically significant correlations. Most of these correlations are spurious and deceive us when we’re trying to understand a situation. Falsity grows exponentially the more data we collect. 

There’s a saying in artificial intelligence circles that techniques like machine learning can very quickly get you 80% of the way to solving just about any (real world) problem, but going beyond 80% is extremely hard, maybe even impossible. The Netflix Challenge is a case in point: hundreds of the best researchers in the world worked on the problem for 2 years and the winning team got a 10% improvement over Netflix’s in-house algorithm.

A corollary of the above saying is that it is very rare for startup companies to ever have a competitive advantage because of their machine learning algorithms.  If a worldwide concerted effort can only improve Netflix’s algorithm by 10%, how likely are 4 people in an R&D department in a startup going to have a significant breakthrough.  Modern machine algorithms are the product of thousands of academics and billions of dollars of R&D and are generally only improved upon at the margins by individual companies.

Some of the best and brightest organizations have recognized that improvement, if any, in machine learning comes from outside the organization. Facebook, Ford, GE and other companies have run contests for data-science challenges on Kaggle, while NASA and other government agencies, as well as the Harvard Business School, have taken the crowdsource route on Topcoder.

Abhishek Shivkumar of IBM Watson Labs has listed the top ten problems for machine learning in 2013. These problems include churn prediction, truth and veracity, scalability and intelligent learning. This doesn’t mean machine learning isn’t ever useful – it just means one needs to apply it to contexts that are fault tolerant:  for example, online ad targeting, ranking search results, recommendations, and spam filtering.  Applying machine learning concepts in the context of persons livelihoods (and, potentially, lives) is problematic, not just for the individual applicant or employee but also for the employer and its potential liability exposure. 

Big Data Networking

Results Are Benchmarked Against Big Data Network

According to Evolv, its network

extracts learnings and insight from the millions of real time talent data points – from across the Evolv client base, or Network – streaming in and out of the Evolv platform every single day, week and month.  

Some of the real time data points streaming in and out of the Evolv Network are data from the use of pre-employment personality tests administered to applicants and employees. As set out in the posts What Are The Issues?, ADA, FFM and DSM and Employment Assessment Are Designed to Reveal An Impairment, tests utilizing the Five-Factor Model of personality may be considered illegal medical examinations under the Americans with Disabilities Act (ADA).

Consequently, information obtained from those tests is confidential medical information, the use of which is subject to strict limits. Regulations require that confidential information be kept on separate forms and in separate files and it may not be intermingled with other information - i.e., shared with third parties on the Evolv network.

Not only are employers subject to claims by applicants and employees alleging breach of the confidentiality provisions of the ADA, the use of confidential medical information in assessment, hiring and other human resource function may have created a virus that has "infected" a variety of databases, applications and software solutions utilized by the employer.

Costs to employers from the illegal use of confidential medical information include damages payable to applicants and employees, defense transaction costs (i.e., legal fees), and costs to "sanitize" infected databases, applications and software solutions. As set out in the Damages and Indemnification Challenges for Employers post, even if companies like Evolv are willing to provide indemnification to all customers, those customers will have to determine whether the company and its insurers have adequate resources to indemnify all customers.

Evaluate

Analyzed Data Reveals Insights That Drive Workforce Performance and Retention

+

Act

The Impact of Insights Are Quantified and Used to Inform Decision-making

As noted in a prior post, prejudice does not rise from malice or hostile animus alone. It may result as well from insensitivity caused by simple want of careful, rational reflection.

For example, take two insights from Evolv:

1. Living in close proximity to the job site and having access to reliable transportation—are correlated with reduced attrition and better performance; and
2. Referred employees have 10% longer tenure than non-referred employees and demonstrate approximately equal performance.

An employer confronted with these two insights might well determine that (i) applicants living beyond a certain distance from the job site (i.e., retail store) should be excluded from employment consideration and (ii) preference in hiring should be extended to applicants referred by existing employees.

Painting with the broad brush of distance from job site will result in well-qualified applicants being excluded, applicants who might have ended up being among the longest tenured of employees. Remember that the Evolv insight is a generalized correlation (i.e., persons living closer to the job site tend to have longer tenure than persons living farther from the job site). The insight says nothing about any particular applicant.

As a consequence, employers will pass over qualified applicants solely because they live (or don't live) in certain areas. Not only does the employer do a disservice to itself and the applicant, they increase the risk of employment litigation, with its consequent costs. How?

A recent New York Time article, "In Climbing Income Ladder, Location Matters," reads, in part:

Her nearly four-hour round-trip [job commute] stems largely from the economic geography of Atlanta, which is one of America’s most affluent metropolitan areas yet also one of the most physically divided by income. The low-income neighborhoods here often stretch for miles, with rows of houses and low-slung apartments, interrupted by the occasional strip mall, and lacking much in the way of good-paying jobs

The dearth of good-paying jobs in low-income neighborhoods means that residents of those neighborhoods have a longer commute. The 2010 Census showed that poverty rates are much higher for blacks and Hispanics. Consequently, hiring decisions predicated on distance, intentionally or not, discriminate against certain races.

Similarly, an employer extending a hiring preference to referrals of existing employees may be further exacerbating the discriminatory impact of its hiring process. Those referrals tend to be persons from the same neighborhoods and socioeconomic backgrounds of existing employees, meaning that workforce diversity, broadly considered, will decline.

With the huge amounts of "bad" data that get generated and stored daily, the failure to understand how to leverage the data in a practical way that has business benefit will increasingly lead to shaky insights and faulty decision-making, with significant costs to applicants, employees , employers  and society.

Optimize

Closed-Loop Optimization Constantly Analyzes and Refines Insights

According to Evolv, "closed-loop optimization is the process of using Big Data analytics to determine the outcomes of the assessments and other data collected, and then using the knowledge gained to make ever more effective assessments." Click on this link for an Evolv video that describes the closed-loop optimization process.

The challenge in using a closed-loop optimization process for hiring and employment decisions is that those decisions do not fit within a closed loop. Take for example the Evolv insight that living in close proximity to the job site are correlated with reduced attrition and better performance. Over time, the closed-loop optimization process for that insight means that a growing percentage of the workforce lives in close proximity to the job site. Excellent. Less attrition and better performance across jobsite.

That closed loop, however, does not account for factors like the element of time and the relative immobility of persons and companies. Businesses tend to be clustered; they are not evenly spread throughout the geography. If all businesses in a particular area focus on hiring applicants in close proximity, costs will increase (greater demand for the same number of applicants), employee turnover will increase (since the number of geographically-proximate employees changes slowly) and profitability will decrease (higher wage costs combined with greater turnover).

When two variables, A and B, are found to be correlated, there are several possibilities:

• A causes B
• B causes A
• A causes B at the same time as B causes A (a self-reinforcing system)
• Some third factor causes both A and B

The correlation is simple coincidence. It is wrong to assume any of these possibilities. Evolv, however, assumes that A (proximity to job site) causes B (reduced attrition and better performance). Therefore, employers should hire applicants who live closer to the job site. 

The correlation could also demonstrate B (reduced attrition and better performance) is caused by C (proximity of job site to applicants homes). Instead of being a hiring insight, the correlation might function better as being a job site location insight. Given the relative immobility of persons and companies, locating a job site (call center, etc.) close to communities with high numbers of lower-income persons could lead to a more sustainable competitive advantage.

As David Brooks wrote, "Data struggles with context. Human decisions are not discrete events. They are embedded in sequences and contexts. ... Data analysis is pretty bad at narrative and emergent thinking, and it cannot match the explanatory suppleness of even a mediocre novel."

Executives and managers frequently hear about some new software billed as the “next big thing.” They call the software provider and say, “We heard you have a great tool and we’d like a demonstration.” The software is certainly seductive with its bells and whistles, but its effectiveness and usefulness depend upon the validity of the information going in and how the people actually work with it over time. Having a tool is great, but remember that a fool with a tool is still a fool (and sometimes a dangerous fool).