Knack Testing Illegal Under ADA?

Wasabi Waiter looks a lot like hundreds of other simple online games. Players acting as sushi servers track the moods of their customers, deliver them dishes that correspond to those emotions, and clear plates while tending to incoming patrons. Unlike most games, though, Wasabi Waiter purportedly analyzes every millisecond of player behavior, measuring conscientiousness, emotion recognition, and other attributes that academic studies show correlate with job performance. The game, designed by startup Knack.it, then scores each player’s likelihood of becoming an outstanding employee.

Knack's assessments are based on games developed by the company that may be "played" on computers and mobile devices. Interesting, but how do persons with disabilities play these games? How would a blind person play these game? How would a persons with limb paralysis play these games? How would a person with diminished mental capacity play these games? How well would a person who may not be computer literate, an older person for example, play these games? What advantage, if any, does a gaming environment provide for one class of persons (young male online gamer ) versus another (mature female non-gamer)?

Screening Out Applicants

Tests that screen out or tend to screen out an individual with a disability or a class of individuals with disabilities are illegal under the Americans with Disabilities Act (ADA) unless the tests are job-related and consistent with business necessity.

Knack testing relies on gamification. Applicants "play" Wasabi Waiter, Balloon Brigade, and other video games to generate the data used by Knack to identify promising applicants. As noted above, however, the reliance on video games screens out persons with disabilities, whether physical disabilities like blindness and limb paralysis or mental disabilities like diminished mental capacity.

Phrased differently, how would physicist Stephen Hawking, clearly an innovator and high performer, fare in taking Knack's Balloon Brigade? Hawking has a motor neurone disease related to amyotrophic lateral sclerosis, a condition that has progressed over the years. He is almost entirely paralysed and communicates through a speech generating device.

From a practical standpoint, legal claims that an individual with a disability has been screened out do not require a statistical showing of disparate impact, or other comparative evidence showing that a group of disabled persons are adversely affected. The plain language of the law – “screen out or tend to screen out” and “an individual with a disability or a class of individuals with disabilities” – confirm that a claim may be supported by evidence that the challenged practice screens out an individual on the basis of their disability.  “In the ADA context, a plaintiff may satisfy the second prong of his prima facie case [impact upon persons with protected characteristic] by demonstrating an adverse impact on himself rather than on an entire group.” Gonzalez v. City of New Braunfels.

Illegal Medical Examination

The ADA prohibits employers, whether directly or via third parties like Knack, from administering pre-employment medical examinations. Guidance by the Equal Employment Opportunity Commission defines medical examination under the ADA by reference to seven factors, any one of which may be sufficient to determine that a test is a medical examination.

Physiological Responses

One of those factors is whether the test measures an applicant's physiological responses to performing a task. EEOC guidance on this issue states:

[I]f an employer measures an applicant's physiological or biological responses to performance, the test would be medical.

According to Knack, its test:

leverages cutting-edge behavioral and cognitive neuroscience, data science, and computer science to build games which produce thousands of data points describing how a player perceives, responds, plans, reacts, thinks, problem-solves, adapts, learns, persists, and performs in a multitude of situations.

Types of physiological responses include a reaction or response - a bodily process occurring due to the effect of some antecedent stimulus or agent. As noted in the prior paragraph, Knack tests create data points that track how an applicant perceives, responds, reacts, adapts, learns and persists. The Knack test, therefore, is an illegal medical examination under the ADA.

Five Factor Model of Personality

Justin Fox, executive editor of the Harvard Business Review Group, took two of the Knack assessments and received information in the following report:

As can be seen by the report, among the factors measured by Knack are conscientiousness, openness and stability. These are elements found in the Five Factor Model of Personality, a model that is currently being challenged in at least seven charges filed with the EEOC. Please see ADA, FFM and DSM.

The ADA prohibits pre-employment medical exams but allows employers to “make pre-employment inquiries into the ability of an applicant to perform job-related functions.” The Knack gaming measurements do not seek job-related information and are not consistent with business necessity. The measurements, designed to reveal information about individuals’ openness, conscientiousness, stability (also referred to as neuroticism), and other factors do not seek information about the ability of an applicant to perform the day-to-day functions of a job.

Knowledge of Disability Not Required

Neither the medical examination claim nor the "screen out" claim under the ADA require that an employer have knowledge that an applicant has a disability, a consistent holding from a number of jurisdictions, including the 7th, 9th, 10th, and 11th Federal Circuit Courts of Appeal.

ADA guidance states, in relevant part:

A covered entity shall not require a medical examination and shall not make inquiries of an employee as to whether such employee is an individual with a disability or as to the nature and severity of the disability, unless such examination or inquiry is shown to be job-related and consistent with business necessity.

According to guidance issued by the EEOC, "This statutory language makes clear that the ADA’s restrictions on inquiries and examinations apply to all employees, not just those with disabilities.”

Sound and Fury, Signifying Nothing

Incorporating elements of gamification, big data, machine learning, and predictive human analytics, Knack is a veritable buzzword oasis. According to Knack, their games are designed to test cognitive skills that employers might want, drawing on some of the latest scientific research. These range from pattern recognition to emotional intelligence, risk appetite and adaptability to changing situations.

John Funge, Knack's CTO, states that "we have used our games to infer cognitive ability, conscientiousness, leadership potential, creativity as well as predict how people would perform as surgeons, management consultants, and innovators." In an Economist article, Chris Chabris, a Knack executive, states that games have huge advantages over traditional recruitment tools, such as personality tests, which can easily be outwitted by an astute candidate. Many more things can be tested quickly and performance can't be faked on Knack's games, he says.

Gary Halfteck, Knack's founder and CEO, says playing a video game can be a better representation of who you are and your skill sets than an employer might get in a one-on-one conversation. "As people, we make many decisions that are biased, whether it's consciously or subconsciously, and we have no good tools to assess and evaluate, let alone predict, what one's potential is," he says.

If Knack's CEO admits that people make many decisions that are biased, what prevents the people at Knack from being biased in the creation, development and implementation of their games? Further, what prevents employers using Knack from being held liable for the biases of those games? The answer to both questions: Nothing.

Algorithmic Illusion

While many companies foster an illusion that scoring/classification is an area of absolute algorithmic rule—that decisions are neutral, organic, and even automatically rendered without human intervention—reality is a far messier mix of technical and human curating. Both the datasets and the algorithms used to analyze the data reflect choices, among others, about connections, inferences, and interpretation.

The recent White House report, “Big Data: Seizing Opportunities, Preserving Values," found that, "while big data can be used for great social good, it can also be used in ways that perpetrate social harms or render outcomes that have inequitable impacts, even when discrimination is not intended."

The fact sheet accompanying the White House report warns:

As more decisions about our commercial and personal lives are determined by algorithms and automated processes, we must pay careful attention that big data does not systematically disadvantage certain groups, whether inadvertently or intentionally. We must prevent new modes of discrimination that some uses of big data may enable, particularly with regard to longstanding civil rights protections in housing, employment, and credit.

Some of the most profound challenges revealed by the White House Report concern how data analytics may lead to disparate inequitable treatment, particularly of disadvantaged groups, or create such an opaque decision-making environment that individual autonomy is lost in an impenetrable set of algorithms. Please see Knack Testing Illegal Under ADA?

Systemic Risk

Workforce assessment systems like Knack's games, designed in part to mitigate risks for employers, are becoming sources of material risk, both to job applicants and employers. The systems create the perception of stability through probabilistic reasoning and the experience of accuracy, reliability, and comprehensiveness through automation and presentation. But in so doing, technology systems draw  attention away from uncertainty and partiality.

While Knack's approach may help reduce an employer's hiring costs and may reduce the impact of overtly biased or discriminatory behavior, the inclusion of one or more potentially "defective components" in the assessments means that employers face the risk that a finding of bias or discrimination of a Knack assessment used by one employer will put all employers that use the assessment at risk. Please see When the First Domino Falls: Consequences to Employers of Embracing Workforce Assessment Solutions.

These "defective components" in assessments may be either design defects (i.e., the adoption and use of certain personality models) or manufacturing defects (i.e., coding errors in the assessment software). The latter is analogous to the coding error at 23andMe that resulted in notices going out to some customers informing them that they had a chronic and life-shortening condition when they did not. Please see On Not Dying Young: Fatal Illness or Flawed Algorithm?

Each day an employer continues to use the Knack assessment, there are more potential plaintiffs with claims against that employer.  Labor and employment laws like Title VII and the ADA, permit an employer to use a third party like Knack to undertake the assessment of job applicants. The use of a third party, however, does not insulate an employer from any claims arising from the assessment usage. Under those laws, an employer is responsible (and liable) for any failures on the part of an assessment or assessment provider to comply with the provisions of those laws.

No Silver Bullet

Just as concerns about scoring systems are heightened, their human element is diminishing. Although software engineers initially identify the correlations and inferences programmed into algorithms, machine learning, predictive analytics, and big data promises to eliminate the human “middleman” at some point in the process.

As Hector J. Levesque, a professor at the University of Toronto and a founding member of the American Association of Artificial Intelligence, wrote:

"As a field, I believe that we tend to suffer from what might be called serial silver bulletism, defined as follows:

the tendency to believe in a silver bullet for AI, coupled with the belief that previous beliefs about silver bullets were hopelessly naıve. 

We see this in the fads and fashions of AI research over the years: first, automated theorem proving is going to solve it all; then, the methods appear too weak, and we favour expert systems; then the programs are not situated enough, and we move to behaviour-based robotics; then we come to believe that learning from big data is the answer; and on it goes."

Similarly, employment assessment companies like Knack market the benefits of science, precision and data over the past fifteen years under the guise of neural networks, artificial intelligence, big data and deep learning, yet what has changed? Employee engagement levels have hardly budged and employee turnover remains a continuing and expensive challenge for employers. Please see Gut Check: How Intelligent is Artificial Intelligence?

The Next Asbestos? The Next FLSA?

Asbestos Litigation

A 2005 RAND report states that asbestos litigation arose as a result of millions of individuals’ exposure to asbestos and as a result of many asbestos product manufacturers’ failure to protect workers against exposure and failure to warn their workers to take adequate precautions against exposure. The history of the litigation has been shaped by the rise of a sophisticated and well-capitalized plaintiff bar, heightened media attention to litigation, and the information technology revolution.

According to the RAND report:

• At least 8,400 entities have been named as asbestos defendants through 2002.
• Defendants are distributed across most U.S. industries.
• Total spending on asbestos litigation through 2002 was about $70 billion, broken down as set out in the diagram below.

FLSA Litigation

The Fair Labor Standards Act (FLSA) is the federal law of broadest application governing minimum wage, overtime pay, and youth employment. Employees who are covered by the FLSA are entitled to be paid at least the Federal minimum wage as well as time and one-half their regular rates of pay for all hours worked over 40 in a workweek, unless an exemption applies.

Although FLSA litigation can involve a variety of claims, two of the most common are misclassification claims—i.e., allegations that an employer has misclassified an employee, or a group of employees, as exempt from the FLSA’s overtime requirements—and “off-the-clock” claims—i.e., allegations that an employee, or group of employees, has not been paid for all of the time they worked for the employers.

These and other claims under the FLSA can be brought individually or on behalf of all “similarly situated” employees and former employees. As a result, FLSA cases can involve a large number of employees and present significant financial exposure for employers. For instance, in 2008 Walmart agreed to pay as much as $640 million to settle 63 federal and state class actions claiming the company cheated hourly workers and forced them to work through breaks.

The multitude of wage and hour claims and lawsuits that workers have filed under the FLSA, and its state law counterparts, have made wage and hour law the nation’s fastest growing type of litigation. All industries (including retail, financial services, hospitality, construction, technology, and communications) have been susceptible to these lawsuits.

As shown in the graph to the left, the number of wage and hour lawsuits increased significantly over the past reporting year to 8,126, up another 4.7% over the prior 12-month period. 

This is the seventh straight year of increases in federal court wage and hour lawsuits and ups the continuing explosion in these cases over the past decade to 237% and since 2000 to 438%. 

Although anecdotal, a partner at a major labor and employment defense law firm believes that those numbers would be substantially greater if wage and hour lawsuits filed in state courts under state pay practices, tip laws, meal and rest break requirements, independent contractor rules, and the like, were added.

Employment Testing Litigation

Employment testing litigation will have many parallels with asbestos and FLSA litigation, but on an even larger scale.

There are potentially tens of millions of plaintiffs

• Any person who takes an assessment, if the assessment is determined to be a medical examination will be a plaintiff. Appellate courts have held consistently that the prohibition on medical examinations extends to all persons, including persons who are not disabled (see, e.g., decisions from the Second, Sixth, Eighth, Tenth and Eleventh Circuits).
• Any class of disabled persons (i.e., those with mental illness) where the assessment tends to screen out those persons from employment consideration. Unlike disparate impact claims under other employment discrimination laws the ADA  does not require statistical evidence if an expert can confirm that the test would screen out persons with disabilities or categories of disabilities.
• Federal and state agencies seeking to recover billions of dollars spent on SSDI/SSI disability awards, Medicare/Medicaid and other costs expended on persons who were illegally and invidiously discriminated against as a consequence of the use of employment assessments.

There are potentially hundreds of thousands of defendants

• Employers utilizing testing will face claims by (A) applicants and employees for both the illegal use of a pre-employment medical examination and the failure to treat information obtained from such medical examination as confidential medical information, (B) federal and state agencies seeking recovery of costs incurred (disability awards, Medicare/Medicaid, etc.) as a consequence of the illegal testing, (C) claims by insurers denying coverage and (D) claims by testing companies denying liability/rejecting indemnification.
• Testing companies will face claims by (A) applicants whose information was not treated as confidential medical information (a separate cause of action that does not require exhausting of remedies with the EEOC), (B) employers seeking indemnification from the testing companies for claims made against the employers by applicants, government agencies and others, and (C) claims by insurers denying coverage. 
• Insurance companies who underwrite policies for employers and testing companies will face claims from those they insure as well as individuals and government agencies making claims against those employers and testing companies.

Costs to employers, testing companies and their insurers will be in the tens (if not hundreds) of billions of dollars, including:

• Defense transaction costs, including the costs of outside counsel, internal management and employee time, public relations, lobbying, etc.
• Gross compensation, including awards to applicants and payment of costs and fees (i.e., counsel, expert witnesses, e-discovery).
• Reputational damage costs, including lost/reduced sales and brand damage.
• Business restructuring and/or “disinfectant” costs – The employers and testing companies retention and use of confidential medical information in violation of the ADA safeguards has resulted in the applications and solutions that illegally use this data. The data derived from the hundreds of millions of assessments over the past years has created a virus that has "infected" the employer and testing company solutions that integrate this data. 

Lovin It (Or Not) - McDonald's and the NLRB

The National Labor Relations Board (NLRB) announced on July 29, 2014 that its Office of General Counsel (OGC) authorized the filing of administrative complaints against franchise giant, McDonald’s USA LLC, for unfair labor practices involving workers at franchisee-owned restaurants.

The OGC said that it had investigated 181 cases of unlawful labor practices at McDonald’s franchise restaurants since 2012 and found sufficient merit in at least 43 cases to name McDonald’s as the workers’ “joint employer” creating a legal basis for holding McDonald’s responsible with the franchise owners for the labor violations.  The OGC's findings were made in the form of an Advice Memo supporting the OGC's legal theory. Since this is a matter of ongoing litigation, disclosure of the Advice Memo will not be made at this time.

The NLRB's rationale is likely found in the new “joint employer” test that it is pressing for in Browning-Ferris Industries of California, Inc., a non-franchise case. In its amicus brief, the OGC urges the NLRB to replace the current “joint employer” standard, which examines a company’s direct control over another company’s essential employment decisions specifically affecting hiring, firing, supervision and direction of employment, with the pre-1984 broader-based “industrial realities” test, which focuses on the “economic dependence” between two companies and assumes that a company effectively controls another company’s labor decisions if it dictates standards for every other variable of its business.

McDonald's HR Practices

Heather Smedstad, senior vice president, human resources, of McDonald’s USA, said in a statement that “this decision to allow unfair labor practice complaints to allege that McDonald’s is a joint employer with its franchisees is wrong. McDonald’s will contest this allegation in the appropriate forum.” In the statement, Ms. Smedstad also says that "McDonald’s does not direct or co-determine the hiring, termination, wages, hours, or any other essential terms and conditions of employment of our franchisees’ employees ..."

Ms. Smedstad's statement that McDonald's does not determine or help determine decisions on employment matters appears at odds with her executive bio on the McDonald's website which reads, in part, that "she has lead responsibility for ... execution of all areas of HR for McDonald’s U.S. business and its 14,000 restaurants." (emphasis added) Assuming the accuracy of Ms. Smedstad's bio, McDonald's appears to play a role in employment matters at franchisees, since 90% of those 14,000 restaurants are franchisee-owned.

McDonald's: Hiring Gatekeeper

The process of applying for an hourly job at a McDonald's franchisee requires the applicant to use the application process found on the McDonald's corporate website.  In addition to providing personal information, applicants are required to complete an online assessment. The assessment goes beyond testing skills and evaluating knowledge, and assesses cultural fit, behavior, and potential.

Applicants are required to choose between pairs of statements, including:

• I am usually a very stable person
• I often am not sure why I fell the way I do about certain things
• I am pretty good at understanding what other people are thinking
• If something very bad happens, it takes time before I'm happy again
• I am sometimes not in touch with my feelings
• Most of the time I am not interested in other people's problems
• I prefer to avoid difficult tasks, in case I end up making mistakes
• When I think about the future, I get worried because I know how difficult life can be
• Sometimes I find it hard to sympathize with others' feelings
• I do not like the idea of change, I like things the way they are
• I have certain ways of doing things which I do not like to change
• I am very disorganized, but it works for me
• New experiences often do not turn out well so I like to do what I already like
• I smile more often than not
• I get frustrated doing things in groups because most people are hard to get along with
• I am not very assertive because I do not want to upset anyone

In most instances, the first time the franchisee is aware of the applicant's interest in a job, the applicant has already had several employment-related actions with McDonald's. The franchisee is made aware of the applicant's interest by a report that not only contains personal information supplied by the applicant, but also the results of the assessment - including a dashboard of predictors, tagging the candidate as qualified or not qualified.

McDonald's active and ongoing control of the applicant intake and assessment portions of the hiring process for the franchisees contrasts sharply with its public claims. It is also at odds with advice being provided by labor and employment lawyers who represent employers. Rochelle Spandorf of Davis, Wright Tremaine, writes, "[T]he best advice for franchisors at the moment is to completely distance all operating advice from anything that could remotely be interpreted as suggesting or recommending particular employment practices." Similarly, John T. Lovett of Frost, Brown, Todd writes, "The more influence a franchisor has over the employment practices of the franchisee, the greater the likelihood that the franchisor will be found to be a "joint employer" with the franchisee."