Sound and Fury, Signifying Nothing

Incorporating elements of gamification, big data, machine learning, and predictive human analytics, Knack is a veritable buzzword oasis. According to Knack, their games are designed to test cognitive skills that employers might want, drawing on some of the latest scientific research. These range from pattern recognition to emotional intelligence, risk appetite and adaptability to changing situations.

John Funge, Knack's CTO, states that "we have used our games to infer cognitive ability, conscientiousness, leadership potential, creativity as well as predict how people would perform as surgeons, management consultants, and innovators." In an Economist article, Chris Chabris, a Knack executive, states that games have huge advantages over traditional recruitment tools, such as personality tests, which can easily be outwitted by an astute candidate. Many more things can be tested quickly and performance can't be faked on Knack's games, he says.

Gary Halfteck, Knack's founder and CEO, says playing a video game can be a better representation of who you are and your skill sets than an employer might get in a one-on-one conversation. "As people, we make many decisions that are biased, whether it's consciously or subconsciously, and we have no good tools to assess and evaluate, let alone predict, what one's potential is," he says.

If Knack's CEO admits that people make many decisions that are biased, what prevents the people at Knack from being biased in the creation, development and implementation of their games? Further, what prevents employers using Knack from being held liable for the biases of those games? The answer to both questions: Nothing.

Algorithmic Illusion

While many companies foster an illusion that scoring/classification is an area of absolute algorithmic rule—that decisions are neutral, organic, and even automatically rendered without human intervention—reality is a far messier mix of technical and human curating. Both the datasets and the algorithms used to analyze the data reflect choices, among others, about connections, inferences, and interpretation.

The recent White House report, “Big Data: Seizing Opportunities, Preserving Values," found that, "while big data can be used for great social good, it can also be used in ways that perpetrate social harms or render outcomes that have inequitable impacts, even when discrimination is not intended."

The fact sheet accompanying the White House report warns:

As more decisions about our commercial and personal lives are determined by algorithms and automated processes, we must pay careful attention that big data does not systematically disadvantage certain groups, whether inadvertently or intentionally. We must prevent new modes of discrimination that some uses of big data may enable, particularly with regard to longstanding civil rights protections in housing, employment, and credit.

Some of the most profound challenges revealed by the White House Report concern how data analytics may lead to disparate inequitable treatment, particularly of disadvantaged groups, or create such an opaque decision-making environment that individual autonomy is lost in an impenetrable set of algorithms. Please see Knack Testing Illegal Under ADA?

Systemic Risk

Workforce assessment systems like Knack's games, designed in part to mitigate risks for employers, are becoming sources of material risk, both to job applicants and employers. The systems create the perception of stability through probabilistic reasoning and the experience of accuracy, reliability, and comprehensiveness through automation and presentation. But in so doing, technology systems draw  attention away from uncertainty and partiality.

While Knack's approach may help reduce an employer's hiring costs and may reduce the impact of overtly biased or discriminatory behavior, the inclusion of one or more potentially "defective components" in the assessments means that employers face the risk that a finding of bias or discrimination of a Knack assessment used by one employer will put all employers that use the assessment at risk. Please see When the First Domino Falls: Consequences to Employers of Embracing Workforce Assessment Solutions.

These "defective components" in assessments may be either design defects (i.e., the adoption and use of certain personality models) or manufacturing defects (i.e., coding errors in the assessment software). The latter is analogous to the coding error at 23andMe that resulted in notices going out to some customers informing them that they had a chronic and life-shortening condition when they did not. Please see On Not Dying Young: Fatal Illness or Flawed Algorithm?

Each day an employer continues to use the Knack assessment, there are more potential plaintiffs with claims against that employer.  Labor and employment laws like Title VII and the ADA, permit an employer to use a third party like Knack to undertake the assessment of job applicants. The use of a third party, however, does not insulate an employer from any claims arising from the assessment usage. Under those laws, an employer is responsible (and liable) for any failures on the part of an assessment or assessment provider to comply with the provisions of those laws.

No Silver Bullet

Just as concerns about scoring systems are heightened, their human element is diminishing. Although software engineers initially identify the correlations and inferences programmed into algorithms, machine learning, predictive analytics, and big data promises to eliminate the human “middleman” at some point in the process.

As Hector J. Levesque, a professor at the University of Toronto and a founding member of the American Association of Artificial Intelligence, wrote:

"As a field, I believe that we tend to suffer from what might be called serial silver bulletism, defined as follows:

the tendency to believe in a silver bullet for AI, coupled with the belief that previous beliefs about silver bullets were hopelessly naıve. 

We see this in the fads and fashions of AI research over the years: first, automated theorem proving is going to solve it all; then, the methods appear too weak, and we favour expert systems; then the programs are not situated enough, and we move to behaviour-based robotics; then we come to believe that learning from big data is the answer; and on it goes."

Similarly, employment assessment companies like Knack market the benefits of science, precision and data over the past fifteen years under the guise of neural networks, artificial intelligence, big data and deep learning, yet what has changed? Employee engagement levels have hardly budged and employee turnover remains a continuing and expensive challenge for employers. Please see Gut Check: How Intelligent is Artificial Intelligence?

The (Non)Predictive Ability of the Gallup TeacherInsight Assessment

Gallup states that the TeacherInsight (TI) assessments have "been thoroughly researched and tested to be sure they identify potentially superior teachers." While it is to be expected that the company marketing the TI assessment would make such a statement, is there any independent support for the predictive ability of the TI assessment?

Establishing the predictive validity of an assessment usually requires that applicants who "pass" the assessment perform satisfactorily in practice, and those who do not pass do not perform satisfactorily in practice. The challenge with assessing the predictive validity of the TI assessment, however, is that teacher applicants who do not meet the cutoff score may not be hired. Consequently, support for the predictive validity of the TI assessment is determined by carefully documenting the evidence from teachers who “pass,” through correlational and regression analyses that compare performance on the target measure, TI scores, with one or more other measures.

Doctoral dissertations by Robert Jacob Koerner and Michael T. Novotny are two of the few independent, published research studies to examine the predictive validity of the TeacherInsight (TI) assessment:  that teachers who score higher on it will be more successful teachers.

Novotny Study


The Novotny study involved 527 teachers hired into a North Texas school district for the 2006-2007 school year. The study analyzed the relationships between the TI assessment scores and the eight Professional Development Appraisal System (PDAS) domain scores for those teachers. PDAS is the instrument used by the State of Texas for appraising its teachers and identifying areas that would benefit from staff development.

The Novotny study concluded that:

The TeacherInsight scores produced a statistically significant correlation with only one of the eight PDAS domain scores. However, even that correlation (r = 0.14) was weak. ... The findings do not support the ability of the TeacherInsight to identify more effective teachers, based on Professional Development Appraisal System scores. 

Koerner Study

The Koerner study examined the relationship between the TI assessment and student achievement as measured by the Texas Growth Instrument (TGI), which is an estimate of a student’s academic growth based on the Texas Assessment of Knowledge and Skills (TASK) scores for two consecutive years.More specifically, the study focused on the predictive validity of the TI assessment, that is, how well teacher TI scores predict student achievement gains, as measured by the TGI in reading, English language arts, and mathematics at the primary and secondary school levels. Participants in the study were 132 teachers from one Texas school district who taught reading, English language arts, or mathematics in Grades 3–11 during the 2005–2006 school year and had taken the online TI assessment.

According to Koerner:

The findings [of his study] provide little support to the validity of TeacherInsight in terms of its ability to predict student achievement scores and its usefulness as a tool for the selection of teachers by school systems. 

Wasting Resources, Eliminating Good Teachers

Both the Koerner and Novotny studies found statistically significant, but very weak, relationships with a small number of variables related to teacher success. Those very low correlations suggest a weak link at best between the TI assessment and student achievement and teacher effectiveness.

As Novotny states in the introduction to his study:

It is critical that schools and districts identify highly effective, highly qualified teachers to raise student achievement. School districts have limited resources such as time, money, and manpower to achieve this task. If standardized interview tools such as the TI are effective at identifying better teachers, the time and money spent on them are worthwhile. However, if these tools are not effective, then the time and money spent could be better utilized elsewhere. Furthermore, if the TI does not effectively identify better teachers it could be preventing good candidates from being hired or from being accepted into alternative certification programs.

Not only are there concerns that the TI assessment may waste resources and screen out good candidates, as set out in Systemic Risks for School Systems Employing TeacherInsight, the use of the TI assessment may also violate the non-discrimination provisions of labor and employment laws like Title VII and the ADA that are enforced by the U.S. Equal Employment Opportunity Commission.

As Dr. Melanie Schneider writes in TeacherInsight and the Selection and Hiring of Bilingual Speakers of English:

Should teachers, teacher educators, and professional teacher organizations passively accept the increasingly pivotal role of online assessment tools in the hiring process, or should they press for more research and a balanced selection process that includes multiples sources of information,including human interaction?

Decision-by-Algorithm: No Silver Bullet

This post contains excerpts from "The Scored Society: Due Process for Automated Predictions," a 2014 law review article authored by Danielle Keats Citron and Frank A. Pasquale III. 

* * * * * * *

Big Data is increasingly mined to rank and rate individuals. Predictive algorithms assess individuals as good credit risks, desirable employees, reliable tenants, and valuable customers. People’s crucial life opportunities are on the line, including their ability to obtain loans, work, housing, and insurance.

The scoring trend is often touted as good news. Advocates applaud the removal of human beings and their flaws from the assessment process. Automated systems are claimed to rate individuals all in the same way, thus averting discrimination. But this account is misleading. Human beings program predictive algorithms. Their biases and values are embedded into the software’s instructions, known as the source code, and predictive algorithms.  Please see What Gets Lost? Risks of Translating Psychological Models and Legal Requirements to Computer Code.

Credit scoring has been lauded as shifting decision-makers’ attention from troubling stereotypes to bias-free assessments of would-be borrowers’ actual records of handling credit. The notion is that the more objective data at a lender’s disposal, the less likely a decision will be based on protected characteristics like race or gender. But far from eliminating existing discriminatory practices, credit-scoring algorithms are instead granting them an imprimatur, systematizing them in hidden ways.

A credit card company uses behavioral-scoring algorithms to rate consumers a worse credit risk because they used their cards to pay for marriage counseling, therapy, or tire-repair services. Online evaluation systems score interviewees, with color-coded rating of red signaling a “poor candidate,” yellow as middling, and green as “hire away.”

Beyond biases embedded into code, some automated correlations and inferences may appear objective, but may in reality reflect bias. Algorithms may place a low score on occupations like migratory work or low paying service jobs. This correlation may have no discriminatory intent, but if a majority of those workers are racial minorities, such variables can unfairly impact consumers’ loan application decisions.

Credit scores are only as free from bias as the software and data behind them. Software engineers construct the datasets mined by scoring systems; they define the parameters of data-mining analyses; they create the clusters, links, and decision trees applied. They generate the predictive models applied. The biases and values of system developers and software programmers are embedded into each and every step of development.

Just as concerns about scoring systems are heightened, their human element is diminishing. Although software engineers initially identify the correlations and inferences programmed into algorithms, Big Data promises to eliminate the human “middleman” at some point in the process.

According to a January 9, 2014 article in CIO.com, IBM says cognitive computing systems like Watson are capable of understanding the subtleties, idiosyncrasies, idioms and nuance of human language by mimicking how humans reason and process information.

Whereas traditional computing systems are programmed to calculate rapidly and perform deterministic tasks, IBM says cognitive systems analyze information and draw insights from the analysis using probabilistic analytics. And they effectively continuously reprogram themselves based on what they learn from their interactions with data.

Said IBM CEO Ginni Rometty, "In 2011, we introduced a new era [of computing] to you. It is cognitive. It was a new species, if I could call it that. It is taught, not programmed. It gets smarter over time. It makes better judgments over time." "It is not a super search engine," she adds. "It can find a needle in a haystack, but it also understands the haystack."

This "new species" of computing has its challenges. According to "IBM Struggles to Turn Watson Computer Into Big Business," a recent Wall Street Journal article:

Watson is having more trouble solving real-life problems than "Jeopardy" questions, according to a review of internal IBM documents and interviews with Watson's first customers. 

For example, Watson's basic learning process requires IBM engineers to master the technicalities of a customer's business—and translate those requirements into usable software. The process has been arduous.

Klaus-Peter Adlassnig is a computer scientist at the Medical University of Vienna and the editor-in-chief of the journal Artificial Intelligence in Medicine. The problem with Watson, as he sees it, is that it’s essentially a really good search engine that can answer questions posed in natural language. Over time, Watson does learn from its mistakes, but Adlassnig suspects that the sort of knowledge Watson acquires from medical texts and case studies is “very flat and very broad.” In a clinical setting, the computer would make for a very thorough but cripplingly literal-minded doctor—not necessarily the most valuable addition to a medical staff.

As Hector J. Levesque, a professor at the University of Toronto and a founding member of the American Association of Artificial Intelligence, wrote:

 "As a field, I believe that we tend to suffer from what might be called serial silver bulletism, defined as follows:

the tendency to believe in a silver bullet for AI, coupled with the belief that previous beliefs about silver bullets were hopelessly naıve. 

We see this in the fads and fashions of AI research over the years: first, automated theorem proving is going to solve it all; then, the methods appear too weak, and we favour expert systems; then the programs are not situated enough, and we move to behaviour-based robotics; then we come to believe that learning from big data is the answer; and on it goes."

Similarly, employment assessment companies have marketed the benefits of "science, precision and data" over the past fifteen years under the guise of neural networks, artificial intelligence, big data and deep learning, yet what has changed? Employee engagement levels have hardly budged and employee turnover remains a continuing and expensive challenge for employers. Please see Gut Check: How Intelligent is Artificial Intelligence?

Employment Challenges for Persons with Serious Mental Illness

This post is comprised of excerpts from testimony by Dr. Gary Bond, Professor of Psychiatry, Dartmouth Psychiatric Research Center to the EEOC at a March 15, 2011 hearing on employment of persons with disabilities. Please click on this link for the complete written version of the testimony, together with all references.

There are many benefits of employment—work enhances skills such as communication, socialization, academics, physical health, and community skills; it factors into how one is perceived by society; it promotes economic well-being; it leads to greater opportunity for upward mobility; and it contributes to greater self-esteem. Yet only 15 percent of those with a mental disability are in the labor market

* * * * * * *

People with serious mental illness are a very heterogeneous group that has included Nobel Prize winners, American Presidents, artists, and other famous persons, as well as many who live in poverty and isolation. You cannot judge a person by their diagnosis. While the public has many negative stereotypes about this group, the take-home message from this testimony is that the research strongly demonstrates that full recovery from mental illness is possible. Working is a crucial element in this recovery process.

What is Serious Mental Illness?

This testimony concerns the population of people with serious mental illness, defined by three criteria: 

(a) Diagnosis: a psychiatric diagnosis of schizophrenia, bipolar disorder, or other major psychiatric disorder;

(b)Disability: significant role impairment, in areas such as independent living, interpersonal functioning, and employment;

(c) Duration: extended involvement with the mental health system (such as admission to psychiatric hospitals, supervised group homes, and mental health case management services). 

This is a large segment of the disability population. For example, over one-third of people of the Social Security disability roles have a serious mental illness.

Employment of People with Serious Mental Illness

Employment rates for people with serious mental illness are very low. Surveys have found that only 10% - 15% of people with serious mental illness receiving community treatment are competitively employed. Rates are even lower, typically less than 5%, in follow-up surveys of people discharged from psychiatric hospitals National and international surveys of community samples, which include respondents with less serious disorders, have reported employment rates of 20% - 25% for people with schizophrenia and related disorders.

The employment rate for people with serious mental illness is less than half the 33% rate for other disability groups Both rates are of course much lower than for the general population. Even during the height of the current recession, the national employment rate for adults in the general population was 72%, according to U.S. Bureau of Labor statistics.

Moreover, people with serious mental illness who are working are often underemployed. Nearly twice as many workers with mental illness earn at or near minimum wage as workers without disabilities. Non-standard jobs (such as temporary employment, independent contracting, and part-time employment) are common among workers with serious mental illness. Such jobs pay lower wages with fewer benefits. Among those employed, people with serious mental illness are overrepresented in unskilled occupations, such as in the service industries and as laborers.

Most People with Serious Mental Illness Want to Work

Despite these dismal employment statistics, most people with severe mental illness want to work. Studies indicate that approximately 2 out of every 3 people with mental illness are interested in competitive employment Moreover, these rates may understate the interest in working in this population, because many mental health professionals discourage clients from pursuing employment goals.

Barriers to Getting and Keeping Jobs

Why, then, is there a wide disparity between employment rates and desire to work? The reason is not that people with serious mental illness cannot work. People with serious mental illness are capable of working if they are matched to appropriate jobs and receive appropriate supports. But attitudinal, service, and system barriers are challenges to their employment. According to a national survey, persons with serious mental illness reported the primary barriers to employment to be:

• stigma and discrimination (45%); 
• fear of losing benefits (40%);
• inadequate treatment of disability (28%);
• lack of vocational services (23%).

Regarding attitudinal barriers, psychiatric disability is the most stigmatizing of all disabilities. One national survey reported that only 19% of those polled were “very comfortable with people with mental illness,” compared to triple than rate for people with a physical disability (National Organization on Disability, 1991). People with serious mental illness experience discrimination and negative attitudes constantly in everyday life. Employers are less likely to hire someone whom they believe has a mental illness.

One major attitudinal barrier is the perception that people with mental illness are violent. News stories involving atrocities committed by people with mental illness reinforce this perception. Based on extensive epidemiological research over the past two decades, we have a much better understanding of the risk factors for violence in the psychiatric population. Violence is exceedingly rare among people with mental illness, and the rare instances that do occur are associated with other factors, such as active substance use or refusing to take medications. Being employed significantly reduces the possibility of violence even further. In sum, a very low proportion of people with mental illness have a history of violence and overall people with mental illness are no more likely to behave violently than people without mental illness.

Another barrier is the fear of losing Social Security disability benefits (MacDonald-Wilson, Rogers, Ellison, & Lyass, 2003) Many of these apprehensions are based on lack of information and misconceptions. In one survey, 85% of Social Security disability beneficiaries incorrectly believed that Medicaid benefits would be terminated if they went to work (MacDonald-Wilson, Rogers, Ellison et al., 2003). Fortunately, when provided accurate information about the impact of employment, beneficiaries substantially increase their employment earnings (Tremblay, Smith, Xie, & Drake, 2006).

Once employed, people with serious mental illness often need ongoing support and accommodations to succeed. But employers are far less willing to accommodate people with psychiatric disabilities than those with physical conditions. Workers with mental health conditions are half as likely to receive accommodations as those with other disabilities. This is true even though most accommodations for psychiatric disabilities cost very little or nothing, in contrast to technological and architectural changes required for other disabilities. According to one employer survey, the kinds of functional limitations they most commonly observe in workers with psychiatric disabilities are cognitive (e.g., following instructions, concentrating) and social (e.g., interacting, reading social cues), and to a lesser extent emotional (e.g., managing symptoms, tolerating stress) and physical (e.g., stamina).

The onset of serious mental illness often occurs in early adulthood, interfering with the completion of education. Over 30% of people with severe mental illness have not completed high school. Low educational attainment contributes to the underemployment of people with serious mental illness. The median annual income of the U.S. population without a high school diploma or equivalent is less than $20,000. Median income increases 33% with completion of high school and more than triples with the completion of a bachelor’s degree.

Conclusion

The dismal rate of employment among people with serious mental illness is a formidable challenge. Nonetheless, we have compelling reasons to be optimistic that people with serious mental health problems can work and that working helps them to recover from mental illness. The major barriers to employment are not immutable clinical or cognitive characteristics but rather attitudes and lack of access to support services and accommodations.

Even in the absence of professional support, employers can play a pivotal role promoting the employment of people with serious mental illness by approaching applicants and workers as individuals and applying sound employment practices. Work accommodations typically involve pragmatic and inexpensive modifications. Employment is a win-win for people with serious mental illness, for employers, and for society at large.

EEOC's Next Step in Preserving Values

The recent White House report, “Big Data: Seizing Opportunities, Preserving Values," found that, "while big data can be used for great social good, it can also be used in ways that perpetrate social harms or render outcomes that have inequitable impacts, even when discrimination is not intended." The fact sheet accompanying the White House report warns:

As more decisions about our commercial and personal lives are determined by algorithms and automated processes, we must pay careful attention that big data does not systematically disadvantage certain groups, whether inadvertently or intentionally. We must prevent new modes of discrimination that some uses of big data may enable, particularly with regard to longstanding civil rights protections in housing, employment, and credit.

In order to address the potential for big data analytics to systematically disadvantage certain groups, the White House report contains the following policy recommendation:

The federal government’s lead civil rights and consumer protection agencies, including the Department of Justice, the Federal Trade Commission, the Consumer Financial Protection Bureau, and the Equal Employment Opportunity Commission, should expand their technical expertise to be able to identify practices and outcomes facilitated by big data analytics that have a discriminatory impact on protected classes, and develop a plan for investigating and resolving violations of law in such cases. In assessing the potential concerns to address, the agencies may consider the classes of data, contexts of collection, and segments of the population that warrant particular attention, including for example genomic information or information about people with disabilities. 

Examples of Discriminatory Big Data Practices and Outcomes

Examples of practices and outcomes facilitated by big data analytics that could have a discriminatory impact on protected classes include:

• Use of location-based data that discriminates against lower-income persons, who are disproportionately Black, Hispanic, women and people with disabilities. Please see  From What Distance is Discrimination Acceptable?

• Use of pre-employment personality tests that discriminate against people with mental disabilities. Please see What Are The Issues, ADA, FFM and DSM, and Employment Assessments Are Designed to Reveal an Impairment.

• Mental illness is no respecter of age, gender, geography, income, occupation, military status, race, religion or sexual orientation.  Please see Tests Discriminate Against Returning Veterans, Tests Discriminate Against New and Expectant Mothers and Employment Tests Discriminate Against LGBTs.

EEOC's Next Step?

The White House report recommends that the EEOC, as one of the federal government's lead civil rights agencies, expand its technical expertise to be able to identify practices and outcomes facilitated by big data analytics that have a discriminatory impact on protected classes. The EEOC's next step may be to call on some of the same resources used by the White House review group led by John Podesta at the three workshops held during the 90-day review period leading to the issuance of the White House report:

• Big Data Privacy: Advancing the State of the Art in Technology and Practice
• Organized and co-hosted by the MIT Big Data Initiative at CSAIL and the MIT Information Policy Project 
• The Social, Cultural & Ethical Dimensions of “Big Data”
• Organized and co-hosted by Data & Society Research Institute and the NYU Information Law Institute
• Big Data: Values and Governance
• Organized and co-hosted by the UC Berkeley School of Information and the Berkeley Center for Law and Technology

The suggestion is that some of the individuals and organizations that co-hosted, presented and supported those the three workshops be persuaded to take their shows on the road, assisting the EEOC and the other federal civil rights agencies (Department of Justice, Federal Trade Commission and Consumer Financial Protection Bureau)  in understanding and preventing new modes of discrimination that some uses of big data may enable, particularly with regard to housing, employment, and credit.

White House: Big Data's Role in Employment Discrimination

In“Big Data: Seizing Opportunities, Preserving Values," a White House review of how the government and private sector use large sets of data found that such information could be used to discriminate against Americans on issues such as employment. As noted in the review, "while big data can be used for great social good, it can also be used in ways that perpetrate social harms or render outcomes that have inequitable impacts, even when discrimination is not intended."

Clear Windshield or Rearview Mirror?

Algorithms embody a profound deference to precedent; they draw on the past to act on (and enact) the future. The apparent omniscience of big data may in truth be nothing more than misdirection. Instead of offering a clear windshield, the big data phenomenon may be more like a big rear-view mirror telling us nothing about the future.

Does this deference to precedent result in a self-reinforcing and self-perpetuating system, where individuals are burdened by a history that they are encouraged to repeat and from which they are unable to escape?

Already burdened segments of the population can become further victimized through the use of sophisticated algorithms in support of the identification, classification, segmentation, and targeting of individuals as members of analytically constructed groups. In creating these groups, the algorithms rely upon correlations that lead to viewing people as members of populations, or categories, or groups, rather than as individuals (i.e., persons who live more than X miles from an employer's location). Please see From What Distance is Discrimination Acceptable?

Just as neighborhoods can serve as a proxy for racial or ethnic identity, there are new worries that big data technologies could be used to “digitally redline” unwanted groups, either as customers, employees, tenants, or recipients of credit. A significant finding of the White House report is that big data could enable new forms of discrimination.

Correlation Does Not Equal Causation

Decisions made or affected by correlation are inherently flawed. Correlation does not equal causation. This point is made vividly by Tyler Vigen, a law student at Harvard who, in his spare time, put together a website that finds very, very high correlations - as shown below - between things that are absolutely not related.

Each of these have correlation coefficents in excess of 0.99, serving to demonstrate the point that a strong correlation isn't nearly enough to make strong conclusions about how two phenomena are related to each other.

Shrouding Opacity In The Guise of Legitimacy

Some of the most profound challenges revealed by the White House Report concern how big data analytics may lead to disparate inequitable treatment, particularly of disadvantaged groups, or create such an opaque decision-making environment that individual autonomy is lost in an impenetrable set of algorithms.

Workforce analytic systems, designed in part to mitigate risks for employers, have become sources of material risk, both to job applicants and employers. The systems create the perception of stability through probabilistic reasoning and the experience of accuracy, reliability, and comprehensiveness through automation and presentation. But in so doing, technology systems draw  attention away from uncertainty and partiality. Please see Workforce Science: A Critical Look at Big Data and the Selection and Management of Employees.

Moreover, they shroud opacity—and the challenges for oversight that opacity presents—in the guise of legitimacy, providing the allure of shortcuts and safe harbors for actors both challenged by resource constraints and desperate for acceptable means to demonstrate compliance with legal mandates and market expectations.

Programming and mathematical idiom (e.g., correlations) can shield layers of embedded assumptions from higher level decisionmakers at an employer who are charged with meaningful oversight and can mask important concerns with a veneer of transparency.

This problem is compounded in the case of regulators outside the firm, who frequently lack the resources or vantage to peer inside buried decision processes. In recognition of this problem, the White House Report states that "[t]he federal government must pay attention to the potential for big data technologies to facilitate discrimination inconsistent with the country’s laws and values" and, as one of the six policy recommendations in the report, :

The federal government’s lead civil rights and consumer protection agencies, including the Department of Justice, the Federal Trade Commission, the Consumer Financial Protection Bureau, and the Equal Employment Opportunity Commission, should expand their technical expertise to be able to identify practices and outcomes facilitated by big data analytics that have a discriminatory impact on protected classes, and develop a plan for investigating and resolving violations of law in such cases. In assessing the potential concerns to address, the agencies may consider the classes of data, contexts of collection, and segments of the population that warrant particular attention, including for example genomic information or information about people with disabilities. 

Big Data: Seizing Opportunities, Preserving Values (Excerpted)

A White House review of how the government and private sector use large sets of data has found that such information could be used to discriminate against Americans on issues such as employment. Findings and recommendations of the review, “Big Data: Seizing Opportunities, Preserving Values," were released today (May 1, 2014). The report is the culmination of a 90-day review by the Obama administration, spearheaded by Counselor John Podesta and including the two Cabinet members - Penny Pritzker (Secretary of Commerce) and Ernest J. Moniz (Secretary of Energy). 

Excerpts from the report relating to big data analytics and employment discrimination follow, along with their location in the report:

A significant finding of this report is that big data analytics have the potential to eclipse longstanding civil rights protections in how personal information is used in housing, credit, employment, health, education, and the marketplace. Americans’ relationship with data should expand, not diminish, their opportunities and potential.  (cover letter to President Obama)

Some of the most profound challenges revealed during this review concern how big data analytics may lead to disparate inequitable treatment, particularly of disadvantaged groups, or create such an opaque decision-making environment that individual autonomy is lost in an impenetrable set of algorithms.  (p. 10)

Regardless of technological advances, the American public retains the power to structure the policies and laws that govern the use of new technologies in a way that protects foundational values. 

Big data is changing the world. But it is not changing Americans’ belief in the value of protecting personal privacy, of ensuring fairness, or of preventing discrimination.  (p. 10)

[T]he civil rights community is concerned that such algorithmic decisions raise the specter of “redlining” in the digital economy—the potential to discriminate against the most vulnerable classes of our society under the guise of neutral algorithms.  (p. 46) 

An important conclusion of this study is that big data technologies can cause societal harms beyond damages to privacy, such as discrimination against individuals and groups. This discrimination can be the inadvertent outcome of the way big data technologies are structured and used. It can also be the result of intent to prey on vulnerable classes.  (p. 51) 

We have taken considerable steps as a society to mandate fairness in specific domains, including employment, credit, insurance, health, housing, and education. Existing legislative and regulatory protections govern how personal data can be used in each of these contexts. Though predictive algorithms are permitted to be used in certain ways, the data that goes into them and the decisions made with their assistance are subject to some degree of transparency, correction, and means of redress. For important decisions like employment, credit, and insurance, consumers have a right to learn why a decision was made against them and what information was used to make it, and to correct the underlying information if it is in error.  

Just as neighborhoods can serve as a proxy for racial or ethnic identity, there are new worries that big data technologies could be used to “digitally redline” unwanted groups, either as customers, employees, tenants, or recipients of credit. A significant finding of this report is that big data could enable new forms of discrimination and predatory practices. 

Whether big data will build greater equality for all Americans or exacerbate existing inequalities depends entirely on how its technologies are applied in the years to come, what kinds of protections are present in the law, and how the law is enforced. (p. 53)

Putting greater emphasis on a responsible use framework has many potential advantages. It shifts the responsibility from the individual, who is not well equipped to understand or contest consent notices as they are currently structured in the marketplace, to the entities that collect, maintain, and use data. Focusing on responsible use also holds data collectors and users accountable for how they manage the data and any harms it causes, (p. 56) 

An important finding of this review is that while big data can be used for great social good, it can also be used in ways that perpetrate social harms or render outcomes that have inequitable impacts, even when discrimination is not intended. Small biases have the potential to become cumulative, affecting a wide range of outcomes for certain disadvantaged groups. Society must take steps to guard against these potential harms by ensuring power is appropriately balanced between individuals and institutions, whether between citizen and government, consumer and firm, or employee and business.have inequitable impacts, even when discrimination is not intended. Small biases have the potential to become cumulative, affecting a wide range of outcomes for certain dis-advantaged groups. (pp. 58-59) 

Policy Recommendations: 

Expand Technical Expertise to Stop Discrimination. The federal government’s lead civil rights and consumer protection agencies should expand their technical expertise to be able to identify practices and outcomes facilitated by big data analytics that have a discriminatory impact on protected classes, and develop a plan for investigating and resolving violations of law. (p. 60) 

We must begin a national conversation on big data, discrimination, and civil liberties.  (p. 64) 

The federal government must pay attention to the potential for big data technologies to facilitate discrimination inconsistent with the country’s laws and values 

RECOMMENDATION: The federal government’s lead civil rights and consumer protection agencies, including the Department of Justice, the Federal Trade Commission, the Consumer Financial Protection Bureau, and the Equal Employment Opportunity Commission, should expand their technical expertise to be able to identify practices and outcomes facilitated by big data analytics that have a discriminatory impact on protected classes, and develop a plan for investigating and resolving violations of law in such cases. In assessing the potential concerns to address, the agencies may consider the classes of data, contexts of collection, and segments of the population that warrant particular attention, including for example genomic information or information about people with disabilities. (p. 65)

Employment Testing: Hot Button Issue for EEOC and OFCCP

On October 30, 2013, the U.S. Department of Labor announced that federal construction contractor M.C. Dean Inc. had settled allegations that it failed to provide equal employment opportunity to 381 African American, Hispanic and Asian American workers who applied for jobs at the company's Dulles headquarters. A review by the department's Office of Federal Contract Compliance Programs determined that the contractor used a set of selection procedures, including invalid tests, which unfairly kept qualified minority candidates from securing jobs as apprentices and electricians.

"Our nation was built on the principles of fair play and equal opportunity, and artificial barriers that keep workers from securing good jobs violate those principles," said OFCCP Director Patricia A. Shiu. "I am pleased that this settlement will provide remedies to the affected workers and that M.C. Dean has agreed to invest significant resources to improve its hiring practices so that this never happens again."

Under the terms of the agreement, M.C. Dean will pay $875,000 in back wages and interest to 272 African American, 98 Hispanic and 11 Asian American job applicants who were denied employment in 2010. The contractor will also extend 39 job offers to the class members as opportunities become available. Additionally, M.C. Dean has agreed to undertake extensive self-monitoring measures and personnel training to ensure that all of its employment practices fully comply with Executive Order 11246, which prohibits federal contractors and subcontractors from discriminating in employment on the bases of race, color and national origin.

As stated in the Affirmative Action & OFCCP Law Advisor:  

This settlement provides (at least) two lessons to all federal contractors.  First, the OFCCP is digging deeper than just the overall applicant-to-hire adverse impact analyses.  Where there is overall applicant-to-hire adverse impact in the hiring process, the Agency will analyze each stage (screen, test, interview, offer, etc.) in the hiring processes for adverse impact.  Second, where there is adverse impact at the testing stage, employers must evaluate the validity of their “tests.”  In these cases, OFCCP will request and send the validation materials to its Industrial-Organization Psychologist for review, so it must be able to withstand scrutiny, including whether the test has been (i) validated recently, (ii) validated for the employer’s specific position, and (iii) that there are not less discriminatory methods for achieving the same predictive results of job performance.   

In particular, employers who are using employment tests that have never been validated, have not been validated for the specific position for which they are being used, have not been validated for their specific company, have not been reviewed by someone other than the testing vendor who created the test, or have not been revalidated as the position changed over time may not realize they may be “at risk” in these audits. 

In short, own each step of your hiring process – even if a third-party testing vendor created and/or administers your test, the employer will be held accountable if the test causes adverse impact and is not properly validated.  Employers need to get in front of these testing issues by analyzing the test’s potential adverse impact and existing validation to minimize exposure during audits.  Notably, this has also become a “hot button” for EEOC, so taking a close look at your tests can help minimize exposure to both OFCCP and EEOC claims.

Positive Trending for Claims Challenging the Legality of Pre-Employment Assessments

A variety of factors are trending in favor of eliminating the use of pre-employment assessments that violate the Americans with Disabilities Act (ADA) and the Rehabilitation Act of 1973, including:

• Implementation of the EEOC Strategic Enforcement Plan for 2013-2016

• The first national priority of the EEOC in the strategic enforcement plan is “eliminating systemic barriers in recruitment and hiring.”
• “[P]eople with disabilities continue to confront discriminatory policies and practices at the recruitment and hiring stages. These include … the use of screening tools (e.g., pre-employment tests …) “

• EEOC Systemic Investigation of Pre-Employment Testing and the ADA

• Stemming from more than six years of litigation by the EEOC against Kroger and Kronos 
• September 14, 2012 Third Circuit Court of Appeals decision in EEOC v. Kronos Incorporated
• It is “a proper inquiry for the EEOC to seek information about how these tests work, including information about the types of characteristics they screen out….“ Third Circuit Court of Appeals (September 14, 2012)
• Transfer of two charges from Atlanta EEOC to the EEOC office leading the systemic investigation

EEOC Focus on Disability Discrimination Litigation

• ADA claims covered the biggest percentage of the EEOC’s yearly litigation filing activity for FY 2013 
• The pie chart below provides a snapshot of the cases filed by the EEOC in the last week of the fiscal year and shows that almost half of the cases filed were based on disability discrimination.

• CVS/Rhode Island ACLU Settlement

• CVS eliminates use of pre-offer assessment as a consequence of claim by ACLU that questions from the assessment could have a discriminatory impact on people with mental impairments or disorders. 
• Please see Challenges to Pre-Employment Assessments

• Karraker Court Decision

• Rejected “form” defenses (e.g., test not reviewed by medical professional) and dismantled distinction between a test that evaluates personality and one that diagnoses mental disorders
• Please see Courts Find Tests To Be Illegal

• Adoption of the Five-Factor Model in DSM-5 by the American Psychiatric Association

• Based on two decades of research demonstrating that the five-factor model - used as the basis for many of the pre-employment personality tests - can be used as a structural model for describing and understanding personality disorders, including those within the Diagnostic and Statistical Manual of Mental Disorders (DSM)
• Please see ADA, FFM and DSM

• Significant Risk of Punitive Damages

• In addition to claims for actual or compensatory damages, which may be nominal on a per person basis, applicants may also seek punitive damages for the reckless behavior of the employers that used illegal pre-employment assessments.
• In State of Arizona v. ASARCO LLC, No. 11-17484 (9th Cir. Oct. 24, 2013), the 9th Circuit Court of Appeals held that a punitive damages award of $125,000 in an employment discrimination case finding no actual damages and $1 in nominal damages was constitutional and "did not raise judicial eyebrows."
• Please see Punitive Damages

• OFCCP issuance of non-discrimination and and affirmative action regulations for individuals with disabilities (IWDs)

• Regulations require federal contractors to achieve a 7%  workforce utilization goal of IWDs. 
• The contractors are required to achieve the 7% in each and every job group of the contractors.

Why Success Is Important

The long-term fiscal stability of the United States of America depends, in part, on ensuring that Americans with disabilities have meaningful opportunities to contribute to our collective well-being and on eliminating outdated policies that keep people in cycles of poverty and dependency.

More than two decades after the passage of the ADA, the unemployment rate for Americans with disabilities stubbornly remains nearly double that of people without disabilities, while their rate of labor force participation has continued to be abysmally low. Figures from the Bureau of Labor Statistics show that labor force participation for workers with disabilities was 20.3 percent, while the total for workers without disabilities was 69.1 percent—more than three times higher. As of April 2012, the unemployment rate for people with disabilities was 12.5 percent, versus 7.6 percent for those without disabilities.

There are many benefits of employment—work enhances skills such as communication, socialization, academics, physical health, and community skills; it factors into how one is perceived by society; it promotes economic well-being; it leads to greater opportunity for upward mobility; and it contributes to greater self-esteem. Yet only 15 percent of those with a mental disability are in the labor market. Please see So Many Job Openings, So Little Hiring.

From What Distance is Discrimination Acceptable?

Some 60% of American workers earn hourly wages. Of these, about half change jobs each year, so firms that employ lots of entry-level workers, such as call centers, supermarkets, home improvement stores and fast-food chains, have to vet million of applications every year.

Xerox Evolv(ing)

A recent article in MIT Technology Review reports that Xerox is screening tens of thousands of applicants for low-wage jobs in its call centers using software from a startup company called Evolv. According to its website, Evolv “is a workforce science software company that harnesses big data, predictive analytics and cloud computing to help businesses improve workplace productivity and profitability” and its customers include 20 of the Fortune 100.

Working with Xerox, Evolv found that one of the best predictors that a customer-service employee will stick with a job is that he lives nearby and can get to work easily.  As Evolv states in its Q3 2013 Workforce Performance Report:

The distance that employees live from work affects how long they choose to stay at a job. Unsurprisingly, employees that live 0-5 miles from their place of work have the longest median tenure. They remain at their jobs 20% longer than employees with the shortest median tenure.

Although it still sells photocopiers, Xerox has also become one of the world’s largest outsourcing companies. It provides services like running customer service centers, handling health claims, and processing credit-card applications that brought in $11.5 billion in revenue last year.

That business relies on a huge workforce of 54,000 customer service agents, and because of high attrition in hourly jobs, Xerox will have to replace 20,000 of them this year, says Teri Morse, vice president for recruiting at Xerox Services.

The Dictatorship of Data

Morse says Xerox today won’t even look at resumes of those who score in the “red” category of Evolv’s initial behavioral assessment, a 30-minute online exam that workers fill out at home. Early on, while piloting the system, Morse says Xerox still hired against the advice of the data. Now, she says, “people who do poorly we no longer hire.”

As stated by the authors in Big Data: A Revolution That Will Transform How We Live, Work, and Think:

We are more susceptible than we may think to the “dictatorship of data” — that is, to letting the data govern us in ways that may do as much harm as good. The threat is that we will let ourselves be mindlessly bound by the output of our analyses even when we have reasonable grounds for suspecting something is amiss. Or that we will become obsessed with collecting facts and figures for data’s sake. Or that we will attribute a degree of truth to the data which it does not deserve.

For more and more companies, like Xerox, the hiring boss is an algorithm. Jobs that were once filled on the basis of work history and interviews are left to personality tests and data analysis. The new hiring tools are part of a broader effort to gather and analyze employee data.

The risks to employers of utilizing online personality tests in their employment application process have been set out in a number of prior posts, including What Are the Issues, Courts Find Tests to be Illegal, The Next Asbestor? The Next FLSA?, Damages and Indemnification Challenges to Employers, and Welcomed as Customers; Rejected as Employers. The remainder of this post sets out the employment discrimination litigation risks to employers (and society) of blindly following the "insights" of Kenexa and Evolv in distance from job site and housing mobility.

Interestingly, while Evolv now touts the distance from job insight for use by its clients, the company expressed a different view in a 2012 Wall Street Journal article, which reads;

Evolv is cautious about exploiting some of the relationships it turns up for fear of violating equal opportunity laws. While it has found employees who live farther from call-center jobs are more likely to quit, it doesn't use that information in its scoring in the U.S. because it could be linked to race.

From What Distance is Discrimination Acceptable?

Kenexa, purchased by IBM in December 2012, will test approximately 40 million applicants this year for thousands of clients. Kenexa believes that a lengthy commute raises the risk of attrition in call-center and fast-food jobs. It asks applicants for call-center and fast-food jobs to describe their commute by picking options ranging from "less than 10 minutes" to "more than 45 minutes."

The longer the commute, the lower their recommendation score for these jobs, says Jeff Weekley, who oversees the assessments.Applicants also can be asked how long they have been at their current address and how many times they have moved. People who move more frequently "have a higher likelihood of leaving," Mr. Weekley said.

Painting with the broad brush of distance from job site, commute time and moving frequency results in well-qualified applicants being excluded, applicants who might have ended up being among the longest tenured of employees. The Kenexa and Evolv findings are generalized correlations (i.e., persons living closer to the job site tend to have longer tenure than persons living farther from the job site). The insights say nothing about any particular applicant.

As a consequence, employers will pass over qualified applicants solely because they live (or don't live) in certain areas. Not only does the employer do a disservice to itself and the applicant, they increase the risk of employment litigation, with its consequent costs. 

Distance From Jobsite

A recent New York Time article, "In Climbing Income Ladder, Location Matters," reads, in part:

Stacey Calvin spends almost as much time commuting to her job — on a bus, two trains and another bus — as she does working part-time at a day care center.  ...

Her nearly four-hour round-trip [job commute] stems largely from the economic geography of Atlanta, which is one of America’s most affluent metropolitan areas yet also one of the most physically divided by income. The low-income neighborhoods here often stretch for miles, with rows of houses and low-slung apartments, interrupted by the occasional strip mall, and lacking much in the way of good-paying jobs

This geography appears to play a major role in making Atlanta one of the metropolitan areas where it is most difficult for lower-income households to rise into the middle class and beyond, according to a new study that other researchers are calling the most detailed portrait yet of income mobility in the United States.

The dearth of good-paying jobs in low-income neighborhoods means that residents of those neighborhoods have a longer commute. The 2010 Census showed that poverty rates are significantly higher for blacks and Hispanics. Consequently, hiring decisions predicated on distance from job site, intentionally or not, discriminate against certain races.

Housing Mobility

As shown in the table below, poor and near-poor families tend to move much more frequently than their higher income neighbors and the general population.

According to a 2011 study by the Center for Public Housing, entitled "Should I Stay or Should I Go? Exploring the Effects of Housing Instability and Mobility on Children," a wide range of often complex forces appears to drive frequent mobility, and residential instability in general — the formation and dissolution of households, an inability to afford one’s housing costs, the loss of employment, the lack of a safety net, lack of quality housing or a safer neighborhood.

Correlation Is Not Causation

When two variables, A and B, are found to be correlated, there are several possibilities:

1. A causes B
2. B causes A
3. A causes B at the same time as B causes A (a self-reinforcing       system)
4. Some third factor causes both A and B

The correlation is simple coincidence. It is wrong to assume any of these possibilities.

Kenexa, Evolv and their clients, however, assume that A (proximity to job site) causes B (reduced attrition and better performance). That assumption leads them to disfavor otherwise qualified applicants who do not live within a five-mile radius.

The correlation could also demonstrate B (reduced attrition and better performance) is caused by C (proximity of job site to applicants homes). Instead of being a hiring insight, the correlation might function better as being a job site location insight. Given the relative immobility of persons and companies, locating a job site (call center, etc.) close to communities with high numbers of lower-income persons could lead to a more sustainable competitive advantage.

Per Kenexa, the correlation between (A) persons who move and (B) shorter job tenure is that A causes B. However, per the 2011 study by the Center of Public Housing, it may well be that (B) shorter job tenure causes (B) persons to move. As shown in the table below, taken from the 2011 study, more than 76% of involuntary moves were a result of job loss.

Although data does give rise to information and insight, they are not the same. Data's value to business relies on human intelligence, on how well managers and leaders formulate questions and interpret results. More data doesn't mean you will get "proportionately" more information. In fact, the more data you have, the less information you gain as a proportion of the data (concepts of marginal utility, signal to noise and diminishing returns).

Maybe It's the Work, Not the Workforce, That Need Analysis ...

Some 60% of American workers earn hourly wages. Of these, about half change jobs each year. The cost to U.S. businesses of worker attrition and lost productivity is $350 billion annually. How much is that?

What should an employer do? Increase starting pay? Since most people work, at least in part, for the money - giving them more money might encourage them to stay longer and work harder. It seems to work reasonably well with corporate executives.  Top executive compensation averaged $9.4 million last year at the 50 largest employers of low-wage workers.

What do many employers do? Retain "workforce science" companies like Evolv and Kenexa to administer personality tests and analyze applicant data in order to address the employee turnover issue.

As noted above, working with Xerox, Evolv found that one predictor that a customer-service employee will stick with a job is that s/he lives nearby and can get to work easily. Kenexa had a similar "insight," and added that people who move more frequently have a higher likelihood of leaving.

Are there any groups of people who might live farther from the work site and may move more frequently than others? Yes, lower-income persons, disproportionately women, black, Hispanic and the mentally ill. They can't afford to live where the jobs are and move more frequently because of an inability to afford housing or the loss of employment.

So, not only are low-income persons poorly paid, many are electronically redlined from hiring consideration. What type of “workforce science” fails to take into account the most important variable (pay) and yet offers “solutions” based on this flawed science?

Are Employer Referral Programs Encouraging Discriminatory Hiring Practices?

According to Evolv, it recently “used rich data on hundreds of thousands of employees to demonstrate that referred workers show measurably better productivity and retention.” The research showed that 65 percent of companies have a referral program and 36 percent filled their last opening through an employee referral. A primary insight from Evolv is that, when it comes to retention, referred workers were around 13 percent less likely to quit.

As noted above, previous insights of Evolv and Kenexa – distance from work and housing mobility – lead to workforce selection processes that discriminate against blacks and Hispanics. Combining the employer referral program insight with the distance from work and housing mobility insights likely exacerbates the discriminatory impact of workforce science and its use in the hiring process.

About 40 percent of white Americans and about 25 percent of non-white Americans are surrounded exclusively by friends of their own race, according to an ongoing Reuters/Ipsos poll. Even looking at a broader circle of acquaintances to include coworkers as well as friends and relatives, 30 percent of Americans are not mixing with others of a different race, the poll showed.

The workforce insights regarding distance from job and employee mobility results in fewer blacks and Hispanics being hired. Consequently, if 36% of job openings are filled by referrals from employees and 30% of those employees do not have friends or relatives of another race, blacks and Hispanics will be underrepresented in the workforce hired as a result of referrals.