A Fool With A Tool Is Still A Fool

In the June 22, 2015 cover story for Time magazine, "Questions to Answer in the Age of Optimized Hiring," author Eliza Gray asks, “Are we truly comfortable with turning hiring–potentially one of the most life-changing experiences that a person can go through–over to the algorithms?” The answer should be no.

When you have algorithms weighing hundreds of factors over a huge data set, you can't really know why they come to a particular decision or whether it really makes sense. As Geoff Nunberg, who teaches at the School of Information at the University of California Berkeley stated in an NPR interview, “big data is no more exact a notion than big hair.”

Decisions made or affected by correlation are inherently flawed. Correlation does not equal causation, as demonstrated by Tyler Vigen on his website Spurious Correlations. For example:

• There is a greater than 99% correlation (0.992558) between the divorce rate in Maine and the per capita consumption of butter in the U.S. over the years 2000-2009;
• There is a greater than 78% correlation (0.78915) between the number of worldwide non-commercial space launches and the number of sociology doctorates awarded in the U.S. over the years 1997-2009; and,
• There is a greater than 66% correlation (0.666004) between the number of films Nicolas Cage appeared in and the number people who drowned by falling into a swimming pool over the years 1999-2009.

And what of the correlation between personality and job performance? In a 2007 article titled, “Reconsidering the Use of Personality Tests in Employment Contexts,” Dr. Neil Schmitt, the University Distinguished Professor at Michigan State University, wrote:

 [A 1965 research paper found that] the average validity of personality tests was 0.09. Twenty-five years later, Barrick and Mount (1991) published a paper in which the best validity they could get for the Big Five [personality model] was 0.13. They looked at the same research. Why are we now suddenly looking at personality as a valid predictor of job performance when the validities still haven’t changed and are still close to zero?

If personality assessments are designed to find those employees with the best fit for the company culture, shouldn't the rising use of those assessments by employers over the past 10-15 years have resulted in a concomitant rise in employee engagement?

Gallup has taken an employee engagement poll annually since 2000. Gallup defines engaged employees as those who are involved in, enthusiastic about and committed to their work and workplace. According to the 2014 Gallup poll, 51% of employees in the U.S. were "not engaged" in their jobs and 17.5% were "actively disengaged." These percentages have changed little over the fifteen years Gallup has been polling.

Gallup’s research shows that employee engagement is strongly connected to business outcomes essential to an organization’s financial success, including productivity, profitability, and customer satisfaction. Yet, the purported benefits of personality assessments have failed to move the needle on employee engagement, meaning companies have not received the promised productivity and profitability "bumps" from using personality assessments.

Laszlo Bock

There are significant risks associated with the use of personality assessments in hiring algorithms, both to the employer and to the job applicant. As Google’s Laszlo Bock state in the Time article, “if [an employer] makes a bad assessment based on an algorithm or a test, that has a major impact on a person’s life–a job they don’t get or a promotion they don’t get.”

For the employer, the risks are at least two-fold. First, people who are “different” will be screened out, denying the employer the benefits that come from having a widely diverse group of employees. As Bock states in the article:

“I imagine someone who has Asperger’s or autism, they will test differently on these things. We want people like that at the company because we want people of all kinds, but they’ll get screened out by this kind of thing.”

The second risk for employers are the liabilities they face under laws like the Americans with Disabilities Act for using personality tests that screen out persons with disabilities, whether it be Asperger’s, autism, bipolar disorder, or other mental health challenges.  The Equal Employment Opportunity Commission (EEOC) currently has two systemic investigations ongoing against employers that used personality tests in their pre-employment screening processes.

The 2014 White House report, “Big Data: Seizing Opportunities, Preserving Values," found that, "while big data can be used for great social good, it can also be used in ways that perpetrate social harms or render outcomes that have inequitable impacts, even when discrimination is not intended." An accompanying fact sheet warns:

As more decisions about our commercial and personal lives are determined by algorithms and automated processes, we must pay careful attention that big data does not systematically disadvantage certain groups, whether inadvertently or intentionally. We must prevent new modes of discrimination that some uses of big data may enable, particularly with regard to longstanding civil rights protections in housing, employment, and credit.

Just as neighborhoods can serve as a proxy for racial or ethnic identity, there are new worries that big data technologies (personality assessments and algorithmic decisionmaking) could be used to “digitally redline” unwanted groups, either as customers, employees, tenants, or recipients of credit.  That is why we should not be comfortable with turning hiring over to the algorithms.

By The Numbers: What Employee Engagement and Stock Price Performance Tell Us About Pre-Employment Assessments

The benefits provided by the use of pre-employment assessments, whether called workforce science, talent analytics or any other name, should be readily apparent and quantifiable. For example, has the rising use of pre-employment assessments created greater employee engagement? If pre-employment assessments are designed to find those employees with the best fit for the company culture, shouldn't companies who use those assessments outperform their peer companies who do not use the assessments?

Employee Engagement

Gallup defines “engaged” employees as those who are involved in, enthusiastic about, and committed to their work and contribute to their organization in a positive manner. The information in this section come from Gallup's State of the American Workforce 2013 report.

The report shows that 70% of American workers are “not engaged” or “actively disengaged” and are emotionally disconnected from their workplaces and less likely to be productive. Currently, 52% of workers are not engaged, and worse, another 18% are actively disengaged in their work. Gallup estimates that these actively disengaged employees cost the U.S. between $450 billion to $550 billion each year in lost productivity.

Having the vast majority of American employees not engaged with their workplaces is troublesome as the country attempts to recover ground lost during the financial crisis and get back on track to pre-recession levels of prosperity. Even more troubling is that workplace engagement levels have hardly budged since Gallup began measuring them in 2000, with fewer than one-third of Americans engaged in their jobs in any given year. 

So, notwithstanding the exponential growth in pre-employment assessments over the past 10-15 years, "workplace engagement levels have hardly budged" since 2000. Contrast the lack of growth in employee engagement with the marketing of pre-employment assessments, like this selection from the Kronos website:

Your employees are the face of your brand and the most vital asset of your business. They drive your productivity and profitability. What’s more important than selecting the right ones? Take the guesswork out of employee selection with industry-specific, behavioral-based assessments and interview guides [from Kronos].

Gallup’s research shows that employee engagement is strongly connected to business outcomes essential to an organization’s financial success, including productivity, profitability, and customer satisfaction. And engaged employees are the ones who are the most likely to drive the innovation, growth, and revenue that their companies desperately need. Yet, the purported benefits of pre-employment assessments have failed to move the needle on employee engagement, meaning companies have not received the promised productivity and profitability "bumps" from using pre-employment assessments.

Stock Price Performance

The chart below compares stock price performance of CVS Caremark (CVS) and Walgreens Co.(WAG) for the period from July 31, 2011 to January 11, 2014.

The reason for selecting the July 31, 2011 start date is that in July 2011, CVS and the Rhode Island Civil Liberties Union (ACLU) entered into a voluntary settlement addressing the ACLU’s complaint challenging CVS’s use of a pre-hire questionnaire that the ACLU claimed could have a discriminatory impact on people with certain mental impairments or disorders.

The settlement came after the Rhode Island Commission for Human Rights had issued a finding in February 2011 that there was "probable cause" to believe that the questionnaire used by CVS violated state anti-discrimination laws that bar employers from eliciting information that pertain to job applicants' mental or physical disabilities.Pursuant to the settlement agreement, CVS agreed to permanently remove the questions at issue from its online application. Since that time, CVS has not utlized online pre-employment assessments as part of its hiring process.

The reasons for comparing CVS stock price performance with Walgreens are (i) that Walgreens and CVS are direct competitors and (ii) Walgreens continues to use online pre-employment assessments, including personality tests similar to those CVS was using prior to the Rhode Island settlement agreement. CVS stock price performance is shown by the black line and Walgreens stock price performance is shown by the brown line.

The numbers don't lie, do they? Since eliminating the use of pre-employment assessments, CVS stock price performance has increased by approximately 90%. Walgreens, in contrast, continues to use pre-employment assessments and its stock price performance has increased by approximately 50%. As an investor, where would you have rather put your money? As a person with a mental illness or their family member, loved one, friend and colleague, where would you have rather shopped, at a company that engages in hiring discrimination against persons with mental illness or at one that does not (CVS)?