Employment Assessments: 21st Century Snake Oil?

The Oxford English Dictionary defines snake oil as "a quack remedy or panacea." The origins of snake oil as a derogatory phrase trace back to the latter half of the 19th century, which saw a dramatic rise in the popularity of "patent medicines." Despite the name, most patent medicines were not officially patented. They were medicines with questionable effectiveness whose contents were usually kept secret.

By the middle of the 19th century the manufacture of patent medicines had become a major industry in America. Often high in alcoholic content and fortified with cocaine, morphine or opium, many of these concoctions were advertised for infants and children. Some level of exoticism in the contents of the preparation was deemed desirable by their promoters and nearly any scientific discovery could inspire a key ingredient or principle in a patent medicine.

From the beginning, some physicians and medical societies were critical of patent medicines. They argued that the remedies did not cure illnesses, discouraged the sick from seeking legitimate treatments, and created negative consequences like alcohol and drug dependency.

By the end of the 19th century, Americans favored laws to force manufacturers to disclose the remedies' ingredients and use more realistic language in their advertising. These laws met with fierce resistance from the manufacturers. Finally, with strong support from President Theodore Roosevelt, a Pure Food and Drug Act was passed by Congress in 1906, paving the way for public health action against unlabeled or unsafe ingredients and misleading advertising.

Are there any similarities between the sales and marketing of snake oil/patent medicine and the sales and marketing of employment assessments? Do the assessments have questionable effectiveness? Are their contents kept secret? Do they market scientific discoveries, in the form of buzzwords, as inspiring key ingredients? Is there public action challenging the assessments?

Questionable Effectiveness?

According to a 2012 study by Oracle and Development Dimensions International (DDI), a global human resources consulting firm whose expertise includes designing and implementing selection systems, more than 250 staffing directors and over 2,000 new hires from 28 countries provided the following perspectives on their organization’s selection processes (the following are excerpts from the study):

• [O]nly 41 percent of staffing directors report that their pre-employment assessments are able to predict better hires.
• Only half of staffing directors rate their systems as effective, and even fewer view them as aligned, objective, flexible, efficient, or integrated. 
• [T]he actual process for making a hiring decision is less effective than a coin toss.

In a 2007 article titled, “Reconsidering the Use of Personality Tests in Employment Contexts”, co-authored by six current or former editors of academic psychological journals, Dr. Kevin Murphy, Professor of Psychology at Pennsylvania State University and Editor of the Journal of Applied Psychology (1996-2002), states:

The problem with personality tests is … that the validity of personality measures as predictors of job performance is often disappointingly low. A couple of years ago, I heard a SIOP talk by Murray Barrick … He said, “If you took all the … [factors], measured well, you corrected for everything using the most optimistic corrections you could possibly get, you could account for about 15% of the variance in performance [between projected and actual performance].” … You are saying that if you take normal personality tests, putting everything together in an optimal fashion and being as optimistic as possible, you’ll leave 85% of the variance unaccounted for. The argument for using personality tests to predict performance does not strike me as convincing in the first place.

Secret Contents?

Using terms like patent-pending, proprietary and trade secret, employment assessment companies claim that they cannot disclose information about their assessment processes. Are these claims legitimate or, like the Wizard of Oz, are the claims used to mask the lack of relevant and legal substance behind the assessments? Or is it a bit of both? 

Even assuming that the confidentiality claims by the assessment companies are appropriate, are there no circumstances in which the companies must disclose information regarding how their assessments are developed and implemented and the results of the assessment usage across a broad population of job applicants? There are. 

In a 2012 decision, a federal appeals court stated the Americans with Disabilities Act (ADA) prohibits employment tests when such tests screen out or tend to screen out disabled people and the use of the test is not job-related for the position in question and consistent with business necessity. The court went on to order that the employer and testing company, Kroger and Kronos, respectively, provide the Equal Employment Opportunity Commission (EEOC) with:

• Any and all documents and data constituting or related to validation studies or validation evidence pertaining to Kronos assessment tests purchased by Kroger, including but not limited to such studies or evidence as they relate to the use of the tests as personnel selection or screening instruments, even if created or performed for other customer(s);
• The user’s manual and instructions for the use of assessment tests used by Kroger;
• Any and all documents (if any) related to Kroger, including but not limited to correspondence, notes, and data files, relating to Kroger; its use of the assessment test; results, ratings, or scores of individual test takers; and any validation efforts made thereto; and
• Any and all documents discussing, analyzing or measuring potential adverse impact on persons with disabilities.

So, it's quick and easy for the regulatory agency (EEOC) to obtain the necessary information, right?No. The EEOC investigation leading to the appeals court decision referenced above has been ongoing for more than six years. It has generated a number of district court and appellate court decisions. None of those decisions have addressed substantive claims of discrimination. They are all decisions relating to the unwillingness of Kroger and Kronos to provide information to the EEOC that would allow the EEOC to determine whether the Kronos assessments illegally discriminated against persons with disabilities.

Please see Kroger and Kronos: Chaos and Disorder.

Negative Consequences?

Employment personality tests discriminate against applicants with mental illness. These applicants are ready, willing and able to work, but are being illegally screened out from employment consideration by personality tests that use a non-validated stereotype of the capabilities of persons with mental illness. Please see What Are The Issues, ADA, FFM and DSM, and Employment Assessments Are Designed to Reveal an Impairment.

Employment discrimination on the basis of mental illness affects all demographic groups. Mental illness is no respecter of age, gender, geography, income, occupation, military status, race, religion or sexual orientation. Persons with mental illness include military veterans returning to the civilian workforce, new and expectant mothers, LGBTs and young adults. Please see Tests Discriminate Against Returning Veterans, Tests Discriminate Against New and Expectant Mothers and Employment Tests Discriminate Against LGBTs.

The illegal screening out of applicants with mental illness has come at a cost of tens of billions of dollars to taxpayers and the U.S. Treasury.  The prevalence of mental disorders has generally remained unchanged over the past 15 years and substantially increased rates of treatment should have resulted in a decline in the percentage of persons receiving disability awards who are diagnosed with mental illness. Sadly, no. People with psychiatric impairments constitute the largest and most rapidly growing subgroup of income support program awards (SSDI, SSI). Please see Costing Taxpayers Billions of Dollars Each Year.

Every year since 1999, more Americans have killed themselves than the year before, making suicide the nation’s greatest untamed cause of death. Being unemployed is associated with a 2-3X increase in the relative risk of death by suicide, compared with being employed. Given that more than 90% of persons who attempt suicide have mental illnesses, a tool like personality testing that illegally excludes persons with mental illness from employment consideration leads to an increase both in perceived burdensomeness and thwarted belongingness/social alienation, two critical elements tied to the risk of suicide. Please see Does the Rising Use of Employment Personality Tests Contribute to An Increase in Suicides?

Marketing by Scientific Buzzwords?

New entrants in the assessment field include ConectCubed, Good Co., Evolv, Knack and Prophesy Sciences They compete with incumbents like Kenexa (IBM), Kronos, SHL, Success Factors (SAP),  and Taleo (Oracle). Marketing claims include:

Our games are fun, but our technology is rock solid. We design and develop our games using state-of-the-art behavioral science, then we use data-mining tools and massive amounts of data to validate and compute the Knacks you earn. (Knack)

We use a powerful combination of cognitive games, biometric signals, and machine learning algorithms to compile actionable insights about you and your teammates. (Prophesy Sciences) 

We have combined decades of academic and business research with sophisticated statistical models to create our Proprietary Psychometric Algorithm. (Good Co.)

Evolv’s patent-pending technology platform unifies and supplements existing data from current systems, then utilizes that dataset to identify fact-based workforce insights that drive measurable ROI. (Evolv)

Kronos helps organizations find value in big data with enhanced analytics. (Kronos)

Public Action?

The EEOC has been attempting to investigate the use of Kronos assessment by Kroger for more than six years. As noted previously, that investigation has resulted in a number of court decisions, not on the substance of the claims of alleged discrimination, but on the requirement of Kronos and Kroger to provide the EEOC with relevant information regarding the assessment and its usage.

That investigation has evolved into a systemic investigation by the EEOC. Systemic investigations involves pattern or practice, policy, and/or class cases where the alleged discrimination has a broad impact on an industry, profession, company, or geographic area. As stated by the court in the 2012 appellate decisions referenced previously, it is “a proper inquiry for the EEOC to seek information about how these tests work, including information about the types of characteristics they screen out….“ 

In connection with systemic investigations, the EEOC’s enforcement tools include issuing broad information requests and subpoenas on employers that are named as respondents in EEOC charges, particularly when the EEOC suspects systemic discrimination, and filing pattern or practice class lawsuits in federal court. For some employers, the potential class size can be measured in the millions of plaintiffs. 

The EEOC's systemic investigation of Kroger and Kronos, as well as its investigation of other employers and their assessment companies, is consistent with the EEOC's implementation of its Strategic Enforcement Plan (SEP) for 2013-2016. The first national priority of the SEP is “eliminating systemic barriers in recruitment and hiring.” The SEP goes on to state that “people with disabilities continue to confront discriminatory policies and practices at the recruitment and hiring stages. These include … the use of screening tools (e.g., pre-employment tests …)."

Disability discrimination, including employment assessment litigation, is a significant element of the EEOC's enforcement activities. As shown in the chart below, ADA claims constituted the largest percentage of the EEOC’s yearly litigation filing activity for FY 2013 - almost half of all cases. 

The Importance of This Issue

The long-term fiscal stability of the United States of America depends, in part, on ensuring that Americans with disabilities have meaningful opportunities to contribute to our collective well-being and on eliminating outdated policies that keep people in cycles of poverty and dependency.

More than two decades after the passage of the ADA, the unemployment rate for Americans with disabilities stubbornly remains nearly double that of people without disabilities, while their rate of labor force participation has continued to be abysmally low. Figures from the Bureau of Labor Statistics show that labor force participation for workers with disabilities was 20.3 percent, while the total for workers without disabilities was 69.1 percent—more than three times higher. 

There are many benefits of employment—work enhances skills such as communication, socialization, academics, physical health, and community skills; it factors into how one is perceived by society; it promotes economic well-being (reducing government expenditures on income support programs, Medicare and Medicaid); it leads to greater opportunity for upward mobility; and it contributes to greater self-esteem. 

From What Distance is Discrimination Acceptable, Redux

There is a vicious cycle of associated with poverty and mental illness. It is a self-reinforcing circle where poverty is linked to greater incidence of mental illness, and mental illness is linked to a greater likelihood of living in poverty.

A 2005 study looked at the health records of 34,000 patients who were hospitalized at least twice for mental illness over a seven-year period. The study looked at whether or not these patients had “drifted down” to less affluent ZIP codes following their first hospitalization. The study found that poverty — acting through economic stressors such as unemployment and lack of affordable housing — is more likely to precede most mental illness.

A prior post discussed location-based "insights" provided by employment assessment companies like Kenexa and Evolv. These companies believe that a greater distance from the jobsite, lengthier commute times and more frequent household moves should weigh against - or even exclude - job applicants from consideration for employment. As the prior post noted, these insights are generalized correlations, they say nothing about any particular applicant.

Employers utilizing these location-based "insights" screen out qualified applicants solely because they live (or don't live) in certain areas. Not only do employers do a disservice to themselves by eliminating qualified persons from consideration for employment, the use of these insights discriminates against the poor, who are disproportionately represented by African-Americans and Hispanics. As shown in the graphic to the right, Blacks are more than two-and-a-half times as likely as Whites to live below the federal poverty level. Similarly, Hispanics are twice as likely as Asians to live below the federal poverty level. The federal poverty level in 2010 for an individual was $10,830 and for a family of four was $22,050. The graphic is taken from a 2011 report issued by the National Center for Health Statistics.

This post reprises certain information from the prior post and then discusses how the use of the location-based insights also discriminates against persons with mental illness.

Reprising "From What Distance is Discrimination Acceptable?"

The assessment company Kenexa, purchased by IBM in December 2012, will test approximately 40 million applicants this year for hundreds of clients. Kenexa believes that a lengthy commute raises the risk of attrition in call-center and fast-food jobs. It asks applicants for call-center and fast-food jobs to describe their commute by picking options ranging from "less than 10 minutes" to "more than 45 minutes."

The longer the commute, the lower their recommendation score for these jobs, says Jeff Weekley, who oversees the assessments. Applicants also can be asked how long they have been at their current address and how many times they have moved. People who move more frequently "have a higher likelihood of leaving," Mr. Weekley said.

As a consequence, Kenexa clients will pass over qualified applicants solely because they live (or don't live) in certain areas. Not only does the employer do a disservice to itself and the applicant, it increases the risk of employment litigation, with its consequent costs. 

Distance From Jobsite

A New York Time article from earlier this year, "In Climbing Income Ladder, Location Matters," reads, in part:

Stacey Calvin spends almost as much time commuting to her job — on a bus, two trains and another bus — as she does working part-time at a day care center.  ...

Her nearly four-hour round-trip [job commute] stems largely from the economic geography of Atlanta, which is one of America’s most affluent metropolitan areas yet also one of the most physically divided by income. The low-income neighborhoods here often stretch for miles, with rows of houses and low-slung apartments, interrupted by the occasional strip mall, and lacking much in the way of good-paying jobs. 

This geography appears to play a major role in making Atlanta one of the metropolitan areas where it is most difficult for lower-income households to rise into the middle class and beyond, according to a new study that other researchers are calling the most detailed portrait yet of income mobility in the United States.

The dearth of good-paying jobs in low-income neighborhoods means that residents of those neighborhoods have a longer commute. 

Housing Mobility

As shown in the table below, poor and near-poor families tend to move much more frequently than their higher income neighbors and the general population.

According to a 2011 study by the Center for Public Housing, entitled "Should I Stay or Should I Go? Exploring the Effects of Housing Instability and Mobility on Children," a wide range of often complex forces appears to drive frequent mobility, and residential instability in general — the formation and dissolution of households, an inability to afford one’s housing costs, the loss of employment, the lack of a safety net, lack of quality housing or a safer neighborhood.

Correlation Is Not Causation

When two variables, A and B, are found to be correlated, there are several possibilities:

1. A causes B
2. B causes A
3. A causes B at the same time as B causes A (a self-reinforcing system)
4. Some third factor causes both A and B

The correlation is simple coincidence. It is wrong to assume any of these possibilities.

According to Kenexa and Evolv, however, the correlation between (A) persons who move more frequently and (B) shorter job tenure is that A causes B. However, a 2011 study by the Center of Public Housing demonstrates that it may well be that (B) shorter job tenure causes (A) persons to move. As shown in the table below, taken from the 2011 study, 76% of involuntary moves were a result of job loss. The second highest factor precipitating involuntary moves is mental health problems.

Socioeconomic Status and Mental Illness

One of the most consistently replicated findings in the social sciences has been the negative relationship of socioeconomic status (SES) with mental illness: The lower the SES of an individual is, the higher is his or her risk of mental illness.

As an example, for the period from 2005-2010, the Centers for Disease Control found that among adults 20–44 and 45–64 years of age, depression was five times as high for those below poverty, about three times as high for those with family income at 100%–199% of poverty, and 60% higher for those with income at 200%–399% of poverty compared with those at 400% or more of the poverty level. 

As noted above, the federal poverty level in 2010 for an individual was $10,830 and for a family of four was $22,050.

People who live in poverty are at increased risk of mental illness compared to their economically stable peers. Their lives are stressful. They are both witness to and victims of more violence and trauma than those who are reasonably well off, and they are at high risk of poor general health and malnutrition. Similarly, when people are mentally ill, they are at increased risk of becoming and/or staying poor. They have higher health costs, difficulty getting and retaining jobs, and suffer the social stigma and isolation of mental illness.

Thousands of employers utilize pre-employment assessments provided by companies like Kenexa and Evolv, whose location-based "insights" discriminate against lower-income persons. According to a 2001 study, lower income Americans had a higher prevalence of 1 or more psychiatric disorders (51% vs 28%): mood disorders (33% vs 16%), anxiety disorders (36% vs 11%), and eating disorders (10% vs 7%). Consequently, pre-employment assessments using these location-based "insights" violate the Americans with Disabilities Act by illegally screening out persons with mental illness.

.

From What Distance is Discrimination Acceptable?

Some 60% of American workers earn hourly wages. Of these, about half change jobs each year, so firms that employ lots of entry-level workers, such as call centers, supermarkets, home improvement stores and fast-food chains, have to vet million of applications every year.

Xerox Evolv(ing)

A recent article in MIT Technology Review reports that Xerox is screening tens of thousands of applicants for low-wage jobs in its call centers using software from a startup company called Evolv. According to its website, Evolv “is a workforce science software company that harnesses big data, predictive analytics and cloud computing to help businesses improve workplace productivity and profitability” and its customers include 20 of the Fortune 100.

Working with Xerox, Evolv found that one of the best predictors that a customer-service employee will stick with a job is that he lives nearby and can get to work easily.  As Evolv states in its Q3 2013 Workforce Performance Report:

The distance that employees live from work affects how long they choose to stay at a job. Unsurprisingly, employees that live 0-5 miles from their place of work have the longest median tenure. They remain at their jobs 20% longer than employees with the shortest median tenure.

Although it still sells photocopiers, Xerox has also become one of the world’s largest outsourcing companies. It provides services like running customer service centers, handling health claims, and processing credit-card applications that brought in $11.5 billion in revenue last year.

That business relies on a huge workforce of 54,000 customer service agents, and because of high attrition in hourly jobs, Xerox will have to replace 20,000 of them this year, says Teri Morse, vice president for recruiting at Xerox Services.

The Dictatorship of Data

Morse says Xerox today won’t even look at resumes of those who score in the “red” category of Evolv’s initial behavioral assessment, a 30-minute online exam that workers fill out at home. Early on, while piloting the system, Morse says Xerox still hired against the advice of the data. Now, she says, “people who do poorly we no longer hire.”

As stated by the authors in Big Data: A Revolution That Will Transform How We Live, Work, and Think:

We are more susceptible than we may think to the “dictatorship of data” — that is, to letting the data govern us in ways that may do as much harm as good. The threat is that we will let ourselves be mindlessly bound by the output of our analyses even when we have reasonable grounds for suspecting something is amiss. Or that we will become obsessed with collecting facts and figures for data’s sake. Or that we will attribute a degree of truth to the data which it does not deserve.

For more and more companies, like Xerox, the hiring boss is an algorithm. Jobs that were once filled on the basis of work history and interviews are left to personality tests and data analysis. The new hiring tools are part of a broader effort to gather and analyze employee data.

The risks to employers of utilizing online personality tests in their employment application process have been set out in a number of prior posts, including What Are the Issues, Courts Find Tests to be Illegal, The Next Asbestor? The Next FLSA?, Damages and Indemnification Challenges to Employers, and Welcomed as Customers; Rejected as Employers. The remainder of this post sets out the employment discrimination litigation risks to employers (and society) of blindly following the "insights" of Kenexa and Evolv in distance from job site and housing mobility.

Interestingly, while Evolv now touts the distance from job insight for use by its clients, the company expressed a different view in a 2012 Wall Street Journal article, which reads;

Evolv is cautious about exploiting some of the relationships it turns up for fear of violating equal opportunity laws. While it has found employees who live farther from call-center jobs are more likely to quit, it doesn't use that information in its scoring in the U.S. because it could be linked to race.

From What Distance is Discrimination Acceptable?

Kenexa, purchased by IBM in December 2012, will test approximately 40 million applicants this year for thousands of clients. Kenexa believes that a lengthy commute raises the risk of attrition in call-center and fast-food jobs. It asks applicants for call-center and fast-food jobs to describe their commute by picking options ranging from "less than 10 minutes" to "more than 45 minutes."

The longer the commute, the lower their recommendation score for these jobs, says Jeff Weekley, who oversees the assessments.Applicants also can be asked how long they have been at their current address and how many times they have moved. People who move more frequently "have a higher likelihood of leaving," Mr. Weekley said.

Painting with the broad brush of distance from job site, commute time and moving frequency results in well-qualified applicants being excluded, applicants who might have ended up being among the longest tenured of employees. The Kenexa and Evolv findings are generalized correlations (i.e., persons living closer to the job site tend to have longer tenure than persons living farther from the job site). The insights say nothing about any particular applicant.

As a consequence, employers will pass over qualified applicants solely because they live (or don't live) in certain areas. Not only does the employer do a disservice to itself and the applicant, they increase the risk of employment litigation, with its consequent costs. 

Distance From Jobsite

A recent New York Time article, "In Climbing Income Ladder, Location Matters," reads, in part:

Stacey Calvin spends almost as much time commuting to her job — on a bus, two trains and another bus — as she does working part-time at a day care center.  ...

Her nearly four-hour round-trip [job commute] stems largely from the economic geography of Atlanta, which is one of America’s most affluent metropolitan areas yet also one of the most physically divided by income. The low-income neighborhoods here often stretch for miles, with rows of houses and low-slung apartments, interrupted by the occasional strip mall, and lacking much in the way of good-paying jobs

This geography appears to play a major role in making Atlanta one of the metropolitan areas where it is most difficult for lower-income households to rise into the middle class and beyond, according to a new study that other researchers are calling the most detailed portrait yet of income mobility in the United States.

The dearth of good-paying jobs in low-income neighborhoods means that residents of those neighborhoods have a longer commute. The 2010 Census showed that poverty rates are significantly higher for blacks and Hispanics. Consequently, hiring decisions predicated on distance from job site, intentionally or not, discriminate against certain races.

Housing Mobility

As shown in the table below, poor and near-poor families tend to move much more frequently than their higher income neighbors and the general population.

According to a 2011 study by the Center for Public Housing, entitled "Should I Stay or Should I Go? Exploring the Effects of Housing Instability and Mobility on Children," a wide range of often complex forces appears to drive frequent mobility, and residential instability in general — the formation and dissolution of households, an inability to afford one’s housing costs, the loss of employment, the lack of a safety net, lack of quality housing or a safer neighborhood.

Correlation Is Not Causation

When two variables, A and B, are found to be correlated, there are several possibilities:

1. A causes B
2. B causes A
3. A causes B at the same time as B causes A (a self-reinforcing       system)
4. Some third factor causes both A and B

The correlation is simple coincidence. It is wrong to assume any of these possibilities.

Kenexa, Evolv and their clients, however, assume that A (proximity to job site) causes B (reduced attrition and better performance). That assumption leads them to disfavor otherwise qualified applicants who do not live within a five-mile radius.

The correlation could also demonstrate B (reduced attrition and better performance) is caused by C (proximity of job site to applicants homes). Instead of being a hiring insight, the correlation might function better as being a job site location insight. Given the relative immobility of persons and companies, locating a job site (call center, etc.) close to communities with high numbers of lower-income persons could lead to a more sustainable competitive advantage.

Per Kenexa, the correlation between (A) persons who move and (B) shorter job tenure is that A causes B. However, per the 2011 study by the Center of Public Housing, it may well be that (B) shorter job tenure causes (B) persons to move. As shown in the table below, taken from the 2011 study, more than 76% of involuntary moves were a result of job loss.

Although data does give rise to information and insight, they are not the same. Data's value to business relies on human intelligence, on how well managers and leaders formulate questions and interpret results. More data doesn't mean you will get "proportionately" more information. In fact, the more data you have, the less information you gain as a proportion of the data (concepts of marginal utility, signal to noise and diminishing returns).

Maybe It's the Work, Not the Workforce, That Need Analysis ...

Some 60% of American workers earn hourly wages. Of these, about half change jobs each year. The cost to U.S. businesses of worker attrition and lost productivity is $350 billion annually. How much is that?

What should an employer do? Increase starting pay? Since most people work, at least in part, for the money - giving them more money might encourage them to stay longer and work harder. It seems to work reasonably well with corporate executives.  Top executive compensation averaged $9.4 million last year at the 50 largest employers of low-wage workers.

What do many employers do? Retain "workforce science" companies like Evolv and Kenexa to administer personality tests and analyze applicant data in order to address the employee turnover issue.

As noted above, working with Xerox, Evolv found that one predictor that a customer-service employee will stick with a job is that s/he lives nearby and can get to work easily. Kenexa had a similar "insight," and added that people who move more frequently have a higher likelihood of leaving.

Are there any groups of people who might live farther from the work site and may move more frequently than others? Yes, lower-income persons, disproportionately women, black, Hispanic and the mentally ill. They can't afford to live where the jobs are and move more frequently because of an inability to afford housing or the loss of employment.

So, not only are low-income persons poorly paid, many are electronically redlined from hiring consideration. What type of “workforce science” fails to take into account the most important variable (pay) and yet offers “solutions” based on this flawed science?

Are Employer Referral Programs Encouraging Discriminatory Hiring Practices?

According to Evolv, it recently “used rich data on hundreds of thousands of employees to demonstrate that referred workers show measurably better productivity and retention.” The research showed that 65 percent of companies have a referral program and 36 percent filled their last opening through an employee referral. A primary insight from Evolv is that, when it comes to retention, referred workers were around 13 percent less likely to quit.

As noted above, previous insights of Evolv and Kenexa – distance from work and housing mobility – lead to workforce selection processes that discriminate against blacks and Hispanics. Combining the employer referral program insight with the distance from work and housing mobility insights likely exacerbates the discriminatory impact of workforce science and its use in the hiring process.

About 40 percent of white Americans and about 25 percent of non-white Americans are surrounded exclusively by friends of their own race, according to an ongoing Reuters/Ipsos poll. Even looking at a broader circle of acquaintances to include coworkers as well as friends and relatives, 30 percent of Americans are not mixing with others of a different race, the poll showed.

The workforce insights regarding distance from job and employee mobility results in fewer blacks and Hispanics being hired. Consequently, if 36% of job openings are filled by referrals from employees and 30% of those employees do not have friends or relatives of another race, blacks and Hispanics will be underrepresented in the workforce hired as a result of referrals.

Big Data and Employment Testing: Correlation Is Not Causation

Imagine you are watching at a railway station. More and more people arrive until the platform is crowded, and then — hey presto — along comes a train. Did the people cause the train to arrive (A causes B)? Did the train cause the people to arrive (B causes A)? No, they both depended on a railway timetable (C caused both A and B).

Some 60% of American workers earn hourly wages. Of these, about half change jobs each year. So firms that employ lots of unskilled workers, such as supermarkets, home improvement stores and fast-food chains, have to vet million of applications every year. Making the process more efficient could yield big payoffs.

Algorithms and big data are powerful tools. Wisely used, they can help match the right people with the right jobs. But they must be designed and used by humans, otherwise they can go terribly wrong.

Big Data

Big Data is a tool in the right hands which can yield insight, help determine paths and alternatives which are more likely to be successful, and lead to improved conditions. But Big Data is just one of a set of tools which can be used to develop successful paths and alternatives. It is best when it is used in conjunction with other tools: intuition, inductive reasoning, statistical analysis to name a few.

One of the reasons why Big Data is in the forefront today is with the advent of new tools, very large data flows, and advanced computing techniques there are real opportunities to manage and use huge data sets.

The theory of big data is to have no theory, at least about human nature. One just gathers huge amounts of information, observes the patterns and estimates probabilities about how people will act in the future. One does not address causality.

As the authors of Big Data state, “Contrary to conventional wisdom, such human intuiting of causality does not deepen our understanding of the world." Instead, they aim to stand back nonjudgmentally and observe linkages: “Correlations are powerful not only because they offer insights, but also because the insights they offer are relatively clear. These insights often get obscured when we bring causality back into the picture.”

But are correlations relatively clear? The authors of Freaknomics discuss correlation and causation in the video below; specifically, the view of medical professionals in the first half of the 20th century that polio was caused by ice cream consumption (since disproved).

When two variables, A and B, are found to be correlated, there are several possibilities:

1. A causes B
2. B causes A
3. A causes B at the same time as B causes A (a self-reinforcing system)
4. Some third factor causes both A and B

The correlation is simple coincidence.

It’s wrong to assume any one of these possibilities. Correlation is a (perhaps strong) hint that there may be a relationship, but identifying the exact nature of that relationship requires more - i.e., a controlled experiments or proper statistical analysis. One needs to examine all of the variables that may influence the relationship and look for evidence supporting or rejecting the influence of each. One also needs to find a mechanism that explains any causal relationship. 

Bad Data

"Bad data" is data that has not been collected accurately or consistently or the data has been defined differently from person to person, group to group and company to company. The huge amount of "bad" data that is regularly served up for analysis may make it irresponsible for one to just "stand back nonjudgmentally and observe linkages," especially in the pre-employment testing process.

In the recruitment and hiring context, unproctored online personality tests are used to allow an applicant to take the test anywhere and anytime. That freedom creates conditions ripe for obtaining "bad' data. As stated by Jim Beaty, PhD, and Chief Science Officer at testing company Previsor:

Applicants who want to cheat on the test can employ a number of strategies to beat the test, including logging in for the test multiple times to practice or get the answers, colluding with another persons while completing the test, or hiring a test proxy to take the test.

And what about the accuracy of tests responses from those who are hired? Analyzing a sample of over 31,000 employees, a data analytic company's researchers found that employees who said they were most likely to follow the rules left the job on average 10% earlier, were 3% less likely to close a sale and were actually not particularly good at following rules.

Bad data exposes a vexing problem for employers. Applicants and employees seek to tell employers what they believe employers want to hear, and employers tend to ask questions that lead applicants employees to answer these questions in the “right” way.

A Simple Want of Careful, Rational Reflection

As noted in a prior post, prejudice rises not from malice or hostile animus alone. It may result as well from insensitivity caused by simple want of careful, rational reflection.

For example, take two insights from Evolv, a data analytics company:

1. Living in close proximity to the job site and having access to reliable transportation—are correlated with reduced attrition and better performance; and
2. Referred employees have 10% longer tenure than non-referred employees and demonstrate approximately equal performance.

An employer confronted with these two insights might well determine that (i) applicants living beyond a certain distance from the job site (i.e., retail store) should be excluded from employment consideration and (ii) preference in hiring should be extended to applicants referred by existing employees. Such a determination may end up being penny wise and pound foolish.

Painting with the broad brush of distance from job site will result in well-qualified applicants being excluded, applicants who might have ended up being among the longest tenured of employees. Remember that the Evolv insight is a generalized correlation (i.e., the pool of persons living closer to the job site tend to have longer tenure than the pool of persons living farther from the job site). The insight says nothing about any particular applicant or employee.

As a consequence, employers will pass over qualified applicants solely because they live (or don't live) in certain areas. Not only does the employer do a disservice to itself and the applicant, they increase the risk of employment litigation, with its consequent costs (attorney fees, damages, reputational harm, etc.). How?

A recent New York Time article, "In Climbing Income Ladder, Location Matters," reads, in part:

Her nearly four-hour round-trip [job commute] stems largely from the economic geography of Atlanta, which is one of America’s most affluent metropolitan areas yet also one of the most physically divided by income. The low-income neighborhoods here often stretch for miles, with rows of houses and low-slung apartments, interrupted by the occasional strip mall, and lacking much in the way of good-paying jobs

The dearth of good-paying jobs in low-income neighborhoods means that residents of those neighborhoods have a longer commute. As to the demographic makeup of low-income families, the 2010 Census showed that poverty rates are much higher for blacks and Hispanics. Consequently, hiring decisions predicated on distance, intentionally or not, discriminate against certain protected classes.

Similarly, an employer extending a hiring preference to referrals of existing employees may be further exacerbating the discriminatory impact of its hiring process. Those referrals tend to be persons from the same neighborhoods and socioeconomic backgrounds of existing employees, meaning that workforce diversity, broadly considered, will decline.

With the huge amounts of "bad" data that get generated and stored daily, the failure to understand how to leverage the data in a practical way that has business benefit will increasingly lead to shaky insights and faulty decision-making, with significant costs both to the employer and society.

A Fool With A Tool Is Still A Fool

Most companies have vast amounts of HR data (employee demographics, performance ratings, talent mobility data, training completed, age, academic history, etc.) but they are in no position to use it. According to Bersin by Deloitte, an HR research and consultancy organization, only 20% believe that the data they capture now is highly credible and reliable for decision-making in their own organization.

Research shows that the average large company has more than 10 different HR applications and their core HR system is over 6 years old. So it will take significant effort and resources (read funding) to bring this data together and make sense of it.

With the huge amounts of "bad" data that get generated and stored daily, the failure to understand how to leverage the data in a practical way that has business benefit will increasingly lead to shaky insights and faulty decision-making, with significant costs both to the employer and society.

As stated by Jim Stikeleather on the Harvard Business Review blog:

Machines don't make the essential and important connections among data and they don't create information. Humans do. Tools have the power to make work easier and solve problems. A tool is an enabler, facilitator, accelerator and magnifier of human capability, not its replacement or surrogate. That's what the software architect Grady Booch had in mind when he uttered that famous phrase: "A fool with a tool is still a fool." 

Understand that expertise is more important than the tool. Otherwise the tool will be used incorrectly and generate nonsense (logical, properly processed nonsense, but nonsense nonetheless). 

Although data does give rise to information and insight, they are not the same. Data's value to business relies on human intelligence, on how well managers and leaders formulate questions and interpret results. More data doesn't mean you will get "proportionately" more information. In fact, the more data you have, the less information you gain as a proportion of the data (concepts of marginal utility, signal to noise and diminishing returns).