Cracking the Labor Market
Predicting Businesses' Brain Power Needs Is Possible
Suppose a fortuneteller could gaze into a crystal ball and foresee which high-value employees companies would lose to competitors in the next year? Could the prediction be used in taking steps to retain valuable workers?
Managers could soon have access not only to forecasts like this, but also to the crystal ball that generates them. In a study in last November’s Information Systems Research, Yuanyang Liu, a professor in the Department of Business Analytics and Statistics, and two colleagues applied conventional data analytics tools such as data scraping, text mining, network analysis, and machine learning to publicly available information. The result? The researchers mapped “human capital flow” between companies including Amazon, Walmart, Ford Motor Co., and IBM from 2000-2014.
Liu, with Gautam Pant (University of Iowa) and Olivia R. L. Sheng (University of Utah), used data from online profiles of 89,943 employees, tracking their careers across 3,467 public businesses over 15 years. Through this analysis, they formulated the human capital flow network (i.e., tracking workers’ movement among firms) and identified “firm-pair” similarities derived from the employee mobility pattern between firms.
Using employees’ reported skill terms, they could associate a firm with its employees’ skills and identify pairs of firms that have employees with similar skill sets. This means those “firm-pairs” are likely to compete in the same human capital pool. Their work in “Predicting Labor Market Competition: Leveraging Interfirm Network and Employee Skills” utilizes these two set of information to predict businesses’ future competitors for high-value employees.
Liu says that while much research has been done on businesses’ competition in the product market, comparatively little has been done on the labor market.
“The competitors in the cookie market, for example, are obvious,” Liu says. “There are many different cookie producers, but all the cookies are on the market shelf where the consumer can see them.” It’s clear which cookies are being picked and therefore easy to identify competitors as well as which are succeeding.
Human capital competition is not so obvious. Liu uses the example of data scientists: Who needs data scientists? Tech companies, and the auto industry’s needs are growing. What may be less obvious is that data scientists can work in the agricultural industry. (Consider precision agriculture.) What makes the labor market competition challenging is that a firm’s competitors are not necessarily the same as its product market competitors. This is because some skills can be applied to produce different products, and a product can be produced in different ways (e.g., natural vs synthetic diamonds, digital camera vs smart phone). Thus, analysis of labor market competition, especially the so-called knowledge worker competition, has been elusive.
“Naturally, we thought about employees’ profiles posted online: we can find their profiles, which shows what skills they have and where they have worked, and we can scrape them for data,” Liu says. They used search engines to search for employees of a given company, returning many LinkedIn profiles.
That simple search gave the analysts a static data picture of employees with certain skill sets at specific companies in a given timeframe. By applying common data analytics tools, the team moved from a single-year snapshot of firms’ human capital assets to a dynamic view of talent movement among the companies over time. From analyzing past talent migration, it’s a short step to predicting future human capital flow, Liu says. It can be done with the same tools his team used—publicly available data and algorithms.
It’s just a question of who wants that crystal ball enough to expend the effort to get it. —Scott McNutt
Note: Liu’s team is not the first to conduct this kind of research. A data analytics firm in California conducted similar scraping to alert its clients when employees might be seeking new jobs. LinkedIn sued, claiming the data in their files was proprietary. A lower court ruled against the social network, but the case is ongoing. Precedent in such suits has the potential to ignite larger ata-ownership dust-ups between social networks and the users who post to them.
