Overcoming U.S. Shortage of Skilled Big Data Talent

The amount of Big Data that is generated daily is massive, and analyzing it and managing this vast amount of data has become crucial to successfully building a competitive advantage. As a result, a new job title of “data scientist” was created to fuse together unique skills such as programming, business, communication, quantification, and visualization. It’s a relatively new field that requires highly specialized skills, and as a result, you really don’t have that many individuals to meet the increasing demand in North America and around the world.

To try and bridge the gap, leading universities like Columbia University and Carnegie Mellon University (and about 28 others) started offering graduate programs and courses in data science and analytics. However, this is not an immediate solution, as educating and training these prospective data scientists takes time. According to a study conducted by McKinsey:

"By 2018, United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.”

Industries Experiencing a Shortage of Data Analyst Talent

Industries currently faced with a talent shortage cover a broad spectrum. Some of the industries struggling to meet the demand are as follows:

• Aerospace
• Finance
• Healthcare
• Insurance
• IT
• Pharmaceuticals

According to a study conducted by Accenture, about 90% of respondents stated that they wanted to hire staff with data science expertise. At the same time, about 41% (of more than 1,000 surveyed) cited a lack of talent is a significant hurdle. As senior managing director at Accenture Analytics, Narendra Mulani pointed out, “it will get worse before it gets better.”

At present, data scientists have a median experience of just six years. In the grand scheme of things, when compared to other occupations, this is really not much. These individuals are highly educated (48% PHD and 92% master’s degrees), and about 36% of them were born overseas.

Outsource Data Discovery to Meet the Demand

With fierce competition for available talent, companies have been forced to look for solutions elsewhere. Some hotbeds for outsourcing data analytical activities are the usual suspects in the outsourcing game:

• China
• Eastern Europe
• Philippines
• India

There is a wide range of tasks that can be outsourced, so depending on your security concerns; companies can outsource all of their significant data needs or just a tiny part of it. This can be anything from data collection and processing to data visualization to developing from scratch or customizing existing analytical platforms to Big Data staff training. Some of the other activities in the field are as follows:

• Data cleaning
• Data evaluation
• Data management and maintenance
• Data monitoring
• Visualization
• Cloud Big Data solutions

Before outsourcing any data analytics, a clear strategy should be developed as there is a high level of risk. Issues such as data security, confidentiality, and intellectual property rights can exist. So it is always best to approach Big Data with a clear strategy with the right partner in another part of the planet. A software development outsourcing vendor should also provide clear guidance and get guarantees on how the data will be stored, managed, and protected.

Destination Eastern Europe

There is a tradition of IT outsourcing technology-related tasks to India, but more and more companies are looking to the former states of the U.S.S.R., such as Ukraine, to fill the Big Data talent vacuum. The former soviet bloc is full of highly educated mathematicians and scientists that can be nurtured in an extensive data spectrum.

Hire talented developers from LATAM, Canada, and Europe

The head of global transaction banking at Deutsche Bank, Daniel Marowitz, has had first-hand experience with outsourcing data analytics. According to Marowitz, there is a lot of talent in Ukraine in particular. The financial conglomerate outsources to both India and Eastern Europe, and it is perceived that both regions are well suited for different things. He further stated that India was good at processing in a factory model (fast and cheap) while Eastern Europe was more suitable for more cost-effective experimental and innovative undertakings.

As the world gets smaller and better connected, the focus will remain on the speed of processing Big Data resources, filtering out the noise, and gaining insight through correlation. With the rise of the Internet of things (IoT), the influx of data is only going to increase, so expect this to be a hot topic for years to come.

Is your company experiencing a shortage of Big Data talent? If yes, how're you addressing the issue?