Is The Data Talent Crisis Here To Stay?

In last month's data talent report, we asked hundreds of enterprises about the trials and tribulations in the world of data talent acquisition and retention today. The results spanned different practices, working solutions, and tools but in general what we discovered was that across the industry there was (and very much still is) a shortage of data talent in every data, AI and anaytical role we asked businesses about.

More worrying still was the fact that this trend is showing no signs of abating anytime soon. A quarter of respondents stated that they expected to still have data scientist shortages in five years' time, with further shortages in both AI engineering (25%) and AI development (32%) also featuring on the list of concerns troubling enterprises today.

The Great Resignation it would seem is in full swing, and across the world enterprises are having to work harder and harder to find and retain the right talent. In the most basic sense, the cause of this shortage is that the demand for data roles far outstrips the supply, and this problem only looks to get worse still as 'AI everywhere' becomes a reality, with businesses continuing to invest heavily in artificial intelligence projects in order to stay ahead. Indeed, the skills needed to facilitate AI adoption and implementation will continue to grow exponentially, leaving many organisations wanting in regards to their data and AI needs.

When we spoke to Ansgar Koene, Global AI Ethics and Regulatory Leader at EY, last month about ethics, explainable AI and upcoming data regulation, he mentioned how entire "academic departments [are] being bought up by big tech companies, meaning that it is increasingly difficult to find the right skills needed to tackle the problems arising in terms of data protection and regulation".

Koene was talking primarily about governments struggling to meet the demands of the 21st century, but his comment hints at the wider reality facing businesss today — that enterprises across the board are unable to obtain the right talent and as such are seeing productivity issues across all levels of their organization.

The picture being painted does not look good. According to the International Data Corporation (IDC), there will be a shortage of 3 million data scientists by 2025, and research from Burning Glass Technologies found that there are currently more than 1 million open positions in the US alone that are for data scientists or analysts. To make matters worse, it is currently estimated that there are less than 50,000 academic data science programs worldwide, meaning that even if the will is there for society to have a larger pool of skilled data scientists to draw upon, what is still lacking is the way to do it.

50% of our respondents stated that the way they were working towards closing the talent gap was via upskilling existing tech workers. That said, 67% of CEOs still aren't satisfied with their workforce's preparedness for AI, meaning that it isn't clear if such a tactic will be particularly viable in overcoming the talent gap at all.

This was a problem that was seemingly a long time coming. According to a survey conducted by the Harvard Business Review (HBR), between 2001 and 2016 the number of companies citing lack of suitable skills as the main barrier preventing further investment in Artificial Intelligence increased by a staggering 450%. A more recent survey by Gartner found that 38% of the organisations they surveyed had limited or no AI implementation because of a lack of available employees with the required skill sets.

As our report showed, this is a serious issue for a vast majority organizations. 58% labelled closing the data talent gap "somewhat of a priority", while a quarter stated it was a "major" one. Only a lucky few (17%) stated it wasn't a problem at all.

If you find your enterprise isn't in that lucky minority, download our latest report, Bridging the Data Talent Gap, to discover some of the ways organizations are experimenting in order to navigate the increasingly barren sea of data talent today.