How Is AI, Data & Analytics Beneficial to Industry?

Respondent Breakdown

The data, AI and analytics leaders we surveyed come from all across the globe and represent a wide range of functions, industries and maturity levels.

Industry Break Down

Respondent Functions

Introduction

The modern world runs on data. From dictating what content, we consume to calculating mortgage rates to matching people on dating apps, data shapes not only our lives, but the societies in which we live. Likewise, major tech companies like Facebook, Apple and Google are spinning the 1.7 MB of data created every second into literal gold and growing into some of the most profitable companies this world has ever seen.

That being said, the vast majority of organizations are a far cry from that reality. While the COVID-19 crisis has pushed companies to accelerate their digital transformation and process automation efforts, many organizations were forced to put some of their most ambitious data science projects on hold as a result.

According to a recent KPMG study that examined how the COVID-19 crisis impacted data strategies, 73% of corporate executives surveyed agreed that “as organizations rushed to survive and thrive, most did not have time to think through the treasure trove of data they were creating for themselves.” 48% also admitted that their data strategies had fallen significantly behind their digital transformation deployments.

But that was 2021.

To shed light on how organizations are envisioning their 2022 data strategies we surveyed 200+ data, analytics and AI leaders to uncover:

• How data and analytics budgets have evolved YOY
• Top enabling tools, technologies and solution providers
• The emerging challenges and strategic priorities that will define 2022

What we found is that data budgets are growing, and organizations seem eager to use data in new, imaginative ways (more on that later). But those same organizations are struggling to lay the technical and strategic groundwork necessary to do just that.

For example, many of our respondents (44%) still lack an enterprise data strategy (chart 1). This is problematic for a number of reasons. Not only do comprehensive data strategies help organizations systematically prioritize and capitalize on their data assets, but they also play a critical role in ensuring data governance, privacy and security.

Without a clear vision and framework for enterprise data success, competing priorities, poor data quality and a lack of technical expertise will always stand in the way of progress. Organizations that fall behind the data maturity curve will not only risk obsoletion and financial failure, but are also more vulnerable to cyber-attacks, data breaches and regulatory/ethical violations.

Despite all these challenges, organizations of all types and sizes are starting to do remarkable things with data. As we enter the golden age of growth-oriented analytics, the industry is poised for incredible growth over the next year and beyond.

Do you currently have an organizational data strategy in place?

Reaping the Rewards of Enterprise Data

Though many organizations are not as mature as they hoped to be by now, that does not mean that they’re not already reaping massive rewards from their existing data and analytics projects.

While the first two were top choices 2021 as well, the third, “facilitating improved decision making across the entire enterprise,” is new and could potentially indicate a number of things: increased trust in the quality of enterprise data, improved data literacy amongst non-technical employees and expanded access to data.

In terms of where respondents see data and analytics adding the most value in 5 years (chart 2), the top 3 areas remained the same (efficiency, digital transformation and decision intelligence).

However, substantially more people selected the following:

1. Boosting overall competitive advantage (40%)

2. Improving financial performance (36%)

3. Identifying & generating new revenue streams (39%)

While they will always play a central role in measuring and optimizing operational performance, organizations are increasingly looking to leverage data and analytics to deliver top-line results.

Though this shift in mindset is certainly exciting, using data to build new products and revenue streams requires both a robust technical infrastructure and strong governance. Otherwise, you risk building low quality, ineffective data products that add little value to the business or the customer.

How are data and analytics currently adding value to your organization? What about 5 years from now?

Similar to 2021, most organizations, if not all, use a mix of internal and external/alternative data sources. Furthermore, the preferred data source remains internal enterprise data. On average, 62% of data is internal data vs 43% of external data (chart 3).

That said, how this split will change over the years is unclear. As companies embrace AI, ML and advanced analytics, the use of synthetic data will likely increase. In addition, the turmoil of the past 18 months (the pandemic, staffing shortages, supply chain disruptions, etc.) have rendered some enterprise data unreliable, pushing companies to invest more in alternative data sources that they might otherwise.

Regardless, internal, enterprise data will likely remain the lifeblood of organizational data strategies, especially as enterprises build out and fortify their data fabrics.

This all spells out increased budgets for data and analytics projects. In fact, 62% of our respondents said they expected their budget to increase over the next year vs. 56% in 2021.

ALSO READ: Key Examples of AI in the Public Sector

What percentage of your data comes from internal sources vs. external/ alternative sources?

• Internal – 62%
• External/Alternative – 43%

Data & Analytics Funding Priorities

With budgets on the rise, where exactly will the money go?

In terms of current tools and solutions, the top 3 areas of investment are (chart 6):

1. Cloud Data Warehouse Solutions (52%)
2. Data integration Solutions (47%)
3. Data Governance Solutions (44%)

The fact that cloud warehouse solutions was the top choice for the second year in a row is no surprise. Afterall, it is the cornerstone of the modern data ecosystem as are data integration and governance solutions. In turn, it makes sense that purchasing new solutions in these areas will become less of a priority going forward.

Looking towards the next 5 years, the top areas of investment are expected to be:

• Deep Learning Frameworks (46%)
• AI/ML Monitoring solutions (44%)
• Predictive & Prescriptive Analytics Solutions (42%)

What these numbers indicate is that companies are, first of all, looking to build larger and more powerful algorithms. Afterall, what differentiates deep learning from machine learning is that it contains 3 or more layers of neural networks. The larger and more complex these algorithms get, the more difficult it becomes to not only track performance but explain the logic behind outputs. Which takes us to our second largest area of growth: AI/ML monitoring solutions (44%).

It cannot be denied that AI is an experimental discipline and will continue to be for the foreseeable future. As organizations roll out more and more AI projects, they’ll be increasingly motivated to embrace AI/ML monitoring tools to not only optimize the ROI of AI but ensure explainability as well.

The third top area of growth, predictive & prescriptive analytics solutions (42%), was also a top area of interest in 2021 and we expect it will continue to be so for years to come.

Powerful and adaptable, predictive & prescriptive analytics are a core component of many of the most buzzed about applications of data science - personalization, content recommendation algorithms, predictive maintenance, smart supply chains, epidemiology, just to name a few.

Given its wide range of applications and high levels of strategic impact, this area is poised for tremendous growth. In fact, according to research by ReportLinker, the global predictive analytics market is expected to double over the next 5 years ballooning from $10.5 billion in 2021 to $28.1 billion by 2026.

In conclusion we should note that these three areas we mentioned (deep learning, predictive analytics, AI monitoring tools) are far from the only ones we expect to see substantial growth. In fact, when compared to 2021’s numbers, percentages increased across all vendor categories both in terms of current and future investments. What this suggests is that organizations are re-focusing their attention on data and analytics. In addition, instead of adopting one solution at a time, organizations are looking to build out multi-faceted, integrated data fabrics whereby multiple technologies work together in harmony.

What data solutions are you actively investing in?

Cloud Strategies for Actioning Data

Over the past decade, cloud computing has taken over the digital landscape. In fact, most studies, including this very research report (chart 7), find that many companies have already adopted cloud services in one way or another.

For the lion’s share of organizations, cloud has proven to be a cost-effective, secure, scalable and reliable way to store enterprise data, especially compared to the standard corporate data center model. However- of course- the technology is not without its drawbacks.

Public clouds, which can be accessed by anyone, anywhere as long as they have the right credentials, can be vulnerable to security threats. In fact, 36% of our respondents say that addressing cloud security is a top strategic priority for 2022 (chart 9). Furthermore, all clouds, whether private or public, do occasionally fail. Human error, networking issues, natural disasters and power blips can all cause cloud outages. Even extreme spikes and surges in usage can significantly impair cloud performance, a frequent occurrence during the early days of the pandemic.

As such, organizations are realizing that they shouldn’t put all of their eggs in one data storage basket. To bolster resiliency and reduce latency, organizations are embracing hybrid cloud strategies that involve a combination of public, private and on-prem data storage solutions. In fact, 39% of our respondents stated they’ve embraced a hybrid cloud strategy, making it the most popular cloud strategy by a large margin.

While some of the 15% who selected on-prem will likely move toward the hybrid model in years to come, this certainly won’t be true for all. As on-prem offers organizations more control over and simpler, faster access to internal data, many, especially those that deal with highly confidential and/or regulated data (i.e., defence, national security, life sciences, etc), will likely stick with at least some percentage on-prem data for the time being.

As with any technology, there are hidden costs associated with the cloud (i.e., hiring new team members, security add-ons, data egress fees, confusing pricing schemes, etc.) that can offset savings. As such, it’s no surprise that 57% of our respondents said that optimizing cloud costs was a top priority over the next year (chart 8).

All in all, though, more than half of our respondents (54%) said they expect their cloud budgets to increase (chart 9) and 57% say that migrating more workloads to the cloud is a top strategic priority (chart 8) for 2022. With that in mind, it’s safe to say that cloud adoption will continue to grow for the foreseeable future.

What are your top 3 strategic cloud priorities for the next year?

Do you expect your final, approved budget for cloud management solutions to increase over the next year?

What percentage of your budget do you expect to devote to cloud management solutions over the next year?

Data Visualization & Virtualization

For organizations looking to democratize data and establish a “single source of truth,” data visualization and virtualization go hand-in-hand. While data virtualization software pulls together structured and unstructured data sources into one central viewing location, it is data visualization that makes these insights comprehensible to the user.

Though data visualization remains a top investment priority and strategic objective heading into 2022 (see chart 15), only 48% of respondents said they expect their budgets for data visualization tools/solutions to increase (chart 12A) vs. 56% in 2021. There was also a slight increase in respondents who said they expect their budgets to remain the same (24% vs 20% in 2021) and those who expect it to decrease (3% vs. 2% in 2021).

As organizations collect more and more data, many businesses are finding that their BI dashboards have s imply grown too complex to be usable. Furthermore, they’re also learning that simply equipping people with data is only half the battle. Ensuring data consumers can effectively apply those insights to real-world scenarios has proven to be challenging.

ALSO READ: Advanced Analytics 2021 Survey

What percentage of your budget is allocated towards data visualization and virtualization solutions?

Instead of stagnant dashboards, organizations are looking to build interactive, self-service data platforms that allow the user to customize what data they see and how. Instead of a collection of charts and graphs, organizations are now trying to build data stories and narratives.

In doing so they’re exploring and embracing the next wave of data visualization tools such as real-time data visualizations, AI-powered data visualization tools, video visualizations and 3D rendering tools. As such, the market for data visualization tools is expected to grow by more than 11% annually, from $5.9 billion in 2021 to $10.2 billion in 2026.

Of course, behind the scenes, data virtualization tools will continue to play a central role in securely and efficiently aggregating all of the data needed for such undertakings.

While a sizable portion of our respondents (43%) expect their budgets for data virtualization tools and technologies to increase over the next year (chart 12B), we do expect this percentage to grow over the next 5 years as organizations look to further capitalize on advanced enterprise data and analytics.

Data ROI & Monetization

In what ways does data deliver value at your organization? Does it help you augment operational performance and reduce costs? Does it enable you to build better, more profitable products? Is your enterprise data a financial asset within itself?

In the past many data strategies were designed to protect and preserve data as well as ensure regulatory compliance. Though all of those things are still major priorities, data strategies are increasingly becoming ROI-centric. Now, in addition to effectively managing data, organizations are looking to use enterprise data to obtain quantifiable economic benefit whether by leveraging it to increase operational efficiency or to identify/exploit new revenue channels.

For many chief data officers (CDOs), their ability to transform data into tangible business results will not only make or break their careers, but also determine the fate of the entire data science practice. Given the high stakes, organizations are more invested in data ROI and monetization than ever before.

In fact, 48% of our respondents expect their budget for data ROI tools and solutions to increase over the next year (chart 13). This represents a sizable boost from 2021 where only 40% said they expect their budgets to increase.

Furthermore, the percentage of budget allocated towards these solutions is also increasing. While only 21% of our 2021 respondents said data monetization tools and services accounted for 11% of their budgets or more. This year, 30% of respondents expect to devote 11% of their budgets or more.

Do you expect your final, approved budget for data ROI/monetization to increase next year?

What percentage of your budget is allocated towards data ROI & monetization tools & services?

Advanced Analytics

If transforming data into revenue is the destination, advanced analytics is the vehicle that will take you there. While standard data analytics techniques show you what happened yesterday, advanced analytics tell you why, what’s happening now and what will likely happen next.

Heading into 2022, the vast majority of respondents have already made sizable investments and are currently leveraging a number of advanced analytics techniques. As established before, data visualization (57%) has already been widely adopted as has data mining (49%) (chart 15), indicating that many organizations are already well entrenched in advanced analytics.

Data mining, also referred to as knowledge discovery in data (KDD), combines machine learning, automation and advanced statistical techniques to identify patterns and extract knowledge from large amounts of data. It is especially adept at identifying anomalies. For example, financial institutions use data mining to help them identify suspicious behaviour that could indicate fraud or other criminal activity.

At this point, most data mining projects focus on structured data such as customer records, IT performance data and sales data. However, as unstructured data processing tools like computer vision, audio analytics and natural language processing (NLP) evolve, we expect this area to evolve as well, if not significantly grow.

What types of advanced analytics are you currently leveraging? And which do you plan to adopt over the next year?

Do you expect your final, approved budget for advanced analytics to increase over the next year?

The third most widely adopted area of advanced analytics is big data analytics (46%).

Though the term is often misused, big data analytics refers to the application of advanced analytics techniques to large, high velocity and hard-to-manage volumes of data - both structured and unstructured data.

However, bigger isn’t always better, especially when it comes to data sets. Managing, cleansing and analysing high volumes of data is not only technically difficult and demands a high level of expertise, but also extremely expensive as it requires enormous amounts of compute power to function. This is why 85% of big data projects fail to deliver anticipated business results.

In an effort to manage high volumes of data more effectively in motion, organizations are increasingly turning towards complex event processing (CEP), a set of technologies that can process and analyse massive streams of data in-motion in real-time. In fact, 42% said they planned to invest in CEP, making it the top area of future growth (chart 15).

As it allows organizations to query data before it enters the database, CEP solutions help organizations overcome many of the challenges associated with big data as well as open the door to new, enhanced real-time analytics capabilities.

One of the most common applications of CEP is real-time cyber security analytics, technology that enables the identification of cyber threats as they occur. Another common use case is personalized marketing on e-commerce sites. The second a known user enters an e-commerce site, the CEP produces real-time, personalized product suggestions based on that person’s previous shopping history, geolocation data, time of year, social network activity, demographic data, and whatever other information they might have on tap.

What percentage of your budget is allocated towards advanced analytics solutions and services?

However, the exploitation of large, fast moving data sets is not the end-all-be-all of advanced analytics. Similar to 2021, many of our respondents are increasingly embracing “small data analytics,” mature techniques that deliver high levels of strategic value but require fewer computing resources than “big data.”

In fact, 40% of our respondents plan to adopt Cohort Analysis in 2022, making it the second largest area of growth. Instead of analysing all data at the same time, Cohort Analysis divides data into categories or groups first. Instead of treating data as a monolith, it allows organizations to filter out the noise and drill down on the areas that matter most to them.

For example, instead of analysing the entirety of their customer base at once, a large retailer might use Cohort Analysis to analyse customer behaviour by age, gender, location, the day/time they visited the store, past purchasing history and so on.

Another top area of growth is Multivariate Statistics (38%), methods that examine the simultaneous effect of multiple variables. For example, an airline company may use multivariate analysis to determine how various factors - such as different fare prices, advertising spend, aircraft selection, amenities, scheduling, etc. actors - impact revenue per flight.

When properly executed, Multivariate Statistics not only deliver richer, but more accurate results also than standard univariate techniques (which only analyse one variable at a time), they do so more efficiently.

Though the innovations we just mentioned (CEP, Cohort Analysis and Multivariate Statistics) are the top areas of future investment, those are far from the only areas of growth. In fact, compared to 2021, we saw significant gains across all types of advanced analytics we inquired about, both in terms of current and future investments. Furthermore, over half (53%) of respondents expect their budgets for advanced analytics to increase over the next year (chart 16).

What this proves is that organizations of all sizes and industries are finally taking their advanced analytics strategies to the next level. While just a few years ago, technologically sophisticated solutions like CEP and machine learning were only used by the most cutting-edge companies, these tools are rapidly going mainstream. In essence, every company is becoming a big data company.

That being said, what “big data” and “the next level” of data analytics capabilities looks like varies greatly from one organization to the other. As organizations build out their own unique data fabrics, it will likely usher in a new era of experimentation and innovation.

Artificial Intelligence

Though many of the artificial intelligence (AI) techniques currently in use have existed for decades, applied AI technology is only just now going mainstream with companies rapidly looking to adopt. Case in point, in less than a year our respondents have already climbed a few rungs up the AI maturity ladder with 60% saying they’re at the exploratory level or above (chart 18) compared to 52% in 2021. Still, only a very small percentage, 4%, has reached the top, transformational level.

While major tech players such as Facebook, Apples and Google have already invested billions into AI, many organizations are still years away from embarking on their first AI project. Given the expense, technical infrastructure and expertise needed to do so, this makes sense.

However, organizations that master the AI puzzle now will likely gain a significant competitive advantage over those who don’t, and this gap will only widen in years to come. This is likely why over half of our respondents (52%) are increasing their AI budgets and only a small minority (6%) are not investing in AI at all (chart 20).

The COVID-19 pandemic has undoubtedly shaped where AI is currently being adopted and where it will be applied next. There are few areas where this is more visible than in customer service. Over the past year and a half, many contact centers have struggled to meet their KPIs due to extreme fluctuations in demand, staffing shortages and, for all too many, an increase in aggressive customer behaviour.

Where does your organization fall on the Artificial Intelligence Maturity Curve?

In what areas/ways is AI technology currently adding value to the business?

However, AI-powered solutions such as chatbots, automated social media responses, agent decision support tools, sentiment analysis and computer vision for object/issue recognition, amongst others, have proved to be invaluable. In addition to taking over some of the tasks a human may have performed in the past, many of these solutions not only enable human workers to focus on the most important and often meaningful customer interactions but enable them to do so faster and more effectively than ever before. Not only does this increase the customer experience, but the employee one as well.

While 44% of our respondents say that AI is currently delivering value in the customer service space, when asked where AI will deliver the most value over the next 5 years, 50% selected AI as one of their top choices.

Do you expect your final, approved budget for applied AI solutions to change over the next year?

What percentage of your budget is allocated towards applied AI solutions and services?

Another area where AI’s value proposition is evolving is “improved strategic decision making” with 44% of people saying AI is already delivering value in this area and 61% selecting it as a top area of growth over the next five years.

In an increasingly complex and data intensive world, making sound, accurate and evidence-based decisions has never been more important. Unlike humans, AI can accurately analyse vast amounts of data in seconds. Though AI can’t always replace human decision making, it can certainly enhance it. Some experts even predict that AI decision making will likely boost economic output by $13 trillion 2030, which is an extra 1.2% of global GDP.

A third area where AI is and will likely continue to add value is increased business process efficiency. While 44% of respondents are currently using AI to optimize business processes, 61% expect to do so in the next 5 years.

At this point most, AI optimizes processes in two ways:

• Smart processes

The use of AI to integrate, orchestrate and scale disparate systems (i.e., intelligent automation and hyper automation)

• Advanced process analytics

The delivery of deeper, more actionable process insights (i.e., simulations, automated process discovery, digital twin technology and advanced process mining).

As companies further digitize their processes, applying AI to optimize them will become increasingly viable. Eventually “smart” processes will evolve into self-healing and self-modifying processes capable of identifying and repairing performance issues on their own.

Data Democratization

The last mile of creating a truly data-centric organization is data democratization. By putting data in the hands of non-technical employees and training them how to use it, enterprises can build an army of citizen analysts capable of making better, more informed decisions.

Given its substantial transformational potential, data democratization continues to be a major strategic goal for our respondents (chart 24). In addition, 44% of our respondents say that they expect their budget for data democratization to increase (chart 26).

That being said, many companies are a long way off from achieving their data democratization goals. In fact, 46% of respondents said less than 10% of their workforce had access to data and analytics. Only a mere 9% said that more than 50% of employees had access to enterprise data and analytics (chart 23).

ALSO READ: Democratizing Data & Analytics

What percentage of your company has access to using or actioning enterprise data & analytics?

Is increasing enterprise access to data and analytics a strategic priority for your organization?

What percentage of your budget is allocated towards data democratization tools & services?

Do you expect your final, approved budget for data democratization tools to increase over the next year?

Achieving true data democratization will likely take most companies years to achieve as it requires robust, integrated data architecture, vigorous data governance and substantial training. However, data democratization is the critical final step in scaling data analytics. Achieving much less maximizing ROI is impossible without it.

Read the Report Here