Why do Business Analytics Projects still fail in 2021?

Table of Contents

The hype of Big Data Analytics has stormed by now quite a lot of Businesses. Only by looking at the amount of literature generated around the topic you can figure out how relevant has become.

Source: Knowledge Graphs and Big Data Processing by Valentina Janev, Dea Pujić, Marko Jelić, Maria-Esther Vidal1

Companies quests for 360 customer view, next best action algorithms, self-service BI, AI and alike Analytics Projects had mostly failed.

Based on a prediction from Gartner2, through 2022, only 20% of analytic insights will deliver business outcomes.

But what are the reasons for what a well funded, sponsored and envisioned project can fail to leave companies not only with a few million less in the bank but also with an empty promise?

Unrealistic Expectations

Analytics requests often include “mission impossible” questions that not even God is ready to answer. Linked to that, companies’ immediate need for seeing returns on their analytics projects concludes with upsetting results. As much as Data & Analytics is a powerful tool, it needs time and patience.

Half of surveyed Senior leaders think that Marketing Analytics did not have the expected influence for the business. Source Gartner3

There is no magic button that you can press and tell you what to do with 100% of accuracy. If that would have existed, the COVID pandemic, for example, shall have never happened.

Weak Promises

We have the system that your company need; this is the future. These are common vendors claims. The fact is that they need to sell and they don’t doubt to offer you something that they know will not work.

Big Data” and other buzzwords created by sales/marketing/BI folks have completely bamboozled companies into creating new teams and positions. “Big Data” is not just a large data set. Around 90% of the professionals pitching Big Data and Social Analytics do not have a grasp of this area.

Vijay Gupta for KDNUGGETS4

Overengenieered, unusable products are now part of companies infrastructure. They were expensive, they don’t work, and it will be even more costly to dismantle them.

Lack of Methodology

The fact that a company have the money, the technology and even the people doesn’t mean that they will succeed using analytics. That is particularly true for corporations with complex structures. The disconnection between functions, divisions, departments, countries, time zones and most crucially, objectives is widespread.

This part is were tools like the GIDAR Analytics Canvas can help put everyone on the same page.

Companies are Not Ready

That includes the large group of:

  • We don’t have the skills
  • We don’t have a budget
  • We are busy with other priorities.

And probably the worst one, We don’t have data!. (Yes some companies haven’t collected and sorted their data and yet want to do AI).

Image of a chart with Top Reason why analytics is not used in Business
Conflicting course of action and Data quality are big problems at the time of using data to inform decisions. Source: Gartner5

According to Venturebeat only one out of every 10 Data Science projects actually makes it into production6.

Change Management

We spoke about technology, processes, and other known issues, but let’s not forget that is people who do analytics in the end. In many cases (self-service BI is a flagrant one), employees are thrown into a buggy system containing wrong and incomplete data and are asked to solve all company problems.

Employees stress levels based on Change parameters. Source: Gartner7

Companies must understand that we are in a speedy period of humanity and that people need time and support to embrace change.

Closing Note

The Big Data Hype will not go away anytime soon. It will probably be substituted by other buzz words like Artificial Intelligence or Machine Learning and vendors will still market it as Saint Grial: The capability that will make your company take over the rest. 

Make sure that you have the right foundations (Goals, Culture, Skills) to make your next analytics projects a success.

Santiago Tacoronte

References:

  1. https://link.springer.com/chapter/10.1007/978-3-030-53199-7_9
  2. https://blogs.gartner.com/andrew_white/2019/01/03/our-top-data-and-analytics-predicts-for-2019/
  3. https://www.gartner.com/en/marketing/insights/articles/gartner-marketing-data-analytics-survey-2020-analytics-fail-expectations
  4. https://www.kdnuggets.com/2014/02/comments-why-your-company-should-not-use-big-data.html
  5. https://www.gartner.com/en/marketing/insights/articles/gartner-marketing-data-analytics-survey-2020-analytics-fail-expectations
  6. https://venturebeat.com/2019/07/19/why-do-87-of-data-science-projects-never-make-it-into-production/
  7. https://www.gartner.com/en/corporate-communications/insights/change-communication

The D in GIDAR Analytics: Data

After defining clear Goals and collecting as much information as possible (clarifying key questions), you will have a good idea of the data that you need.
Here you want to be methodical. That is the technical part that requires extracting data, access to data sets, databases, build data models and data cleansing. That means that even if you or your team are not in charge of this part, you need to prepare a full brief.

What data do you need?

In the previous step, information, you should have collected the key questions to answer. They are the starting point for the data collection.

You need to write down the questions if you haven’t already and figure out which data you need to answer them.

Don’t be shy, take this initial list as a wish list, so put everything you think could help answer your key questions. 

How to collect and store the data 

In a spreadsheet, list all the questions, data sources and fields, you need to answer the key questions you gathered. 

The next step is the data gap analysis. For this section, we can use any validated gap analysis technique. If you want to keep it simple, you can start with, availability, accessibility and DQI (Data quality Index).

  • Availability refers to the mere existence of the data; do we have competitors data? It is usually a binary result, either yes or no.
  • Accessibility is a step after confirming availability, and it answers the question, can we access the data? Here you might get yes, no or yes but. For example, yes, but we cannot use specific fields due to privacy.
  • DQI or Data Quality Index. That is a quality of measure for the data we will use, and it answers the question. Can we rely on this data? DQI can become complicated, but I recommend you to make sure that the information is accurate, complete and unique if you want to know what these mean have a look into the Introduction to Business Analytics course. And if you don’t have much time, look at the data quality charts presented here.

Examples of Data points

  • Examples of Data  for “Jumpy Shoes”:
  • Total sales of “Jumpy Shoes” by hour of the day, day, week, month, quarter of year.
  • Total sales of “Jumpy Shoes” by model
  • Total share of “Jumpy Shoes” sales by model.
  • Customer that bought “Jumpy Shoes”
  • Demographics
  • Psycographics
  • Price history of “Jumpy Shoes”

In a more DATA friendly format:

ModelDateTimePriceSizeChannel  Oder IDCustomer ID
Shoe Gx20/03/20201:23pm76.538Web1214914Customer1
Shoe Gy24/03/20209:34m4942In Store85895798Customer2
Example of a table with sales data
Customer IDAgeGenderPost CodeNumber of PurchasesLast PurchaseCustomer Lifecycle Value
Customer124Female4057319/01/2020146.74
Customer244Male4914120/05/201956
Example of a table with Customer data

Final Remark

By the end of this step, you should have collected and prepared the data you will use for the analysis.

The (first) A in GIDAR Analytics: Analysis

Data analysis refers to the identification of patterns in data to extract conclusions and insights.

Data analysis is usually the role of a data analyst who will perform the activities needed to provide with actionable intelligence. Frequent activities done by a data analyst are data cleaning, data mining, statistical analysis, diagnostic analysis, regression analysis, data visualization, text analysis, cluster analysis.

Some of these are specialities of Data science and some others are more traditional data analytics techniques. Regardless, the objective of this step is to provide with insights that can be turned into actions (next step of GIDAR).

Elements of an actionable Data Analysis

Clear Goals

As stated in the first step of the method, Goals are intrinsic to businesses and they need to be clearly defined. Everyone involved must clearly understand what is that we are trying to achieve.

Image of a ball in the net after scoring a goal
Make sure everyone is clear on where is the GOAL

Lack of clear goals is a frequent issue with data analysis and one of the main reasons of futile efforts.

Analysis Scope

Also, a frequent source of trouble is the scope. At the beginning of the data analysis, we need to objectively define what are the boundaries. What are the areas we are going to cover? What is excluded? What are the metrics and KPIs that you will use? Which countries are included?

Analysis Methodology

The data analysis methodology refers to the set of techniques and conditions that you will apply on top of the dataset. You need to declare the data analysis process that you followed. A good methodology includes:

  • Category of the data analysis: Descriptive Analytics, Predictive Analytics or Prescriptive analytics. You can have more than one.
  • Analysis technique: data mining, machine learning, statistical analysis.
  • Name of the data analyst.
  • Date ranges for the analysis and data.
  • Sources of data: Data Warehouse, Salesforce, Google Analytics, Shopify, etc.
  • Collected data: Ecommerce sales, customer purchases, lead report, campaign X.
  • Caveats or known issues: Excluded Data, missing countries, seasonality, etc.
  • Data quality: Is the data complete, accurate and unique?

Data Quality

Data quality is a common problem at the time of believing in the conclusions of a Data Analytics project. It is frequent that someone will raise the flag of non-reliable data along with the project (especially if they don’t like the outcome).

TO avoid this common pitfall is important to invest time on data quality. If you are planning to recommend changes you have to make sure that your conclusion is reliable.

Partnering with data engineers is always a good way, they can analyze the quality of the collected data.

Data Analysis Technique

Depending on the category of the analysis, descriptive, predictive or prescriptive analytics you will need to pick one or another analysis method as well as the analysis tools.

Image of python code used for data analysis
Python has become the main analysis tool for advanced analytics

For advanced analytics cases (prescriptive and predictive analytics) is common for a data scientist to use data analysis methods like data mining, machine learning and tools like Python or R.

There are a lot of good articles, discussions and even job opportunities around data science at the website Analytics Vidhya.

image of a dashboard used for data analysis
Dashboards are very useful to analyze data, not so much to present results

For descriptive analytics, the data analysis technique is more exploratory data analysis, descriptive statistics with tools like Microsoft Excel, Tableau or Power BI.

Analysis Output

Now that you have collected and analyzed quantitative and qualitative data is time to put it into a format that can be digested by everyone. I say digested because is common to see reports that are only understandable by the creator and that is a huge mistake. The report should be self-explanatory and contain clear insights.

The template below is from

Image of a data analysis presentation
Image of a data analysis presentation
Image of a data analysis presentation

How to produce a strong Data Analysis with Actionable Insights

Go back to your Goals

It is easy to get lost in the sea of DATA. Have a copy of your goals always in front of you. It is your north star and will put you back on track when you get lost.

Look at the Key questions

At the end of the step information (useful information), you should have collected a set of key questions like Why customers buy this or that? How can we measure the quality of the leads? What is the impact of a change in price?. These are good pathfinders for your data analysis.

Blend information with Data

Take the questions above and answer them with data or use the information to filter your data. For example, if you know that your sales at Christmas are always excellent you might want to exclude that period from the analysis.

Choose the right Analysis Technique

Chose the right data analysis tool, not all are born equal. If you are going for statistic, make sure that your dataset has statistical significance. Your data analysis skills could worth nothing if you choose the wrong methods.

When using visuals avoid complex visualisations like a scatter plot. They are very powerful but some get confused.

Dashboards are very good to monitor but for analysis, it is better to have single charts with captions. Less is more!

Master the Output

A great data analysis contains a scope, methodology, executive summary, visuals, captions, and conclusions.

Once you have all inside give it a beautiful final touch. People are more likely to pay attention if your output is nice. Make your conclusions and key charts with big and bold letters. If your output is PowerPoint use the 10, 20, 30 technique. It never fails.

Conclusion

There are many more things to say about data analysis, it can get dense, complex and long but it can be much lighter and easier if you rely on the previous 2 steps of the framework. Focus on your goals and use the information.

The G in GIDAR Analytics: Business Goals

The G in GIDAR: BUSINESS GOALS

Goals are intrinsic to any business, even if we do not explicitly declare them, there are always goals behind any activity. A known problem in Business is, however, the inability to articulate Business Goals properly.

Table of Contents

Business Goals are the first and essential step in the entire GIDAR framework. Without purpose, there is no project. If after your stakeholder’s meeting or Business Goals Workshop you don’t have a clear understanding of what you are trying to achieve, you shouldn’t probably be spending resources. In that case, it would help if you revisited your company vision, mission and strategy.

The best way to set a successful Goal is to use SMART goals.

Business Goals must be:

● Specific: If possible numeric

● Measurable: You should be able to track it

● Achievable: Must be something doable (Avoid x10’s)

● Relevant: Have to bring a positive impact.

● Time-bound: Must have a beginning and an end in time.

One more thing your goal should contain is as much as possible a baseline. If you are to increase sales, you should know where your sales were last year or last month.

The more thoroughly you define these steps, the more likely you are to succeed.

Examples of poorly defined Analytics Goal:

● Increase customer NPS (Net promoter Score)

Example of well defined Analytics Goal:

● Increase average customer NPS from 4 to 25 – Specific

● We will use the customer surveys to obtain the data and process it into a monthly report. – Measurable

● We are confident to reach 25 because that is the industry average. – Attainable

● Customers with NPS above 20 spend, on average $100 more per year. – Relevant

● The analytics part of this project will be completed between 20 and 25 days. After delivering Insights and recommendation, we expect four months to implement actions and another 60 days before we measure the results. – Time-bound

Examples of poorly defined Analytics Goal:

● Increase sales

Example of well defined Analytics Goal:

●Increase sales for “Jumpy Shoes” Category by 5% from 10,000 units/month to 10.500 Units month. – Specific and with a baseline

● We will use the ERP data to measure the increment and Marketing Data to assess the campaigns’ impact- Measurable.

● The 5% is based on the industry forecast for these products next year and the competitive market share.- Attainable

● “Jumpy Shoes” category A represents 20% of our global sales. Moreover, a 5% increase will enable the launch of a new shoe category – Relevant.

● We will measure results on a bi-weekly basis during the next six months. There is the first milestone of +2.5% on month three and a final report after month 6.

Note how we have gone from a broad question to a more specific query. When you have worked out such detailed Business Goals, it is more evident for all parties what they are trying to achieve. 

Business Sponsor and Analytics teams can now move to the next phase of GIDAR, Information.