Why Machine Learning Projects Fail Part 2

December 27, 2021 | By Rahul Surasinghe

machine learning failures

Voiced by Amazon Polly

machine learning projects

Machine Learning (ML) has greatly improved the insights organizations can gather from their data. It’s not easy, though. ML implementations are incredibly complex and require a lot of moving parts working together harmoniously to produce meaningful results. So much can go wrong that 85% of all ML projects are predicted to fail. ML projects can fail for the same reason that other projects doincluding poor support mechanisms and insufficient requirement definition. But more often than not, ML project failure occurs when organizations fail to identify key process gaps from the very beginning. From the onset of any ML project, organizations should:

Define the business problem first

The greatest challenge to ML implementation is strategy. The benefits of ML can be such a strong motivator that it makes organizations implement projects without the proper planning to make them successful. In other words, some organizations buy a hammer before knowing where or what to nail. But before implementation can begin, organizations must define and understand the problems they are trying to solve and the business value that ML will bring to them. Organizations often simply aren’t sure why they want an ML practice or are rushing to adopt ML practices before their business is ready for it. This also stems from a lack of shared vision, seamless team-level collaboration, and an effective ML adoption & implementation blueprint.

In such cases, developers are tasked with designing and coding solutions that do not add value to the business or match its long-term goals. On the contrary, ML requires an understanding of business, strategy, and consumer needs to develop a focused approach towards the development of domain-specific models to operate effectively. In the absence of goals and an efficient adoption blueprint, development and data science teams can waste precious resources in creating models without the right objectives.

machine learning projectsIdentify Technical Gaps

ML initiatives rely on resource-intensive infrastructure typically hosted on platforms that not only require specific skills and experience to operate but which can also incur substantial recurring costs. Ironically, the costs of maintaining poorly planned ML frameworks that fail to identify technical gaps, in the beginning, can raise costs and harm operational efficiency.

As such, organizations can often greatly miscalculate their financial projections and fail to account for challenges due to various unknown factors. Those can include system entanglement, unstable data dependencies, and dead experimental code paths. If left unchecked, the project may turn out to increase technical debt and rely on inadequate infrastructure capacity to operate an ML project.

Understand the Importance of Specialized Skills (and their low supply)

machine learning

While ML and pattern recognition technologies have been in existence for a while, the recent spike in ML adoption has created a supply gap of specialized skills necessary to

handle the underlying tools and technologies. As unskilled resources are more prone to introduce manual errors into the system, organizations should include both replacing manual efforts with automation and reskilling employees as key criteria for long term ML project success.

As a result, professionals with legacy skills are required to undertake extensive training to adopt skills before designing and developing ML-driven applications. Factually, this aspect remains one of the most critical challenges that firms face as they struggle to attract the right talent or upskill their existing pool of resources, creating an entry barrier into the ML field.

Conclusion

ML projects are hard and often end in failure. But organizations can take steps during their ML project planning that can help ensure they ultimately return meaningful results. They must understand where their gaps are in strategy, staffing, and technical support all.

This was the second in a series of articles dedicated to discussing the fundamental challenges ML projects face and the steps organizations can take to give themselves a greater chance of success. This first focused on how problems with data can doom projects to failure.

Explore part 1 of Why Machine Learning Projects Fail


FAQ’s

Why do we need machine learning?

Machine Learning (ML) helps organizations across a wide range of industries make critical decisions with insights gained from their data and its discoverable patterns. Most enterprises have far too much data to examine for them to do it with human eyes. ML leverages mathematical algorithms for analyzing and interpreting patterns in data, which over time enables learning, reasoning, and (hopefully) better decision making.

What problems Cannot be solved by machine learning?

Machine Learning under the hood is mathematics, specifically, probability, linear algebra, and statistics. Therefore, the type of problems that ML can’t solve include problems that have inadequate datasets, poorly scoped problems, and high uncertainty.

ML utilizes algorithms to find patterns within datasets. Therefore, when there isn’t enough abundance or quality data — no matter what question is attempted in being solved — ML will not be able to solve the problem.

Furthermore, a poorly defined or vague question will lead to an unsolvable solution. However, this is universal to any tool or process, not just ML.

Finally, there are some problems where ML algorithms will have a high uncertainty regarding being accurate. These types of problems can be a consequence from low-quality data or ambiguous questions. But also, these types of problems can arise from cutting edge research where researchers are trying to draw conclusions between two unrelated factors such as the correlation between gray matter and psychiatric disorders. These types of research-based questions are speculative and ML may not be able to produce an accurate model.

Why do most machine learning projects fail?

From Gartner: https://www.gartner.com/en/newsroom/press-releases/2020-10-19-gartner-identifies-the-top-strategic-technology-trends-for-2021

Gartner research shows only 53% of projects make it from artificial intelligence (AI) prototypes to production. CIOs and IT leaders find it hard to scale AI projects because they lack the tools to create and manage a production-grade AI pipeline. The road to AI production means turning to AI engineering, a discipline focused on the governance and life cycle management of a wide range of operationalized AI and decision models, such as machine learning or knowledge graphs.

AI engineering stands on three core pillars — DataOps, ModelOps and DevOps. A robust AI engineering strategy will facilitate the performance, scalability, interpretability and reliability of AI models while delivering the full value of AI investments.

How can enterprises avoid failing at machine learning?

Organizations can avoid failure with ML by investing time in the following areas: business strategy, preparation, and adopting a ML lifecycle management such as ML Ops.

ML is a great tool. However, a single tool can’t solve every problem. Therefore, one’s business strategy should account for which problems ML can’t solve and ensure that the outcome of their ML initiative provides business value.

Gartner Says Nearly Half of CIOs Are Planning to Deploy Artificial Intelligence
https://www.gartner.com/en/newsroom/press-releases/2018-02-13-gartner-says-nearly-half-of-cios-are-planning-to-deploy-artificial-intelligence

Why machine learning strategies failhttps://venturebeat.com/2021/02/25/why-machine-learning-strategies-fail/

Gartner Identifies the Top Strategic Technology Trends for 2021
https://www.gartner.com/en/newsroom/press-releases/2020-10-19-gartner-identifies-the-top-strategic-technology-trends-for-2021

What Does Technical Debt Mean?
https://www.techopedia.com/definition/27913/technical-debt

Who We Are & What We Do

As passionate technologists, we love to push the envelope. We act as strategists, practitioners and coaches to enable enterprises to adopt modern technology and accelerate innovation.

We help customers win by meeting their business objectives efficiently and effectively.

icon         icon        icon

Newsletter Signup:

Join tens of thousands of your peers and sign-up for our best technology content curated by our experts. We never share or sell your email address!