Before starting a ML project, ask whether ML is necessary of cost-effective.
Machine learning is an approach to (1) learn (2) complex patterns from (3) existing data and use these patterns to make (4) predictions on (5) unseen data.
When to use machine learning
(1) Learn
The ML system should have the capacity to learn, i.e., there must be something for it to learn from. In most cases, ML systems learn from data.
(2) Complex Patterns
ML solutions are only useful when there are patterns to learn. In contrast to traditional software which require specified patterns to calculate outputs, ML solutions learn patterns from inputs and outputs. However, the existence of a pattern might not be obvious, and the dataset / ML algorithms might not be sufficient to capture them.
(3) Existing Data
There must be data for ML to learn from. Some exceptional cases include:
- Zero-shot learning: It is possible for a ML system to have good performance without being trained on data for the task. However, the system is previously trained on data for other tasks (often related to the task of interest), meaning it still requires data to learn.
- Continual learning: ML models can be deployed without having been trained on any data; they will learn from incoming data in production. However, serving insufficiently trained models comes with risks (e.g., poor customer experience).
Without data and continual learning, one might consider “fake-it-til-you-make-it” approach, where a product is launched predicting based on heuristics (instead of ML models), with the hope of using generated data to train ML models later on.
(4) Predictions
ML models solve problems that require predictive answers. Several problems can be reframed as predictive problems, including computation-intensive problems (e.g., image denoising, screen-space shading).
As of 2024, I do not believe this should be considered an important factor in deciding the usage of ML in projects, as generative models such as ChatGPT and DALL·E are not exactly models that solve ‘predictive’ problems, yet they provide so many values and can be applied to various different tasks.
(5) Unseen Data
The patterns the model learns from existing data are only useful if unseen data also share the same patterns. In other words, unseen data and training data should come from similar distributions.
ML solutions are especially great at solving problems with the following additional properties.
(6) Repetitive
Most ML algorithms require many examples to learn a pattern. Repetitive task implies the repetition of the intrinsic patterns, making it easier for ML models to learn the patterns.
(7) Low Cost of Wrong Predictions
Since it is highly unlikely to develop a model with 100% performance, the model will make mistakes. Consequently, ML is suitable when the cost of a wrong prediction is low, or at least the benefits of correct predictions outweigh the cost of wrong predictions on average.
(8) At Scale
ML solutions require significant investment on data, computation, infrastructure, and talent. Such solutions would only benefit if the problem is at scale, i.e., a lot of predictions are made from the solution. This also implies that there is a lot of data to collect, which is useful for training and improving the model.
(9) Constantly Changing Patterns
Problems involving one or more constantly changing patterns makes hardcoded rule-based solutions become obsolete quickly. Updating the rules can be too expensive or even impossible. In contrast, ML solutions can be updated accordingly without such limitations.
When not to use ML
Unethical Problems
Simpler Solutions Suffice
Not Cost-effective
Note that if you wait for the technology to prove its worth to the rest of the industry before jumping in, you might end up years behind your competitors.