Casey Rovinelli

The first rule of machine learning: Do not start with machine learning. Unfortunately, to a hammer, everything looks like a nail. Whenever I get a call from a company to help them, they have already made their mind up: they want to use machine learning. It's always tempting to start the conversation there. Yet, machine learning is usually not the best way to solve most problems. Here is a better approach: Step 1 - Start with simple heuristics Heuristics is a fancy word. I prefer to call them "simple rules." You should start most problems by writing simple rules. It may not be perfect, but it's a better solution than jumping straight into machine learning. There are two significant advantages to this approach. First, you'll learn much more about the problem you need to solve. Second, you'll have a baseline to compare against any future machine-learning solution. For example, imagine you are building an online store and you want to show your users a list of recommended products. Instead of thinking about machine learning, start with a fixed list of recommendations. You can upgrade that to a list sorted by popularity. You can solve this problem in many ways before thinking of building a model. Step 2 - Replace heuristics with a simple model There's a point where you don't want to keep adding complexity to your rules. Maintaining a simple machine-learning model is easier than a codebase of complex rules. When your rules get out of hand, train a simple model and replace your heuristics with it. People usually recommend starting with simple algorithms like linear regression or decision trees. This makes sense, but simplicity doesn't always refer to the algorithm but how easy it is to use. For example, a ResNet-50 model is a complex convolutional neural network. But you can download a pre-trained version and start using it fast. Step 3 - Increase complexity There's a point where your simple model can't give you better performance. Here is when you should start exploring more complex solutions. Sometimes, a better model can improve your results. Sometimes, you need to combine a few models together. If I've learn one thing over the years, is that simplicity always wins.