Simple Model Choice Checklist

1. Be clear about the problem

  • What are we trying to predict or find?
  • What does “good enough” look like?
  • What happens if we get it wrong?

2. Start with the simplest thing

  • For predictions: try a small, simple model first.
  • For search: try keyword search (BM25) with a few improvements.
  • Build something fast and easy.
  • If the simple version hits the goal → stop. You’re done.

3. If the simple model fails, ask why

Bad data?
Fix the data before touching the model.

Missing information?
Add better features, fields, or structured data.

Model too weak?
Only then consider upgrading.

4. Only use bigger models if you REALLY need them

Move to advanced models only if:

  • The patterns are too complicated for a simple model
  • You must hit a very high accuracy target
  • Mistakes are expensive
  • You have lots of data
  • Users ask vague questions and you need to understand meaning

If you can’t tick at least two of these → stay simple.

5. Check the real-world impact

  • Is it cheap enough to run?
  • Is it fast enough?
  • Can we monitor it easily?
  • Can the team look after it?
  • Is it worth the extra complexity?

6. Improve in the cheapest order

Before using a big AI model:

  1. Clean your data
  2. Improve your features or chunking
  3. Combine a few simple models
  4. Then — only then — use the heavy stuff

Why This Matters

Most engineering teams over-engineer their ML solutions. They jump straight to LLMs, transformers, or complex neural networks when a simple logistic regression or BM25 search would solve the problem perfectly.

Start simple. Upgrade only when you have evidence you need to.

This checklist keeps you focused on solving the actual problem, not just using fancy technology.

↑