Book Review: Designing Machine Learning Systems — Chip Huyen

There is a small paradox in the AI industry that few talk about directly:

You can learn machine learning the “right” way—through Coursera, fast.ai, or Stanford lectures on YouTube—training models with 94% accuracy, writing clean code, and creating beautiful notebooks. But when it comes to the real world, or trying to actually deploy that model—you don’t know where to start.

It’s not because you aren’t smart enough. It’s because that part—the part that determines whether your model actually works in reality—is almost never taught anywhere.

“Designing Machine Learning Systems” by Chip Huyen is a rare book that speaks directly to that gap.

---

Who is Chip Huyen — and Why Does It Matter?

Her real name is Huynh Nhat Anh, originally from Hanoi. A Stanford graduate (Computer Science), she has worked at Netflix and Snorkel AI, and later co-founded Claypot AI—a company recently acquired by Databricks. She teaches the CS329S (Machine Learning Systems Design) course at Stanford.

And more importantly: O’Reilly chose her to write this book.

O’Reilly is no ordinary publisher. They are the creators of the reference books that engineers at Google, Amazon, and Meta keep on their desks. To be published there, you have to truly know what you’re talking about—you can’t just be someone summarizing content from elsewhere.

The book was published in 2022 and is currently used as a textbook in many American universities. It now has a Vietnamese version—translated by Do Huu Thieu and Nguyen Sy Khanh.

---

What Is This Book About?

It’s not about training models. It’s not about deep learning or neural networks.

It’s about what happens after the model is trained—and that is why it differs from 90% of other AI books on the market.

Chip Huyen breaks the book down into problems that any team bringing AI into a real product must face:

1. Framing the problem correctly from the start

Before thinking about the model, you need to answer a much harder question: what are you optimizing for? A recommendation system can be optimized for click-through rate, watch time, or user satisfaction—and those three metrics often conflict. If you pick the wrong objective from the start, a model with perfect accuracy is useless.

2. Data is not fuel — data is architecture

Most ML books teach you how to handle data after you already have it. Chip Huyen goes deeper: how do you design a pipeline so that the data you collect today is still valuable 6 months from now? How do you detect mislabeled data? How do you handle imbalanced data without skewing the model?

This is the part I find most lacking in traditional courses—and it’s also the part that takes up the most time in real-world work.

3. Feature engineering still matters — even if you use deep learning

There is a common misconception that deep learning “learns features automatically.” True—but not entirely. Chip Huyen clearly explains why feature leakage (accidentally including information from the future in your training data) remains one of the most common mistakes causing models to fail in production.

4. Monitoring — the part most teams ignore until it’s too late

Your model is deployed. It runs well for the first week. By the third month, accuracy starts to drop. By the sixth month, the results are meaningless. But no one knows, because there’s no alerting system.

Chip Huyen spends nearly three chapters discussing data distribution shift—the phenomenon where the world changes but the model still “thinks” based on old data—and how to build monitoring to detect it before users start complaining.

The examples she gives are very practical: a fraud detection model trained on pre-COVID data. After March 2020, user transaction behavior changed completely. The model started blocking a massive number of legitimate transactions because they looked “strange” compared to the old data. Real damage. Not just theory.

5. Continual learning — how to keep models from becoming “fossilized”

Traditional model deployment: train once, deploy, and forget. Chip Huyen presents why this is an anti-pattern—and how to design systems so that models can be updated continuously without causing downtime.

---

My Favorite Part — The One Few People Talk About

The final chapter: “The Human Side of Machine Learning.”

This is a section that technical books usually don’t have. Its presence here is why I think Chip Huyen is not just a technical expert—she is someone who understands why the technology exists.

She writes bluntly: AI is not a neutral tool. Every decision in the process of building an ML system—which data to choose, which metrics to define, who to deploy it to—is a decision with hidden ethical weight.

One example she uses is Amazon’s automated recruitment model from 2018. The accuracy was high. However, because the training data was Amazon’s hiring history over 10 years—which was already biased toward men in the tech industry—the model learned that being female was a feature of a less suitable candidate. Amazon had to shut it down.

“Works correctly” has never been enough.

---

The Actual Downsides

I don’t write reviews to flatter. There are two points I need to be direct about:

First: This book is not for absolute beginners. If you don’t know what machine learning is yet, or have never used scikit-learn or PyTorch, this is not the place to start. You will read it without understanding why these problems are important.

Second: Some sections on infrastructure (Kubernetes, distributed training) are quite heavy and may become outdated quickly. I read those to grasp the concepts rather than memorize technical details—since those are things you look up when you face a real problem, not something to learn by heart.

---

Who Is This Book For?

Read it if:

• You are studying AI/ML and want to know what the job actually looks like—before your first interview or offer.
• You are a Product Manager or Business Analyst and want to understand what the ML team is struggling with.
• You are preparing to transition into ML Engineering or AI Product Management.
• You already have experience with data and want to understand why ML projects fail—even with good models.

Don’t read it if:

• You don’t know Python and have never worked with data.
• You want to learn how to train a model from scratch (there are hundreds of better courses for that).
• You are looking for a “light” read—this isn’t that kind of book.

---

Overall Verdict

If I had to choose one book to describe the difference between someone who knows ML and someone who can use ML in practice—I would choose this one.

Not because it covers everything, but because it hits directly on what most other ML learning resources ignore: the part where real products are actually built.

There is a Vietnamese version. The translation is quite good. Highly readable.

---

Buy the book at Tiki — nationwide delivery available