Designing machine learning systems means designing the full path from data to decision. The model matters, but the system around it matters just as much. That is the part many people miss at first.
I start with a simple view. A machine learning system has a few moving parts: data intake, training, evaluation, deployment, and monitoring. If one part is weak, the whole system can fail in a quiet way. The model may still produce output, but the output may no longer fit the real world.
That is why system design is not just about picking an algorithm. It is about deciding where the data comes from, how it is cleaned, how features are built, how the model is trained, and how results are used after deployment. In practice, the hard work is often in the pipeline, not the model. The code can be fine while the data is wrong. That happens more often than people like to admit.
The first real design choice is the problem itself. I need to know what the model is supposed to predict, how often it must run, and what a good result means. A fraud model, a search ranker, and an image tagger all need different shapes of data and different tradeoffs. If the goal is vague, the system will be vague too.
Then comes data. This is the part that sounds simple and turns out not to be simple at all. Training data is only useful if it matches the kind of data the model will see later. If the live data shifts, the model can drift away from reality. That shift is often called data drift. It means the input data in production no longer looks like the data used for training.
I treat drift as a normal risk, not a rare event. Data changes. Users change. Products change. Sensors change. A system that never checks for that is blind. Good design includes a reference set, a current set, and a way to compare them. If the live data starts to look different, the team needs to know before the model makes too many bad calls.
Monitoring is where machine learning systems become real systems. I care about more than accuracy here. I want to watch latency, errors, input quality, prediction patterns, and, when possible, model quality after labels arrive. Some failures show up fast. Others take time. A model may look fine on day one and slowly get worse over weeks.
This is also where many teams run into a hard limit. In the real world, labels often arrive late, or not at all. That makes it hard to know right away if the model is right. So the system has to watch proxies. It watches input drift, output drift, schema problems, and data gaps. These are not perfect signs, but they are often the only early signs.
I think of deployment as a contract. Once a model is in production, it is no longer a notebook result. It is part of a live service. That means versioning matters. Data versions matter. Feature code matters. Model artifacts matter. If none of these are tracked, debugging becomes guesswork.
Experiment tracking is also part of the design, even if it feels like a lab tool. It helps answer a basic question: what changed? Without that, it is hard to tell whether a model got better because of a better feature, a better split, or just luck. Machine learning systems need more discipline here than many software systems do, because training is not always repeatable in the same neat way.
There is one honest limit I keep in mind. No design removes uncertainty from machine learning. A model learns patterns from past data, and the future is not forced to match the past. Even a careful system can fail when the world shifts, the labels are noisy, or the goal itself was framed too loosely. That is not a bug in one tool. It is a fact of the field.
So when I say designing machine learning systems, I mean designing for the full life of the model. Not just training day. Not just launch day. The useful system is the one that still works after the data moves a little. That is the real job, and it is usually less glamorous than the slide deck.
That is also the kind of work I try to keep in view with The Model Log: one practical AI concept, one working example, and one honest look at what actually works.
