Part 5: Types of Machine Learning. Supervised to RL

series: AI from zero to expert | article 5 of 30

Three different ways to learn: with correct answers, without them, or by trial and error.

Three ways to learn

In 1959, IBM engineer Arthur Samuel published a paper with a title that sounds obvious today and was a complete novelty back then: “Some Studies in Machine Learning Using the Game of Checkers.” That’s where the phrase “machine learning” shows up for the first documented time: a program that played checkers and, instead of following fixed hand-written rules, got better game after game by comparing its own positions against a record of past outcomes. Samuel defined it with a line that remains the best short definition out there: a computer’s ability to learn without being explicitly programmed for the specific task it ends up solving.

Sixty-seven years later, “machine learning” is no longer an IBM lab curiosity, it’s phase 1 of this series, and the ground where almost everything we today call “serious” AI lives (the kind that isn’t an expert system or a fixed-rule algorithm, the type you saw in chapter 1). But before getting into concrete algorithms in chapter 6, this chapter maps out the types of machine learning: the three fundamentally different ways a machine can learn, supervised, unsupervised, and reinforcement. These aren’t three algorithms, they’re three schools of thought, three types of problem that call for different approaches, and almost everything you’ll see in the rest of this series fits into one of the three. Each one assumes a different amount and type of information available upfront (already-solved examples, unsolved examples, or neither of those, just the ability to act and observe the outcome), and that starting assumption determines which algorithms make sense to use and which don’t, long before a single line of code gets written.

Supervised learning

Supervised learning is, by far, the most intuitive of the three, and also the most used in real applications. The core idea: you show the model a bunch of already-solved examples, each one with its correct answer attached, and the model learns to generalize the pattern so it can respond to new examples it’s never seen. “Supervised” doesn’t mean a human watches the training happen in real time, it means the training data already comes labeled with the correct answer, like an exam with the answer key attached.

The spam filter from chapters 1 and 4 is textbook supervised learning: it’s trained on thousands of emails already hand-classified as “spam” or “not spam,” the model learns which patterns (words, senders, structure) are associated with each label, and when a new, unlabeled email arrives, it predicts which of the two categories it belongs to. This type of problem, where the answer is one of several fixed categories, is called classification. There’s a second family within supervised learning, regression, where instead of predicting a category you predict a number: a house’s price based on its size, location, and age, or a company’s estimated revenue for next quarter. Classification and regression are, at bottom, the same idea (learning from labeled examples) applied to two different kinds of question, “which of these categories?” versus “what number?” Another classification example, less cited than spam but just as common: a system that reviews an X-ray and predicts whether there are signs of a fracture, trained on thousands of X-rays already diagnosed by human radiologists. When you get to chapter 12 on convolutional networks you’ll see how this works concretely with images, but the underlying principle (labeled examples, a learned pattern, predictions on new cases) is exactly the same as the spam filter.

The cost of this approach is exactly what makes it powerful: someone, at some point, had to label all that data by hand. Labeling thousands or millions of examples with the correct answer is expensive, slow, and sometimes requires specialized expertise (a radiologist labeling X-rays, not an intern). This limitation, more than any other, is what pushes the industry toward the other two paradigms once hand-labeling stops being viable at the scale needed.

It’s not enough for the model to get the training examples right, what actually matters is whether it gets new examples right, ones it’s never seen, a distinction you’ll come back to by name in chapter 7. That’s why, in practice, labeled data never all goes toward training: a portion gets set aside, untouched, to check afterward whether the model actually learned the general pattern or just memorized the specific examples it was shown.

Unsupervised learning

Unsupervised learning flips the setup: there are no correct answers attached to the data, no labels of any kind. The model gets a bunch of unsolved examples and has to find, on its own, structure or hidden patterns in them. Nobody tells it “this is a type A customer and this is a type B customer,” the algorithm itself decides how many groups make sense and which examples go where, based purely on how similar they are to each other.

The most common task in this paradigm is grouping, or clustering, and the reference algorithm is k-means, formally described in 1967 by James MacQueen in “Some Methods for Classification and Analysis of Multivariate Observations,” presented at the Berkeley Symposium on Mathematical Statistics and Probability. The intuition is simple: given a set of points (customers, images, whatever, already represented as vectors, as you saw in chapter 4), the algorithm splits them into k groups so that points within the same group end up as close together as possible. You’ll see the exact mechanism, step by step, in chapter 6. For now, the idea is enough: an online store can run clustering over its customers’ purchase history (what they buy, how much they spend, how often they come back) to discover, without anyone telling it in advance, natural groups of shoppers with similar habits, say occasional big-ticket buyers versus frequent low-ticket ones, and tailor its campaigns to each group separately. Nobody decided in advance how many groups existed or what defined them, the algorithm found them on its own, purely from the structure of the data.

Another common unsupervised task is anomaly detection: instead of grouping, the goal is to identify which examples fall outside the general pattern of the rest. A bank can train a model on an account’s history of legitimate transactions (with no “this is fraud” label anywhere), and use it afterward to flag any transaction that strays too far from that account’s usual behavior. Nobody had to label thousands of confirmed fraud cases for the system to work, which is exactly the limitation supervised learning didn’t handle well.

A third, less flashy unsupervised task is dimensionality reduction: taking data represented with a huge number of variables (the thousand-number vectors from chapter 4) and compressing it into a smaller version that keeps the essential information, useful both for visualizing complex data on a simple two-axis chart and for speeding up the training of whatever model comes next.

Reinforcement learning

The third paradigm is the most different of the two above, and also the one that most literally resembles how an animal learns by trial and error. There are no labeled examples upfront (like in supervised learning) and no fixed dataset to extract patterns from (like in unsupervised learning). Instead, there’s an agent that acts within an environment, gets a reward (positive or negative) after each action, and adjusts its behavior with the sole goal of maximizing accumulated reward over time. The formal framework for this paradigm, with its standard terminology (agent, environment, state, action, reward, policy), was laid out in the field’s reference book, “Reinforcement Learning: An Introduction,” by Richard Sutton and Andrew Barto, first published in 1998 and still the text any serious course on the subject builds on. A simple example, no Go board required: a robot vacuum that learns entirely on its own, by reinforcement, would be an agent that at every moment observes its position and the state of the room (the state), decides to move in a direction or turn on suction (the action), gets a small positive reward for every area cleaned and a penalty if it bumps into furniture or runs out of battery far from its dock, and over time, run after run, adjusts its overall strategy (the policy) to maximize the accumulated reward for the whole session. Nobody hand-programmed the optimal route, it discovered one through trial and error, exactly the way AlphaGo discovered its moves.

The most cited large-scale case of reinforcement learning is AlphaGo, the DeepMind system that in 2016 defeated Lee Sedol, world champion of Go, a game long considered too complex for a machine to master because of the sheer combinatorial explosion of possible moves. DeepMind’s own team, led by David Silver, published the technical details in Nature (“Mastering the Game of Go with Deep Neural Networks and Tree Search,” 2016): AlphaGo combined initial supervised learning (games from expert human players, to start at a reasonable level) with subsequent reinforcement learning, playing millions of games against itself and adjusting its strategy according to whether it won or lost each one. That combination of the first two paradigms to train the third, instead of treating them as fully separate boxes, is an idea you’ll see come up again and again for the rest of this series. Two years later, the same team went a step further with AlphaZero (also published in Science, “A General Reinforcement Learning Algorithm That Masters Chess, Shogi, and Go Through Self-Play,” 2018): this second version didn’t even start from human reference games, it learned chess, shogi, and Go from scratch, using only the rules of each game and millions of games of self-play, pure reinforcement learning with no supervised crutch to start.

Outside of games, reinforcement learning is also the conceptual backbone of much of modern robotics (a robotic arm that learns to grasp objects of different shapes through simulated trial and error) and of the most sophisticated recommendation systems, where the “environment” is the user themselves and the “reward” is a click, a full watch, or a return visit the next day. Worth an honest caveat on this last example: a system trained to maximize clicks or watch time has no internal mechanism that distinguishes “content the user is genuinely interested in” from “content that hooks them even if it isn’t good for them,” because both generate the exact same measurable reward. This phenomenon, when a system optimizes exactly the metric it was given but in a way nobody actually wanted, is known as reward hacking or specification gaming, and it’s one of the central problems in the chapter on safety and alignment in phase 4.

The three schools aren’t fully sealed compartments, as you just saw with AlphaGo. There’s also a middle ground between supervised and unsupervised, semi-supervised learning (a small slice of labeled data alongside a large mass of unlabeled data), and its cousin most relevant to the rest of this series, self-supervised learning, where the data generates its own label without any human involvement, for instance by hiding a word in a sentence and asking the model to guess it from context. That last trick, deceptively simple, is exactly the mechanism that makes it possible to train the language models you’ll see in phase 3 of this series, without anyone having to hand-label even a fraction of all the internet text they consume. In a way, it’s the paradigm that solves the exact problem the supervised learning section opened with: if hand-labeling data is the bottleneck, self-supervised learning lets the text itself, already existing in enormous quantities, do the labeling.

How to choose

How do you know, faced with a new problem, which type of machine learning applies? The practical starting question isn’t “which algorithm sounds more advanced,” it’s much more boring than that:

Do you have data already labeled with the correct answer? If yes, supervised.

Do you have data, but nobody has labeled it and labeling it by hand would likely be prohibitively expensive or impossible? Unsupervised.

Do you not even have a fixed dataset, but rather a system that can act and receive a success or failure signal after each action, in an environment that responds to what it does? Reinforcement.

The choice of paradigm almost never comes down to technical preference, it comes down to the shape of the data and the problem sitting on the table.

With this map of the three schools in place, next chapter goes down to concrete ground: the classic algorithms (regression, decision trees, SVM, k-means) that have spent decades solving real supervised and unsupervised learning problems, long before any neural network entered the picture.

Part 5: Supervised, unsupervised, and reinforcement learning

Three ways to learn

Supervised learning

Unsupervised learning

Reinforcement learning

How to choose

Sources