ML vs Deep Learning vs Generative AI

An 8-scene walkthrough of the terminology, the engineering differences, and the kinds of problems each approach is good at.

In AI, machine learning means building software by fitting a function to examples instead of writing the rules by hand. The machine learning vs deep learning question is a question about nesting, while generative AI is a question about output, which is why the three cannot be ranked on a single ladder. Machine learning is a way of building software: instead of writing the rules, you fit a function to examples. Deep learning is a subset of machine learning that uses many-layered neural networks and learns its own intermediate features from raw inputs such as pixels, audio or text. Generative AI is defined by what the system produces: new content (text, images, code, audio) sampled from a learned model, rather than a label or a number. Most generative systems you meet today, including large language models (see LLM Overview), are deep learning models. But the category is defined by the output, not the architecture: an n-gram text generator is generative machine learning without deep learning, and fraud scoring, churn prediction and demand forecasting are machine learning that is neither deep nor generative.

That is why the right unit of decision is the task. The rest of this chapter follows one product through three tasks, shows the mechanism behind each family with small calculations you can reproduce, and ends with a way to choose, combine and test them. For the field-level definition of AI itself, see What Is AI?.

An online store runs three pieces of work that all get called "AI" in planning meetings.

1. 1

   Will this customer churn in the next 90 days?

   Input: a row of numbers and categories (tenure, support tickets, late deliveries, discount usage). Output: a probability between 0 and 1. Data: a few thousand to a few million labelled rows, "churned" or "stayed", taken from history. The shape of the data is tabular and the output is a score.
2. 2

   Is the package in this photo damaged?

   Input: an image, which is a grid of, say, 224 × 224 × 3 pixel values (150,528 numbers) with no column that says "dent". Output: a label, or a box around the damage. Nothing in the raw numbers is individually meaningful; the useful signal is in patterns of edges, textures and shapes spread across many pixels.
3. 3

   Write a reply to "My order was charged twice; when will the refund arrive?"

   Input: free text plus facts from your systems. Output: a new paragraph that never existed before and has no single correct answer, only better and worse ones. A classifier cannot produce it, because the output space is open-ended.

The objective and the data structure, not fashion, pick the family. A table of numbers with a yes/no label points to classical machine learning. Raw pixels or audio point to deep learning, because writing features by hand for them is impractical. An open-ended text or image output is a generative task. And notice that the three tasks do not compete: a real product uses all three, with different models, different evaluation and different failure costs. Choosing "the best AI" for the store is a category error; each task needs its own choice.

In short, AI is the broad field, ML (machine learning) is the part of AI that learns from data, and DL (deep learning) is the part of ML built on many-layered neural networks.

You will often see AI > ML > DL > GenAI drawn as four nested circles. The first three are accurate: every deep learning model is a machine learning model, and every machine learning system is AI. The fourth does not nest. Generative AI is not a smaller or more advanced slice of deep learning; it is a different axis, classifying systems by what they output. Treating it as "the newest, most advanced ML" hides two things that matter in practice.

First, generative modelling predates deep learning. An n-gram language model that counts which word followed which in a corpus and samples from those counts generates text, and it is machine learning without a neural network. Hidden Markov models and Gaussian mixture models are classical generative models too. Second, some generation involves no learning at all: a mail-merge template or a grammar-based generator produces new strings and belongs to AI-adjacent software only loosely. Conversely, many deep learning models are not generative: an image classifier or a speech recogniser consumes content and returns a label or transcript. The ellipse in the figure therefore straddles regions rather than living inside one.

The practical consequence is about transfer. Properties of deep learning (large data appetite, GPU training, weak interpretability) transfer to a gradient-boosted tree only by accident, and properties of one generative system (fluent but unverified output) do not describe a churn model. Ask which family a given claim is about before accepting it.

Classical supervised learning has a clear division of labour. A person decides what the model sees, the features, and the algorithm learns how to combine them. For churn, an analyst turns raw order and support logs into four numbers per customer: tenure in months, support tickets in the last 30 days, late deliveries in the last 90 days, and whether a discount code was used. The label is 1 if the customer left within 90 days and 0 otherwise. A training set is simply a table of such rows with labels, and the algorithm searches for parameters that map features to labels with low error on rows it has not seen.

A logistic-regression score, computed by hand

Logistic regression is the simplest widely used learner. It computes a weighted sum z = b + w₁x₁ + w₂x₂ + w₃x₃ + w₄x₄ and squashes it to a probability with the sigmoid p = 1 / (1 + e−z). Suppose training produced the weights below (illustrative values, not from a real dataset): bias b = −1.0, tenure −0.08 per month, tickets +0.45 each, late deliveries +0.60 each, discount used −0.90. The signs encode what the data said: long tenure and a discount reduce churn risk, tickets and late deliveries increase it.

import math

w = {"tenure": -0.08, "tickets": 0.45, "late": 0.60, "discount": -0.90}
b = -1.0

def churn_probability(c):
    z = b + sum(w[k] * c[k] for k in w)
    return z, 1 / (1 + math.exp(-z))

customers = {
    "A": {"tenure": 5,  "tickets": 3, "late": 2, "discount": 0},
    "B": {"tenure": 36, "tickets": 0, "late": 0, "discount": 1},
    "C": {"tenure": 14, "tickets": 1, "late": 1, "discount": 0},
}
for name, c in customers.items():
    z, p = churn_probability(c)
    print(name, round(z, 2), round(p, 3))
# A 1.15 0.76
# B -4.78 0.008
# C -1.07 0.255

Customer A is a five-month customer with three tickets and two late deliveries: 76% predicted churn. Customer B, three years in, no tickets, a discount used, scores 0.8%. The model is a few arithmetic operations; it can be read, audited and explained to a regulator: "each late delivery adds 0.60 to the log-odds". Whether the 0.76 is a trustworthy probability is a calibration question answered on held-out data, not by the formula.

Change the four inputs in the figure below to see which term moves z and how the sigmoid turns it into a probability; the defaults reproduce customer A.

Trees, forests and boosting

A decision tree learns a sequence of threshold questions: is tickets_30d > 2? If yes, is late_90d > 1? Each leaf stores the churn rate of the training rows that reached it. Trees capture interactions and non-linear thresholds that a single weighted sum cannot, with no feature scaling, but one deep tree memorises its training data. A random forest averages many trees, each trained on a random sample of rows and features, which cancels much of that variance. Gradient boosting builds trees sequentially, each one fitted to the errors left by the ensemble so far; libraries such as XGBoost, LightGBM and CatBoost implement refined versions. On tables with mixed numeric and categorical columns, boosted trees are frequently very hard to beat, even by deep networks, which is why they remain the default baseline for churn, credit risk, pricing and fraud. Reasons are structural: tabular columns have no spatial or sequential structure for a deep network's inductive biases to exploit, the datasets are often only thousands to millions of rows, and trees handle heterogeneous column scales and missing values gracefully.

For the damaged-package photo, no analyst can write the feature "dent shape" as a function of 150,528 pixel values. Deep learning replaces hand-written features with learned ones. A neural network is a stack of layers; each layer multiplies its input vector by a weight matrix, adds a bias, and applies a non-linear activation function. Without the non-linearity, stacked layers collapse into a single linear map; with it, depth lets later layers build on earlier ones.

A two-neuron hidden layer, forward pass

Take input x = (1.0, 2.0). Hidden neuron h₁ has weights (0.5, 0.25) and bias 0.1; h₂ has weights (−0.6, 0.2) and bias 0. The activation is ReLU, max(0, v). Then h₁ = ReLU(0.5·1.0 + 0.25·2.0 + 0.1) = ReLU(1.1) = 1.1 and h₂ = ReLU(−0.6·1.0 + 0.2·2.0) = ReLU(−0.2) = 0. The output neuron has weights (0.8, 0.5) and bias −0.3, so z = 0.8·1.1 + 0.5·0 − 0.3 = 0.58 and the sigmoid gives p ≈ 0.641. Notice that h₂ is switched off for this input: each hidden unit responds only to inputs in a region of the space, and a wide, deep network is a vast collection of such learned detectors. Real models have millions to billions of parameters arranged this way, but the arithmetic is the same.

Step through the same forward pass below, one neuron at a time.

How the weights get learned

Training compares the output with the label using a loss function, for example binary cross-entropy for a yes/no task. Backpropagation applies the chain rule layer by layer from the output backwards, producing for every weight its gradient: how much the loss would change if that weight changed slightly. An optimiser then nudges each weight a small step against its gradient, w ← w − η · ∂L/∂w with learning rate η, and the loop repeats over many batches. The numerical mechanics of one such step are the subject of Training vs Inference.

Architecture supplies the structure that makes learning feasible on high-dimensional inputs. A convolutional network reuses the same small filter at every image position, so "an edge" is learned once and detected anywhere, a built-in assumption that nearby pixels relate. Transformers use attention to let every token read from every other token in a sequence, which suits language and, with patches or audio frames as tokens, images and speech. See Transformer Basics for that one architecture in detail.

"Learned features" does not mean "no human decisions". Deep learning pipelines still need data cleaning, label quality control, normalisation of inputs, tokenisation or image resizing, augmentation, train/validation splits that avoid leakage, and architecture and loss choices. What changes is that the representation inside the model is no longer written by hand, and that the price is data, compute and reduced transparency.

A discriminative model learns P(y | x): given an input x, how likely is each label y? The churn and damaged-package models are discriminative. A generative model learns how the data itself is distributed, P(x), or, conditioned on some context c, P(x | c), and creates outputs by sampling from it. A chat model approximates P(next token | previous tokens) and chains those draws into a reply; an image diffusion model learns to turn noise into an image, optionally conditioned on a text prompt. The distinction is illustrative rather than exclusive: Naive Bayes is a generative classifier, and a language model can be scored on a classification prompt, so use the labels for the dominant purpose of the system.

The smallest runnable generative model

You can see "learn a distribution, then sample" without any neural network. Count, for each word in a tiny corpus, which word followed it, normalise the counts into P(next | previous), and sample.

from collections import Counter, defaultdict
import random

corpus = [
    "your refund will arrive in five days",
    "your refund was sent today",
    "your order will arrive in two days",
    "your order was sent today",
]
counts = defaultdict(Counter)
for line in corpus:
    toks = ["<s>"] + line.split() + ["</s>"]
    for a, b in zip(toks, toks[1:]):
        counts[a][b] += 1

def p_next(prev):
    c = counts[prev]
    n = sum(c.values())
    return {w: k / n for w, k in c.items()}

print(p_next("your"))    # {'refund': 0.5, 'order': 0.5}
print(p_next("refund"))  # {'will': 0.5, 'was': 0.5}

def sample(rng):
    out, cur = [], "<s>"
    while True:
        d = p_next(cur)
        cur = rng.choices(list(d), weights=list(d.values()))[0]
        if cur == "</s>":
            return " ".join(out)
        out.append(cur)

rng = random.Random(7)
for _ in range(5):
    print(sample(rng))
# your refund was sent today
# your order will arrive in five days   <- not in the corpus

Two things appear immediately. The model produced "your order will arrive in five days", a sentence that is in no training line: generation recombines learned statistics rather than retrieving stored text. And that sentence may be false for your order, because "five" was learned as a plausible continuation after "in" within the refund sentence, not as a fact about this order. A modern language model replaces the lookup table with a transformer that conditions on thousands of earlier tokens and has billions of parameters, giving vastly more coherent text, but the contract is the same: the output is a sample from a learned distribution, fluent by construction and correct only to the extent the context and training support it. How that distribution is produced is covered in Next-Token Prediction.

Predictive AI, such as the churn model, returns a measured score, while generative AI returns new text; they do different jobs.

A language model can write a convincing paragraph about churn. Paste a customer's history into a prompt and it will say "this customer appears at high risk because of recent delivery problems". It is tempting to retire the separate churn model and use the chat model for everything. Compare what each approach gives you for customers A and C above.

The logistic model returns 0.76 and 0.255 every time, for identical inputs, in microseconds, at negligible cost. You can sweep a threshold, measure precision and recall on 12 months of held-out customers, schedule a retention offer for everyone above 0.5 and track whether the score stays calibrated month by month. The language model returns prose; to get a number you must instruct it to produce one, parse it, and discover that the same prompt can yield 0.7 on one call and 0.8 on the next, and that its numbers were never fitted against your outcome labels. It has no way to learn from your 90-day churn history unless you train it on it, at which point you are doing supervised learning with a heavier tool. It does add something the score lacks: a readable explanation. The sound design is to let the predictive model decide and the generative model describe the decision, using the score and the model's top contributing features as inputs to the text. That also keeps the number auditable, because the sentence cannot override it.

Classical models usually learn from structured labels: someone or something recorded the outcome (the customer left; the payment was fraudulent). Labels are specific and valuable, but they are also scarce, delayed (you only know who churned after 90 days) and shaped by past decisions. Deep image or audio classifiers need many labelled examples too, often tens of thousands or more, unless they start from a pretrained network and are adapted with far fewer.

Large generative models rely on self-supervised learning, where the label is manufactured from the data itself: hide the next token and predict it, or add noise to an image and predict the noise. No human labels every example, which is what lets training use enormous unstructured corpora. This is followed by adaptation: supervised fine-tuning on curated demonstrations and preference-based alignment to make the model follow instructions (see Fine-Tuning). Pretraining a large model is a very large compute project, run on clusters of accelerators for weeks or months. Training a gradient-boosted churn model, by contrast, typically takes seconds to minutes on a laptop-class CPU for datasets of this kind.

An important consequence for most developers: consuming a hosted generative model involves no training at all. Calling an API runs inference; the weights never change, whatever you put in the prompt, and nothing you send teaches the model unless a provider explicitly offers and you opt into a tuning service. Your engineering work is then prompt, context and evaluation design, not optimisation, and your cost profile is per-request inference rather than a training bill. The resource contrast for training and serving at scale is taken up in Training Infrastructure and Inference Infrastructure.

"Accuracy" is a defensible metric for balanced classification and a misleading one for the churn task. Suppose 1,000 customers contain 50 who churn (5%). A model that predicts "stays" for everyone is right on 950 rows: 95% accuracy, and zero churners found. Now take a real model with this confusion matrix:

TP = 30   # churners flagged
FP = 40   # stayers flagged
FN = 20   # churners missed
TN = 910  # stayers not flagged

accuracy  = (TP + TN) / 1000        # 940 / 1000 = 0.94
precision = TP / (TP + FP)          # 30 / 70    = 0.429
recall    = TP / (TP + FN)          # 30 / 50    = 0.60
f1 = 2 * precision * recall / (precision + recall)   # 2(0.429)(0.60)/1.029 = 0.50

Precision answers "of the customers we flagged, how many really churn?" (43%, so most retention offers go to people who would have stayed). Recall answers "of the churners, how many did we flag?" (60%). F1 is their harmonic mean, 0.50 here. Which matters depends on cost: a cheap email favours recall, an expensive phone call favours precision. ROC-AUC summarises ranking quality across all thresholds, as the probability that a random churner is scored above a random stayer; under strong imbalance it can look comfortably high while precision at a usable threshold is poor, so check precision-recall curves too. Because logistic and tree models output scores, the threshold is a business decision made after training.

The figure below lays the same confusion matrix out as a grid and shows each metric next to the trivial baseline.

Regression tasks use different measures. For delivery-time prediction, with predictions (3, 5, 2, 6) days and actual (4, 5, 4, 9) days, the absolute errors are 1, 0, 2, 3 and the mean absolute error is (1+0+2+3)/4 = 1.5 days: a number in the units of the target, easy to explain to operations.

Generated content has no single ground truth, so evaluation changes character. You measure task success(did the reply resolve the ticket, did the code pass unit tests), factuality (is every claim in the reply supported by the order record or policy you supplied, checked by humans or a carefully validated automated judge) and rubric-based human evaluation for tone and helpfulness, on a fixed test set re-run whenever the prompt or model changes. Text-overlap scores can be useful for narrow tasks but a reply can reuse none of the reference's words and still be better. No single metric spans the three families; stating "the model is 94% accurate" without naming the task and the baseline communicates nothing.

It is true that deep models can reach quality classical models cannot on unstructured data. It does not follow that they are better for a given product; each dimension below can reverse the decision.

• Data size and quality. A boosted tree can give a useful churn model with a few thousand clean rows. A deep network trained from scratch on that data will usually memorise it. If you have little labelled data in images or text, start from a pretrained model and adapt it, rather than training from scratch. Noisy or biased labels hurt all families, and larger models can fit the noise more thoroughly.
• Compute and latency. A tree ensemble scores a row in well under a millisecond on a CPU. A large generative model decodes token by token and takes hundreds of milliseconds to seconds per reply, on accelerators. Latency budgets (a checkout fraud decision may have tens of milliseconds) can rule models out before accuracy is considered.
• Interpretability. Linear coefficients and shallow trees can be explained to a reviewer; deep network internals cannot be summarised as readily, and a generative model's stated reasoning is itself generated text, not a readout of its computation. Regulated decisions (credit, hiring) often require reasons a person can verify.
• Maintenance and drift. Every learned model degrades when the world changes: new products, a new carrier, a pricing change. Classical models are cheap to retrain on a schedule. Hosted generative models change underneath you when a provider updates a version, so pin versions and keep regression tests.
• Cost and privacy. Training costs recur with retraining; inference costs scale with traffic and, for language models, with the number of input and output tokens. Sending customer data to an external model needs a data-handling review; an on-premise tree model never leaves your infrastructure.

The result is not a ranking but a fit. The accurate summary is that deep and generative models extend what is possible on unstructured inputs and open-ended outputs, while classical models remain the economical, controllable choice for structured prediction.

Real products compose families, giving each the part it does best. Consider a returns assistant that, when a customer reports a damaged item, decides whether to refund automatically and writes the response.

The boosted model is trained on historical refund outcomes and outputs a score for abusive patterns. The image encoder, a deep network, turns the photo into a vector, and a nearest-neighbour search over past confirmed-damage photos supplies evidence (see Embeddings Explained and Vectors & Similarity Search). A plain-code decision service applies thresholds and policy: refund automatically if risk is below a set cutoff, similarity to confirmed damage is high, and the amount is under a limit; otherwise queue for a person. Only then does a language model receive the decision, the order id, amount and policy text, and write the customer message. If it were also asked to decide, an unmeasured, unstable component would sit on the money path.

Data boundaries matter here. The language model should receive only fields it needs, not the whole customer record, and its output is checked (amount and order id must match the decision) before sending. When the reply needs facts from documents rather than from structured data, add retrieval, described in RAG Overview.

Start from the task, not the technology: what exactly is the output, what data exists, what error is tolerable, how will you measure it, what is the budget. Then walk down the cheapest option first.

Four cases show the reasoning, including when no model is the right answer.

• Fraud scoring. Labelled historical transactions in tabular form, a latency budget of tens of milliseconds, and a requirement to explain declines. Boosted trees with engineered features (velocity, device change, amount versus history) are the natural fit; add hard rules for known-bad patterns that must never be missed. A language model adds nothing to the score itself. Without enough labelled fraud, start with rules and anomaly detection, and add supervised learning as labels accumulate.
• Product recommendations. Start with popularity and "bought together" counts, which are plain code and strong baselines. When there is enough interaction history, a learned ranker (boosted trees on user/item features, or matrix factorisation or neural two-tower retrieval at larger scale) can raise relevance. Evaluate with online experiments, since offline accuracy only approximates revenue. A generative model might later write the explanation, not choose the item.
• Image defect detection. Raw pixels, so deep learning, usually a pretrained vision network fine-tuned on a few thousand labelled photos of your own defects. Classical image features are a baseline only if conditions are tightly controlled. If the defect is simply "colour differs from reference" under fixed lighting, a threshold on pixel statistics may be enough and needs no training.
• Report generation. Open-ended prose from structured facts: generative. If the report is a fixed template with filled-in numbers, a template engine is more reliable and costs nothing per call. Use a language model when wording must adapt, and constrain it with the data to be used.

A non-AI solution is preferable whenever the rule is stable and fully specifiable, the cost of an error is high and the logic must be exact (tax calculation, permission checks), or no labelled data and no way to evaluate exist. An approach you cannot measure is a liability, whatever its label.

Every learned model fails in characteristic ways, and the test you write should target the way, not just overall accuracy.

Overfitting on small data. An unrestricted decision tree trained on 200 customers with 40 features can reach 100% training accuracy and, say, 61% on a held-out set (illustrative numbers): it memorised noise. Always report performance on data the model never fitted, with the split made by time rather than random shuffling when the use is forecasting, so that future rows never inform past predictions. Mitigations are limiting tree depth, regularisation, cross-validation, fewer features and more data.

Biased or leaky labels. If past agents granted refunds more generously to some regions, a model trained on "refund granted" learns that history, not fairness or need. Check metrics per segment, not only overall, audit how labels were produced, and remove features that encode the outcome after the fact.

Domain shift. A damage classifier trained on warehouse photos can fail on customer phone photos with different lighting and angles; a churn model trained before a pricing change can misjudge afterwards. Test on a sample from the real deployment distribution, monitor input statistics and outcome rates, and retrain on fresh data. Observability for AI systems is expanded in AI Observability.

Hallucination and unstable output. A generative model can state a refund policy that does not exist, because the sentence is plausible, as the bigram example showed in miniature. Run it against a fixed test set of real tickets, check each factual claim against the record supplied, require outputs in a schema you validate, ground answers in retrieved documents and have it say when information is absent. Sampling makes output vary between calls; lower temperature reduces variation but does not guarantee correctness (see Hallucinations and Temperature). Where the output triggers an action such as a refund, keep the decision in deterministic code or with a person.

Everything above rests on a lifecycle distinction we have used loosely: fitting parameters from data, versus running the fitted model on new inputs. The tabular model, the image network and the language model all separate those two phases, with very different cost, risk and tooling on each side, which is the subject of the next lesson.

1. 1

   Choose by task, objective and data structure, not by label: tabular rows with a yes/no outcome point to classical ML, raw pixels or audio to deep learning, and open-ended prose to generative models.

2. 2

   Do not swap a language model in for a fitted scorer. The logistic churn model returns the same 0.76 for the same input in microseconds at negligible cost, while a prompted LLM's 'probability' is a formatted guess that was never fitted to your labels.

3. 3

   Accuracy misleads on imbalanced classes. Compare against the trivial baseline and report precision and recall; generated content needs task-success and groundedness checks instead of a single ground truth.

4. 4

   Consuming a hosted generative model involves no training, but you still own evaluation, data boundaries, drift monitoring and per-request cost.

5. 5

   Real products compose families: a boosted model for the abuse score, a vision encoder for photos, a language model for the explanation, with only the needed fields sent to the language model.

6. 6

   Each family fails differently, so test for the failure: overfitting on small data, biased or leaky labels, domain shift, and hallucination or unstable output. A plain rule is the right answer when the rule is stable, specifiable and errors are costly.