AI Recommendation Systems

Watch a recommendation feed move from behavior signals to candidates, ranking, and final placement, then see how the same architecture pattern appears in streaming, commerce, and social products.

AI recommendation systems work by turning user and item signals into a short list in stages: candidate generation narrows the catalog, a ranking model scores each candidate, re-ranking shapes the final slate, and the user's reactions flow back as feedback. A recommendation system is a pipeline, not a single model. Given a user and a moment (device, time, page), it first retrieves a few hundred candidate items out of a catalog of millions using cheap, approximate methods; then a more expensive ranking model predicts, for each candidate, how likely the user is to respond in the ways the product cares about; then a re-ranking stage arranges the winners into a slate that obeys diversity, safety, inventory and fairness rules; and finally the slate is served and the user's reactions are logged to train the next version. The models in the middle are classic predictive machine learning (matrix factorisation, two-tower neural networks, gradient-boosted trees or deep rankers) trained on behaviour logs. A large language model is not what chooses your next video; where generative AI appears, it usually explains or summarises a choice the ranker already made.

This lesson follows one fictional app, StreamLoop, through that pipeline. Everything about StreamLoop, including every score, latency and count, is invented for illustration. Where real services are mentioned, only the broad, publicly described pattern (candidate generation followed by ranking) is meant; nothing here describes any company's internal system. Recommending items to a person is a different problem from answering a question from documents, which is the topic of AI Search Systems; the two share embeddings and nearest-neighbour search but differ in what they optimise and how they are judged.

Maya and Dev both open StreamLoop at 8:40 p.m. on the same day. The catalog is identical for both: roughly four million videos. Their home screens share almost nothing. Maya has watched forty minutes of cooking videos every evening this week on her living-room TV, finishes long videos, and rarely uses search. Dev is on a phone during a commute, skips most things within eight seconds, but watched two short science explainers to the end yesterday.

Both slates contain a few shared items because some videos are broadly good, but the order, the mix and the length of each item differ. It is tempting to describe this as "the AI understands each person". The mechanism is more mundane and more inspectable: the system holds a representation of each user (recent behaviour, aggregates, learned vectors) and of each item (metadata, engagement statistics, learned vectors), looks for items whose representation fits the user, predicts a response, and then applies product rules. Personalisation is candidate retrieval plus ranking plus policy and business constraints. Each of the three can be wrong independently, and each is debugged differently: a missing video is a retrieval problem, a badly ordered one is a ranking problem, and five near-identical videos in a row is a slate problem.

The response for Maya's request is small. It carries not only the item ids but the request id and the model and feature versions that produced it; that is what makes the feedback joinable later and an incident debuggable. Note that a cooking fan sees a travel video first: that follows from the toy scoring weights worked through later, which deliberately reward freshness and novelty.

{
  "request_id": "r_9f2c",
  "user_id": "u_maya",
  "ranker_version": "ranker-2026-10-14",
  "feature_set_version": "fs-v12",
  "slate": [
    { "video_id": "v_lisbon",    "position": 1, "source": "two_tower", "score": 0.78 },
    { "video_id": "v_tides",     "position": 2, "source": "content",   "score": 0.73 },
    { "video_id": "v_sourdough", "position": 3, "source": "co_visit",  "score": 0.65 }
  ],
  "fallback_used": false
}

A recommender never observes what a person likes. It observes behaviour, and infers from it. Explicit signals are deliberate statements: a thumbs-up, a star rating, a "not interested" tap, a follow. They are clear but scarce, because most people rate almost nothing. Implicit signals are by-products of using the product: impressions (what was shown), clicks, watch duration and completion fraction, skips, replays, searches, purchases, shares, add-to-list, plus context such as device, local time, session length and connection quality. Implicit data is abundant but ambiguous, and the central craft is deciding what each signal means.

Consider three events on StreamLoop. Maya watches 11 minutes of a 12-minute video: that is strong positive evidence. Dev watches 11 minutes of a 3-hour podcast: perhaps interest, but it is only 6% of the item. A raw "minutes watched" label would rank long videos above short ones for everyone, a length bias that comes from the label rather than from taste. A better target normalises, for example completion fraction or "watched at least 70% or at least 5 minutes". The same care applies to clicks: a click on a misleading thumbnail followed by a three-second exit is a negative outcome that a pure click-through objective counts as a success.

A signal is not a preference. A person who watches a documentary about a disaster may be curious rather than wanting more of it; a parent watching cartoons on a shared account describes the child; a late-night binge may reflect insomnia rather than taste. Implicit feedback also lacks true negatives: an unwatched video might be disliked, unseen, or simply buried. This is why systems record impressionsas well as interactions. "Shown and ignored" is information; "never shown" is not.

Selection bias and position bias

The logs contain only what earlier recommendations chose to show, so the data is a biased sample of reality (selection bias). Within what was shown, items in the first slot are examined far more often than items in the fifth, so a click rate partly measures position (position bias). Suppose, illustratively, that users examine slot 1 with probability 0.40 and slot 2 with probability 0.15, and that two videos X and Y have the same true appeal: 10% of people who actually look at them click. If X always sits in slot 1 and Y in slot 2, over 1,000 impressions each X receives 1,000 × 0.40 × 0.10 = 40 clicks and Y receives 1,000 × 0.15 × 0.10 = 15. A trainer that treats clicks per impression as quality sees X at 4.0% and Y at 1.5% and concludes X is 2.7 times better, although they are identical. Dividing by the examination probability (inverse propensity weighting) recovers 40 / 0.40 = 100 and 15 / 0.15 = 100 per 1,000 impressions. Production systems handle this by logging position, training with position as a feature that is neutralised at serving time, using propensity-weighted losses, or running small amounts of randomised exposure to measure true appeal. The point of the arithmetic is that bias does not announce itself; it must be modelled.

The pipeline has an offline half that prepares things slowly and an online half that must answer in a fraction of a second. Events flow into a durable log; a streaming job keeps fast-changing aggregates fresh while batch jobs recompute heavy ones; features are stored in a feature store; retrieval narrows the catalog; a ranker scores candidates; re-ranking shapes the slate; the serving layer responds; and what the user does next re-enters the log.

Two properties of this design deserve attention. First, the heavy work (training embeddings, building the index, computing long-window aggregates) is moved out of the request path; only lookups and scoring remain online. Second, the loop closes: what is logged depends on what was shown, which is the root of the feedback problems discussed later.

Offline and online features must agree

A ranking model is trained on feature values computed offline and applied on feature values fetched online. If the two computations differ, the model is fed inputs it never learned from, which is training-serving skew. Suppose watch_minutes_7d is computed in the training pipeline from a SQL query over complete days, but online it is maintained by a streaming counter that includes today's partial day and double-counts a replayed event. The ranker learned that 140 on one scale means "heavy viewer", and at serving 140 means something else. Nothing crashes; the recommendations just quietly degrade. The remedy is to define each feature once and materialise it to both an offline store and an online store, so that training and serving read the same definition.

The offline store keeps history so training examples can be assembled as of the time of each logged impression: the feature values the system would have had at that moment, and no later ones. Joining on today's value instead leaks the future into training, a subject taken up again under evaluation.

The ranker is too expensive to run on the whole catalog. If scoring one candidate costs 0.05 ms of model time, scoring four million is 200 seconds, while a request must finish in a fraction of one second. Candidate generation exists to select roughly the right few hundred cheaply, optimising recall: do not lose the video the user would love. Precision comes later. Because no single method finds everything, production systems run several in parallel and merge the results.

Popularity and trending. The most-watched items in a region, genre or the last hour. This is simple, robust, needs no user history, and is the standard fallback; it is also identical for everyone and reinforces what is already popular.

Collaborative filtering uses patterns across users instead of item content: people whose histories overlap with yours liked X, so X may suit you. Classical matrix factorisation learns a vector for each user and each item such that the dot product approximates the observed interaction. With three-dimensional illustrative vectors, a user u = (0.8, 0.1, 0.6) and an item a = (0.9, 0.0, 0.5) give 0.8×0.9 + 0.1×0.0 + 0.6×0.5 = 0.72 + 0 + 0.30 = 1.02, while item b = (0.1, 0.9, 0.1) gives 0.08 + 0.09 + 0.06 = 0.23. The learned dimensions are not human-labelled genres; they are whatever structure best explains the data.

Content-based retrieval matches item attributes (genre, language, creator, title and transcript embeddings) against a profile built from what the user consumed. It works for brand-new items because it needs only their content, but it tends to recommend more of the same.

Co-visitation and graph methods use item-to-item relationships: "people who watched this next watched that", computed from sessions as a graph. They are fast, explainable, and strong for the "up next" row.

Two-tower retrieval

The two-tower model generalises matrix factorisation. One neural network (the user tower) maps user features and recent behaviour to a vector; another (the item tower) maps item features to a vector in the same space. The score is the dot product. Training uses observed positive pairs and treats other items in the same batch as negatives. For the numbers above, with in-batch candidates a, b and an extra item c with dot product 0.90, the softmax over logits (1.02, 0.23, 0.90) gives probabilities (0.427, 0.194, 0.379), so the loss for the true positive a is −ln 0.427 ≈ 0.850. Gradient descent pushes the positive logit up and the sampled negatives down. A known subtlety is that in-batch negatives are drawn in proportion to popularity, which penalises popular items too much unless the logits are corrected for sampling frequency.

The towers are separate so that item vectors can be computed in advance and indexed; only the user tower runs at request time. The price is that the model cannot see user and item features together, which is the job of the ranker. The vector machinery is the same as in Embeddings and Similarity Search, but here the vectors are trained on behaviour (who engaged with what), not on text meaning, and the query is a user rather than a sentence. Approximate search trades a small, measurable loss in recall for orders-of-magnitude speed.

Merging sources

Each source returns ids with its own score on its own scale; two-tower dot products and co-visit counts are not comparable, so the merge layer deduplicates and applies quotas rather than sorting by raw score. The ranker then sees one comparable feature set per candidate.

{
  "user_id": "u_maya",
  "budget": 400,
  "sources": [
    { "name": "two_tower", "returned": 200, "kept": 180 },
    { "name": "co_visit",  "returned": 120, "kept": 100 },
    { "name": "content",   "returned":  80, "kept":  60 },
    { "name": "trending_region", "returned": 100, "kept": 60 }
  ],
  "unique_after_dedupe": 372,
  "top_example": [
    { "video_id": "v_night_market", "sources": ["two_tower", "content"] },
    { "video_id": "v_sourdough",    "sources": ["co_visit"] }
  ]
}

Cold start

Collaborative signals need history, so two cases are hard. A new user such as Priya on her first visit has no plays. StreamLoop can use context it legitimately has (region, language, device, time), an optional onboarding choice of a few topics, and popular items, then adapt quickly: after three plays the user tower has something to work with. A new item such as a video uploaded one minute ago has no engagement data, so it has no collaborative vector. Its content embedding, creator history and metadata let content-based retrieval propose it, and a small exploration allowance gives it initial impressions so that real behaviour can accumulate. Treating cold start as an afterthought produces a feed in which nothing new ever surfaces.

The ranker receives the ~400 candidates and, for each, builds a row of features combining user, item, context and cross features: the user's completion rate on this creator, the item's 24-hour engagement rate, how long since the item was published, whether the user has already seen it, device type, the hour of day, and the candidate's retrieval source and score. Because it sees user and item together, it can capture interactions that two towers cannot. Typical model families are gradient-boosted trees and deep neural networks; the choice is about latency, data volume and feature types, not about one being universally better.

The output is usually not one number but several predictions per candidate: probability of a click, expected fraction watched, probability of a like, probability of a hide or "not interested", probability of a return visit. Optimising a single objective is dangerous, since click prediction alone rewards sensational thumbnails. So these heads are combined into one ranking value, often a weighted sum or product, with weights tuned through experiments and sometimes constrained by business rules such as minimum quality or age limits. The weights are a product decision expressed numerically.

To see the mechanics, take a deliberately tiny model. For Maya's request the ranker receives five candidates, each with three features scaled to the 0–1 range: relevance (predicted fit with Maya's taste), freshness (1 for brand new, decaying with age) and diversity (how different the item is from what Maya's recent history already contains). The toy score is

score = 0.6 * relevance + 0.2 * freshness + 0.2 * diversity   # all inputs in [0, 1]

A  Night Market Noodles   rel 0.90  fresh 0.30  div 0.20  genre cooking
B  Lisbon in 48 Hours     rel 0.80  fresh 0.90  div 0.60  genre travel
C  Sourdough Basics       rel 0.85  fresh 0.50  div 0.20  genre cooking
D  Tides in 6 Minutes     rel 0.60  fresh 0.95  div 0.90  genre science
E  Hidden Trails of Peru  rel 0.70  fresh 0.40  div 0.80  genre travel

Calculating two candidates in full:

A: 0.6*0.90 + 0.2*0.30 + 0.2*0.20 = 0.54 + 0.06 + 0.04 = 0.64
B: 0.6*0.80 + 0.2*0.90 + 0.2*0.60 = 0.48 + 0.18 + 0.12 = 0.78

Remaining: C = 0.51 + 0.10 + 0.04 = 0.65
           D = 0.36 + 0.19 + 0.18 = 0.73
           E = 0.42 + 0.08 + 0.16 = 0.66
Order by score: B 0.78, D 0.73, E 0.66, C 0.65, A 0.64

The most relevant video, A (0.90), ends up last: its low freshness and low novelty relative to Maya's habits cost it more than its relevance gains. That is the trade-off the weights encode. Moving the relevance weight from 0.6 to 0.8 (and the others to 0.1 each) would reorder these five, which is why weights are tuned by experiment, not argued.

Production ranking differs from this toy in nearly every respect. The inputs are hundreds of features, not three. Relevance is not a given number; it is the output of a model trained on logged outcomes, with several heads. Combination is often multiplicative, so that a video with a high click probability but a high hide probability is suppressed. The values interact (a highly fresh item may only be good for users who like news). And the weights are applied to predicted outcomes with uncertainty, so well-run systems validate them against long-term metrics rather than the immediate ones the model can see.

Sorting by score gives B, D, E, C, A. The top three contain two travel videos, and cooking, Maya's dominant interest, would appear only in positions 4 and 5. The best individual items do not necessarily make the best list, because the value of an item depends on what sits beside it: a second very similar video adds less than the first, and a feed of near-duplicates is tiring. Re-ranking treats the slate as the object being optimised.

A simple and widely used pattern is greedy selection: at each position, choose the candidate with the highest adjusted score, where the score is reduced by a penalty for similarity to what is already selected. With a toy penalty of 0.10 per already-selected video of the same genre, the process runs as follows:

pick 1: B 0.78, D 0.73, E 0.66, C 0.65, A 0.64        -> B (travel)
pick 2: D 0.73, E 0.66-0.10=0.56, C 0.65, A 0.64      -> D (science)
pick 3: C 0.65, E 0.56, A 0.64                        -> C (cooking)
pick 4: E 0.66-0.10=0.56, A 0.64-0.10=0.54            -> E (travel)
pick 5: A 0.64-0.10=0.54                              -> A (cooking)

Final slate: B, D, C, E, A   (three different genres in the first three slots)

The penalty cost the list a little predicted relevance, since C scored 0.65 and E 0.66, and bought a more varied head of the list. Real re-rankers use more principled methods (maximal marginal relevance, determinantal point processes, or a learned slate model), but the shape is the same. Beyond diversity, this stage typically enforces rules that are not about predicted engagement at all:

• Deduplication and recency: remove near-duplicates, reuploads and videos shown several times without engagement.
• Novelty: reserve room for items outside the user's established pattern.
• Safety and policy: exclude or demote content that violates policy or the user's age or region restrictions, enforced by deterministic filters, since a ranker's score is no guarantee.
• Inventory and business constraints: promotions, licensing windows, items that are no longer available.
• Fairness and exposure: avoid systematically starving creators or groups of any exposure, for example by guaranteeing minimum exposure for new creators or capping a single creator per slate.

Each rule has a cost in predicted engagement, and each should be monitored as a metric of its own (for example, the share of impressions going to creators with fewer than 1,000 followers) so that the trade-off is visible rather than implicit.

All of this must happen while the page loads. StreamLoop's illustrative target is a 150 ms server-side budget at the 95th percentile (p95). A single request's critical path might look like this:

Several design decisions keep this feasible. Item vectors are searched with approximate nearest-neighbour indexes, because exact search over millions of vectors would blow the budget. Frequently needed data is cached: item metadata, popular-region candidate lists and sometimes whole slates for short periods, with the trade-off that a cached slate is stale for exactly as long as its time-to-live; the general trade-offs are in Caching in AI Systems. Feature freshness is chosen per feature: "what did the user just watch" must update within seconds, while "genre affinity over 90 days" can be a day old. Scoring runs on batched candidates so that hundreds of rows are evaluated in one model call.

Tail latency requires deliberate handling. If each of four parallel retrieval sources independently meets its deadline 95% of the time, then all four meet it together only 0.95⁴ ≈ 81% of the time, so waiting for the slowest one pushes the request's p95 well beyond any single source's. Stage p95s do not add up arithmetically either, but they compound. The standard defence is a per-source deadline (say 25 ms) after which the request continues with whichever sources have answered. A late source degrades the candidate pool slightly; a blocked request degrades the whole page.

Anonymous or new users have no history to fetch, so the path degrades gracefully: skip user features, retrieve from trending-by-region and content-based sources using only context, rank with a model that does not require personal features, and mark the response fallback_used so dashboards can show how often it happens. The same ladder covers failures: if the ranker times out, serve candidates in retrieval-score order; if retrieval fails, serve a precomputed regional list. A degraded feed is far better than an error, and being able to see how often the ladder is used is a health metric in its own right. Whether the ranker runs on CPU or GPU, how it is batched and how it is scaled are questions for Inference Infrastructure.

A recommender generates its own training data. It shows items, users react only to what was shown, and the reactions train the next model. Without correction the loop reinforces itself.

Popularity bias arises because popular items collect more exposure, therefore more engagement, therefore higher predicted quality, and so more exposure. The earlier arithmetic with items X and Y shows it in miniature: identical appeal, a persistent two-and-a-half-fold gap in observed performance created entirely by where each was placed. A filter bubble is the same loop at the level of topics. If Maya watches cooking, the model shows cooking, she watches more cooking, and evidence of any other interest never accumulates because nothing else is shown. The system is not wrong according to its own data; its data is simply a narrow slice.

Exploration is the deliberate breaking of the loop: occasionally show something the model would not otherwise choose, in order to learn. The tension is exploitation (show what is believed best now) versus exploration (gain information that may be worth more later). Practical implementations include reserving a small share of slots, say 1 slot in 8, for items sampled from those with high uncertainty or from new content; epsilon-greedy selection; and bandit methods such as Thompson sampling, which sample from the model's uncertainty so that uncertain items get tried more. Whichever is used, the system should log the probability with which each item was chosen(its propensity). With 1 in 8 slots chosen uniformly from a pool of 500 exploration candidates, a given explored item has propensity roughly (1/8) × (1/500) ≈ 0.00025 per slot, and its later weight in training is the inverse. That log is what allows an unbiased estimate of how good an item really is.

There is also the question of what the loop is optimising. Clicks, plays and minutes watched are proxies for the real goal, which is long-term value to the user and the business: satisfaction, return visits weeks later, reduced regret. A system optimised for immediate engagement can raise next-minute metrics while degrading next-quarter retention, for example by over-serving sensational content. The defence is to measure long-horizon outcomes in experiments, track guardrail metrics (hide rate, "not interested" rate, complaints, diversity of consumption), and include survey-based or explicit satisfaction signals in the objective. Whether a design really improves things is answered by controlled experiments, not by the offline model's own score.

Evaluation has two layers: cheap offline measurements that screen candidate models, and online experiments that decide whether a change ships. Neither replaces the other.

Ranked-list metrics, with a calculation

Take a held-out window in which Maya's behaviour is known after the fact. Four videos were genuinely relevant to her, with graded relevance 3 (watched to the end and liked), 2, 1 and 1. The system showed five slots, and what happened in each slot was graded 3, 0, 2, 0, 1 (the one other relevant item with grade 1 was not retrieved at all).

• Precision@5 is the fraction of the shown five that were relevant: 3 of 5 slots (positions 1, 3, 5) = 0.60.
• Recall@5 is the fraction of all relevant items that were shown: 3 of 4 = 0.75.
• NDCG@5 rewards putting highly relevant items near the top. With gain 2^grade − 1 and a log discount, DCG = Σ gain_i / log₂(i + 1). DCG = 7/1 + 0/1.585 + 3/2 + 0/2.322 + 1/2.585 = 7 + 0 + 1.5 + 0 + 0.387 = 8.887. The ideal list orders the relevant items 3, 2, 1, 1: IDCG = 7 + 3/1.585 + 1/2 + 1/2.322 = 7 + 1.893 + 0.5 + 0.431 = 9.823. NDCG@5 = 8.887 / 9.823 ≈ 0.905.

Order matters: if the same five items had appeared as grades 0, 2, 0, 3, 1 (the grade-3 video in slot 4), precision and recall would be unchanged at 0.60 and 0.75, while NDCG@5 would fall to about 0.539. A list that places the best item last is worse than one that places it first even though the same items appear. The computation is short enough to check yourself:

import math

def dcg(gains):
    return sum((2 ** g - 1) / math.log2(i + 2) for i, g in enumerate(gains))

def ndcg_at_k(shown, all_relevant, k):
    ideal = sorted(all_relevant, reverse=True)[:k]
    return dcg(shown[:k]) / dcg(ideal)

shown = [3, 0, 2, 0, 1]       # grade of what happened in each shown slot
heldout = [3, 2, 1, 1]        # every relevant item in the test window
k = 5

hits = sum(g > 0 for g in shown[:k])
print(hits / k)                           # precision@5 = 0.6
print(hits / len(heldout))                # recall@5    = 0.75
print(round(ndcg_at_k(shown, heldout, k), 3))   # 0.905

Offline pitfalls

Offline metrics are only as honest as the data split. The most common failure is data leakage: features or labels that contain information from after the moment being predicted. Splitting interactions randomly puts a user's later behaviour into training while evaluating on earlier behaviour; joining features by today's value rather than as of the impression time lets the model see the future. Both inflate offline NDCG and then collapse online. Evaluation should split by time and use point-in-time features. The second pitfall is that held-out data is itself the product of the old recommender: items it never showed have no recorded outcomes, so offline evaluation favours models that resemble the one in production. A model that would have introduced better, previously unseen items cannot be credited for them offline.

Online experiments and guardrails

The deciding evidence is an A/B test: randomly assign users to the current system or the candidate, run long enough to cover weekly cycles and enough users for statistical power, and compare pre-declared metrics. A primary metric (for example, satisfied watch sessions per user) answers "did this help". Guardrail metrics answer "did it hurt something else": hide and "not interested" rate, report rate, session abandonment, latency p95, error rate, share of impressions to new creators, and consumption diversity. A change that raises watch time and doubles the hide rate should not ship. Retention and satisfaction take weeks to show, so teams often keep a long-running holdout group served by an older or simpler system to measure cumulative effects. Interference is an additional complication: if users interact (shared social graph) or items are a shared resource (limited-supply inventory), the treatment and control groups are not independent, and experiments need to be designed accordingly.

A recommender is built on a detailed record of behaviour, which makes privacy an architectural requirement, not an addendum. These are enforced in data pipelines and access controls, never by a model prompt or instruction.

• Consent and purpose: record what the user agreed to and let users turn personalisation off, clear history or remove items from their profile; the system must then really stop using that data, including for training.
• Retention and minimisation: keep raw events only as long as needed, aggregate or delete afterwards, and collect only features that improve the product. Not every available attribute should be a feature.
• Protected and sensitive attributes: do not use attributes such as health, religion or sexual orientation directly, and watch for proxies that reveal them indirectly (a viewing pattern can imply them). Legal requirements differ by jurisdiction and should be settled with counsel, not assumed.
• Access control and tenancy: restrict who can read user-level logs and embeddings, since an embedding derived from behaviour is personal data; see AI Security.
• Children and shared accounts: stricter defaults for minors and detection of profile mixing.

Operationally, models degrade silently. User tastes shift, catalog composition changes, upstream pipelines alter a field's meaning, and the ranker keeps returning confident, worse results. Driftmonitoring compares live feature distributions with those at training time and tracks prediction distributions and online metrics by segment, which belongs in AI Observability. Every model, feature set and index should be versioned and recorded in each response, as in the JSON earlier, so that a drop in a metric can be tied to a deploy. Releases move through shadow scoring, a small experiment and a gradual ramp, with an automatic rollback to the last good version. Finally, a safe fallback (a simple non-personalised ranking that is itself tested) must always exist, so an incident means a less personal feed instead of an empty one.

The following pseudocode ties the stages together with deadlines, fallbacks and instrumentation. It is illustrative pseudocode, not a runnable program or a specific framework's API; the helper functions stand for services described above.

async function recommend(req: Request): Promise<Response> {
  const t0 = now();
  const user = await features.getUser(req.userId, { timeoutMs: 15 }).catch(() => null);
  const ctx = { device: req.device, localHour: req.localHour, region: req.region };

  // 1. Retrieve: independent sources, each with its own deadline
  const sources = user
    ? [twoTower, coVisit, contentBased, trending]
    : [trending, contentBased];                    // anonymous: context only
  const lists = await Promise.all(
    sources.map(s => s.fetch(user, ctx, { timeoutMs: 25 }).catch(() => [])),
  );
  const candidates = mergeAndDedupe(lists, { budget: 400, exclude: user?.watched });

  // 2. Score: batched, with a fallback if the ranker is slow or down
  let ranked;
  try {
    const rows = await features.getItems(candidates, user, ctx);
    ranked = await ranker.score(rows, { timeoutMs: 45 });   // multi-head, combined value
  } catch {
    ranked = candidates.sort(byRetrievalScore);
    metrics.inc('fallback.ranker');
  }

  // 3. Re-rank: deterministic policy, diversity, exploration slot
  const slate = reranker.build(ranked, {
    size: 8, maxPerGenre: 2, maxPerCreator: 1, exploreSlots: 1, blocklist: policy.blocked(user),
  });

  // 4. Serve and log everything needed for feedback and debugging
  log.impression({ requestId: req.id, userId: req.userId, slate,
    rankerVersion: ranker.version, featureSetVersion: features.version,
    explorationPropensity: slate.explorePropensities });
  metrics.observe('latency.total_ms', now() - t0);
  return { requestId: req.id, slate, fallbackUsed: !user };
}

1. 1

   Trace Maya's request

   Features load in about 15 ms, four sources return a merged set of 372 unique candidates, the ranker scores them, and the re-ranker applies one-per-creator and two-per-genre rules.
2. 2

   Trace a failure

   If the ranker call exceeds 45 ms, the request serves candidates in retrieval order, increments a fallback counter, and still returns inside its budget; the slate is lower quality but valid.
3. 3

   Close the loop

   The logged impression joins later with plays and hides by request id, and the job that builds training data uses features as of the logged time.

Recommender ML versus generative AI

A recommender ranks a fixed catalog of existing items using learned behavioural patterns and is evaluated with ranking metrics and experiments. Generative AI produces new text, images or audio from a prompt and is evaluated very differently. They can be combined, but the roles stay distinct. A language model can write the sentence under a row ("Because you watched Sourdough Basics"), summarise a video for the card, or interpret a free-text request ("something light to watch with my kids") into structured filters and seed items that feed the same retrieval pipeline. In each case the model produces text or a query, and the item still comes from the catalog through retrieval and ranking. If a model is allowed to generate titles directly, it may invent items that do not exist; generated explanations must also be true to the actual reason the item was chosen, which means grounding them in the logged retrieval source and features, not in the model's guess, and checking them with the usual safeguards for generated text. See Hallucinations for why that matters. Conversational interfaces that ask follow-up questions belong with Chatbots and Assistants, and the broader decision of when to use a predictive model, a generative model or both is the subject of AI Product Design Patterns.

Having seen how products choose what to show, the next kind of AI application aims at a very different user: a developer who needs code understood, changed and verified.

1. 1

   Recommenders learn from behaviour, not preferences, and logs only show what earlier recommendations chose to display, so training data carries selection bias.

2. 2

   Use a funnel: cheap, high-recall candidate generation (popularity, collaborative, content-based, co-visitation, two-tower) narrows millions to hundreds, then a ranker scores those. Scoring four million items at 0.05 ms each would take 200 seconds.

3. 3

   Train-serve skew breaks rankers: compute features the same way offline and online, and build training examples as of each impression.

4. 4

   Sorting by score is not a good list: re-ranking adds diversity, novelty, deduplication, policy and exposure rules, each with an engagement cost that you monitor separately.

5. 5

   Serving has a tail-latency budget. Parallel sources that each hit their deadline 95% of the time rarely all do together, so set deadlines, fall back to popularity, and degrade gracefully for new users.

6. 6

   Offline metrics (precision, recall, NDCG) screen candidates but A/B tests decide, and a loop that trains on its own output needs exploration and privacy controls such as consent and personalisation opt-out.