AI Content Generation

Watch a single prompt travel through a trained generative model, get broken into concepts, and gradually form into a finished image, then see the same underlying pattern power text, audio, and video generation.

A production AI content generation workflow turns a creative brief into a typed specification, generates each medium with its own model, runs medium-specific quality checks, records provenance and rights, and publishes only what passes review; the checks are what make the output usable. A product team is launching a feature and asks for four assets: a 120-word written summary, a hero illustration, a 30-second narrated explainer, and a 15-second silent clip for social media. In the product this looks like one request with one button. Underneath, it is at least four different generation problems. Text is produced one token at a time by a language model. An image is produced by a model that starts from noise and refines it under the guidance of a text description. Speech is produced by a model that maps text to an acoustic representation and then to a waveform. Video adds time, so the model must keep objects, lighting and motion consistent across many frames. Each family has different inputs, different failure modes, a different unit of cost and a different way of being wrong.

A generative AI application is therefore mostly an orchestrator. It converts a human intent into specifications that each model can act on, chooses a model for each branch, validates every output, stores results with enough lineage to reproduce or roll them back, and presents editing tools so people can correct what the models get wrong. This chapter follows the launch campaign through that machinery, explains what each model family does at a level useful for engineering decisions, and spends most of its time on the parts that decide whether the output is usable: contracts, quality gates, rights and cost. The text mechanics build on how LLMs generate text; this lesson does not repeat them.

1. 1

   Brief

   Capture intent, audience, constraints and rights as a structured contract.
2. 2

   Generate

   Route each requested medium to its own model branch.
3. 3

   Quality checks

   Apply a different automated check per medium, then route doubtful outputs to a person.
4. 4

   Provenance and rights

   Record lineage, licences and disclosure labels with each asset.
5. 5

   Publish

   Release only approved assets, keeping the ability to roll back.

Every content-generation product repeats the same outline: capture the intent, turn it into a specification, pick a model, generate, check, edit, and store and deliver the result. The outline is common. The boxes inside it are not. A text branch can check facts against a source and re-run in a second. An image branch needs a vision-capable checker or a person (image synthesis is taught in Text-to-Image Generation, and speech recognition in Audio to Text). A speech branch can be measured for duration and word accuracy by running recognition on its own output. A video branch can take minutes and cost orders of magnitude more per attempt, so its checks must be cheap proxies applied before the expensive step.

The text branch also feeds the others. The narration script is text, the image prompt is derived from text, and the video prompt usually reuses the same scene description. Making text the first branch, and treating its output as input to the rest, keeps the campaign consistent and is why the diagram shows one orchestrator rather than four independent calls.

A language model generates a response one token at a time, each token sampled from a probability distribution conditioned on the prompt and everything produced so far. The sampling rule, covered in temperature and sampling, decides how adventurous the wording is. For marketing copy a moderate temperature gives varied phrasing. For a product summary that must quote a version number, a low temperature is safer, but it still does not guarantee the number is right, because the model produces plausible continuations, not verified statements.

Applications therefore add constraints in layers. The prompt states the audience, tone and length. The output format is constrained (a JSON schema, a fixed set of fields, a maximum word count) so downstream code can parse it. The facts are supplied rather than recalled: the feature description, version and release date go into the prompt as source material, which is the idea behind retrieval-augmented generation. Finally the draft is verified. For the launch summary, a verifier extracts each numeric and named claim and checks it against the source fields. If the draft says "available on all plans" and the source says "Pro and Enterprise", the draft is rejected with that specific discrepancy and the model is asked to revise, rather than silently shipped.

Many current image generators are built on diffusion or closely related flow-based methods, though the family is broad and individual products differ in architecture, so this section describes the general mechanism and not a specific vendor's system. During training, real images are progressively corrupted with noise, and the model learns to predict how to undo a small amount of that corruption. At generation time the process runs in reverse: begin with random noise and repeatedly apply the learned denoising step, so structure emerges gradually, coarse layout first and fine detail later.

Step through the schematic below to see structure emerge from noise; it is a teaching illustration, not output from a real model.

The text prompt steers each denoising step. A text encoder converts the prompt into vectors, and the image model attends to those vectors as it denoises, so regions of the emerging image are pulled toward the described concepts. Many systems do this in a compressed latent space, working on a much smaller representation than raw pixels and decoding to pixels at the end, which is what makes high resolutions affordable. A guidance setting controls how strongly the output follows the prompt: low guidance gives varied, loosely related images, and high guidance follows the text closely but can produce oversaturated or repetitive results.

For product work the practical issues are not the mathematics but the controls. Aspect ratio and resolution are set before generation, and a model trained mostly on certain sizes tends to compose better at them, so the hero image for a wide banner and the square social image are separate generations or an outpainting step, not a crop. Composition and text rendering remain weak points: requested layout ("logo top left, product centre") is followed approximately, and any text inside the image is often misspelled, so brand wordmarks and legible copy are better added in a design layer after generation. Seeds make a result repeatable on the same model version and settings, and image-to-image or masked editing lets the team change one region while keeping the rest, which is how visual consistency across a campaign is maintained.

Audio systems get lumped together, but they solve distinct tasks. Text-to-speech turns a script into a voice. Speech recognition goes the other way, turning audio into text, and is useful here as a checker because transcribing generated audio and comparing it to the script reveals mispronounced or dropped words. Speech-to-speech changes the voice, language or style of existing audio while trying to keep timing and emotion. Music and sound generation produces non-speech audio from a description. The conversational side, where audio flows in both directions in real time, belongs to AI Voice Systems.

A typical text-to-speech pipeline first normalises text (expanding "v3.2" or "$4.5M" into spoken words), predicts a compact acoustic representation such as a spectrogram or a sequence of audio codec tokens, with prosody (pace, pitch, stress) modelled along the way, and then a vocoder or codec decoder converts that into a waveform. Offline generation can take the whole script and produce polished audio. Streaming generation emits audio chunks while text is still arriving, which matters for assistants but trades a little consistency for latency. For a recorded explainer, offline is the natural choice.

The constraint that bites most in a campaign is timing. A narration track has a fixed slot, and speech duration is determined by the words, not the designer's wishes. At a natural pace of about 2.5 words per second, a 30-second slot holds roughly 75 words. A script of 92 words will run about 37 seconds, so something must give: speed the voice up, which sounds hurried, or rewrite shorter. That loop is demonstrated in the worked example below. Voice identity adds a second constraint: using a specific person's voice needs their documented consent, and many providers require it, which is covered with rights below.

A video is a sequence of frames, and a generator must produce frames that agree with each other. A person must keep the same face, clothing and proportions from frame to frame, objects must move with plausible physics, and the camera must follow a path rather than jitter. This temporal coherence is the central difficulty. Many video models extend image-generation ideas by denoising a block of frames together in a compressed space and using attention across both space and time, though designs vary and details are mostly not public.

Three consequences shape applications. First, compute scales with the number of frames, resolution and clip length, so a 15-second clip costs far more than an image, and clips are commonly short. Second, coherence decays over length: the longer the clip, the more opportunities for identity drift, objects appearing or vanishing, and physics errors, so products often generate several short shots and cut between them instead of one long take. Third, conditioning is richer than text. A video request can supply a first frame or reference image to anchor identity, a motion description, a camera instruction, or a previous clip to extend, and each anchor reduces drift at the price of flexibility. Scene transitions are usually done in an editor, not by the model.

Audio is a separate matter. Some models generate sound with the video, but for brand work the safer pipeline is silent video plus separately controlled narration and music, mixed and aligned in assembly. That is why the campaign's social clip is generated silent: the team controls the soundtrack exactly, and a timing mistake in speech does not force re-rendering expensive video.

A free-text request like "make a punchy launch pack" leaves each model to interpret it separately, and they will interpret it differently. A structured brief is the contract that all branches share. It records the facts, the brand constraints and the technical constraints once, and the orchestrator derives per-model prompts from it. Treating it as a schema also lets the application reject an incomplete request before spending any money.

{
  "campaign_id": "launch-2026-q3-reports",
  "language": "en-US",
  "facts": {
    "feature": "Scheduled reports",
    "available_on": ["Pro", "Enterprise"],
    "release_date": "2026-09-15"
  },
  "text": { "summary_words": 120, "tone": "clear, confident, no hype" },
  "image": { "aspect_ratio": "16:9", "style": "flat illustration, brand palette", "no_text_in_image": true },
  "audio": { "duration_seconds": 30, "voice_id": "brand-voice-01", "pace_wps": 2.5 },
  "video": { "duration_seconds": 15, "shots": 1, "silent": true, "aspect_ratio": "9:16" },
  "forbidden_claims": ["guaranteed savings", "fastest on the market", "available on Free plan"],
  "review": { "required": true, "approver_role": "marketing-lead" }
}

The important point is that these are requests, not guarantees. A model may be told "no text in the image" and render a garbled sign anyway, told "available on Pro and Enterprise" and write "all plans", or told 30 seconds and produce 34. The brief is useful because every constraint in it can be checked afterwards by something other than the generator: a word counter, a claim verifier, a text-detection pass on the image, an audio duration reader. A constraint with no corresponding check is a hope, not a control.

Generated media is only as useful as the record of how it was made. Each asset is stored with the brief version, the exact prompt, the model identifier and version, the settings (including the seed where the model supports one), the parent assets it was derived from, and the checks it passed. That record is the lineage. It answers practical questions: which assets used the old claim list, which version is live, what do we roll back to?

Reproducibility is weaker than it looks. A seed makes an image repeatable only on the same model version, same settings and often the same software and hardware stack. Hosted models change versions, and language-model sampling is not guaranteed to be deterministic even with a fixed temperature. The practical rule is to treat the stored asset as the source of truth rather than expecting to regenerate it identically, and to keep the prompt and settings as documentation and as a starting point for a revised version. After generation come ordinary media steps: resize and crop for each channel, transcode to delivery formats, burn in or attach captions, add metadata, and open the result in an editor for human corrections.

There is no single quality score for generated media, because what counts as correct differs between formats. Combining automated checks with human review is the norm, with automation handling the cheap, objective items and people the judgement calls.

For text, the main check is factual consistency with the supplied source (every number, name and availability claim), followed by length, tone and forbidden phrases. For images, the questions are whether the picture matches the brief (a vision model or a person compares the image with the requested subject and style), whether it contains unwanted artefacts such as malformed hands or garbled text, and whether it is acceptable under brand rules. For speech, intelligibility is measurable: transcribe the audio, compute the word error rate against the script, and measure duration, pauses and loudness. A listener still has to judge naturalness and whether the tone fits. For video, check temporal consistency (does the subject keep its identity, does anything flicker), brief alignment, and that audio is synchronised. Across all of them sits accessibility: captions for video and audio, alt text for images, and sufficient contrast for any text overlays.

A quality gate combines these into a decision. Hard failures, such as a forbidden claim or an unlicensed voice, block publishing. Soft scores route an asset to human review when they fall below a threshold. The thresholds should be calibrated on past human decisions, and the gate should log why each asset passed or failed so reviewers can see what the automation caught. Evaluation methods for model outputs in general are covered in AI Product Design Patterns.

Generated media raises questions that text-only software rarely meets, and the answers depend on jurisdiction, the provider's terms, and the specific use, so what follows is a list of engineering concerns, not legal advice.

Licensing and ownership. Providers publish terms describing what you may do with outputs and whether your inputs are used for training. Read them per model, because they differ, and record which terms applied to each asset in its lineage. Reference images, fonts and music supplied by the team must have their own licences. Consent. Cloning or imitating a real person's voice or likeness requires that person's documented consent, and the pipeline should refuse to use a voice or face without a consent record. Safety. Run generated outputs through content-safety checks appropriate to the audience, and also check inputs, because a brief or reference file can itself request prohibited content. Disclosure and provenance. Some platforms and regulations expect synthetic media to be labelled in certain contexts; for example, the EU AI Act sets transparency obligations for some AI-generated content (Regulation (EU) 2024/1689), and requirements differ by jurisdiction and use. Standards such as C2PA content credentials let tools attach signed metadata that records how an asset was created and edited, though not every platform preserves or displays it. Watermarking methods exist but are not a complete defence against removal. Retention. Decide how long prompts, reference files and drafts are kept, since they may contain unreleased product information or personal data.

Each medium bills in a different unit. Text is priced per token, usually with separate rates for input and output. Images are typically priced per image, often varying with size and quality. Speech is commonly priced per character or per second of audio. Video is typically priced per second of output, sometimes by resolution. Storage and transcoding are small per-asset costs but accumulate. The most important cost fact is that attempts multiply: the number of tries you need to get an acceptable asset matters more than the unit price of any one try.

The calculation below uses assumed unit prices chosen for clarity. They are not vendor quotes and real prices vary widely and change often, so substitute your own.

# Assumed unit prices
text_in   = 2.00  / 1_000_000   # dollars per input token
text_out  = 10.00 / 1_000_000   # dollars per output token
image     = 0.04                # dollars per image
tts_char  = 15.00 / 1_000_000   # dollars per character
video_sec = 0.50                # dollars per second of generated video
post      = 0.05                # transcode + storage for the campaign

text   = 3 * 1200 * text_in + 3 * 600 * text_out      # 3 drafts
images = 8 * image                                     # 8 hero candidates
speech = 3 * 450 * tts_char                            # 3 takes of ~450 characters
video  = 3 * 15 * video_sec                            # 3 attempts of 15 s

total = text + images + speech + video + post
print(round(text, 4), round(images, 2), round(speech, 4), round(video, 2), round(total, 2))
# 0.0252 0.32 0.0203 22.5 22.92

Change the attempt counts to see how quickly video dominates; the defaults reproduce the $22.92 total above.

That asymmetry decides the architecture. Spend generously on cheap checks before the expensive step: validate the brief, preview a still frame from the video prompt with the image model, approve the narration timing, and only then render video. Latency follows a similar pattern: text returns in seconds, images in seconds to tens of seconds, and video in minutes, so the workflow runs branches in parallel and reports progress per asset instead of blocking on the slowest one.

The trace below follows the launch campaign. Numbers are illustrative; the control flow is the point.

1. 1

   Validate the brief

   The orchestrator checks the schema: facts present, forbidden claims listed, durations set. It stores the brief as version 3.
2. 2

   Generate text

   The text model receives the facts and tone. Draft 1 says "available on all plans". The claim verifier compares it to available_on = [Pro, Enterprise], rejects it, and requests a revision. Draft 2 passes.
3. 3

   Derive the branches

   From the approved summary, the orchestrator derives the image prompt, the video prompt and a narration script, all reusing the same scene description.
4. 4

   Run image and video previews in parallel

   Eight hero candidates are generated and filtered for garbled text; a reviewer picks candidate 5. A single still of the video prompt is previewed before any video is rendered.
5. 5

   Generate narration and measure it

   The script is 92 words. Speech generation returns 36.8 seconds of audio against a 30-second slot. The timing check fails.
6. 6

   Rewrite instead of speeding up

   The text model is asked to shorten the script to at most 75 words while keeping the three key points. The result is 72 words, which produces 28.8 seconds of audio and passes. The recognition check shows no dropped words.
7. 7

   Render video and assemble

   Two 15-second silent clips are generated, trimmed to the narration, mixed with the audio and music, and captioned from the transcript.
8. 8

   Quality gate and publish

   Hard checks pass; the marketing lead approves. The mix is published with its lineage record, and the previous version stays available for rollback.

Speech duration is estimated here at 2.5 words per second (92 / 2.5 = 36.8 s; 72 / 2.5 = 28.8 s). Real voices vary with punctuation, numbers and the chosen speaking style, which is why the check measures the produced audio and not the estimate. The pattern, measure the actual output and loop at the cheapest stage that can fix it, applies equally to a text draft that violates a claim rule or an image that fails a text-detection check.

1. 1

   Content generation is an orchestration problem: a typed brief becomes per-medium specifications, each output is validated, and lineage and rights are recorded before anything publishes.

2. 2

   Constraints narrow the distribution but do not replace checking. A brief is a request, not a guarantee, so check text for factual consistency with the supplied source and measure produced audio and video rather than trusting the estimate.

3. 3

   Quality gates differ per medium. Hard failures such as a forbidden claim or an unlicensed voice block publishing, and soft scores route to human review against thresholds calibrated on past human decisions.

4. 4

   Store brief version, prompt, model version, settings and parent assets with every asset. Reproducibility is weaker than it looks, since seeds hold only for the same model version, settings and stack.

5. 5

   Video dominates cost, so spend on cheap checks first: validate the brief, preview a still frame, approve narration timing, and only then render. A 30-second slot holds about 75 words at 2.5 words per second.

6. 6

   Rights, consent, disclosure and retention are engineering concerns that vary by jurisdiction. Record the terms applied to each asset, require a consent record before using a real person's voice or likeness, and retain provenance metadata.

Content generation is a pipeline problem with a human at the end. Customer support has the opposite shape: the input arrives from users at volume, and the system must decide when to answer, when to act and when to hand over to a person.