Generating consistent AI music videos from text prompts alone — no reference images, no image-to-video chains — produces dramatically better results than any image-first pipeline we have tested. After weeks of wrestling with image-to-video models that warped faces, ignored composition, and injected visual artifacts, we abandoned the approach entirely and built a pure text-to-video pipeline on Seedance 2.0 that is faster, cheaper, and more visually coherent than anything we achieved with reference images.

We are building SyncFrame, an AI music video generator that takes an audio clip and produces a scroll-stopping short-form video — the kind of content that stops a thumb mid-scroll on TikTok or Instagram Reels. The engineering challenge is not generating a single pretty clip. It is generating multiple clips that feel like they belong in the same world, cut to the rhythm of the music, and maintain a consistent visual identity across every segment.

Why Image-to-Video Pipelines Break Down for Music Videos

Our original pipeline was architecturally clean: analyse the audio, generate a creative concept, produce start and end frames for each segment using an image model, then feed those frames into an image-to-video model to render motion. On paper, the reference images should guarantee visual consistency between clips. In practice, they guaranteed nothing.

Image-to-video models like Kling 3.0 suffer from well-documented identity drift and visual warping — the generated video drifts away from the source image within the first few seconds, distorting faces and shifting proportions. We got uncanny-valley faces, limbs that morphed mid-motion, and lighting that shifted randomly between frames. And that was the good case.

The worse case: when we tried Seedance 2.0 through third-party providers, uploaded images containing human faces were rejected entirely. The content moderation layer flagged any recognisable face in a reference image, which meant our entire i2v pipeline was dead on arrival for the most common music video use case — a human performer on screen.

What a "World Bible" Solves That Reference Images Cannot

The breakthrough came when we stopped thinking about visual consistency as an image problem and started treating it as a language problem. Instead of pinning the model to reference pixels, we pin it to reference descriptions — what we call a World Bible.

A World Bible is a forensic description block that defines every recurring visual element in hyper-specific text: locations, objects, materials, atmosphere, lighting colour temperature, time of day, weather conditions, surface textures. When Seedance 2.0 processes the same description block across multiple generation calls, the model's internal vector space interprets those words into consistent visual representations. Same words, same latent space neighbourhood, same visual output — without a single pixel of reference imagery.

This is not a hack. It is how the model's architecture actually works. Text-to-video transformers encode prompts into a continuous vector space before generating frames. Two prompts that share identical descriptive passages land in overlapping regions of that space, producing visually coherent outputs. We tested this extensively and found the consistency to be stronger than what i2v models achieved with actual reference images — because the text encoding is deterministic in a way that image conditioning is not.

How We Structure Prompts for Multi-Clip Coherence

The prompting format draws from how professional creative directors write shot lists and visual treatments. Every generation prompt follows a strict structure: Visual Style block, World Elements (the Bible), Main Character definition, and a timestamped shot structure that maps specific actions to specific time windows.

For clips under 15 seconds, we use a single prompt with no World Bible — the clip is short enough that internal consistency is handled by the model natively. For clips over 15 seconds, we split into two prompts that share the same World Bible but feature different characters or camera angles. This avoids the continuity dependency trap: we never ask the model to pick up exactly where a previous clip left off, because that is where text-to-video models fail hardest.

The shot structure uses a labelled timestamp format that we found the model respects remarkably well:

Shot 1 — Wide Establishing (0-3s): Camera slowly pushes through fog...
Shot 2 — Medium Close-Up (3-7s): Character turns toward camera...
Shot 3 — Detail Insert (7-10s): Hands gripping a rusted chain...

Each shot label tells the model both the framing and the temporal window. The model does not always nail the exact timestamps, but it consistently produces the right number of shots in the right order with the right framing — which is what actually matters for a music video edit.

The Tom Hardy Problem: Why AI Models Default to Celebrity Archetypes

This was one of the stranger discoveries. When we described a main character as "muscular build, grime-streaked skin, exhausted but determined expression, industrial setting" — the model consistently produced someone who looked like Tom Hardy. Not vaguely. Recognisably.

This makes sense if you think about training data distribution. A massive percentage of film and media imagery matching that description features a handful of actors. The model has learned a strong prior: "muscular + grime + determination + industrial = this cluster of faces." The latent space is not uniformly distributed — it has celebrity-shaped gravity wells.

We solved this with what we call anti-archetype rules. Instead of describing what the character looks like in general terms, we force specific diverging physical traits that push the generation away from any known face cluster. Wider jaw than typical, slightly asymmetric brow ridge, a specific scar placement, an unusual hair texture. And we append an explicit instruction: "Original character, no resemblance to any known actor or public figure."

The combination of hyper-specific unique traits and an explicit anti-resemblance directive reliably produces original faces. We tested this across dozens of generations and the celebrity gravity well effect disappeared completely.

Why Dropping the Image Generation Step Made Everything Better

Beyond visual quality, the pure text-to-video approach collapsed our pipeline complexity and cost. The original i2v pipeline had four expensive stages: audio analysis, concept generation, image generation (two images per segment), and video rendering. Every stage added latency, cost, and failure modes.

The text-only pipeline has three stages: audio analysis, concept generation (which now outputs prompts directly instead of image descriptions), and video rendering. We cut an entire generation step — and the most expensive one at that, since generating high-quality reference images required multiple iterations to get right.

The concept-to-video turnaround dropped significantly. More importantly, creative iteration became fast. Changing a visual direction means editing text, not regenerating images and hoping the i2v model interprets them correctly. We can adjust the World Bible — swap "abandoned steel mill" for "rain-soaked Tokyo alley" — and regenerate every clip in the sequence with full consistency, in minutes rather than hours. As we documented in our work on audio-first AI video pipeline design, getting the generation sequence right is where most of the production quality comes from.

What We Would Tell Another Team Building AI Video Pipelines

First: do not assume that giving a model more visual information produces better results. We chose text-over-image conditioning because the image-to-video path introduced more variance than it removed. Reference images feel like they should be the safer bet — they are not, at least not with the current generation of models.

Second: invest heavily in prompt architecture. The difference between a vague prompt and a forensic one is not incremental — it is the difference between unusable output and production-quality footage. The World Bible pattern, the anti-archetype rules, the timestamped shot structure — these are not nice-to-haves. They are the core IP of any serious AI film and video pipeline.

Third: design for independence between clips. The moment you create a dependency chain — where clip 2 needs to visually match clip 1's output — you have introduced a brittle coupling that will break at scale. Shared descriptions are robust. Shared pixels are not.

The Pattern That Emerged

AI music video generation with Seedance 2.0 works best when you treat the model as a cinematographer who reads scripts, not a compositor who traces photographs. The World Bible provides the shared visual language. Anti-archetype rules prevent celebrity drift. Timestamped shot structures give the model a professional shot list to execute. And keeping clips independent — sharing a world, not sharing frames — is what makes the whole system scale. The text-to-video pipeline is not a compromise. For music video generation today, it is the architecturally correct approach.