Video Generation

A generated clip is a short piece of moving video, made in two steps: first a still image, the keyframe, is generated from an image prompt, then a video model animates it according to a motion prompt. Knowing that explains almost everything about how clips behave.

Describing a clip

from imaginaryarts import VideoAssetBuilder

launch = VideoAssetBuilder(
    prompt="A Falcon 1 rocket on a launch pad on a small tropical island, palm trees, azure water, a warehouse marked SpaceX",
    video_prompt="Fire erupts from the base and the rocket lifts off; the camera pulls back and up as it climbs",
    duration=8000,
    audio=True,
)

Two prompts because two models: the image model needs a composition, the video model needs a verb. Put the look in prompt and the change in video_prompt.

VideoAssetBuilder.from_scene(scene, duration=8000, audio=None) builds one from an authored video scene, which already has both prompts; audio=None means "with sound if the scene's direction described a soundscape".

Generating

Clips are generated with the same AssetsBuilder as images, which takes a VideoModel for them:

from imaginaryarts import AssetsBuilder, VideoModel

assets = AssetsBuilder(
    project=project,
    video_model=VideoModel(name="veo31", resolution="720p"),
    direction=ImageDirection(art_style=ImageDirectionStyle.Realistic()),
    assets=[launch, ImageAssetBuilder(prompt="...")],
).build()

assets[0].path         # the mp4
assets[0].duration     # 8000
assets[0].has_audio    # True

The plan printed before generation includes a cost estimate, since clips are the expensive part of a video.

Video models

Two models are available. Both animate a keyframe from a motion prompt; they differ in how long a clip can be and in what they are good at.

The builder keeps the length you asked for; the model is chosen on AssetsBuilder, so the length is settled when the assets are built. At that point duration is rounded up to the nearest length the model offers: on Veo 3.1 a duration=5000 becomes a 6-second clip, on Seedance 2.5 a duration=5500 becomes a 6-second clip and duration=12000 is honoured as is. The built clip's .duration is the length that was actually generated. Asking for more than the model's maximum raises, and so does a resolution the model does not offer, such as VideoModel(name="veo31", resolution="480p").

A rule of thumb: Veo 3.1 for short, high-fidelity moments; Seedance 2.5 when a scene needs one continuous shot longer than eight seconds, or when the narration for a scene should be covered by a single clip rather than a looped one. The model is set on AssetsBuilder and applies to every clip it builds, so use two builders to mix models in one video.

Editing the keyframe

.edit() on a video builder edits the keyframe, and the clip is animated from the edited image:

launch.edit("add a crowd of engineers watching from a safe distance")

Continuity between clips

By default every clip is generated on its own. continuity ties a clip to the asset that comes right before it in the list:

Use "follows" for a cut within the same scene and "extends" to make one long shot from two or three clips.

Placing a clip

A clip has a natural length, so it can be placed at a moment or stretched over an interval:

video.add_video(assets[0], at=12000)                                   # once, 8 s
video.add_video(assets[0], interval=[scene.start, scene.end], fit="loop")   # fill the scene

Narration for a scene often runs longer than the clip, so fit matters: "loop" replays the clip, "freeze" holds its last frame, "none" shows the background. Nothing slows a clip down. If a scene needs fifteen seconds of continuous motion, generate a fifteen-second clip with Seedance 2.5, or use two Veo 3.1 clips.

A clip generated with sound plays it at volume=1.0; scale it down under narration:

video.add_video(assets[0], interval=[scene.start, scene.end], volume=0.4)

.trim([start, end]) takes a window of the clip, as with any file.

Directing video scenes with the writer

When the scene writer authors a video scene, its direction can say what moves and what it sounds like:

VideoSceneBuilder(
    voiceover="The lighthouse had stood for two hundred years.",
    direction=VideoSceneDirection(
        subject="a lighthouse on a rocky cliff at dusk",
        motion="waves roll in; the beacon sweeps; the camera drifts forward",
        audio="waves crashing, distant gulls",
    ),
)

The writer produces both prompts, and VideoAssetBuilder.from_scene picks them up. A described audio turns sound generation on for that clip unless you say otherwise.