Grok Imagine Video v1.5 Reference-to-Video API by xAI

1. Introduction

Grok Imagine Video V1.5 is a frontier-tier video generation model developed by xAI that produces short clips of up to 15 seconds with natively generated, synchronized audio — including dialogue, lip-sync, sound effects, and ambient music — in a single inference pass.

This README applies to the following API model identifier:

• xai/grok-imagine-video-v1.5/reference-to-video

Reference-to-video is the identity-preserving mode: you supply 1–7 reference images that contribute specific people, products, or visual styles. The prompt then directs how those references behave, using <IMAGE_N> tags to bind each reference to its role in the scene.

Built on xAI's Aurora engine — an autoregressive mixture-of-experts (MoE) network that jointly models text, image, video, and audio tokens — the model produces tightly coupled audiovisual output rather than dubbing audio in post-processing.

2. Key Features

• 1–7 Reference Images: Multiple references can be combined in a single generation — a person from one image, a product from another, a setting or style from a third. Each is addressed independently in the prompt.

• Tag-Based Binding: References are bound to scene roles explicitly through <IMAGE_0> … <IMAGE_N> tags in the prompt, so the model is told which reference plays which part rather than inferring it.

• Native Synchronized Audio Generation: Audio is generated jointly with video tokens in a single inference pass, producing event-aligned sound effects and natural lip-sync.

• Granular Duration Control (1–15 seconds): Clips can be requested at any integer second from 1 to 15.

• Broad Format Support: Outputs H.264 MP4 at 24 FPS across seven aspect ratios (1:1, 16:9, 9:16, 4:3, 3:4, 3:2, 2:3), at 480p or 720p.

3. Using References

Reference images are passed in image_urls and are addressed in array order: the first entry is <IMAGE_0>, the second <IMAGE_1>, and so on.

Reference voices

voice_ids selects up to 3 xAI preset voices to supply the speaking voice for generated dialogue, e.g. "voice_ids": ["ara"]. These are first-party preset identifiers drawn from the same catalog as the text-to-speech API:

altair, ara, atlas, carina, castor, celeste, cosmo, eve, helios, helix, iris, kepler, leo, lumen, luna, lux, naksh, orion, perseus, rex, rigel, sal, sirius, ursa, zagan, zenith

Custom or uploaded audio is not supported — the voice must be one of the presets above. Omit voice_ids to let the model choose a voice itself.

A prompt that uses the tags explicitly performs considerably better than one that describes the references in prose:

"Handheld UGC-style product review. The creator from <IMAGE_0> holds the skincare jar from <IMAGE_1>, looks at the camera, and talks about the product. Casual phone-camera framing, short social clip."

Practical guidance:

• One subject per reference image. Clean, well-lit, unobstructed references preserve identity best; crowded or low-resolution references degrade likeness.
• Name every reference you pass. An image supplied but never tagged in the prompt may be ignored or blended unpredictably.
• Reference-to-video is capped at 720p. Use text-to-video or image-to-video when 1080p output is required.

4. Model Architecture & Technical Details

xai/grok-imagine-video-v1.5/reference-to-video is built on xAI's Aurora engine, an autoregressive mixture-of-experts network that predicts next tokens across an interleaved sequence of text, image, video, and audio modalities. This is architecturally distinct from the diffusion-transformer designs used by OpenAI Sora and Google Veo, and is the mechanism by which V1.5 produces joint video+audio output in a single forward pass.

Key infrastructure and lineage points:

• Training infrastructure: Trained on xAI's Colossus 2 supercomputer, a ~2 GW, ~555,000 NVIDIA GPU facility — the largest known single-site AI training cluster.
• R&D lineage: The video pipeline incorporates technology from Hotshot, a video generation startup acquired by xAI in March 2025.
• Aurora foundation: The underlying Aurora image model was first released on December 9, 2024, with video capability progressively layered on top through Imagine 0.9 (October 2025), Imagine 1.0 (February 2026), multi-image and extension support (March 2026), and the V1.5 line from May 2026.
• Joint token modeling: Because audio and video tokens are produced in the same autoregressive stream, lip-sync and event-aligned SFX emerge from the model rather than from separate alignment models.

xAI has not published a technical report, parameter count, training-data disclosure, or formal model card for V1.5, so finer architectural details are not publicly documented.

5. Use Cases

• UGC-Style Product Marketing: Combine a creator reference and a product reference to produce authentic-feeling social product clips at volume — the flagship use case for this mode.

• Consistent Spokesperson Content: A fixed person reference yields a recognisable presenter across an entire content series.

• Brand and Product Placement: Products, packaging, and brand assets can be carried into generated scenes with their real appearance preserved rather than approximated.

• Character Continuity Across Shots: Reusing the same reference set across multiple generations keeps a character visually consistent between clips in a sequence.

• Style Transfer: A reference image supplied purely for look — palette, texture, art direction — steers the aesthetic of an otherwise freely composed scene.

• Localized Variants: The same visual references with different prompts produce market-specific versions of one creative concept.

The model is less well-suited to prompt-only ideation (use text-to-video), animating one specific image as the literal first frame (use image-to-video), and any delivery requiring 1080p output.