Ambience AI

Guide to Prompting WAN 2.1 for Video Generation

Master AI video creation with text-to-video and image-to-video prompting techniques

Introduction to WAN 2.1 Video Prompting

WAN 2.1 is a state-of-the-art AI model from Alibaba that can generate short video clips from either text descriptions or images. It powers Ambience AI's video creation pipeline, enabling creators to produce high-quality videos with rich motion and detail.

In Ambience AI, the typical workflow involves a two-step process: first creating a key image (using a text-to-image model like Flux), and then animating that image into a video with WAN 2.1.

This guide will explain how WAN 2.1 interprets text and image inputs, and walk through best practices for prompting the model – covering both text-to-video and image-to-video scenarios.

Understanding WAN 2.1's Video Generation

WAN 2.1 is a diffusion-based generative model trained on over a billion video clips. Think of it as a "mini movie director" – you give it a script (text prompt) or a starting image (plus optional text), and it produces a short video clip.

How WAN 2.1 Interprets Your Prompts

Text-to-Video Mode

When you provide a text prompt alone, WAN 2.1 will imagine the entire scene and action from scratch. It parses the prompt for subjects, actions, and style cues, then synthesizes a sequence of frames that depict the described scene in motion.

Image-to-Video Mode

When you provide an image plus a prompt, WAN 2.1 uses the image as the starting point and animates it according to the text instructions. The model preserves key elements of the image while introducing movement or changes guided by the prompt.

Technical Capabilities

  • Generates up to ~5-second clips at 480p or 720p resolution
  • Can generate legible text within videos (use sparingly)
  • Follows complex instructions and adheres to physical principles
  • Uses 14B-parameter model for best quality in Ambience AI

Best Practices and Key Elements

The golden rule for prompting WAN 2.1 is to be clear and sufficiently detailed in describing the scene and action you want. The more precise and rich your prompt, the closer the video will match your vision.

Essential Prompting Elements

1. Subject & Scene Setup

Describe who/what and where - the main elements and setting of your video.

2. Action & Motion

Specify the movement or activity that should occur during the video.

3. Camera Movement

Include camera directions like "camera follows," "smooth pan," or "close-up."

4. Style & Atmosphere

Set the mood with lighting, atmosphere, and artistic style descriptors.

Example: Structured Prompt

"A knight in shining armor stands by a medieval castle gate at dusk. He mounts a dragon and takes off into the sky as the camera pulls back. Cinematic lighting, glowing sunset clouds."

Subject & Scene: Knight, castle gate, dusk

Action: Mounts dragon, takes off

Camera: Camera pulls back

Style: Cinematic lighting, sunset clouds

Key Motion & Camera Keywords

Subject Motion

walking, running, flying, dancing, rotating, transforming

Camera Movement

pan, tilt, zoom in, orbit, follows, push-in, pull back

Shot Types

close-up, wide shot, drone shot, first-person view, bird's eye

Text-to-Video vs Image-to-Video Prompting

Understanding the differences between these two modes helps you choose the right approach and craft more effective prompts.

AspectText-to-VideoImage-to-Video
Starting pointText prompt onlyImage + text prompt
Prompt focusComplete scene descriptionMotion and changes to image
Visual consistencyDepends on prompt clarityHigh - anchored by input image
Best forCreative freedom, new scenesSpecific subjects, brand consistency
Typical workflowSingle-step generationTwo-step: Image → Video

Text-to-Video Tips

  • Be vivid and unambiguous
  • Include concrete nouns and active verbs
  • Stick to one main scene per clip
  • Consider using prompt enhancement tools

Image-to-Video Tips

  • Ensure prompt aligns with image content
  • Focus on motion/animation description
  • Frame subjects well in the input image
  • Use for consistent branding/products

Prompt Templates for Common Use Cases

Different creative goals call for different prompt styles. Here are actionable templates for popular use cases.

📱 Marketing/Product Video

Template:

[Shot type] of [Product] in [Scene/Background], [Movement or camera action], [Lighting style], [Background details].

Example:

"Close-up shot of a new smartphone on a reflective black surface, camera slowly rotates around the phone. Studio lighting catches the metal edges and the screen's glow, against a dark blurred background."

Key Tips:

  • Use adjectives like "sleek," "professional," "high-definition"
  • Keep camera movement simple (rotating, gentle slides)
  • Specify professional lighting and clean backgrounds

📱 Social Media Content

Template:

[Subject/Trend] [Action], [Context or background], [Camera style], [Mood/Filters].

Example:

"First-person view skateboarding down a street, camera GoPro style on the skateboard. Fast motion, slight fisheye lens effect, urban afternoon setting, thrilling mood."

Key Tips:

  • Include trendy descriptors like "viral," "aesthetic"
  • Embrace imperfection (handheld camera, shaky cam)
  • Use vibrant colors and high-energy motion

🎬 Short Cinematic Scene

Template:

[Subject] in [Setting], [Action]; [Camera angle/movement]; [Atmosphere]; [Style]

Example:

"A lone astronaut wanders through an alien forest at twilight. The camera tracks from behind through misty trees. Soft bioluminescent glow from plants lights the scene, creating a mysterious, awe-inspiring atmosphere. 4K cinematic detail."

Key Tips:

  • Mention time of day and lighting conditions
  • Use cinematic camera language (wide-angle, tracking shot)
  • Include genre-specific style cues ("film noir," "epic")

Technical Settings and Optimization

Understanding technical parameters helps you optimize both quality and generation speed.

Key Settings

Guidance Scale

Controls how strictly the model follows your prompt. Recommended: 5-7

Diffusion Steps

Quality vs speed trade-off. Typical range: 20-30 steps per frame

Resolution

480p for speed, 720p for quality. Higher resolution = longer generation time

Negative Prompts

Use negative prompts to avoid unwanted artifacts:

Common Negative Prompt:

"no text, no watermark, no blur, no distortion, no logos, no subtitles"

Add specific terms if you encounter unwanted elements in your generations.

Performance Considerations

  • A 5-second 720p clip can take several minutes on high-end GPU
  • Higher guidance can cause flickering between frames
  • Plan for concise clips focusing on single scenes or actions

Troubleshooting Common Issues

Even with the best prompting techniques, you may encounter certain challenges. Here are solutions to common issues:

Video is Off-Topic

Problem: Generated video doesn't match the prompt

Solution: Make prompt more explicit, increase guidance scale, or break complex scenes into simpler components

Flickering Between Frames

Problem: Video has jittery, unstable motion

Solution: Lower guidance scale (try 5 instead of 7), increase diffusion steps, or simplify the motion

Unwanted Text/Artifacts

Problem: Random text or visual artifacts appear

Solution: Add specific negative prompts like "no text, no watermark, no blur"

Inconsistent Subject Appearance

Problem: Subject changes appearance mid-video

Solution: Use image-to-video mode for consistency, or make subject description more detailed and specific

Slow Generation

Problem: Video takes too long to generate

Solution: Use 480p resolution, reduce diffusion steps to 20, or create shorter clips

Poor Motion Quality

Problem: Motion looks unnatural or choppy

Solution: Use clearer motion keywords, describe motion that fits the timeframe (3-5 seconds), or try image-to-video for better control

Start Creating Video Content with WAN 2.1

By following this guide, you can harness WAN 2.1's capabilities to create compelling video content for marketing, social media, and creative projects. Remember that effective video prompting combines clear scene description, specific motion direction, and appropriate technical settings.

Whether you're using text-to-video for creative freedom or image-to-video for brand consistency, the key is to practice and iterate. Each prompt is an opportunity to refine your approach and develop intuition for what works best with WAN 2.1.

Start with simple prompts following our templates, then gradually experiment with more complex scenes as you become comfortable with the model's capabilities and limitations.

Sources & Citations

This guide has been compiled based on research and expert insights from the following sources:

Ready to Create Amazing Videos with WAN 2.1?

Put your new video prompting skills to use with our AI video generator. Create compelling video content using the techniques you've just learned.

Learn MoreStart Creating Free