Skip to main content

Blog

How to Make an AI Dance Video From a Photo With Motion Control

Make an AI dance video from a photo: match the two inputs, pick the right orientation, and know when Pro is worth it.

Written by
trendshiftlabsEditorial
Published
Reading time
8 min
Sections
08
How to Make an AI Dance Video From a Photo With Motion Control

You've seen the clips. A still photo of someone, or a character somebody drew, suddenly doing the dance that's everywhere that week. You try it yourself and get a figure whose legs melt into the floor by second four. Making an AI dance video from a photo has less to do with the prompt than you'd think. It comes down to two inputs agreeing with each other.

This guide uses Kling 2.6 Motion Control, which runs in the Video to Video tool in two tiers, Standard and Pro. Kling has since released a 3.0 version of Motion Control. We don't run that one, so everything below is about 2.6.

How an AI dance video from a photo works: one clip for motion, one image for the look

Motion control takes two files. A reference video supplies the movement: the steps, the arm swings, the head turns, the timing. A character image supplies what you see: the face, the outfit, the body shape. The model reads the pose in each frame of the video and redraws your character in that pose.

That split explains most of what you'll get. The model never invents choreography. If the reference dancer spins, your character spins. If the reference is shaky, blurry, or half off screen, the model is guessing, and guesses show up as wobbly hands and arms that swap sides.

There's an optional prompt on top. Kling's own tutorial says to use it for "production direction, not choreography", meaning the light, the style, the setting. Something like "neon-lit rooftop at night, light rain, cinematic" works. "She kicks left then spins twice" fights the reference instead of helping it.

Woman in a long white dress sitting in sand dunes
Placeholder image. A full-figure photo gives the model a whole body to work with.

Pick a reference clip the model can read

The reference clip decides more of the result than your photo does. Kling's tutorial asks for 3 to 30 seconds of motion with the full body clearly visible. A clip that passes these checks is far easier for the model:

  • One dancer. A second person crossing the frame muddles which body to follow.
  • No cuts.
  • The whole body in frame the entire time, feet included.
  • A steady camera and a background that barely moves.
  • Moderate speed. Fast footwork and quick hand flicks blur, and blurred limbs become guessed limbs.
  • Arms that don't stay wrapped around the body for long, since the model can't track what it can't see.

Trending dance clips often fail two or three of these at once: jump cuts on the beat, a phone camera swinging around, a friend walking into shot. You'll usually do better filming yourself or a friend doing the routine against a plain wall, in one take, with the phone propped on a shelf. It also settles the question of whose performance you're copying.

Match your photo to the clip

This is the rule people skip, and Kling puts it plainly:

If your video shows a full-body shot of a dancer, your image must also be a full-body shot.

Kling, AI motion transfer tutorial

A head-and-shoulders portrait paired with a full-body dance forces the model to invent legs, hips, and shoes it has never seen, and it invents them a little differently in every frame. That's where the melting comes from. Check the photo against the clip before you upload:

CheckWhy it matters
Same framing: full body with full body, half body with half bodyMismatched framing is the most common cause of warped bodies
Head, arms, and legs all visibleHidden parts get invented, and invented parts drift
Similar camera angleA photo shot from above next to an eye-level clip stretches proportions
Arms apart from the torsoVisible gaps help the model keep limbs separate
Sharp, well lit faceSoft detail in the photo turns into shimmer once it moves
Short edge at least 300pxKling's stated minimum. Bigger is better
Run through this before every upload. Most failed dance clips break at least one row.

If the photo you have doesn't fit, fix the photo instead of hoping. You can try extending a cropped portrait into a full-body shot, or changing an awkward pose, in Image to Image, then bring the result over. The cost is an extra generation and the risk that the edit shifts the face slightly, so compare the two side by side before you commit.

Orientation: follow the video or follow the image

The composer has a Character orientation setting with two choices. The name sounds technical. The difference isn't.

  • Follows video is the default. Which way the character faces, its expressions, and the camera framing all come from the reference clip. If the dancer turns their back, so does your character. The studio accepts a reference clip up to 30 seconds in this mode.
  • Follows image still takes the moves from the video, but keeps the character's orientation anchored to your photo, and lets you steer the camera with the prompt. The reference limit drops to 10 seconds.

For dance, pick Follows video. Dances turn, step sideways, and face away, and this mode lets the character do all of it. Follows image suits cases where the photo's pose matters more than matching every turn, like a front-facing mascot that should stay front-facing while it bobs to a beat. You pay for that with the shorter limit and some stiffness when the dancer rotates hard.

Standard vs Pro

Both tiers take the same inputs and the same settings. Pro is the pricier one, and it's aimed at higher fidelity: cleaner detail, and hands, faces, and fabric that hold together better through fast moves. Standard is cheaper and good enough to find out whether a photo and clip pair works at all.

Our recommendation: run the first attempt on Standard with a short cut of the reference, around 5 to 8 seconds. If the body tracks the dance and the face holds, run the full clip on Pro. If Standard falls apart, don't expect Pro to save it. A framing mismatch or a blurry reference breaks both tiers, so fix the inputs first.

In both tiers the cost scales with the length of the reference clip. The composer shows the credit estimate for your exact clip before you press Generate, so a 25 second run on Pro is something you choose on purpose, not something you find out about later.

Sound, length, and loops for social

Motion control keeps the reference clip's original sound by default. For a dance that's what you want, because the beat lines up with the moves: they came from the same video. There's a toggle in the composer to turn it off if you plan to add a track in an editor. If you do swap the music, keep the same tempo, or the steps will land off the beat.

Length is where most people overspend. A social dance post rarely needs 30 seconds. The recognizable part of a routine is usually a short phrase, so cut the reference down to that before you upload. Shorter clips cost less and leave less time for small errors to pile up.

For a loop, trim the reference so the dancer finishes in roughly the pose they started in. The output follows the reference pose by pose, so a clean loop in the source tends to give you a clean loop in the result. And if the post is for a phone screen, keep both inputs vertical so they agree on shape as well as framing.

Common failures and fixes

What you seeLikely causeFix
Legs or feet melt or flickerPhoto cropped tighter than the referenceUse a full-body photo, or a reference framed like your photo
Arms swap sides or merge into the bodyLimbs overlap the torso in the clip or the photoPick a clip with clear gaps between arms and body
Face changes partway throughSmall or soft face in the photo, or long back-facing sectionsUse a sharper photo with a larger face; skip clips where the dancer faces away for long
Character jitters on the spotHandheld, shaky referenceFilm on a tripod or a shelf
The wrong body gets trackedTwo people in the referenceOne dancer only
Stiff, flat turnsOrientation set to Follows imageSwitch to Follows video
Odd moves the dancer never madePrompt describes choreographyDescribe the setting and light, not the steps

When a run fails, change one thing and rerun on Standard with a short clip. Swap the photo, the clip, and the prompt all at once and you learn nothing about which one was the problem.

Rider in a mint helmet sitting on a mint motorcycle
Placeholder image. A full-body subject with a large prop. The model has no motion in the reference for the prop, so it has to guess how it moves.

The short version

  • Motion control takes the moves from a video and the look from a photo. The two have to agree.
  • Use a one-person, one-take reference with the whole body in frame and a steady camera.
  • Full-body photo for a full-body dance.
  • Follows video for dance, up to 30 seconds. Follows image when the photo's pose matters more, up to 10 seconds.
  • Test on Standard with a short cut, then run the final on Pro.
  • Only animate people who agreed to it.

Open Video to Video and try a short test with your own photo, or look through the other video tools first. If you want a still to move without a reference clip, Image to Video is the one for that.

Filed under

  • ai dance video from photo
  • kling motion control tutorial
  • motion transfer ai
  • make a photo dance ai
  • ai dance video generator
All posts

Keep reading

View all posts

Start now

Your next idea is one prompt away.

Browse the studio free, then pick a plan when you want to generate. Image, video, and 3D tools share one account and gallery.