Skip to content
Get packs

First Order Motion Model

Animates a still face or object image using motion from a driving video, from the first-order motion model papAliaksandrSiarohin/first-order-model

GitHub Download source
Stars
15k
Forks
3.2k
Licence
MIT
Language
Jupyter Notebook
Last update
2.0 years ago
Type
AI model / agent

No commits in over a year — check for forks or alternatives before adopting.

What it is

This is the source code for a research paper on image animation. You give it one still image (the source) and a video of motion (the driving video), and it produces a video where the still image moves the way the driving video moves. The README shows examples on faces, fashion and other datasets. It is released as research code, with a pretrained checkpoint and a demo.

  • Animate a single portrait image using a driving video of a person talking
  • Run the provided demo to produce a result.mp4 from your own source image and driving video
  • Train the model on your own dataset by following the README's dataset preparation steps
  • Evaluate video reconstruction quality on a checkpoint using the reconstruction mode

Great for

  • Researchers reproducing or building on the NeurIPS 2019 first-order motion paper
  • Developers who want to experiment with keypoint-based image animation
  • Students learning how deep-learning motion transfer works
  • People who can cropped their own driving videos and source images to the method's requirements

Not ideal for

  • Casual users wanting a polished app: the README describes a command-line demo and a Colab/Kaggle notebook, not a consumer product
  • Anyone without a GPU for training: the README references multi-GPU training on Titan-X cards
  • Users needing face-swap out of the box: the README says it can be modified for face-swap with segmentation masks and points to a separate project for that
Step-by-step guide: how to use First Order Motion ModelInstall, first run, examples, tips and FAQs

How to get it

  1. 1

    Clone and build from source

    You'll need: Python 3 · pip · Docker 19.03+ and nvidia-docker (only if using the Docker route)

    Install dependencies

    pip install -r requirements.txt

    Build Docker container (optional)

    docker build -t first-order-model .
  2. 2

    Run it

    • Clone the repository and install dependencies with pip install -r requirements.txt
    • Download a pretrained checkpoint from the Google Drive or Yandex Disk link in the README
    • Prepare a cropped driving video and a source image
    • Run python demo.py with --config, --driving_video, --source_image, --checkpoint, --relative and --adapt_scale
    • Find the output in result.mp4
  3. 3

    Get the most out of it

    • Use the relative keypoint mode (--relative) when the source and driving objects have matching poses, as the README suggests for better results
    • Use crop-video.py with ffmpeg to get semi-automatic crop suggestions for driving videos
    • Check config/taichi-256.yaml for a description of each parameter before changing settings
    • Use the Colab or Kaggle demo notebook if you cannot run locally
    • See the linked Motion Co-Segmentation project if you need face-swap or supervised video editing

Downloads

Source code (.zip)master branch · straight from GitHubOfficial website & docsaliaksandrsiarohin.github.io/first-order-model-website/

Strengths

  • Open-source code tied to a published paper with documented commands
  • Includes a pretrained checkpoint link and a Colab/Kaggle demo for trying it without local setup
  • Provides Docker instructions for users who have library compatibility problems on Linux

Trade-offs

  • Inputs need to be cropped beforehand, and the README points to a helper script and an external library for that step
  • Relative-coordinate animation requires the driving and source objects to start in the same pose, per the README
  • Checkpoints are hosted on Google Drive and Yandex Disk, which may be inconvenient to access

How it compares

ProjectStarsTypeLevelUpdatedBest for
first-order-model (this)15kAI model / agentintermediate2.0 years agoResearchers reproducing or building on the NeurIPS 2019 first-order motion paper
denoising-diffusion-pytorch10.7kLibraryintermediate1 months agoResearchers and engineers who want a readable PyTorch reference for diffusion models
CLIP34.6kAI model / agentintermediate8 months agoPython developers experimenting with image classification using text labels
JARVIS25.5kAI model / agentadvanced1.3 years agoResearchers studying LLM-driven task planning and tool use
LaTeX-OCR16.6kAI model / agentintermediate1.8 years agoStudents who want to turn a handwritten or printed formula photo into editable LaTeX
facenet14.4kAI model / agentadvanced3.3 years agoResearchers reproducing the FaceNet paper with TensorFlow

Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.