MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model
Paper • 2405.20222 • Published • 11
Updates 🔥🔥🔥
We have released the Gradio demo for Hybrid (Trajectory + Landmark) Controls HERE!
Introduction
This repo provides the inference Gradio demo for Trajectory Control of MOFA-Video.
Environment Setup
pip install -r requirements.txt
Download checkpoints
Download the pretrained checkpoints of SVD_xt from huggingface to
./ckpts.Download the checkpint of MOFA-Adapter from huggingface to
./ckpts.
The final structure of checkpoints should be:
./ckpts/
|-- controlnet
| |-- config.json
| `-- diffusion_pytorch_model.safetensors
|-- stable-video-diffusion-img2vid-xt-1-1
| |-- feature_extractor
| |-- ...
| |-- image_encoder
| |-- ...
| |-- scheduler
| |-- ...
| |-- unet
| |-- ...
| |-- vae
| |-- ...
| |-- svd_xt_1_1.safetensors
| `-- model_index.json
Run Gradio Demo
python run_gradio.py
Please refer to the instructions on the gradio interface during the inference process.
Paper
arxiv.org/abs/2405.20222
- Downloads last month
- -
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support