--- license: mit pipeline_tag: text-to-image library_name: diffusers ---

Kandinsky 5.0: A family of diffusion models for Video & Image generation

In this repository, we provide a family of diffusion models to generate a video or an image given a textual prompt and/or image. https://github.com/user-attachments/assets/b06f56de-1b05-4def-a611-1a3159ed71b0 ## Kandinsky 5.0 Image Lite Kandinsky 5.0 Image Lite is a line-up of 6B image generation models with the following capabilities: * 1K resulution (1280x768, 1024x1024 and others). * High visual quality * Strong text-writing * Russian concepts understanding ### Model Zoo | Model | config | NFE | Checkpoint | Latency* | |-------------------------------------|--------|-----|------------|----------------| | Kandinsky 5.0 T2I Lite |configs/k5_lite_t2i_sft_hd.yaml| 100 | 🤗 [HF](https://huggingface.co/kandinskylab/Kandinsky-5.0-T2I-Lite)| 13 s | | Kandinsky 5.0 T2I Lite pretrain |-| 100 | 🤗 [HF](https://huggingface.co/kandinskylab/Kandinsky-5.0-T2I-Lite-pretrain)| 13 s | *Latency was measured after the second inference run. The first run of the model can be slower due to the compilation process. Inference was measured on an NVIDIA H100 GPU with 80 GB of memory, using CUDA 12.8.1 and PyTorch 2.8. For 5-second models Flash Attention 3 was used. ### Examples:

### Results: #### Side-by-Side evaluation


Comparison with FLUX.1 dev	Comparison with Qwen-Image

## Kandinsky 5.0 Image Editing Kandinsky 5.0 Image Editing is a line-up of 6B image editing models with the following capabilities: - 1K resulution (1280x768, 1024x1024 and others). - High visual quality - Strong text-writing - Russian concepts understanding ### Model Zoo | Model | config | NFE | Checkpoint | Latency* | |-------------------------------------|--------|-----|------------|----------------| | Kandinsky 5.0 T2I Editing |configs/k5_lite_i2i_sft_hd.yaml| 100 | 🤗 [HF](https://huggingface.co/kandinskylab/Kandinsky-5.0-I2I-Lite) | - | | Kandinsky 5.0 T2I Editing pretrain |-| 100 | 🤗 [HF](https://huggingface.co/kandinskylab/Kandinsky-5.0-I2I-Lite-pretrain) | - | *Latency was measured after the second inference run. The first run of the model can be slower due to the compilation process. Inference was measured on an NVIDIA H100 GPU with 80 GB of memory, using CUDA 12.8.1 and PyTorch 2.8. For 5-second models Flash Attention 3 was used. ### Examples:


Change this to a cowboy hat.	Turn this into a neon sign hanging on a brick wall in a cool modern office.

Swap your sweatshirt for a se- quined evening dress, add some bright jewelry, and brighten your lips and eyes. Keep the angle.	Turn this into a real photograph of the same dog.

### Results: #### Side-by-Side evaluation


Comparison with FLUX.1 Kontext [dev]	Comparison with Qwen-Image-Edit-2509

## Quickstart #### Installation Clone the repo: ```sh git clone https://github.com/kandinskylab/kandinsky-5.git cd kandinsky-5 ``` Install dependencies: ```sh pip install -r requirements.txt ``` To improve inference performance on NVidia Hopper GPUs, we recommend installing [Flash Attention 3](https://github.com/Dao-AILab/flash-attention/?tab=readme-ov-file#flashattention-3-beta-release). #### Model Download ```sh python download_models.py ``` use `models` argument to download some specific models, otherwise all models will be downloaded example to download only `kandinskylab/Kandinsky-5.0-T2V-Lite-sft-5s` and `kandinskylab/Kandinsky-5.0-T2V-Pro-sft-5s`: ```sh python download_models.py --models kandinskylab/Kandinsky-5.0-T2V-Lite-sft-5s,kandinskylab/Kandinsky-5.0-T2V-Pro-sft-5s ``` #### Run Kandinsky 5.0 T2I Lite ```sh python test.py --config ./configs/k5_lite_t2i_sft_hd.yaml --prompt "A dog in a red hat" --width=1280 --height=768 ``` ### T2I Inference ```python import torch from kandinsky import get_T2I_pipeline device_map = { "dit": torch.device('cuda:0'), "vae": torch.device('cuda:0'), "text_embedder": torch.device('cuda:0') } pipe = get_T2I_pipeline(device_map, conf_path="configs/k5_lite_t2i_sft_hd.yaml") images = pipe( seed=42, save_path='./test.png', text="A cat in a red hat with a label 'HELLO'" ) ``` ### I2I Inference ```python import torch from kandinsky import get_I2I_pipeline device_map = { "dit": torch.device('cuda:0'), "vae": torch.device('cuda:0'), "text_embedder": torch.device('cuda:0') } pipe = get_I2I_pipeline( resolution=1024, offload=True, device_map=device_map, ) out = pipe( "Replace the cat with a husky, leave the rest unchanged", image='./assets/cat_in_hat.png' ) ``` Please, refer to [examples](examples) folder for more examples in various notebooks. ### Distributed Inference For a faster inference, we also provide the capability to perform inference in a distributed way: ``` NUMBER_OF_NODES=1 NUMBER_OF_DEVICES_PER_NODE=1 / 2 / 4 python -m torch.distributed.launch --nnodes $NUMBER_OF_NODES --nproc-per-node $NUMBER_OF_DEVICES_PER_NODE test.py ``` ### Optimized Inference #### Offloading For less memory consumption you can use **offloading** of the models. ```sh python test.py --prompt "A dog in red hat" --offload ``` #### Magcache Also we provide [Magcache](https://github.com/Zehong-Ma/MagCache) inference for faster generations (now available for sft 5s and sft 10s checkpoints). ```sh python test.py --prompt "A dog in red hat" --magcache ``` #### Qwen encoder quantization To reduce GPU memory needed for Qwen encoder we provide option to use NF4-quantized version from [bitsandbytes](https://github.com/bitsandbytes-foundation/bitsandbytes). ```sh python test.py --prompt "A dog in red hat" --qwen_quantization ``` #### Attention engine selection Depending on your hardware you can use the follwing full attention algorithm implementation: * PyTorch [SDPA](https://docs.pytorch.org/docs/stable/generated/torch.nn.functional.scaled_dot_product_attention.html) * [Flash Attention 2](https://github.com/Dao-AILab/flash-attention) * [Flash Attention 3](https://github.com/Dao-AILab/flash-attention/tree/main/hopper) * [Sage Attention](https://github.com/thu-ml/SageAttention) The attention algorithm can be selected using an option "--attention_engine" of test.py script for 5 second (and less) video generation. For 10-second generation we use sparse attention algorithm [NABLA](https://arxiv.org/abs/2507.13546). Note that currently (19 Oct. 2025) version build from source contains a bug and produces noisy output. A temporary workaround to fix it is decribed [here](https://github.com/thu-ml/SageAttention/issues/277). ```sh python test.py --prompt "A dog in red hat" --attention_engine=flash_attention_3 ``` ```sh python test.py --prompt "A dog in red hat" --attention_engine=flash_attention_2 ``` ```sh python test.py --prompt "A dog in red hat" --attention_engine=sdpa ``` ```sh python test.py --prompt "A dog in red hat" --attention_engine=sage ``` By default we use option --attention_engine=auto which enables automatic selection of the most optimal algorithm installed in your system. ### ComfyUI See the instruction [here](comfyui) ### CacheDiT cache-dit offers Fully Cache Acceleration support for Kandinsky-5 with DBCache, TaylorSeer and Cache CFG. Visit their [example](https://github.com/vipshop/cache-dit/blob/main/examples/pipeline/run_kandinsky5_t2v.py) for more details. ### Beta testing You can apply to participate in the beta testing of the Kandinsky Video Lite via the [telegram bot](https://t.me/kandinsky_access_bot). # Authors Core Contributors: - Video: Alexey Letunovskiy, Maria Kovaleva, Lev Novitskiy, Denis Koposov, Dmitrii Mikhailov, Anastasiia Kargapoltseva, Anna Dmitrienko, Anastasia Maltseva - Image & Editing: Nikolai Vaulin, Nikita Kiselev, Alexander Varlamov - Pre-training Data: Ivan Kirillov, Andrey Shutkin, Nikolai Vaulin, Ilya Vasiliev - Post-training Data: Julia Agafonova, Anna Averchenkova, Olga Kim - Research Consolidation & Paper: Viacheslav Vasilev, Vladimir Polovnikov Contributors: Yury Kolabushin, Kirill Chernyshev, Alexander Belykh, Mikhail Mamaev, Anastasia Aliaskina, Kormilitsyn Semen, Tatiana Nikulina, Olga Vdovchenko, Polina Mikhailova, Polina Gavrilova, Nikita Osterov, Bulat Akhmatov Track Leaders: Vladimir Arkhipkin, Vladimir Korviakov, Nikolai Gerasimenko, Denis Parkhomenko Project Supervisor: Denis Dimitrov # Citation ``` @misc{arkhipkin2025kandinsky50familyfoundation, title={Kandinsky 5.0: A Family of Foundation Models for Image and Video Generation}, author={Vladimir Arkhipkin and Vladimir Korviakov and Nikolai Gerasimenko and Denis Parkhomenko and Viacheslav Vasilev and Alexey Letunovskiy and Nikolai Vaulin and Maria Kovaleva and Ivan Kirillov and Lev Novitskiy and Denis Koposov and Nikita Kiselev and Alexander Varlamov and Dmitrii Mikhailov and Vladimir Polovnikov and Andrey Shutkin and Julia Agafonova and Ilya Vasiliev and Anastasiia Kargapoltseva and Anna Dmitrienko and Anastasia Maltseva and Anna Averchenkova and Olga Kim and Tatiana Nikulina and Denis Dimitrov}, year={2025}, eprint={2511.14993}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2511.14993}, } @misc{mikhailov2025nablanablaneighborhoodadaptiveblocklevel, title={$\nabla$NABLA: Neighborhood Adaptive Block-Level Attention}, author={Dmitrii Mikhailov and Aleksey Letunovskiy and Maria Kovaleva and Vladimir Arkhipkin and Vladimir Korviakov and Vladimir Polovnikov and Viacheslav Vasilev and Evelina Sidorova and Denis Dimitrov}, year={2025}, eprint={2507.13546}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2507.13546}, } ``` # Acknowledgements We gratefully acknowledge the open-source projects and research that made Kandinsky 5.0 possible: - [PyTorch](https://pytorch.org/) — for model training and inference. - [FlashAttention 3](https://github.com/Dao-AILab/flash-attention) — for efficient attention and faster inference. - [Qwen2.5-VL](https://github.com/QwenLM/Qwen3-VL) — for providing high-quality text embeddings. - [CLIP](https://github.com/openai/CLIP) — for robust text–image alignment. - [HunyuanVideo](https://huggingface.co/tencent/HunyuanVideo) — for video latent encoding and decoding. - [MagCache](https://github.com/Zehong-Ma/MagCache) — for accelerated inference. - [ComfyUI](https://github.com/comfyanonymous/ComfyUI) — for integration into node-based workflows. We deeply appreciate the contributions of these communities and researchers to the open-source ecosystem.