November 12, 2020

558 words 3 mins read



RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation

repo name hzwer/arXiv2020-RIFE
repo link
language Python
size (curr.) 7915 kB
stars (curr.) 596
created 2020-11-12
license MIT License

RIFE Video Frame Interpolation

arXiv | Project Page | Reddit | YouTube_v1.2

11.20 News: I optimize the parallel processing, get 60% speedup!

Date of recent model update: 2020.11.19, v1.2

You can easily use colaboratory to have a try and generate the above youtube demo.

Our model can run 30+FPS for 2X 720p interpolation on a 2080Ti GPU. Currently our method supports 2X,4X,8X interpolation for 1080p video, and multi-frame interpolation between a pair of images. Everyone is welcome to use our alpha version and make suggestions!

16X interpolation results from two input images:

Demo Demo


We propose RIFE, a Real-time Intermediate Flow Estimation algorithm for Video Frame Interpolation (VFI). Most existing methods first estimate the bi-directional optical flows and then linearly combine them to approximate intermediate flows, leading to artifacts on motion boundaries. RIFE uses a neural network named IFNet that can directly estimate the intermediate flows from images. With the more precise flows and our simplified fusion process, RIFE can improve interpolation quality and have much better speed. Based on our proposed leakage distillation loss, RIFE can be trained in an end-to-end fashion. Experiments demonstrate that our method is significantly faster than existing VFI methods and can achieve state-of-the-art performance on public benchmarks.


$ pip3 install tqdm
$ pip3 install torch
$ pip3 install numpy
$ pip3 install opencv-python


  • Download the pretrained models from here. We are optimizing the visual effects and will support animation in the future.

(我们也提供了百度网盘链接: 密码:88bu,把压缩包解开后放在 train_log/*.pkl)

  • Unzip and move the pretrained parameters to train_log/*.pkl

The models under different setting is coming soon.

Video 2x Interpolation

You can use our demo video or use your own video to process.

$ python3 --exp=1 --video=video.mp4 

(generate video_2X_xxfps.mp4, you can use this script repeatly to get 4X, 8X…)

$ python3 --exp=2 --video=video.mp4

(we specificly support exp=2 for 4X interpolation)

$ python3 --exp=2 --video=video.mp4 --fps=60

(add slomo effect)

$ python3 --video=video.mp4 --montage --png

(if you want to montage the origin video, and save the png format output)

$ python3 --exp=2 --video=video.mp4

(Try our parallel process, may be useful on your device.)

The warning info, ‘Warning: Your video has *** static frames, it may change the duration of the generated video.’ means that your video has changed the frame rate by adding static frames, it is common if you have processed 25FPS video to 30FPS.

Image Interpolation

$ python3 --img img0.png img1.png --exp=4

(2^4=16X interpolation results) After that, you can use pngs to generate mp4:

$ ffmpeg -r 10 -f image2 -i output/img%d.png -s 448x256 -c:v libx264 -pix_fmt yuv420p output/slomo.mp4 -q:v 0 -q:a 0

You can also use pngs to generate gif:

$ ffmpeg -r 10 -f image2 -i output/img%d.png -s 448x256 -vf "split[s0][s1];[s0]palettegen=stats_mode=single[p];[s1][p]paletteuse=new=1" output/slomo.gif


First you should download RIFE model reported by our paper.

We will release our training and benchmark validation code soon.

Vimeo90K Download Vimeo90K dataset at ./vimeo_interp_test

$ python3
(You will get 35.695PSNR and 0.9788SSIM)


  title={RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation},
  author={Huang, Zhewei and Zhang, Tianyuan and Heng, Wen and Shi, Boxin and Zhou, Shuchang},
  journal={arXiv preprint arXiv:2011.06294},


Optical Flow: ARFlow pytorch-liteflownet RAFT

Video Interpolation: DAIN CAIN AdaCoF-pytorch

comments powered by Disqus