Open In Github

Python scripts for performing optical flow estimation using the RAFT model in ONNX

Original video:https://youtu.be/3wdsE1UgP6k

Requirements

• Check the requirements.txt file. Additionally, pafy and youtube-dl are required for youtube video inference.

Installation

pip install -r requirements.txt
pip install pafy youtube_dl>=2021.12.17

ONNX model

The original models were converted to different formats (including .onnx) by PINTO0309, download the models from his repository and save them into the models folder.

Original Pytorch model

The Pytorch pretrained model was taken from the original repository.

Examples

• Image inference:

 python image_flow_estimation.py

• Video inference:

 python video_flow_estimation.py

• Webcam inference:

 python webcam_flow_estimation.py

Inference video Examples: https://youtu.be/GNwyuhYu7ZI

Cheetah

Original video: https://youtu.be/-KheqfpUpr0

Water drop

Original video: https://youtu.be/gS_tU6chC4A

Plant

Original video: https://youtu.be/cNr_cttSf4U

Blink

Original video: https://youtu.be/lkJ4p__ZByU

Fall

Original video: https://youtu.be/RKTXn_c2tyQ

Baseball player

Original video: https://youtu.be/t_vEJu3jmpw

Dance

Original video: https://youtu.be/1WIA6Yvj8Yg

References:

• RAFT model: https://github.com/princeton-vl/RAFT
• PINTO0309’s model zoo: https://github.com/PINTO0309/PINTO_model_zoo
• PINTO0309’s model conversion tool: https://github.com/PINTO0309/openvino2tensorflow
• OpticalFlowToolkit toolkit: https://github.com/liruoteng/OpticalFlowToolkit
• Original paper: https://arxiv.org/abs/2003.12039