Call for Paper - January 2024 Edition
IJCA solicits original research papers for the January 2024 Edition. Last date of manuscript submission is December 20, 2023. Read More

A Comprehensive Study on Novel Video Frame Interpolation Methods

Print
PDF
 
Download Full Text
International Journal of Computer Applications
Foundation of Computer Science (FCS), NY, USA
Volume 183 - Number 15
Year of Publication: 2021
Authors:
Hrishikesh Mahajan, Yash Shekhadar, Shebin Silvister, Dheeraj Komandur, Nitin Pise
10.5120/ijca2021921472

Hrishikesh Mahajan, Yash Shekhadar, Shebin Silvister, Dheeraj Komandur and Nitin Pise. A Comprehensive Study on Novel Video Frame Interpolation Methods. International Journal of Computer Applications 183(15):6-10, July 2021. BibTeX

@article{10.5120/ijca2021921472,
	author = {Hrishikesh Mahajan and Yash Shekhadar and Shebin Silvister and Dheeraj Komandur and Nitin Pise},
	title = {A Comprehensive Study on Novel Video Frame Interpolation Methods},
	journal = {International Journal of Computer Applications},
	issue_date = {July 2021},
	volume = {183},
	number = {15},
	month = {Jul},
	year = {2021},
	issn = {0975-8887},
	pages = {6-10},
	numpages = {5},
	url = {http://www.ijcaonline.org/archives/volume183/number15/32000-2021921472},
	doi = {10.5120/ijca2021921472},
	publisher = {Foundation of Computer Science (FCS), NY, USA},
	address = {New York, USA}
}

Abstract

Video Frame Interpolation is the process of generating frames between two or more frames of a video. This process helps in the generation of slow-motion videos or helps in increasing the framerate of the video. Today, methods such as Optical Flow, Depth mapping and Visibility Mapping techniques are used to interpolate frames of high quality with less emphasis on Learning-Based methods. Thissurvey demonstrates a comprehensive overview of major research contributions in this domain. This paper provides an overview of 18 research papers along with novel architectures. The papers are compared with respect to two benchmark datasets: UCF 101 and Vimeo 90k across two metrics: Peak signal-to-noise ratio(PSNR) and Structural Similarity Index(SSIM).

References

  1. WenboBao, Wei-Sheng Lai, Chao Ma, Xiaoyun Zhang, Zhiyong Gao, Ming-Hsuan Yang; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 3703-3712
  2. Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, Jan Kautz; Super SloMo: High-Quality Estimation of Multiple Intermediate Frames for Video Interpolation, CVPR 2018.
  3. Huang, Z., Zhang, T., Heng, W., Shi, B., & Zhou, S. (2020). RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation. ArXiv, abs/2011.06294.
  4. Niklaus, Simon & Mai, Long & Liu, Feng. (2017). Video Frame Interpolation via Adaptive Separable Convolution. 261-270. 10.1109/ICCV.2017.37.
  5. Tran QN, Yang S-H. Efficient Video Frame Interpolation Using Generative Adversarial Networks. Applied Sciences. 2020; 10(18):6245. https://doi.org/10.3390/app10186245
  6. Meyer, Simone &Djelouah, Abdelaziz&Mcwilliams, Brian &Sorkine-Hornung, Alexander & Gross, Markus & Schroers, Christopher. (2018). PhaseNet for Video Frame Interpolation. 498-507. 10.1109/CVPR.2018.00059.
  7. S. Meyer, O. Wang, H. Zimmer, M. Grosse, and A. SorkineHornung. Phase-based frame interpolation for video. In Computer Vision and Pattern Recognition, pages 1410– 1418, 2015
  8. Y. Bengio, J. Louradour, R. Collobert, and J. Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pages 41–48. ACM, 2009
  9. Davischallenge.org. 2021. DAVIS: Densely Annotated Video Segmentation. [online] Available at: https://davischallenge.org.
  10. Niklaus, Simon & Liu, Feng. (2018). Context-aware Synthesis for Video Frame Interpolation.
  11. Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, Deep Residual Learning for Image Recognition, Dec 2015.
  12. D. Sun, X. Yang, M. Liu and J. Kautz, "PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 8934-8943, DOI: 10.1109/CVPR.2018.00931.
  13. Karen Simonyan, Andrew Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition, September 2014.
  14. T. Peleg, P. Szekely, D. Sabo and O. Sendik, "IM-Net for High-Resolution Video Frame Interpolation," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 2393-2402, DOI: 10.1109/CVPR.2019.00250.
  15. Vimeo.com. 2021. Vimeo | The world's only all-in-one video solution. [online] Available at: https://vimeo.com.
  16. T. Xue, B. Chen, J. Wu, D. Wei, and W. T. Freeman. Video enhancement with the task-oriented flow, 2017. arXiv preprint arXiv:1711.09078
  17. Choi, M., Kim, H., Han, B., Xu, N., & Lee, K. M. (2020). Channel Attention Is All You Need for Video Frame Interpolation. Proceedings of the AAAI Conference on Artificial Intelligence, 34(07), 10663-10671. https://doi.org/10.1609/aaai.v34i07.6693
  18. S. Wen, W. Liu, Y. Yang, T. Huang and Z. Zeng, "Generating Realistic Videos From Keyframes With Concatenated GANs," in IEEE Transactions on Circuits and Systems for Video Technology, vol. 29, no. 8, pp. 2337-2348, Aug. 2019, DOI: 10.1109/TCSVT.2018.2867934.
  19. Z. Liu, R. A. Yeh, X. Tang, Y. Liu and A. Agarwala, "Video Frame Synthesis Using Deep Voxel Flow," 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 4473-4481, DOI: 10.1109/ICCV.2017.478.
  20. X. Chen, W. Wang and J. Wang, "Long-term video interpolation with bidirectional predictive network," 2017 IEEE Visual Communications and Image Processing (VCIP), 2017, pp. 1-4, DOI: 10.1109/VCIP.2017.8305029.
  21. X. Chen, W. Wang and J. Wang, "Long-term video interpolation with bidirectional predictive network," 2017 IEEE Visual Communications and Image Processing (VCIP), 2017, pp. 1-4, DOI: 10.1109/VCIP.2017.8305029.
  22. Jean Bégaint, Franck Galpin, Philippe Guillotel, Christine Guillemot. Deep frame interpolation for video compression. DCC 2019 - Data Compression Conference, Mar 2019, Snowbird, United States. pp.1-10, ff10.1109/DCC.2019.00068ff. ffhal-02202172f
  23. Liu, Xiaozhang& Liu, Hui & Lin, Yuxiu. (2020). Video frame interpolation via optical flow estimation with image inpainting. International Journal of Intelligent Systems. 35. 10.1002/int.22285.
  24. Liu, Yu-Lun& Liao, Yi-Tung & Lin, Yen-Yu & Chuang, Yung-Yu. (2019). Deep Video Frame Interpolation Using Cyclic Frame Generation. Proceedings of the AAAI Conference on Artificial Intelligence. 33. 8794-8802. 10.1609/AAAI.v33i01.33018794.
  25. T. Jayashankar, P. Moulin, T. Blu and C. Gilliam, "Lap-Based Video Frame Interpolation," 2019 IEEE International Conference on Image Processing (ICIP), 2019, pp. 4195-4199, DOI: 10.1109/ICIP.2019.8803484.
  26. Niklaus, S., & Liu, F. (2020). Softmax Splatting for Video Frame Interpolation. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 5436-5445.

Keywords

Video Frame Interpolation, Deep Learning, Optical Flow, Video Processing