Reconstruction of three-dimensional scenes based on video flow data

Keywords: video flow, odometry, neural network, computer vision

Abstract

This work is dedicated to the application of modern algorithms for reconstructing spatial scenes from images to restore spatial information from video. The work is looking at a variety of modern methods, approaches, algorithms and trends in the field. The attention was paid to the sequence of development of approaches to the completion of the task. While researching the field and results related to three-dimensional reconstruction based on images and video streams, an algorithm was invented that allows constructing dense depth maps using information from all video frames. The idea is to use ready-made, commonly accepted, and tested solutions to solve two problems: COLMAP for visual odometry, and RAFT for computing optical flow. The algorithm shows quite accurate results and reconstructs the depth map in detail on arbitrary static scenes.

Downloads

Download data is not yet available.

References

Deep Two-View Structure-from-Motion Revisited. URL: https://openaccess.thecvf.com/content/CVPR2021/papers/Wang_Deep_Two-View_Structure-FromMotion_Revisited_CVPR_2021_paper.pdf

Xuan Luo, Jia-Bin Huang, Richard Szeliski, Kevin Matzen, and Johannes Kopf. 2020. Consistent video depth estimation. ACM Trans. Graph. 39, 4, Article 71 (August 2020), https://doi.org/10.1145/3386569.3392377

T. Caselitz, B. Steder, M. Ruhnke and W. Burgard, "Monocular camera localization in 3D LiDAR maps," 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016, pp. 1926-1931, DOI: https://doi.org/10.1109/IROS.2016.7759304

C. Campos, R. Elvira, J. J. G. Rodríguez, J. M. M. Montiel and J. D. Tardós, "ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM," in IEEE Transactions on Robotics, vol. 37, no. 6, pp. 1874-1890, Dec. 2021, https://doi.org/10.1109/TRO.2021.3075644

Teed Zachary, Lipson Lahav, Deng Jia, “Deep Patch Visual Odometry”. arXiv e-print, 2022, DOI:https://doi.org/10.48550/arXiv.2208.04726

Lahav Lipson, Zachary Teed, Jia Deng, "RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching", arXiv e-print, 2021, https://doi.org/10.48550/arXiv.2109.07547

Jiankun Li, Peisen Wang, Pengfei Xiong, Tao Cai, Ziwei Yan, Lei Yang, Jiangyu Liu, Haoqiang Fan, Shuaicheng Liu, "Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation", arXiv e-print, 2022, https://doi.org/10.48550/arXiv.2203.11483

Richard Hartley, Andrew Zisserman, “Multiple View Geometry in Computer Vision”, 2nd Edition, Cambridge University Press, 2003.

Weirong Chen, Suryansh Kumar, Fisher Yu, "Uncertainty-Driven Dense Two-View Structure from Motion", arXiv e-print , 2023, https://doi.org/10.48550/arXiv.2302.00523

Denys Hrulov (2024) Analysis of Three-dimensional Scenes based on Video flow data (master diploma) V. N. Karazin Kharkiv National University

Published
2024-09-11
Cited
How to Cite
Denys Hrulov, Anastasiia Morozova, Petro Dolia, & Liliia Bielova. (2024). Reconstruction of three-dimensional scenes based on video flow data. Computer Science and Cybersecurity, (1), 66-75. https://doi.org/10.26565/2519-2310-2024-1-06
Section
Статті