Arash Azimi Fard; Ali Hosseininaveh Ahmad Abadian
Abstract
Extended Abstract
Introduction
Due to the complexity of frame processing used for positioning and mapping in visual odometry (VO) and visual simultaneous localization and mapping (VSLAM) algorithms, key-frame selection methods have been introduced to improve the performance and decrease the number ...
Read More
Extended Abstract
Introduction
Due to the complexity of frame processing used for positioning and mapping in visual odometry (VO) and visual simultaneous localization and mapping (VSLAM) algorithms, key-frame selection methods have been introduced to improve the performance and decrease the number of frames required for processing while maintaining accuracy and robustness of the algorithms. Selected key-frames in these methods make a very good representation of all available frames. The current key-frame selection methods rely on heuristic thresholds in their selection procedure. Researchers have used several datasets to find optimum values for these thresholds through trial and error. In fact, proposed methods may not work as expected with a new dataset due to changes occurring in the sensor, environment and the platform.
Materials & Method
The present study has proposed an improved geometric and photogrammetric key-frame selection method built upon ORB-SLAM3, as the state of the art visual SLAM algorithm. The proposed Photogrammetric Key-frame Selection (PKS) algorithm has replaced inflexible heuristic thresholds with photogrammetric principles and thus guaranteed the robustness of the algorithm and the quality of the point cloud obtained from the key-frames. First, an adaptive threshold decides the allowable number of points whose line of sight zone has changed on a four-zone cone built upon each point. Increased number of points whose line of sight zone has changed means increased changes and displacements of the frame and thus, increased need for a new key-frame. Then, a 3*3 grid was formed in each frame and the number of points with a more than 30-degree change in line of sight angle (effective points) in each cell were counted. Later, the Equilibrium of Center Of Gravity (ECOG) criterion decides whether the distribution of points is appropriate using the center of gravity of the points inside the frame. Appropriate distribution of effective points within the frame shows a high geometric strength and thus will improve the strength of key-frames network. IMU sensor is not dependent on the position of the frames and the camera sensor. Thus, it independently obtains the key-frame in case significant changes occur in acceleration. The threshold value of acceleration has been experimentally considered equal to 1 meter per square second, which entirely depends on the type of robot. For ground robots with slower moving speeds, this threshold must be reset.
Results & Discussion
The present study has employed data collected by the European Robotics Challenge (EuRoC) flying robot containing the information collected by the synchronized camera and IMU information, as well as the ground truth data such as the robot trajectory and point cloud formed by the laser scanner. To evaluate the proposed method, extensive experiments have been implemented on the EuRoC dataset in mono-inertial and stereo-inertial modes. Then, trajectory of each algorithm was compared with the reference trajectory and point clouds formed by the key-frames were also compared. Apart from these qualitative evaluations, absolute trajectory error (ATE) obtained from running the PKS and ORB-SLAM3 algorithm 10 times were also compared quantitatively and finally, the error histogram was used to evaluate the point clouds. The processing time of each algorithm was also evaluated for each EuRoC dataset sequence. Results indicated that the proposed algorithm has improved ORB-SLAM3 accuracy in stereo-inertial by 18.1% and in the mono-inertial mode by 20.4% producing a more complete and accurate point cloud and thus, extracting more details from the environment. Furthermore, despite higher density of the point cloud, the error histogram has not changed significantly and fewer errors were observed in the ORB-SLAM3 algorithm.
Conclusion
Findings indicated that the PKS method has succeeded in extracting key-frames using photogrammetric and geometric principles. Apart from improving the positioning accuracy of the robot, the method has produced a much more complete and dense point cloud as compared to the ORB-SLAM3 algorithm. Also, dependency of the PKS method on the environment conditions and the type of system used (stereo camera or mono camera) was greatly reduced. Future studies can expand our key-frame selection method to include fisheye cameras or visual-only systems. More geometric conditions (near and far point condition and the vertex angle in the triangle formed by the points in the current frame, the camera and the corresponding points in the last key-frame) can also be added to the key-frame selection method.
