Mojdeh Ebrahimikia; Ali HosseiniNaveh
Abstract
Extended Abstract
Introduction
Today, orthophotos are one of the most widely used products in the field of spatial information, and they are often created from aerial or satellite images, so paying attention to their accuracy and quality is essential in order to have both geometric and radiometric ...
Read More
Extended Abstract
Introduction
Today, orthophotos are one of the most widely used products in the field of spatial information, and they are often created from aerial or satellite images, so paying attention to their accuracy and quality is essential in order to have both geometric and radiometric information. The point clouds and the digital surface model used to build them are the two most important aspects that affect the quality of these images. On true orthophotos, there are some distortions on the structural edges of buildings, which is due to defects in these areas in the point cloud used in the digital surface model. This problem is greater for orthophotos that have been made from UAV images in urban areas because of their lower altitude. Additionally, because of the presence of occluded regions and radiometric changes between overlapping images, approaches for creating point clouds based on image matching are unable to produce complete point clouds and contain flaws, particularly towards the outer edges of objects with high height differences. Before interpolation of the point cloud and preparation of the digital surface model and then preparation of orthophotos of it, it is necessary to complete the point cloud in areas with defects. Some studies have shown that adding edge points has the effect of decreasing the distortion of true orthophotos. In this study, a new method for completing point clouds is proposed and explained in detail.
Materials & Methods
In this study, the imaging of the Yazd region was done with a Phantom 4 drone equipped with a DJI camera. The SfM algorithm has been used to calibrate the camera, estimate the internal and external camera parameters, and produce images without distortion and low-density point clouds, and SGM has been used to produce dense point clouds. In the proposed method, the purpose is to complete the incomplete points of the building. Assuming that the points on the roof of each building are predetermined, without noise, and have incomplete edges, these point clouds were used to complete them, and then added to the existing point clouds in their actual coordinates. The final point cloud was used in the preparation of digital models to produce irregular and then regular surfaces and in the preparation of true orthophotos using camera parameters and undistorted images. One of the images with buildings marked as numbers 1 to 4 was selected to perform tests and prepare orthophotos.
Results & Discussion
The lack of structural edge points on any roof, which is the distance between severe height differences between levels, causes the greatest amount of distortion on the edge of the roof and around it. Adding these points with edge line recognition and reconstruction algorithms to the point cloud improves the resulting digital surface model. Since the quality and accuracy of the digital elevation model directly affects the resulting orthophoto, using a more accurate digital elevation model improves these images. These point clouds have been modified in the proposed method, and quantitative and qualitative comparisons demonstrate improved results in eliminating distortion in the majority of the buildings studied. The reasons for the superiority of the proposed method over previous methods include determining and calculating a more complete and precise form of the roof of each building and considering the outermost edges of the buildings.
Conclusion
The biggest amount of distortion on the edge of the roofs and their surroundings is caused by the lack of points on the structural edge of each roof, which is the boundary between dramatic height variations between the levels. By integrating these points with algorithms for recognizing and repairing edge lines, the resulting digital elevation model will be improved. This study presented a new method for completing the point cloud that enhanced the quality of true orthophoto edges, which was tested on a large number of building images and compared to the results of existing methods. In addition to implementing a new method for improving point clouds for orthophoto creation, the degree of distortion on the selected edge of four buildings has been greatly reduced when compared to the previous method. The success of the results with the latest proposed method of true orthophoto enhancement indicates an improvement of about 62% and 55% in the distortion decreasing of the structural edges and maintaining their coordinate accuracy.
The proposed method did not uniformly reduce the distortions at the structural edges, and future advanced models could possibly improve it.
Arash Azimi Fard; Ali Hosseininaveh Ahmad Abadian
Abstract
Extended Abstract
Introduction
Due to the complexity of frame processing used for positioning and mapping in visual odometry (VO) and visual simultaneous localization and mapping (VSLAM) algorithms, key-frame selection methods have been introduced to improve the performance and decrease the number ...
Read More
Extended Abstract
Introduction
Due to the complexity of frame processing used for positioning and mapping in visual odometry (VO) and visual simultaneous localization and mapping (VSLAM) algorithms, key-frame selection methods have been introduced to improve the performance and decrease the number of frames required for processing while maintaining accuracy and robustness of the algorithms. Selected key-frames in these methods make a very good representation of all available frames. The current key-frame selection methods rely on heuristic thresholds in their selection procedure. Researchers have used several datasets to find optimum values for these thresholds through trial and error. In fact, proposed methods may not work as expected with a new dataset due to changes occurring in the sensor, environment and the platform.
Materials & Method
The present study has proposed an improved geometric and photogrammetric key-frame selection method built upon ORB-SLAM3, as the state of the art visual SLAM algorithm. The proposed Photogrammetric Key-frame Selection (PKS) algorithm has replaced inflexible heuristic thresholds with photogrammetric principles and thus guaranteed the robustness of the algorithm and the quality of the point cloud obtained from the key-frames. First, an adaptive threshold decides the allowable number of points whose line of sight zone has changed on a four-zone cone built upon each point. Increased number of points whose line of sight zone has changed means increased changes and displacements of the frame and thus, increased need for a new key-frame. Then, a 3*3 grid was formed in each frame and the number of points with a more than 30-degree change in line of sight angle (effective points) in each cell were counted. Later, the Equilibrium of Center Of Gravity (ECOG) criterion decides whether the distribution of points is appropriate using the center of gravity of the points inside the frame. Appropriate distribution of effective points within the frame shows a high geometric strength and thus will improve the strength of key-frames network. IMU sensor is not dependent on the position of the frames and the camera sensor. Thus, it independently obtains the key-frame in case significant changes occur in acceleration. The threshold value of acceleration has been experimentally considered equal to 1 meter per square second, which entirely depends on the type of robot. For ground robots with slower moving speeds, this threshold must be reset.
Results & Discussion
The present study has employed data collected by the European Robotics Challenge (EuRoC) flying robot containing the information collected by the synchronized camera and IMU information, as well as the ground truth data such as the robot trajectory and point cloud formed by the laser scanner. To evaluate the proposed method, extensive experiments have been implemented on the EuRoC dataset in mono-inertial and stereo-inertial modes. Then, trajectory of each algorithm was compared with the reference trajectory and point clouds formed by the key-frames were also compared. Apart from these qualitative evaluations, absolute trajectory error (ATE) obtained from running the PKS and ORB-SLAM3 algorithm 10 times were also compared quantitatively and finally, the error histogram was used to evaluate the point clouds. The processing time of each algorithm was also evaluated for each EuRoC dataset sequence. Results indicated that the proposed algorithm has improved ORB-SLAM3 accuracy in stereo-inertial by 18.1% and in the mono-inertial mode by 20.4% producing a more complete and accurate point cloud and thus, extracting more details from the environment. Furthermore, despite higher density of the point cloud, the error histogram has not changed significantly and fewer errors were observed in the ORB-SLAM3 algorithm.
Conclusion
Findings indicated that the PKS method has succeeded in extracting key-frames using photogrammetric and geometric principles. Apart from improving the positioning accuracy of the robot, the method has produced a much more complete and dense point cloud as compared to the ORB-SLAM3 algorithm. Also, dependency of the PKS method on the environment conditions and the type of system used (stereo camera or mono camera) was greatly reduced. Future studies can expand our key-frame selection method to include fisheye cameras or visual-only systems. More geometric conditions (near and far point condition and the vertex angle in the triangle formed by the points in the current frame, the camera and the corresponding points in the last key-frame) can also be added to the key-frame selection method.
Mina Karimi; Abolghasem Sadeghi Niaraki; Ali Hosseininaveh Ahmadabdian
Abstract
Extended Abstract Introduction Underground infrastructure such as electricity, gas, telecommunications, water and sewage are managed by different organizations. Since most projects in these organizations require drilling,and imprecise excavations will endanger infrastructure and result in extensive financial ...
Read More
Extended Abstract Introduction Underground infrastructure such as electricity, gas, telecommunications, water and sewage are managed by different organizations. Since most projects in these organizations require drilling,and imprecise excavations will endanger infrastructure and result in extensive financial and physical losses, drilling projects require having accurate information about the infrastructure status. However, reaching accurate position of facilities such as pipes and cables is difficult due to their being concealed underground.Nowadays, ubiquitous computing and new developments in Geospatial Information Systems (GIS) can be an appropriate solution to such problems. This new generation of GIS is called the Ubiquitous Geospatial Information System (UBGIS). New technologies such as Augmented Reality (AR) can visualize this infrastructure on platforms like smart phones or tablets. Such technologies show spatial and descriptive attributes of these utilities more interactively, and thus can be applied as a modern solution for this problem. One of the major features of AR is identifying and locating real-world objects with respect to the person’s head or a camera. To have an accurate Augmented Reality, the position and orientation (pose) of the camera should be estimated with high accuracy. Therefore, exterior orientation parameters of the camera are required for AR and tracking. Different methods are used to calculate these exterior orientation parameters. One of the most common methods applies different sensors,such as Global Positioning System (GPS) and Inertial Measuring Unit (IMU),embedded in smart phones or tablets to calculate these parameters. These sensors include accelerometers, gyroscopes, magnetic sensors and compasses. Althoughsimple and fast, this method is not suitable for accurate cases, because sensors of mobile phones or tabletscannot provide such high accuracy. Vision-based (sometimes called image-based) method is another way of estimating exterior orientation parameters. In this method, fixed or dynamic images are used to determine the position and orientation of camera. The method is more complex and slower, but more accurate than the first one. Materials and Methods Regarding previously mentioned issues, the present article aims to visualize underground infrastructure using both sensor-based and vision-based approaches of Augmented Reality. Since the sensors embedded in a mobile phone or tablet do not provide such an accuracy (an accuracy of a few centimeters considering diameter of pipes and width of streets and pavements), a novel vision-based approach is proposed. In this method, image-based techniques and special kinds of targets, known as coded targets, are used to estimate camera’s position and orientation along with space resection method. In photogrammetry,space resection involves determining the spatial position and orientation of an image based on thesize of ground control points appearing on the image. Since space resection is a nonlinear problem, existing methods involve linearization of the collinearity condition and the use of an iterative process to determine the final solution using the least squares method. The process also requires determination of the initial approximate values of the unknown parameters, some of which must be estimated using another least squares solution. In order to obtain suitable initial values for space resection procedure, data received from GPS, accelerometers, and magnetic sensors are used and a low-pass filter is applied to reduce noise and increase precision. Then, due to improved camera pose parameters, the resulting virtual model is overlaid at its correct real worldplanimetriclocation. The planimetric coordinates are shown graphically on the ground and the Z coordinate (depth) is presented as a descriptive parameter. Results and Discussion Both proposed methods were implemented and tested in an Android Operating System. Camera pose parameters were estimated and the virtual modelwas overlaid at its correct real world planimetric location and shown on camera. Then, the results were compared and evaluatedusingthe well-known photogrammetry software, Agisoft, with the aim of modelling and precise measuring based on basic photogrammetry and machine vision. For sensor-based method, mean accuracy of the position parameters equals 4.2908±3.951 meters and mean accuracy of orientation parameters equals 6.1796±1.478 degrees,whilein vision-based method,these decreases to 0.1227±0.325 meters and 2.2017±0.536 degrees, respectively. Thus, results indicate that the proposed methodimprove accuracy and efficiency of AR technologies. Conclusion Augmented Reality is a technology that can be used to visualize underground facilities. Although,processing in sensor-based methods is sufficiently fast and simple, they lack the precision required for this purpose. Despite the fact that noise elimination and sensor integration using Kalman filter improves accuracy to some degree, it still does not reach the required accuracy. The present article sought to improve the accuracy of augmented reality in underground infrastructureusing targets. Results indicated that the machine vision and vision-based methods improve the accuracy. In drillings, third dimension (accuracy of height measurements) is as crucial as other parameters, thusit is suggested that future researches consider this not as a descriptive parameter, but as a three dimensional parameter to reach 3dimensional visualization.
