Scientific- Research Quarterly of Geographical Data (SEPEHR)

Scientific- Research Quarterly of Geographical Data (SEPEHR)

A new way to decrease the distortion of the edges of buildings on true orthophotos

Document Type : Research Paper

Authors
Mojdeh Ebrahimikia 1
Ali HosseiniNaveh 2
1 PhD Student of Photogrammetry, The Department of Geodesy and Geomatics Engineering, The Faculty of Surveying Engineering, K.N. Toosi University of Technology, Tehran, Iran
2 Associate professor, Department of Geodesy and Geomatics Engineering, The Faculty of Surveying Engineering, K.N. Toosi University of Technology, Tehran, Iran
10.22131/sepehr.2022.699911
Abstract
Extended Abstract
 Introduction
Today, orthophotos are one of the most widely used products in the field of spatial information, and they are often created from aerial or satellite images, so paying attention to their accuracy and quality is essential in order to have both geometric and radiometric information. The point clouds and the digital surface model used to build them are the two most important aspects that affect the quality of these images. On true orthophotos, there are some distortions on the structural edges of buildings, which is due to defects in these areas in the point cloud used in the digital surface model. This problem is greater for orthophotos that have been made from UAV images in urban areas because of their lower altitude. Additionally, because of the presence of occluded regions and radiometric changes between overlapping images, approaches for creating point clouds based on image matching are unable to produce complete point clouds and contain flaws, particularly towards the outer edges of objects with high height differences. Before interpolation of the point cloud and preparation of the digital surface model and then preparation of orthophotos of it, it is necessary to complete the point cloud in areas with defects. Some studies have shown that adding edge points has the effect of decreasing the distortion of true orthophotos. In this study, a new method for completing point clouds is proposed and explained in detail. 
Materials & Methods
In this study, the imaging of the Yazd region was done with a Phantom 4 drone equipped with a DJI camera. The SfM algorithm has been used to calibrate the camera, estimate the internal and external camera parameters, and produce images without distortion and low-density point clouds, and SGM has been used to produce dense point clouds. In the proposed method, the purpose is to complete the incomplete points of the building. Assuming that the points on the roof of each building are predetermined, without noise, and have incomplete edges, these point clouds were used to complete them, and then added to the existing point clouds in their actual coordinates. The final point cloud was used in the preparation of digital models to produce irregular and then regular surfaces and in the preparation of true orthophotos using camera parameters and undistorted images. One of the images with buildings marked as numbers 1 to 4 was selected to perform tests and prepare orthophotos.
Results & Discussion
The lack of structural edge points on any roof, which is the distance between severe height differences between levels, causes the greatest amount of distortion on the edge of the roof and around it. Adding these points with edge line recognition and reconstruction algorithms to the point cloud improves the resulting digital surface model. Since the quality and accuracy of the digital elevation model directly affects the resulting orthophoto, using a more accurate digital elevation model improves these images. These point clouds have been modified in the proposed method, and quantitative and qualitative comparisons demonstrate improved results in eliminating distortion in the majority of the buildings studied. The reasons for the superiority of the proposed method over previous methods include determining and calculating a more complete and precise form of the roof of each building and considering the outermost edges of the buildings.   
Conclusion
The biggest amount of distortion on the edge of the roofs and their surroundings is caused by the lack of points on the structural edge of each roof, which is the boundary between dramatic height variations between the levels. By integrating these points with algorithms for recognizing and repairing edge lines, the resulting digital elevation model will be improved. This study presented a new method for completing the point cloud that enhanced the quality of true orthophoto edges, which was tested on a large number of building images and compared to the results of existing methods. In addition to implementing a new method for improving point clouds for orthophoto creation, the degree of distortion on the selected edge of four buildings has been greatly reduced when compared to the previous method. The success of the results with the latest proposed method of true orthophoto enhancement indicates an improvement of about 62% and 55% in the distortion decreasing of the structural edges and maintaining their coordinate accuracy.
The proposed method did not uniformly reduce the distortions at the structural edges, and future advanced models could possibly improve it.
 
Keywords
orthophoto
point cloud
edge distortion
1- Agisoft, 2021. AgisoftPhotoScan User Manual: Professional Edition, Version 1.7. Agisoft LLC, St Petersburg, Russia. 187 pages.
2- Barazzetti, L., Brumana, R., Oreni, D., Previtali, M., &Roncoroni, F. (2014). True-orthophoto generation from UAV images: implementation of a combined photogrammetric and computer vision approach. ISPRS Annals of Photogrammetry, Remote Sensing & Spatial Information Sciences, 2(5).
3- Boccardo, P., Dequal, S., Lingua, A., &Rinaudo, F. (2001). True digital orthophoto for architectural and archaeological applications. INTERNATIONAL ARCHIVES OF PHOTOGRAMMETRY AND REMOTE SENSING, 34(5/W1), 50-55.
4- Canny, J. (1986). A computational approach to edge detection. IEEE transactions on pattern analysis and machine intelligence(6), 679-698.
5- Chang, Y., Jung, C., &Xu, Y. (2021). FinerPCN: High fidelity point cloud completion network using pointwise convolution. Neurocomputing, 460, 266-276.
6- Chen, X., Chen, B., &Mitra, N. J. (2019). Unpaired point cloud completion on real scans using adversarial training. arXiv preprint arXiv:1904.00069.
7- Chen, Y., Briese, C., Karel, W., & Pfeifer, N. (2014). True orthophoto generation using multi-view aerial images. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 40(3), 67.
8- Chibane, J., Mir, A., & Pons-Moll, G. (2020). Neural unsigned distance fields for implicit function learning. arXiv preprint arXiv:2010.13938.
9- Czerniawski, T., Ma, J. W., &Leite, F. (2021). Automated building change detection with amodal completion of point clouds. Automation in Construction, 124, 103568.
10- Dai, A., Diller, C., &Nießner, M. (2020). Sg-nn: Sparse generative neural networks for self-supervised scene completion of rgb-d scans. Paper presented at the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
11- Dai, A., Ruizhongtai Qi, C., &Nießner, M. (2017). Shape completion using 3d-encoder-predictor cnns and shape synthesis. Paper presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
12- Dunbar, D., & Humphreys, G. (2006). A spatial data structure for fast Poisson-disk sample generation. ACM Transactions on Graphics (TOG), 25(3), 503-508.
13- Ebrahimikia, M., &Hosseininaveh, A. (2022). True Orthophoto Generation Based on Unmanned Aerial Vehicle Images Using Reconstructed Edge Points. The Photogrammetric Record.
14- Gkioxari, G., Malik, J., & Johnson, J. (2019). Mesh r-cnn. Paper presented at the Proceedings of the IEEE/CVF International Conference on Computer Vision.
15- Groueix, T., Fisher, M., Kim, V., Russell, B., &Aubry, M. (1802). AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation. arXiv 2018. arXiv preprint arXiv:1802.05384.
16- Hirschmuller, H. (2008). Stereo processing by semiglobal matching and mutual information. IEEE Trans Pattern Anal Mach Intell, 30(2), 328-341. doi:10.1109/TPAMI.2007.1166
17- Hou, J., Dai, A., &Nießner, M. (2020). Revealnet: Seeing behind objects in rgb-d scans. Paper presented at the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
18- Huang, Z., Yu, Y., Xu, J., Ni, F., & Le, X. (2020). Pf-net: Point fractal network for 3d point cloud completion. Paper presented at the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
19- Kato, A., Moskal, L., Schiess, P., Calhoun, D., & Swanson, M. E. (2010). True orthophoto creation through fusion of LiDAR derived digital surface model and aerial photos: na.
20- Krauss, T., &Reinartz, P. (2010). Urban object detection using a fusion approach of dense urban digital surface models and VHR optical satellite stereo data.
21- Li, C.-L., Zaheer, M., Zhang, Y., Poczos, B., &Salakhutdinov, R. (2018). Point cloud gan. arXiv preprint arXiv:1810.05795.
22- Liu, M., Sheng, L., Yang, S., Shao, J., & Hu, S.-M. (2020). Morphing and sampling network for dense point cloud completion. Paper presented at the Proceedings of the AAAI conference on artificial intelligence.
23- Mohammadi, H., Samadzadegan, F., &Reinartz, P. (2019). 2D/3D information fusion for building extraction from high-resolution satellite stereo images using kernel graph cuts. International Journal of Remote Sensing, 40(15), 5835-5860. doi:10.1080/01431161.2019.1584417
24- Nielsen, M. Ø. (2004). True orthophoto generation: InformatikogMatematiskModellering, DanmarksTekniskeUniversitet.
25- Noronha, S., &Nevatia, R. (2001). Detection and modeling of buildings from multiple aerial images. IEEE transactions on pattern analysis and machine intelligence, 23(5), 501-518.
26- Qi, C. R., Su, H., Mo, K., &Guibas, L. J. (2017). Pointnet: Deep learning on point sets for 3d classification and segmentation. Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition.
27- Qin, C., You, H., Wang, L., Kuo, C.-C. J., & Fu, Y. (2019). Pointdan: A multi-scale 3d domain adaption network for point cloud representation. arXiv preprint arXiv:1911.02744.
28- Sarmad, M., Lee, H. J., & Kim, Y. M. (2019). Rl-gan-net: A reinforcement learning agent controlled gan network for real-time point cloud shape completion. Paper presented at the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
29- Siddiqui, F. U., Teng, S. W., Awrangjeb, M., & Lu, G. (2016). A Robust Gradient Based Method for Building Extraction from LiDAR and Photogrammetric Imagery. Sensors, 16, 1110.
30- Tchapmi, L. P., Kosaraju, V., Rezatofighi, H., Reid, I., &Savarese, S. (2019). Topnet: Structural point cloud decoder. Paper presented at the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
31- Venkatesh, R., Karmali, T., Sharma, S., Ghosh, A., Jeni, L. A., Babu, R. V., & Singh, M. (2021). Deep Implicit Surface Point Prediction Networks. arXiv preprint arXiv:2106.05779.
32- Wang, N., Zhang, Y., Li, Z., Fu, Y., Liu, W., & Jiang, Y.-G. (2018). Pixel2mesh: Generating 3d mesh models from single rgb images. Paper presented at the Proceedings of the European Conference on Computer Vision (ECCV).
33- Wang, Q., Yan, L., Sun, Y., Cui, X., Mortimer, H., & Li, Y. (2018). True orthophoto generation using line segment matches. The Photogrammetric Record, 33(161), 113-130.
34- Wang, X., AngJr, M. H., & Lee, G. H. (2020). Cascaded refinement network for point cloud completion. Paper presented at the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
35- Wang, X., Jin, Y., Cen, Y., Wang, T., & Li, Y. (2021). Attention Models for Point Clouds in Deep Learning: A Survey. arXiv preprint arXiv:2102.10788.
36- Wei, S. (2008). Building boundary extraction based on lidar point clouds data. Proceedings of the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 37, 157-161.
37- Widyaningrum, E., Lindenbergh, R., Gorte, B., & Zhou, K. (2018). Extraction of building roof edges from LiDAR data to optimize the digital surface model for true orthophoto generation. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 42, 2.
38- Wu, J., Jie, S., Yao, W., &Stilla, U. (2011). Building boundary improvement for true orthophoto generation by fusing airborne LiDAR data. Paper presented at the Urban Remote Sensing Event (JURSE), 2011 Joint.
39- Xiang, P., Wen, X., Liu, Y.-S., Cao, Y.-P., Wan, P., Zheng, W., & Han, Z. (2021). SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer. arXiv preprint arXiv:2108.04444.
40- Yang, Y., Feng, C., Shen, Y., &Tian, D. (2018). Foldingnet: Point cloud auto-encoder via deep grid deformation. Paper presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
41- Yu, L., Li, X., Fu, C.-W., Cohen-Or, D., &Heng, P.-A. (2018). Pu-net: Point cloud upsampling network. Paper presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
42- Yuan, W., Khot, T., Held, D., Mertz, C., & Hebert, M. (2018). Pcn: Point completion network. Paper presented at the 2018 International Conference on 3D Vision (3DV).
43- Zhang, Y., Liu, Z., Li, X., &Zang, Y. (2019). Data-Driven Point cloud objects completion. Sensors, 19(7), 1514.