Extended Abstract
Introduction
Accurate land use classification using deep learning models has become a crucial aspect of remote sensing and geographic information systems (GIS), enabling precise monitoring and management of natural and urban environments. In recent years, convolutional neural networks (CNNs) have gained significant attention in image segmentation and classification tasks, particularly due to their ability to automatically extract hierarchical spatial features. Among these, U-Net and ResNet-34 have demonstrated remarkable success in remote sensing applications, with U-Net excelling in pixel-wise segmentation tasks and ResNet-34 offering deeper feature extraction capabilities. Despite the success of deep learning models, their performance is highly influenced by the choice of optimization algorithm. Optimizers play a fundamental role in adjusting model parameters, impacting convergence speed, computational efficiency, and generalization ability. Selecting an appropriate optimizer is crucial to ensuring that the model effectively learns complex spatial patterns while minimizing classification errors. However, the comparative impact of different optimizers on U-Net and ResNet-34 for land use classification remains underexplored. This study aims to evaluate the effectiveness of six optimizers—SGD, Adam, RMSprop, Adagrad, Nadam, and AdamW—when applied to U-Net and ResNet-34 for land use classification using high-resolution aerial imagery. The research primarily focuses on assessing how different optimization techniques influence classification accuracy, model stability, and computational efficiency in remote sensing applications. The findings provide insights into the most suitable optimizer for training deep learning models for land use classification, thereby assisting future research and practical implementations.
Materials and Methods
The dataset used in this study comprises aerial images from selected regions in Poland, which were obtained from high-resolution remote sensing data sources. These images were pre-processed and divided into three sets: training, validation, and testing. In total, 616 labeled images (256×256 pixels each) were utilized, with 462 images allocated for training, 154 for validation, and 15 for final testing. Given the limited dataset size, data augmentation techniques such as horizontal and vertical flipping, rotation at different angles, and brightness adjustments were applied to enhance model generalization and prevent overfitting. Both U-Net and ResNet-34 were implemented using Python, TensorFlow, and Keras, and the training was conducted in Google Colab's cloud environment to utilize GPU acceleration for efficient computation. Each model was trained using the six selected optimizers, with hyperparameters tuned to achieve optimal performance. Model evaluation was conducted based on multiple performance metrics, including Overall Accuracy, Kappa Coefficient, F1-score, Jaccard Index, Mean Absolute Error, and Allocation Discrepancy. To better understand the impact of each optimizer on model convergence and training stability, training and validation loss curves were analyzed, allowing for an assessment of optimization efficiency and the prevention of issues such as gradient vanishing or overfitting. Furthermore, the training epochs were scaled using a custom epoch compression technique, ensuring that the optimization progress was clearly visualized without excessive data compression.
Results and Discussion
The experimental results confirmed that ResNet-34 consistently outperformed U-Net across all evaluation metrics, emphasizing the importance of deeper architectures in enhancing classification accuracy. The superior performance of ResNet-34 can be attributed to its residual learning framework, which facilitates efficient feature propagation and mitigates gradient vanishing issues, making it particularly effective for remote sensing image classification. Among the evaluated optimizers, RMSprop and Adam yielded the highest classification accuracies, ensuring faster convergence and lower classification errors. The best accuracy achieved in ResNet-34 was 87.54% using RMSprop, whereas in U-Net, the highest accuracy was 77.17%. These results suggest that adaptive optimizers like RMSprop and Adam dynamically adjust learning rates, leading to more efficient weight updates and improved generalization. Conversely, Adagrad demonstrated the weakest performance, achieving 83.71% accuracy in ResNet-34 and 77.87% in U-Net, which can be attributed to its aggressive learning rate decay, causing stagnation in later training epochs. Similarly, SGD exhibited slower convergence, resulting in lower classification accuracy compared to adaptive methods. These findings align with previous research, which suggests that adaptive optimizers, particularly Adam and RMSprop, enhance deep learning model generalization and classification precision. Additionally, comparisons with prior studies revealed that many recent works utilize Mean Intersection over Union (MIoU) and Frequency-Weighted IoU (FWIoU) as key performance indicators. Although these metrics were not directly employed in the present study, their relevance in remote sensing classification underscores the importance of optimizing segmentation models for both pixel-wise accuracy and spatial consistency.
Conclusion
This study demonstrated that the choice of optimizer significantly impacts the accuracy, efficiency, and stability of U-Net and ResNet-34 models in land use classification. ResNet-34 consistently achieved higher accuracy compared to U-Net, reinforcing the advantage of deeper architectures in aerial image processing. Among the optimizers, RMSprop and Adam emerged as the most effective, delivering faster convergence rates, higher classification accuracies, and improved generalization. Meanwhile, SGD and Adagrad exhibited slower convergence and lower classification accuracy, indicating that their static learning rate mechanisms are less suitable for complex remote sensing datasets.
For future research, it is recommended to explore hybrid optimization strategies, where SGD is employed during early training stages to enhance generalization, followed by adaptive optimizers like Adam or RMSprop to accelerate convergence and stabilize learning. Moreover, further investigations into emerging optimization techniques, such as Lion, AdaBelief, and RAdam, could provide valuable insights into their potential applications in deep learning-based remote sensing tasks. Additionally, the integration of advanced architectures, such as U-Net++, UNet3+, and Transformer-based segmentation models, could further improve classification precision by capturing multi-scale spatial features more effectively. Expanding the dataset to include multi-resolution and multi-spectral aerial imagery would also enable a more comprehensive assessment of optimizer adaptability across diverse geospatial environments. Overall, these findings emphasize that careful selection of the optimizer is essential for maximizing deep learning model performance in remote sensing applications. By incorporating advanced optimization techniques and architectural innovations, future studies can further enhance the accuracy and robustness of land use classification models, ultimately contributing to more efficient environmental monitoring and urban planning.