Efficient Side-Tuning for Remote Sensing: A Low-Memory Fine-Tuning Framework

• Published in: IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
• Main Authors: Haichen Yu, Wenxin Yin, Hanbo Bi, Chongyang Li, Yingchao Feng, Wenhui Diao, Xian Sun
• Format: Article
• Language: English
• Published: IEEE 2025-01-01
• Subjects: Fine-tuning; memory efficienct; object detection; remote sensing; semantic segmentation
• Online Access: https://ieeexplore.ieee.org/document/10974700/
• ISSN: 1939-1404; 2151-1535

Fine-tuning pretrained models for remote sensing tasks often demands substantial computational resources. To reduce memory requirements and training costs, this article proposes a low-memory fine-tuning framework, called efficient side-tuning (EST), for remote sensing downstream tasks. EST attaches a parallel network to the backbone of the model, and only fine-tunes the parameters of the parallel network during the training phase. The proposed EST Block is the main component of the parallel network, which uses the multichannel adapter fusion module, gate layer and depthwise convolution to achieve feature selection and enhancement effects. In the evaluation, on six remote sensing datasets including object detection and semantic segmentation, EST achieved SOTA performance results using only less than 40<inline-formula><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> of the memory expenditure of full fine-tuning, which is better than all current parameter efficient fine-tuning methods. In addition, experiments on backbones of various sizes and classes show that the generalizability of EST is also reliable. EST thus offers a highly efficient and effective approach for efficient transfer learning in remote sensing, unlocking new possibilities for advanced remote sensing applications.