Change Captioning Meets SAR Imagery: A Multimodal Dataset and Bitemporal Modeling Framework
IEEE Geoscience and Remote Sensing Letters, cilt.23, 2026 (SCI-Expanded, Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 23
- Basım Tarihi: 2026
- Doi Numarası: 10.1109/lgrs.2026.3687910
- Dergi Adı: IEEE Geoscience and Remote Sensing Letters
- Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Aerospace Database, Compendex, Geobase, INSPEC, Materials Science & Engineering Collection (ProQuest), Technology Collection (ProQuest)
- Anahtar Kelimeler: Change captioning (CC), multimodal learning, multispectral (MS) imagery, remote sensing, synthetic aperture radar (SAR)
- Yıldız Teknik Üniversitesi Adresli: Evet
Özet
Most change captioning (CC) research relies on optical imagery, which is vulnerable to atmospheric and illumination effects that synthetic aperture radar (SAR) can mitigate through its active, all-weather sensing capabilities. In this work, we introduce MOSAIC-SEN12-CC, the first remote sensing CC dataset with co-registered Sentinel-2 multispectral (MS) and Sentinel-1 SAR imagery, enabling the study of SAR-based and multimodal CC. Temporal change is commonly modeled via feature differencing between pre- and post-event observations; however, for MS-SAR inputs, differences in sensing physics can amplify modality-specific artifacts rather than semantic change. To address this limitation, we propose a bitemporal modeling framework that replaces direct differencing with a structured change representation that decouples change strength from semantic content and regulates temporal information flow. Experiments show that SAR is most effective as a complementary source when integrated through stable modality fusion and temporal design, leading to more consistent performance in different types of change. Our approach outperforms unimodal baselines and conventional fusion strategies across standard captioning metrics and an LLM-as-a-Judge (LaaJ) protocol, while reducing computational overhead relative to benchmark architectures. The public release will be made available at https://github.com/ChangeCapsInRS/MOSAIC-SEN12-CC and will include the dataset assets required to reproduce the proposed methodology together with the training, evaluation, and LaaJ codebase.