Abstract
The infrared polarization (IRP) Division-of-Focal-Plane (DoFP) imaging technology has received much attention, but the insufficient resolution caused by sensor size has hindered the development and application of IRP imaging technology. Meanwhile, high-resolution (HR) visible light (VIS) is relatively easy to obtain. Therefore, using VIS images as auxiliary priors to enhance the super-resolution (SR) of infrared polarization imaging is of great significance. However, compared with infrared SR, the SR of DoFP infrared polarization imaging is more challenging due to the need to reconstruct accurate polarization information. In this paper, we propose an effective multi-modal SR network and achieve end-to-end IRP SR based on the designed loss function. In addition, we establish the first benchmark dataset with a focus on multi-modal IRP SR. The dataset contains 1559 pairs of registered images, including buildings, streets, and pedestrians. Experiments on this dataset show that the proposed method and designed loss function can effectively utilize VIS images and restore the polarization information of IRP images, achieving 4x SR. The experimental results also indicate that the proposed method outperforms the competing methods in both quantitative evaluation and visual effect evaluation.
In the figure above, HR refers to high-resolution images, LR refers to images downsampled four times from HR, and SR refers to super-resolution images. The first column displays infrared polarization DoFP images, the second column displays DoLP images, the third column displays AoP images, and the last column displays guided visible light images. These images were captured continuously at a specific dock.
GitHub
The code will soon be open-source code on GitHub.
Our relative work.
SwinIPISR: A Super-Resolution Method for Infrared Polarization Imaging Sensors via Swin Transformer
Abstract
The performance of the emerging infrared polarization remote sensing systems is limited by the use of infrared polarization imaging sensors and cannot produce high-resolution (HR) infrared polarization images. The lack of HR infrared polarization imaging sensors and systems hinders the development and application of infrared polarization imaging technology. The existing infrared image super-resolution (SR) methods fail to improve the resolution of infrared polarization images (IRPIs) while preserving the infrared polarization information inherent in the IRPIs; thus, aiming at obtaining accurate HR infrared polarization images, this study proposed a deep-learning-based SR method, SwinIPISR, to improve infrared polarization image resolution and preserve the infrared polarization information of the target or scene. The performance of the proposed SwinIPISR was verified and compared with existing SR methods. In contrast to other methods, SwinIPISR not only improves image resolution but also retains polarization information of the scene and objects in the polarization image. Further, the impact of the network depth of SwinIPISR on the SR performance was evaluated through experiments. The experimental results confirmed the effectiveness of the proposed SwinIPISR in enhancing the image resolution and visual effects of infrared polarization images without compromising the polarization information.
