Vol. 54, Issue 4, pp. 551-565 (2024)

Abstract

The multispectral imaging system using the filter array can capture the multispectral information of the scene in one snapshot and reconstruct the complete multispectral image by demosaicing. However, the sparse sampling rate makes image captured by demosaicing a challenging problem. Although a lot of demosaicing algorithms have been developed, the existing well-performing methods have limitations in modeling non-local dependencies which lead to artifacts. To solve this problem, this paper proposes a transformer-based multispectral image demosaicing model to address the problem. The proposed model comprises a pseudo-panchromatic image generation network and a transformer-based multispectral image reconstruction network. Additionally, we designed a fusion module to combine the pseudo-panchromatic image with the raw mosaic image captured by the camera, leveraging the correlation between the band of multispectral images to improve the performance of the model. The experimental results show that the proposed method has the advantages of high reconstruction precision, strong anti-noise interference ability, and small calculation amount, which provides a better image reconstruction solution for constructing a high-quality multispectral imaging system applied to multiple scenes.