WSformer: Wavelet-Based Sparse Transformer for Blind Image Restoration

WSformer: Wavelet-Based Sparse Transformer for Blind Image Restoration

Zhonggui Sun^a,b, Can Zhang^a, Jie Li^b, Mingzhu Zhang^c, and Xinbo Gao^c

^aLiaocheng University, School of Mathematical Sciences, Liaocheng, China, 252000

^bXidian University, Video and Image Processing System Laboratory, School of Electronic Engineering, Xi’an, China, 710126

^cChongqing University of Posts and Telecommunications, College of Computer Science and Technology, Chongqing, China, 400065

As a fundamental task in image processing, blind image restoration (BIR) faces significant challenges due to the unknown nature of the degradation process. While transformerbased methods have shown promise in various applications, they encounter difficulties in BIR. One key challenge is that the complexity of degradation easily leads to incorporate irrelevant information into their attention mechanisms, thereby hindering restoration performance. To address this challenge, sparsification strategies have been commonly adopted. However, existing sparse transformer-based methods typically determine sparse members through fixed patterns such as constant thresholds or predefined sources, making their sparsification strategies too rigid. To tackle this issue, we propose WSformer, a Waveletbased Sparse transformer tailored for BIR, which offers three key advantages. First, we design a Sparse Reciprocal Multi-head Self-Attention (SR-MSA) mechanism in the attention layer. This mechanism employs sparse and reciprocal strategies to adaptively select reliable information, while operating across channels to reduce computational complexity. Second, recognizing that feedforward networks in existing transformer blocks fail to effectively leverage global information, we develop a Recalibrated FeedForward Network (RFFN). It fully exploits the fusion of local and global information, enhancing the robustness of feature learning. Finally, to mitigate the increased computational burden introduced by these innovations, we equip WSformer with wavelet transform. Combined with a U-shaped architecture, it enables WSformer to achieve an optimal balance between performance and inference time. Extensive experiments on multiple BIR tasks validate WSformer’s effectiveness in both quantitative metrics and visual quality. The code is available at https://github.com/CanZhang01/WSformer

Z. Sun#, C. Zhang, J. Li, M. Zhang, X. Gao*. WSformer: Wavelet-Based Sparse Transformer for Blind Image Restoration. IEEE Trans. on Image Processing (IEEE-TIP)，2026. code