Towards Bringing Advanced Restoration Networks into Self-Supervised Image Denoising

TL;DR — We explore migrating the recent advances in image restoration (\eg, SwinIR, Restormer, NAFNet, and HAT) into self-supervised image denoising.

Abstract

Self-supervised image denoising (SSID) has witnessed significant progress in recent years. Therein, most methods focus on exploring blind-spot techniques while only employing a simple network architecture (\eg, plain CNN or U-Net) as a denoising backbone. However, with the ongoing advancements in image restoration networks, these architectures have become somewhat outdated. In this work, we aim to migrate the advanced restoration network designs (\eg, SwinIR, Restormer, NAFNet, and HAT) into SSID methods. We begin by conducting an analysis of the fundamental concepts in existing typical blind-spot networks (BSN). Subsequently, we introduce a series of approaches to adapt restoration networks into various blind-spot ones. In particular, we suggest effective adjustment for window attention to mimic the convolution layers in BSN. And we discourage the adoption of channel attention, as it can potentially lead to the leakage of blind-spot information, consequently impeding performance. Experiments on both synthetic and real-world RGB noisy images demonstrate our methods substantially enhance SSID performance. Furthermore, we hope this study could enable SIDD methods to keep pace with the progress in restoration networks, and serve as benchmarks for future works. The code and pre-trained models will be publicly available.