Poisson denoising using Poisson2Sparse

This code shows how to restore a single image corrupted by Poisson noise using Poisson2Sparse, without requiring external training or knowledge of the noise level.

This method is based on the paper “Poisson2Sparse” Ta et al.[1] and restores an image by learning a sparse non-linear dictionary parametrized by a neural network using a combination of Neighbor2Neighbor Huang et al.[2], of the negative log Poisson likelihood, of the \(\ell^1\) pixel distance and of a sparsity-inducing \(\ell^1\) regularization function on the weights.

import deepinv as dinv
import torch

Load a Poisson corrupted image

This example uses an image from the microscopy dataset FMD Zhang et al.[3].

# Seed the RNGs for reproducibility
torch.manual_seed(0)
torch.cuda.manual_seed(0)

device = dinv.utils.get_freer_gpu() if torch.cuda.is_available() else "cpu"

physics = dinv.physics.Denoising(dinv.physics.PoissonNoise(gain=0.01, normalize=True))

x = dinv.utils.demo.load_example(
    "FMD_TwoPhoton_MICE_R_gt_12_avg50.png", img_size=(256, 256)
).to(device)
x = x[:, 0:1, :64, :64]
x = x.clamp(0, 1)
y = physics(x)

Selected GPU 0 with 3772.25 MiB free memory

Define the Poisson2Sparse model

backbone = dinv.models.ConvLista(
    in_channels=1,
    out_channels=1,
    kernel_size=3,
    num_filters=512,
    num_iter=10,
    stride=2,
    threshold=0.01,
)

model = dinv.models.Poisson2Sparse(
    backbone=backbone,
    lr=1e-4,
    num_iter=200,
    weight_n2n=2.0,
    weight_l1_regularization=1e-5,
    verbose=True,
).to(device)

Run the model

Note that we do not pass in the physics model as Poisson2Sparse assumes a Poisson noise model internally and does not depend on the noise level.

x_hat = model(y)

# Compute and display PSNR values
learning_free_psnr = dinv.metric.PSNR()(y, x).item()
model_psnr = dinv.metric.PSNR()(x_hat, x).item()
print(f"Measurement PSNR: {learning_free_psnr:.1f} dB")
print(f"Poisson2Sparse PSNR: {model_psnr:.1f} dB")

# Plot results
dinv.utils.plot(
    [y, x_hat, x],
    titles=["Measurement", "Poisson2Sparse", "Ground truth"],
    subtitles=[f"{learning_free_psnr:.1f} dB", f"{model_psnr:.1f} dB", ""],
)

  0%|          | 0/200 [00:00<?, ?it/s]
  2%|▏         | 3/200 [00:00<00:07, 24.89it/s]
  4%|▍         | 8/200 [00:00<00:05, 36.45it/s]
  6%|▋         | 13/200 [00:00<00:04, 38.73it/s]
  9%|▉         | 18/200 [00:00<00:04, 41.57it/s]
 12%|█▏        | 23/200 [00:00<00:04, 42.11it/s]
 14%|█▍        | 28/200 [00:00<00:04, 42.75it/s]
 16%|█▋        | 33/200 [00:00<00:03, 42.23it/s]
 19%|█▉        | 38/200 [00:00<00:03, 42.65it/s]
 22%|██▏       | 43/200 [00:01<00:03, 43.47it/s]
 24%|██▍       | 48/200 [00:01<00:03, 44.40it/s]
 26%|██▋       | 53/200 [00:01<00:03, 43.92it/s]
 29%|██▉       | 58/200 [00:01<00:03, 43.70it/s]
 32%|███▏      | 63/200 [00:01<00:03, 44.12it/s]
 34%|███▍      | 68/200 [00:01<00:03, 43.39it/s]
 36%|███▋      | 73/200 [00:01<00:02, 44.15it/s]
 39%|███▉      | 78/200 [00:01<00:02, 43.91it/s]
 42%|████▏     | 83/200 [00:01<00:02, 44.89it/s]
 44%|████▍     | 88/200 [00:02<00:02, 44.04it/s]
 46%|████▋     | 93/200 [00:02<00:02, 44.90it/s]
 49%|████▉     | 98/200 [00:02<00:02, 44.56it/s]
 52%|█████▏    | 103/200 [00:02<00:02, 43.52it/s]
 54%|█████▍    | 108/200 [00:02<00:02, 44.84it/s]
 56%|█████▋    | 113/200 [00:02<00:01, 44.04it/s]
 59%|█████▉    | 118/200 [00:02<00:01, 43.90it/s]
 62%|██████▏   | 123/200 [00:02<00:01, 44.15it/s]
 64%|██████▍   | 128/200 [00:02<00:01, 43.70it/s]
 66%|██████▋   | 133/200 [00:03<00:01, 44.06it/s]
 69%|██████▉   | 138/200 [00:03<00:01, 44.13it/s]
 72%|███████▏  | 143/200 [00:03<00:01, 43.06it/s]
 74%|███████▍  | 148/200 [00:03<00:01, 43.97it/s]
 76%|███████▋  | 153/200 [00:03<00:01, 42.93it/s]
 79%|███████▉  | 158/200 [00:03<00:00, 43.81it/s]
 82%|████████▏ | 163/200 [00:03<00:00, 43.37it/s]
 84%|████████▍ | 168/200 [00:03<00:00, 42.97it/s]
 86%|████████▋ | 173/200 [00:04<00:00, 42.51it/s]
 89%|████████▉ | 178/200 [00:04<00:00, 43.19it/s]
 92%|█████████▏| 183/200 [00:04<00:00, 44.35it/s]
 94%|█████████▍| 188/200 [00:04<00:00, 43.98it/s]
 96%|█████████▋| 193/200 [00:04<00:00, 44.94it/s]
 99%|█████████▉| 198/200 [00:04<00:00, 44.59it/s]
100%|██████████| 200/200 [00:04<00:00, 43.49it/s]
Measurement PSNR: 27.5 dB
Poisson2Sparse PSNR: 30.3 dB

References:

[1]

Calvin-Khang Ta, Abhishek Aich, Akash Gupta, and Amit K Roy-Chowdhury. Poisson2sparse: self-supervised poisson denoising from a single image. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 557–567. Springer, 2022.

[2]

Tao Huang, Songjiang Li, Xu Jia, Huchuan Lu, and Jianzhuang Liu. Neighbor2neighbor: self-supervised denoising from single noisy images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 14781–14790. 2021.

[3]

Yide Zhang, Yinhao Zhu, Evan Nichols, Qingfei Wang, Siyuan Zhang, Cody Smith, and Scott Howard. A poisson-gaussian denoising dataset with real fluorescence microscopy images. In CVPR. 2019.