Poisson denoising using Poisson2Sparse

This code shows how to restore a single image corrupted by Poisson noise using Poisson2Sparse, without requiring external training or knowledge of the noise level.

This method is based on the paper “Poisson2Sparse” Ta et al.[1] and restores an image by learning a sparse non-linear dictionary parametrized by a neural network using a combination of Neighbor2Neighbor Huang et al.[2], of the negative log Poisson likelihood, of the \(\ell^1\) pixel distance and of a sparsity-inducing \(\ell^1\) regularization function on the weights.

import deepinv as dinv
import torch

Load a Poisson corrupted image

This example uses an image from the microscopy dataset FMD Zhang et al.[3].

# Seed the RNGs for reproducibility
torch.manual_seed(0)
torch.cuda.manual_seed(0)

device = dinv.utils.get_freer_gpu() if torch.cuda.is_available() else "cpu"

physics = dinv.physics.Denoising(dinv.physics.PoissonNoise(gain=0.01, normalize=True))

x = dinv.utils.demo.load_example(
    "FMD_TwoPhoton_MICE_R_gt_12_avg50.png", img_size=(256, 256)
).to(device)
x = x[:, 0:1, :64, :64]
x = x.clamp(0, 1)
y = physics(x)

Define the Poisson2Sparse model

backbone = dinv.models.ConvLista(
    in_channels=1,
    out_channels=1,
    kernel_size=3,
    num_filters=512,
    num_iter=10,
    stride=2,
    threshold=0.01,
)

model = dinv.models.Poisson2Sparse(
    backbone=backbone,
    lr=1e-4,
    num_iter=200,
    weight_n2n=2.0,
    weight_l1_regularization=1e-5,
    verbose=True,
).to(device)

Run the model

Note that we do not pass in the physics model as Poisson2Sparse assumes a Poisson noise model internally and does not depend on the noise level.

x_hat = model(y)

# Compute and display PSNR values
learning_free_psnr = dinv.metric.PSNR()(y, x).item()
model_psnr = dinv.metric.PSNR()(x_hat, x).item()
print(f"Measurement PSNR: {learning_free_psnr:.1f} dB")
print(f"Poisson2Sparse PSNR: {model_psnr:.1f} dB")

# Plot results
dinv.utils.plot(
    [y, x_hat, x],
    titles=["Measurement", "Poisson2Sparse", "Ground truth"],
    subtitles=[f"{learning_free_psnr:.1f} dB", f"{model_psnr:.1f} dB", ""],
)

  0%|          | 0/200 [00:00<?, ?it/s]
  0%|          | 1/200 [00:00<00:29,  6.64it/s]
  1%|          | 2/200 [00:00<00:28,  6.99it/s]
  2%|▏         | 3/200 [00:00<00:27,  7.12it/s]
  2%|▏         | 4/200 [00:00<00:26,  7.31it/s]
  2%|▎         | 5/200 [00:00<00:26,  7.42it/s]
  3%|▎         | 6/200 [00:00<00:25,  7.50it/s]
  4%|▎         | 7/200 [00:00<00:25,  7.55it/s]
  4%|▍         | 8/200 [00:01<00:25,  7.58it/s]
  4%|▍         | 9/200 [00:01<00:25,  7.58it/s]
  5%|▌         | 10/200 [00:01<00:25,  7.59it/s]
  6%|▌         | 11/200 [00:01<00:25,  7.55it/s]
  6%|▌         | 12/200 [00:01<00:25,  7.52it/s]
  6%|▋         | 13/200 [00:01<00:25,  7.44it/s]
  7%|▋         | 14/200 [00:01<00:25,  7.43it/s]
  8%|▊         | 15/200 [00:02<00:24,  7.47it/s]
  8%|▊         | 16/200 [00:02<00:24,  7.53it/s]
  8%|▊         | 17/200 [00:02<00:24,  7.41it/s]
  9%|▉         | 18/200 [00:02<00:24,  7.33it/s]
 10%|▉         | 19/200 [00:02<00:24,  7.35it/s]
 10%|█         | 20/200 [00:02<00:24,  7.34it/s]
 10%|█         | 21/200 [00:02<00:24,  7.32it/s]
 11%|█         | 22/200 [00:02<00:24,  7.29it/s]
 12%|█▏        | 23/200 [00:03<00:24,  7.36it/s]
 12%|█▏        | 24/200 [00:03<00:23,  7.41it/s]
 12%|█▎        | 25/200 [00:03<00:23,  7.42it/s]
 13%|█▎        | 26/200 [00:03<00:23,  7.43it/s]
 14%|█▎        | 27/200 [00:03<00:23,  7.42it/s]
 14%|█▍        | 28/200 [00:03<00:23,  7.42it/s]
 14%|█▍        | 29/200 [00:03<00:22,  7.46it/s]
 15%|█▌        | 30/200 [00:04<00:22,  7.48it/s]
 16%|█▌        | 31/200 [00:04<00:22,  7.47it/s]
 16%|█▌        | 32/200 [00:04<00:22,  7.33it/s]
 16%|█▋        | 33/200 [00:04<00:22,  7.36it/s]
 17%|█▋        | 34/200 [00:04<00:22,  7.46it/s]
 18%|█▊        | 35/200 [00:04<00:21,  7.54it/s]
 18%|█▊        | 36/200 [00:04<00:21,  7.51it/s]
 18%|█▊        | 37/200 [00:04<00:21,  7.43it/s]
 19%|█▉        | 38/200 [00:05<00:21,  7.44it/s]
 20%|█▉        | 39/200 [00:05<00:21,  7.46it/s]
 20%|██        | 40/200 [00:05<00:21,  7.53it/s]
 20%|██        | 41/200 [00:05<00:20,  7.60it/s]
 21%|██        | 42/200 [00:05<00:20,  7.64it/s]
 22%|██▏       | 43/200 [00:05<00:20,  7.60it/s]
 22%|██▏       | 44/200 [00:05<00:20,  7.63it/s]
 22%|██▎       | 45/200 [00:06<00:20,  7.62it/s]
 23%|██▎       | 46/200 [00:06<00:20,  7.64it/s]
 24%|██▎       | 47/200 [00:06<00:20,  7.55it/s]
 24%|██▍       | 48/200 [00:06<00:20,  7.58it/s]
 24%|██▍       | 49/200 [00:06<00:19,  7.62it/s]
 25%|██▌       | 50/200 [00:06<00:19,  7.55it/s]
 26%|██▌       | 51/200 [00:06<00:19,  7.51it/s]
 26%|██▌       | 52/200 [00:06<00:19,  7.47it/s]
 26%|██▋       | 53/200 [00:07<00:20,  7.33it/s]
 27%|██▋       | 54/200 [00:07<00:19,  7.36it/s]
 28%|██▊       | 55/200 [00:07<00:19,  7.36it/s]
 28%|██▊       | 56/200 [00:07<00:19,  7.42it/s]
 28%|██▊       | 57/200 [00:07<00:19,  7.25it/s]
 29%|██▉       | 58/200 [00:07<00:19,  7.24it/s]
 30%|██▉       | 59/200 [00:07<00:19,  7.25it/s]
 30%|███       | 60/200 [00:08<00:19,  7.26it/s]
 30%|███       | 61/200 [00:08<00:19,  7.27it/s]
 31%|███       | 62/200 [00:08<00:18,  7.28it/s]
 32%|███▏      | 63/200 [00:08<00:18,  7.32it/s]
 32%|███▏      | 64/200 [00:08<00:18,  7.37it/s]
 32%|███▎      | 65/200 [00:08<00:18,  7.49it/s]
 33%|███▎      | 66/200 [00:08<00:17,  7.56it/s]
 34%|███▎      | 67/200 [00:09<00:17,  7.60it/s]
 34%|███▍      | 68/200 [00:09<00:17,  7.57it/s]
 34%|███▍      | 69/200 [00:09<00:17,  7.51it/s]
 35%|███▌      | 70/200 [00:09<00:17,  7.54it/s]
 36%|███▌      | 71/200 [00:09<00:17,  7.57it/s]
 36%|███▌      | 72/200 [00:09<00:16,  7.61it/s]
 36%|███▋      | 73/200 [00:09<00:16,  7.65it/s]
 37%|███▋      | 74/200 [00:09<00:16,  7.62it/s]
 38%|███▊      | 75/200 [00:10<00:16,  7.59it/s]
 38%|███▊      | 76/200 [00:10<00:16,  7.66it/s]
 38%|███▊      | 77/200 [00:10<00:16,  7.66it/s]
 39%|███▉      | 78/200 [00:10<00:15,  7.66it/s]
 40%|███▉      | 79/200 [00:10<00:15,  7.63it/s]
 40%|████      | 80/200 [00:10<00:15,  7.59it/s]
 40%|████      | 81/200 [00:10<00:15,  7.57it/s]
 41%|████      | 82/200 [00:10<00:15,  7.49it/s]
 42%|████▏     | 83/200 [00:11<00:15,  7.57it/s]
 42%|████▏     | 84/200 [00:11<00:15,  7.62it/s]
 42%|████▎     | 85/200 [00:11<00:15,  7.66it/s]
 43%|████▎     | 86/200 [00:11<00:14,  7.65it/s]
 44%|████▎     | 87/200 [00:11<00:14,  7.62it/s]
 44%|████▍     | 88/200 [00:11<00:14,  7.63it/s]
 44%|████▍     | 89/200 [00:11<00:14,  7.61it/s]
 45%|████▌     | 90/200 [00:12<00:14,  7.54it/s]
 46%|████▌     | 91/200 [00:12<00:14,  7.48it/s]
 46%|████▌     | 92/200 [00:12<00:14,  7.46it/s]
 46%|████▋     | 93/200 [00:12<00:14,  7.51it/s]
 47%|████▋     | 94/200 [00:12<00:14,  7.50it/s]
 48%|████▊     | 95/200 [00:12<00:13,  7.57it/s]
 48%|████▊     | 96/200 [00:12<00:13,  7.60it/s]
 48%|████▊     | 97/200 [00:12<00:13,  7.60it/s]
 49%|████▉     | 98/200 [00:13<00:13,  7.62it/s]
 50%|████▉     | 99/200 [00:13<00:13,  7.56it/s]
 50%|█████     | 100/200 [00:13<00:13,  7.59it/s]
 50%|█████     | 101/200 [00:13<00:12,  7.63it/s]
 51%|█████     | 102/200 [00:13<00:12,  7.59it/s]
 52%|█████▏    | 103/200 [00:13<00:12,  7.60it/s]
 52%|█████▏    | 104/200 [00:13<00:12,  7.64it/s]
 52%|█████▎    | 105/200 [00:14<00:12,  7.64it/s]
 53%|█████▎    | 106/200 [00:14<00:12,  7.54it/s]
 54%|█████▎    | 107/200 [00:14<00:12,  7.58it/s]
 54%|█████▍    | 108/200 [00:14<00:12,  7.60it/s]
 55%|█████▍    | 109/200 [00:14<00:12,  7.57it/s]
 55%|█████▌    | 110/200 [00:14<00:11,  7.60it/s]
 56%|█████▌    | 111/200 [00:14<00:11,  7.58it/s]
 56%|█████▌    | 112/200 [00:14<00:11,  7.65it/s]
 56%|█████▋    | 113/200 [00:15<00:11,  7.67it/s]
 57%|█████▋    | 114/200 [00:15<00:11,  7.70it/s]
 57%|█████▊    | 115/200 [00:15<00:11,  7.68it/s]
 58%|█████▊    | 116/200 [00:15<00:10,  7.70it/s]
 58%|█████▊    | 117/200 [00:15<00:10,  7.71it/s]
 59%|█████▉    | 118/200 [00:15<00:10,  7.68it/s]
 60%|█████▉    | 119/200 [00:15<00:10,  7.67it/s]
 60%|██████    | 120/200 [00:15<00:10,  7.61it/s]
 60%|██████    | 121/200 [00:16<00:10,  7.66it/s]
 61%|██████    | 122/200 [00:16<00:10,  7.68it/s]
 62%|██████▏   | 123/200 [00:16<00:10,  7.65it/s]
 62%|██████▏   | 124/200 [00:16<00:09,  7.67it/s]
 62%|██████▎   | 125/200 [00:16<00:09,  7.58it/s]
 63%|██████▎   | 126/200 [00:16<00:10,  7.37it/s]
 64%|██████▎   | 127/200 [00:16<00:10,  7.27it/s]
 64%|██████▍   | 128/200 [00:17<00:09,  7.27it/s]
 64%|██████▍   | 129/200 [00:17<00:09,  7.25it/s]
 65%|██████▌   | 130/200 [00:17<00:09,  7.17it/s]
 66%|██████▌   | 131/200 [00:17<00:09,  7.22it/s]
 66%|██████▌   | 132/200 [00:17<00:09,  7.21it/s]
 66%|██████▋   | 133/200 [00:17<00:09,  7.24it/s]
 67%|██████▋   | 134/200 [00:17<00:09,  7.28it/s]
 68%|██████▊   | 135/200 [00:18<00:08,  7.31it/s]
 68%|██████▊   | 136/200 [00:18<00:08,  7.37it/s]
 68%|██████▊   | 137/200 [00:18<00:08,  7.46it/s]
 69%|██████▉   | 138/200 [00:18<00:08,  7.55it/s]
 70%|██████▉   | 139/200 [00:18<00:08,  7.58it/s]
 70%|███████   | 140/200 [00:18<00:07,  7.53it/s]
 70%|███████   | 141/200 [00:18<00:07,  7.51it/s]
 71%|███████   | 142/200 [00:18<00:07,  7.49it/s]
 72%|███████▏  | 143/200 [00:19<00:07,  7.51it/s]
 72%|███████▏  | 144/200 [00:19<00:07,  7.48it/s]
 72%|███████▎  | 145/200 [00:19<00:07,  7.50it/s]
 73%|███████▎  | 146/200 [00:19<00:07,  7.53it/s]
 74%|███████▎  | 147/200 [00:19<00:07,  7.50it/s]
 74%|███████▍  | 148/200 [00:19<00:06,  7.56it/s]
 74%|███████▍  | 149/200 [00:19<00:06,  7.65it/s]
 75%|███████▌  | 150/200 [00:19<00:06,  7.73it/s]
 76%|███████▌  | 151/200 [00:20<00:06,  7.76it/s]
 76%|███████▌  | 152/200 [00:20<00:06,  7.78it/s]
 76%|███████▋  | 153/200 [00:20<00:06,  7.78it/s]
 77%|███████▋  | 154/200 [00:20<00:05,  7.80it/s]
 78%|███████▊  | 155/200 [00:20<00:05,  7.76it/s]
 78%|███████▊  | 156/200 [00:20<00:05,  7.71it/s]
 78%|███████▊  | 157/200 [00:20<00:05,  7.62it/s]
 79%|███████▉  | 158/200 [00:21<00:05,  7.58it/s]
 80%|███████▉  | 159/200 [00:21<00:05,  7.63it/s]
 80%|████████  | 160/200 [00:21<00:05,  7.66it/s]
 80%|████████  | 161/200 [00:21<00:05,  7.71it/s]
 81%|████████  | 162/200 [00:21<00:04,  7.70it/s]
 82%|████████▏ | 163/200 [00:21<00:04,  7.70it/s]
 82%|████████▏ | 164/200 [00:21<00:04,  7.76it/s]
 82%|████████▎ | 165/200 [00:21<00:04,  7.80it/s]
 83%|████████▎ | 166/200 [00:22<00:04,  7.76it/s]
 84%|████████▎ | 167/200 [00:22<00:04,  7.72it/s]
 84%|████████▍ | 168/200 [00:22<00:04,  7.64it/s]
 84%|████████▍ | 169/200 [00:22<00:04,  7.54it/s]
 85%|████████▌ | 170/200 [00:22<00:03,  7.54it/s]
 86%|████████▌ | 171/200 [00:22<00:03,  7.53it/s]
 86%|████████▌ | 172/200 [00:22<00:03,  7.50it/s]
 86%|████████▋ | 173/200 [00:23<00:03,  7.42it/s]
 87%|████████▋ | 174/200 [00:23<00:03,  7.10it/s]
 88%|████████▊ | 175/200 [00:23<00:03,  6.93it/s]
 88%|████████▊ | 176/200 [00:23<00:03,  7.03it/s]
 88%|████████▊ | 177/200 [00:23<00:03,  7.08it/s]
 89%|████████▉ | 178/200 [00:23<00:03,  7.13it/s]
 90%|████████▉ | 179/200 [00:23<00:02,  7.18it/s]
 90%|█████████ | 180/200 [00:23<00:02,  7.33it/s]
 90%|█████████ | 181/200 [00:24<00:02,  7.36it/s]
 91%|█████████ | 182/200 [00:24<00:02,  7.30it/s]
 92%|█████████▏| 183/200 [00:24<00:02,  7.34it/s]
 92%|█████████▏| 184/200 [00:24<00:02,  7.38it/s]
 92%|█████████▎| 185/200 [00:24<00:02,  7.37it/s]
 93%|█████████▎| 186/200 [00:24<00:01,  7.53it/s]
 94%|█████████▎| 187/200 [00:24<00:01,  7.61it/s]
 94%|█████████▍| 188/200 [00:25<00:01,  7.61it/s]
 94%|█████████▍| 189/200 [00:25<00:01,  7.65it/s]
 95%|█████████▌| 190/200 [00:25<00:01,  7.70it/s]
 96%|█████████▌| 191/200 [00:25<00:01,  7.52it/s]
 96%|█████████▌| 192/200 [00:25<00:01,  7.34it/s]
 96%|█████████▋| 193/200 [00:25<00:00,  7.29it/s]
 97%|█████████▋| 194/200 [00:25<00:00,  7.31it/s]
 98%|█████████▊| 195/200 [00:26<00:00,  7.29it/s]
 98%|█████████▊| 196/200 [00:26<00:00,  7.13it/s]
 98%|█████████▊| 197/200 [00:26<00:00,  7.22it/s]
 99%|█████████▉| 198/200 [00:26<00:00,  7.29it/s]
100%|█████████▉| 199/200 [00:26<00:00,  7.35it/s]
100%|██████████| 200/200 [00:26<00:00,  7.45it/s]
100%|██████████| 200/200 [00:26<00:00,  7.49it/s]
Measurement PSNR: 27.3 dB
Poisson2Sparse PSNR: 31.1 dB

References:

[1]

Calvin-Khang Ta, Abhishek Aich, Akash Gupta, and Amit K Roy-Chowdhury. Poisson2sparse: self-supervised poisson denoising from a single image. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 557–567. Springer, 2022.

[2]

Tao Huang, Songjiang Li, Xu Jia, Huchuan Lu, and Jianzhuang Liu. Neighbor2neighbor: self-supervised denoising from single noisy images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 14781–14790. 2021.

[3]

Yide Zhang, Yinhao Zhu, Evan Nichols, Qingfei Wang, Siyuan Zhang, Cody Smith, and Scott Howard. A poisson-gaussian denoising dataset with real fluorescence microscopy images. In CVPR. 2019.