Pretrained Models#

Pretrained reconstructors#

Some methods do not require any training and can be quickly deployed to your problem.

See also

See the example Use a pretrained model for how to get started with these models on various problems.

These models can be set-up in one line and perform inference in another line:

>>> import deepinv as dinv
>>> x = dinv.utils.load_example("butterfly.png")
>>> physics = dinv.physics.Downsampling()
>>> y = physics(x)
>>> model = dinv.models.RAM(pretrained=True) # or any of the models listed below
>>> x_hat = model(y, physics) # Model inference
>>> dinv.metric.PSNR()(x_hat, x)
tensor([22.9831])
Table 7 Pretrained reconstructors#

Name

Family

Modality/Physics

Speed

RAM

Feedforward

General; physics must be linear

Fast

DPIR

Plug-and-play w/ pretrained denoiser

General

Medium

DDRM

Diffusion w/ pretrained denoiser

General; physics must be decomposable

Slow

DiffPIR

Diffusion w/ pretrained denoiser

General

Slow

DPS

Diffusion w/ pretrained denoiser

General

Slow

Pretrained denoisers

Feedforward

Denoising

Fast

Tip

If you want to fine-tune these models on your own measurements (without or with ground truth) or physics, check out Inference and fine-tune a foundation model.

See also

See below for a full list of denoisers that can be plugged into iterative/sampling algorithms.

Description of weights#

For each model (Denoiser or Reconstructor) that has pretrained weights, we briefly summarize the origin of the weights, associated reference and relevant details. All pretrained weights are hosted on HuggingFace.

Click on the model name to learn more about the type of model and use pretrained=True to use the pretrained weights.

Table 8 Summary of pretrained weights#

Model

Type

Weight

deepinv.models.DnCNN

Denoiser

Default weights from Learning Maximally Monotone Operators trained on noise level 2.0/255: DnCNN grayscale weights, DnCNN color weights.

Alternative weights trained on noise level 2.0/255 with Lipschitz constraint to ensure approximate firm nonexpansiveness: Non-expansive DnCNN grayscale weights, Non-expansive DnCNN color weights.

deepinv.models.DRUNet

Denoiser

Default weights trained with deepinv (logs), trained on noise levels in [0, 20]/255 and on the same dataset as DPIR: DRUNet grayscale weights, DRUNet color weights.

Alternative weights from DPIR, trained on noise levels in [0, 50]/255. DRUNet original grayscale weights, DRUNET original color weights.

deepinv.models.GSDRUNet

Denoiser

Weights from Gradient-Step PnP, trained on noise levels in [0, 50]/255: GSDRUNet color weights and GSDRUNet grayscale weights.

deepinv.models.SCUNet

Denoiser

Weights from SCUNet, trained on images degraded with synthetic realistic noise and camera artefacts. SCUNet color weights.

deepinv.models.SwinIR

Denoiser

Weights from SwinIR, trained on various noise levels levels in {15, 25, 50}/255, in color and grayscale. The weights are automatically downloaded from the authors’ project page.

deepinv.models.DiffUNet

Denoiser

Default weights from Ho et al. trained on FFHQ (128 hidden channels per layer): DiffUNet weights.

Alternative weights from Dhariwal and Nichol trained on ImageNet128 (256 hidden channels per layer): DiffUNet OpenAI weights.

deepinv.models.EPLLDenoiser

Denoiser

Weights estimated with deepinv on 50 mio patches from the training/validation images from BSDS500 for grayscale and color images. Code for generating the weights for the example Patch priors for limited-angle computed tomography is contained within the demo.

deepinv.models.Restormer

Denoiser

Weights from Restormer: Efficient Transformer for High-Resolution Image Restoration: Restormer weights, also available on the deepinverse Restormer HuggingfaceHub.

deepinv.models.RAM

Reconstructor & Denoiser

Weights from Terris et al. [1]. Pretrained weights from RAM HuggingfaceHub.