Pretrained Models#
Pretrained reconstructors#
Some methods do not require any training and can be quickly deployed to your problem.
See also
See the example Use a pretrained model for how to get started with these models on various problems.
These models can be set-up in one line and perform inference in another line:
>>> import deepinv as dinv
>>> x = dinv.utils.load_example("butterfly.png")
>>> physics = dinv.physics.Downsampling()
>>> y = physics(x)
>>> model = dinv.models.RAM(pretrained=True) # or any of the models listed below
>>> x_hat = model(y, physics) # Model inference
>>> dinv.metric.PSNR()(x_hat, x)
tensor([22.9831])
Name |
Family |
Modality/Physics |
Speed |
---|---|---|---|
Feedforward |
General; physics must be linear |
Fast |
|
Plug-and-play w/ pretrained denoiser |
General |
Medium |
|
Diffusion w/ pretrained denoiser |
General; physics must be decomposable |
Slow |
|
Diffusion w/ pretrained denoiser |
General |
Slow |
|
Diffusion w/ pretrained denoiser |
General |
Slow |
|
Feedforward |
Denoising |
Fast |
Tip
If you want to fine-tune these models on your own measurements (without or with ground truth) or physics, check out Inference and fine-tune a foundation model.
See also
See below for a full list of denoisers that can be plugged into iterative/sampling algorithms.
Description of weights#
For each model (Denoiser
or Reconstructor
) that has pretrained weights, we briefly summarize the origin of the weights,
associated reference and relevant details. All pretrained weights are hosted on
HuggingFace.
Click on the model name to learn more about the type of model and use pretrained=True
to use the pretrained weights.
Model |
Type |
Weight |
---|---|---|
Denoiser |
Default weights from Learning Maximally Monotone Operators trained on noise level 2.0/255: DnCNN grayscale weights, DnCNN color weights. |
|
Alternative weights trained on noise level 2.0/255 with Lipschitz constraint to ensure approximate firm nonexpansiveness: Non-expansive DnCNN grayscale weights, Non-expansive DnCNN color weights. |
||
Denoiser |
Default weights trained with deepinv (logs), trained on noise levels in [0, 20]/255 and on the same dataset as DPIR: DRUNet grayscale weights, DRUNet color weights. |
|
Alternative weights from DPIR, trained on noise levels in [0, 50]/255. DRUNet original grayscale weights, DRUNET original color weights. |
||
Denoiser |
Weights from Gradient-Step PnP, trained on noise levels in [0, 50]/255: GSDRUNet color weights and GSDRUNet grayscale weights. |
|
Denoiser |
Weights from SCUNet, trained on images degraded with synthetic realistic noise and camera artefacts. SCUNet color weights. |
|
Denoiser |
Weights from SwinIR, trained on various noise levels levels in {15, 25, 50}/255, in color and grayscale. The weights are automatically downloaded from the authors’ project page. |
|
Denoiser |
Default weights from Ho et al. trained on FFHQ (128 hidden channels per layer): DiffUNet weights. |
|
Alternative weights from Dhariwal and Nichol trained on ImageNet128 (256 hidden channels per layer): DiffUNet OpenAI weights. |
||
Denoiser |
Weights estimated with deepinv on 50 mio patches from the training/validation images from BSDS500 for grayscale and color images. Code for generating the weights for the example Patch priors for limited-angle computed tomography is contained within the demo. |
|
Denoiser |
Weights from Restormer: Efficient Transformer for High-Resolution Image Restoration: Restormer weights, also available on the deepinverse Restormer HuggingfaceHub. |
|
Reconstructor & Denoiser |
Weights from Terris et al. [1]. Pretrained weights from RAM HuggingfaceHub. |