Note
New to DeepInverse? Get started with the basics with the 5 minute quickstart tutorial.
5 minute quickstart tutorial#
Follow this example to get started with DeepInverse in under 5 minutes.
Contents
1. Install#
First, install and import the latest stable release of deepinv
:
pip install deepinv
We then get the device (CPU in the case of this example).
import deepinv as dinv
import torch
device = dinv.utils.get_freer_gpu() if torch.cuda.is_available() else "cpu"
2. Physics#
In DeepInverse, x
are images:
x = dinv.utils.load_example("butterfly.png", device=device)
Images are tensors of shape B, C, ...
where B
is batch size, C
are channels and ...
are spatial dimensions:
print(x.shape)
torch.Size([1, 3, 256, 256])
Imaging forward operators are called physics
and simulate
measurements y
from x
.
DeepInverse implements many different types of physics across various imaging modalities. Physics also possess noise models such as Gaussian or Poisson noise.
physics.noise_model = dinv.physics.GaussianNoise(sigma=0.1)
y = physics(x)
dinv.utils.plot({"GT": x, "Noisy inpainting measurement": y})

Many physics also take
physics parameters such as mask
, filter
, sigma
etc.:
# Blur with Gaussian filter parameter
filter = dinv.physics.blur.gaussian_blur((5, 5))
physics = dinv.physics.BlurFFT(x.shape[1:], filter=filter, device=device)
# Simulate measurements
y = physics(x)
You can easily use your own params by passing these into the physics
,
or you can use a generator
to generate random params:
# Blur kernel random generator
physics_generator = dinv.physics.generator.MotionBlurGenerator(
psf_size=(31, 31), num_channels=3, device=device
)
# Generate a dict of random params {"filter": ...}
params = physics_generator.step()
# Update physics during forward call
y2 = physics(x, **params)
dinv.utils.plot(
{
"GT": x,
"Blurred...": y,
"... with Gaussian kernel": filter,
"Blurred ...": y2,
"...with motion kernel": params["filter"],
}
)

Physics are powerful objects and have many methods, for example a pseudo-inverse:
# You can also update params like so
physics.update(filter=filter.to(device))
x_pinv = physics.A_dagger(y)
As it is well-known in the field of inverse problems, the pseudo-inverse can give good results if the problem is noiseless, but it completely fails in the presence of noise - this is why we need reconstructors!
physics.noise_model = dinv.physics.GaussianNoise(sigma=0.1)
y = physics(x)
x_pinv_noise = physics.A_dagger(y)
dinv.utils.plot({"Pseudoinv w/o noise": x_pinv, "Pseudoinv with noise": x_pinv_noise})

Tip
Want to use DeepInverse with your own physics operator? Check out Bring your own physics for a tutorial!
3. Models#
In DeepInverse, a model
is a reconstruction algorithm that
reconstructs images from y
and knowledge of physics
.
Tip
Many models, such as Reconstruct Anything Model
, are pretrained reconstructors and can
be used out of the box. See Use a pretrained model for a full example.
Downloading: "https://huggingface.co/mterris/ram/resolve/main/ram.pth.tar" to /home/runner/.cache/torch/hub/checkpoints/ram.pth.tar
0%| | 0.00/136M [00:00<?, ?B/s]
2%|▏ | 2.75M/136M [00:00<00:05, 23.5MB/s]
4%|▎ | 5.00M/136M [00:00<00:10, 13.2MB/s]
5%|▍ | 6.50M/136M [00:00<00:12, 10.7MB/s]
6%|▌ | 8.12M/136M [00:00<00:10, 12.3MB/s]
27%|██▋ | 36.2M/136M [00:00<00:01, 86.1MB/s]
44%|████▍ | 60.4M/136M [00:00<00:00, 131MB/s]
67%|██████▋ | 91.9M/136M [00:00<00:00, 187MB/s]
88%|████████▊ | 119M/136M [00:01<00:00, 214MB/s]
100%|██████████| 136M/136M [00:01<00:00, 125MB/s]
Plot the image x
, the measurement y
and the reconstructed image
x_hat
and compute metrics:
metric = dinv.metric.PSNR()
psnr_y = metric(y, x).item()
psnr_x_hat = metric(x_hat, x).item()
dinv.utils.plot(
{
f"Measurement\n {psnr_y:.2f} dB": y,
f"Reconstruction\n {psnr_x_hat:.2f} dB": x_hat,
"GT": x,
}
)

Some models are only denoisers that denoise
images from y
and sigma
, which can be used to build many
model-based reconstruction algorithms.
denoiser = dinv.models.DRUNet(device=device)
x_denoised = denoiser(y, sigma=0.1)
model = dinv.optim.DPIR(sigma=0.1, denoiser=denoiser, device=device)
x_hat = model(y, physics)
dinv.utils.plot(
{"Measurement": y, "Denoised": x_denoised, "Reconstructed": x_hat, "GT": x}
)

Downloading: "https://huggingface.co/deepinv/drunet/resolve/main/drunet_deepinv_color_finetune_22k.pth?download=true" to /home/runner/.cache/torch/hub/checkpoints/drunet_deepinv_color_finetune_22k.pth
0%| | 0.00/125M [00:00<?, ?B/s]
10%|▉ | 12.4M/125M [00:00<00:00, 129MB/s]
32%|███▏ | 39.2M/125M [00:00<00:00, 218MB/s]
53%|█████▎ | 66.1M/125M [00:00<00:00, 247MB/s]
76%|███████▌ | 94.4M/125M [00:00<00:00, 266MB/s]
99%|█████████▉| 124M/125M [00:00<00:00, 281MB/s]
100%|██████████| 125M/125M [00:00<00:00, 259MB/s]
DeepInverse covers many frameworks of reconstruction algorithms including deep model architectures, iterative algorithms, sampling algorithms (e.g. diffusion models), and unfolded models.
# Reconstruct Anything Model foundation model
model = dinv.models.RAM(pretrained=True, device=device)
Tip
Want to use DeepInverse with your own network? Just inherit from the reconstructor base class deepinv.models.Reconstructor
!
4. Datasets#
You can use DeepInverse with dataset, for testing or training. First, define a ground-truth dataset. We implement wrappers for many popular imaging datasets across domains including natural images, medical imaging, satellite imaging, etc.
Tip
It’s easy to use your own dataset with DeepInverse. See Bring your own dataset for a tutorial.
dataset = dinv.datasets.SimpleFastMRISliceDataset(
"data", anatomy="brain", download=True
)
0%| | 0/820534 [00:00<?, ?it/s]
100%|██████████| 801k/801k [00:00<00:00, 18.6MB/s]
Datasets return either x
, tuples x, y
or x, y, params
of images,
measurements, and optional physics parameters. Given a ground-truth
dataset, you can simulate a dataset with random physics:
physics = dinv.physics.MRI(device=device)
physics_generator = dinv.physics.generator.RandomMaskGenerator(
(320, 320), device=device
)
path = dinv.datasets.generate_dataset(
dataset,
physics,
save_dir="data",
physics_generator=physics_generator,
device=device,
)
dataset = dinv.datasets.HDF5Dataset(path, load_physics_generator_params=True)
Dataset has been saved at data/dinv_dataset0.h5
You can use this dataset to test or train a model:

0%| | 0/2 [00:00<?, ?it/s]
Test: 0%| | 0/2 [00:00<?, ?it/s]
Test: 0%| | 0/2 [00:04<?, ?it/s, PSNR=41.4, PSNR no learning=31.4]
Test: 50%|█████████████████████████████████████▌ | 1/2 [00:04<00:04, 4.43s/it, PSNR=41.4, PSNR no learning=31.4]
Test: 50%|█████████████████████████████████████▌ | 1/2 [00:04<00:04, 4.43s/it, PSNR=41.4, PSNR no learning=31.4]
Test: 50%|█████████████████████████████████████▌ | 1/2 [00:08<00:04, 4.43s/it, PSNR=40.2, PSNR no learning=30.5]
Test: 100%|███████████████████████████████████████████████████████████████████████████| 2/2 [00:08<00:00, 4.43s/it, PSNR=40.2, PSNR no learning=30.5]
Test: 100%|███████████████████████████████████████████████████████████████████████████| 2/2 [00:08<00:00, 4.43s/it, PSNR=40.2, PSNR no learning=30.5]
Test results:
PSNR no learning: 30.459 +- 0.940
PSNR: 40.170 +- 1.241
{'PSNR no learning': np.float64(30.458812713623047), 'PSNR no learning_std': np.float64(0.939930729178376), 'PSNR': np.float64(40.16980743408203), 'PSNR_std': np.float64(1.2413168748421881)}
Tip
Want to use DeepInverse with your own dataset? Check out Bring your own dataset for a tutorial!
🎉 Well done, you now know how to use DeepInverse!
What’s next?#
Try more basic examples, including how to inference a pretrained model, how to use your own dataset, or how to use your custom physics operator.
Dive deeper into our full library of examples.
Read the User Guide for further details on the concepts introduced here.
Want help? Open an issue ask a message on our Discord or get in touch with our MAINTAINERS.
Total running time of the script: (0 minutes 31.913 seconds)