Physics#

class deepinv.physics.Physics(A=<function Physics.<lambda>>, noise_model=<function Physics.<lambda>>, sensor_model=<function Physics.<lambda>>, max_iter=50, tol=0.001)[source]#

Bases: Module

Parent class for forward operators

It describes the general forward measurement process

\[y = N(A(x))\]

where \(x\) is an image of \(n\) pixels, \(y\) is the measurements of size \(m\), \(A:\xset\mapsto \yset\) is a deterministic mapping capturing the physics of the acquisition and \(N:\yset\mapsto \yset\) is a stochastic mapping which characterizes the noise affecting the measurements.

Parameters:
  • A (Callable) – forward operator function which maps an image to the observed measurements \(x\mapsto y\).

  • noise_model (Callable) – function that adds noise to the measurements \(N(z)\). See the noise module for some predefined functions.

  • sensor_model (Callable) – function that incorporates any sensor non-linearities to the sensing process, such as quantization or saturation, defined as a function \(\eta(z)\), such that \(y=\eta\left(N(A(x))\right)\). By default, the sensor_model is set to the identity \(\eta(z)=z\).

  • max_iter (int) – If the operator does not have a closed form pseudoinverse, the gradient descent algorithm is used for computing it, and this parameter fixes the maximum number of gradient descent iterations.

  • tol (float) – If the operator does not have a closed form pseudoinverse, the gradient descent algorithm is used for computing it, and this parameter fixes the absolute tolerance of the gradient descent algorithm.

A(x, **kwargs)[source]#

Computes forward operator \(y = A(x)\) (without noise and/or sensor non-linearities)

Parameters:

x (torch.Tensor,list[torch.Tensor]) – signal/image

Returns:

(torch.Tensor) clean measurements

A_dagger(y, x_init=None)[source]#

Computes an inverse as:

\[x^* \in \underset{x}{\arg\min} \quad \|\forw{x}-y\|^2.\]

This function uses gradient descent to find the inverse. It can be overwritten by a more efficient pseudoinverse in cases where closed form formulas exist.

Parameters:
  • y (torch.Tensor) – a measurement \(y\) to reconstruct via the pseudoinverse.

  • x_init (torch.Tensor) – initial guess for the reconstruction.

Returns:

(torch.Tensor) The reconstructed image \(x\).

A_vjp(x, v)[source]#

Computes the product between a vector \(v\) and the Jacobian of the forward operator \(A\) evaluated at \(x\), defined as:

\[A_{vjp}(x, v) = \left. \frac{\partial A}{\partial x} \right|_x^\top v.\]

By default, the Jacobian is computed using automatic differentiation.

Parameters:
Returns:

(torch.Tensor) the VJP product between \(v\) and the Jacobian.

__add__(other)[source]#

Stacks two linear forward operators \(A(x) = \begin{bmatrix} A_1(x) \\ A_2(x) \end{bmatrix}\) via the add operation.

The measurements produced by the resulting model are deepinv.utils.TensorList objects, where each entry corresponds to the measurements of the corresponding operator.

Parameters:

other (deepinv.physics.Physics) – Physics operator \(A_2\)

Returns:

(deepinv.physics.Physics) stacked operator

__mul__(other)[source]#

Concatenates two forward operators \(A = A_1\circ A_2\) via the mul operation

The resulting operator keeps the noise and sensor models of \(A_1\).

Parameters:

other (deepinv.physics.Physics) – Physics operator \(A_2\)

Returns:

(deepinv.physics.Physics) concantenated operator

forward(x, **kwargs)[source]#

Computes forward operator

\[y = N(A(x), \sigma)\]
Parameters:

x (torch.Tensor, list[torch.Tensor]) – signal/image

Returns:

(torch.Tensor) noisy measurements

noise(x, **kwargs)[source]#

Incorporates noise into the measurements \(\tilde{y} = N(y)\)

Parameters:
  • x (torch.Tensor) – clean measurements

  • noise_level (None, float) – optional noise level parameter

Return torch.Tensor:

noisy measurements

sensor(x)[source]#

Computes sensor non-linearities \(y = \eta(y)\)

Parameters:

x (torch.Tensor,list[torch.Tensor]) – signal/image

Returns:

(torch.Tensor) clean measurements

set_noise_model(noise_model)[source]#

Sets the noise model

Parameters:

noise_model (Callable) – noise model

update(**kwargs)[source]#

Update the parameters of the forward operator.

Parameters:

kwargs (dict) – dictionary of parameters to update.

Examples using Physics:#

Single photon lidar operator for depth ranging.

Single photon lidar operator for depth ranging.

Stacking and concatenating forward operators.

Stacking and concatenating forward operators.

Reconstructing an image using the deep image prior.

Reconstructing an image using the deep image prior.

Creating your own dataset

Creating your own dataset

Creating a forward operator.

Creating a forward operator.

Training a reconstruction network.

Training a reconstruction network.

3D diffraction PSF

3D diffraction PSF

A tour of forward sensing operators

A tour of forward sensing operators

Image deblurring with custom deep explicit prior.

Image deblurring with custom deep explicit prior.

Saving and loading models

Saving and loading models

A tour of blur operators

A tour of blur operators

Random phase retrieval and reconstruction methods.

Random phase retrieval and reconstruction methods.

Image deblurring with Total-Variation (TV) prior

Image deblurring with Total-Variation (TV) prior

Image inpainting with wavelet prior

Image inpainting with wavelet prior

Plug-and-Play algorithm with Mirror Descent for Poisson noise inverse problems.

Plug-and-Play algorithm with Mirror Descent for Poisson noise inverse problems.

Vanilla PnP for computed tomography (CT).

Vanilla PnP for computed tomography (CT).

DPIR method for PnP image deblurring.

DPIR method for PnP image deblurring.

Regularization by Denoising (RED) for Super-Resolution.

Regularization by Denoising (RED) for Super-Resolution.

PnP with custom optimization algorithm (Condat-Vu Primal-Dual)

PnP with custom optimization algorithm (Condat-Vu Primal-Dual)

Uncertainty quantification with PnP-ULA.

Uncertainty quantification with PnP-ULA.

Image reconstruction with a diffusion model

Image reconstruction with a diffusion model

Building your custom sampling algorithm.

Building your custom sampling algorithm.

Implementing DPS

Implementing DPS

Implementing DiffPIR

Implementing DiffPIR

Learned Iterative Soft-Thresholding Algorithm (LISTA) for compressed sensing

Learned Iterative Soft-Thresholding Algorithm (LISTA) for compressed sensing

Vanilla Unfolded algorithm for super-resolution

Vanilla Unfolded algorithm for super-resolution

Learned iterative custom prior

Learned iterative custom prior

Deep Equilibrium (DEQ) algorithms for image deblurring

Deep Equilibrium (DEQ) algorithms for image deblurring

Learned Primal-Dual algorithm for CT scan.

Learned Primal-Dual algorithm for CT scan.

Unfolded Chambolle-Pock for constrained image inpainting

Unfolded Chambolle-Pock for constrained image inpainting

Expected Patch Log Likelihood (EPLL) for Denoising and Inpainting

Expected Patch Log Likelihood (EPLL) for Denoising and Inpainting

Patch priors for limited-angle computed tomography

Patch priors for limited-angle computed tomography

Image transformations for Equivariant Imaging

Image transformations for Equivariant Imaging

Self-supervised learning with measurement splitting

Self-supervised learning with measurement splitting

Self-supervised MRI reconstruction with Artifact2Artifact

Self-supervised MRI reconstruction with Artifact2Artifact

Self-supervised denoising with the UNSURE loss.

Self-supervised denoising with the UNSURE loss.

Self-supervised denoising with the SURE loss.

Self-supervised denoising with the SURE loss.

Self-supervised denoising with the Neighbor2Neighbor loss.

Self-supervised denoising with the Neighbor2Neighbor loss.

Self-supervised learning with Equivariant Imaging for MRI.

Self-supervised learning with Equivariant Imaging for MRI.

Self-supervised learning from incomplete measurements of multiple operators.

Self-supervised learning from incomplete measurements of multiple operators.

Imaging inverse problems with adversarial networks

Imaging inverse problems with adversarial networks

Radio interferometric imaging with deepinverse

Radio interferometric imaging with deepinverse