Operators & Noise

Operators

Operators describe the forward model \(z = A(x,\theta)\), where \(x\) is the input image and \(\theta\) are the parameters of the operator. The parameters \(\theta\) can be sampled using random generators, which are available for some specific classes. Using automatic differentiation, we can compute derivatives w.r.t to both the input \(x\) or the parameters \(\theta\). This is particular useful when dealing with blind inverse problems or parameter estimation.

Tip

The operator you are looking for is not on this list? See Bring your own physics for how to implement your own physics operator.

Table 1 Operators, Definitions, and Generators

Family	Operators	Generators
Pixelwise	deepinv.physics.Denoising deepinv.physics.Inpainting deepinv.physics.Demosaicing deepinv.physics.Decolorize	BernoulliSplittingMaskGenerator GaussianSplittingMaskGenerator MultiplicativeSplittingMaskGenerator Phase2PhaseSplittingMaskGenerator Artifact2ArtifactSplittingMaskGenerator
Blur & Super-Resolution	deepinv.physics.Blur deepinv.physics.BlurFFT deepinv.physics.SpaceVaryingBlur deepinv.physics.Downsampling deepinv.physics.Upsampling deepinv.physics.DownsamplingMatlab	MotionBlurGenerator DownsamplingGenerator DiffractionBlurGenerator ProductConvolutionBlurGenerator ConfocalBlurGenerator3D gaussian_blur, sinc_filter bilinear_filter, bicubic_filter
Magnetic Resonance Imaging (MRI)	deepinv.physics.MRIMixin deepinv.physics.MRI deepinv.physics.MultiCoilMRI deepinv.physics.DynamicMRI deepinv.physics.SequentialMRI The above all also natively support 3D MRI.	GaussianMaskGenerator RandomMaskGenerator EquispacedMaskGenerator PolyOrderMaskGenerator The above all also support k+t dynamic sampling.
Tomography	deepinv.physics.Tomography deepinv.physics.TomographyWithAstra
Remote Sensing & Multispectral	deepinv.physics.Pansharpen deepinv.physics.HyperSpectralUnmixing deepinv.physics.CompressiveSpectralImaging
Compressive	deepinv.physics.CompressedSensing deepinv.physics.StructuredRandom deepinv.physics.SinglePixelCamera
Radio Interferometric Imaging	deepinv.physics.RadioInterferometry
Single-Photon Lidar	deepinv.physics.SinglePhotonLidar
Dehazing	deepinv.physics.Haze
Phase Retrieval	deepinv.physics.PhaseRetrieval RandomPhaseRetrieval StructuredRandomPhaseRetrieval Ptychography PtychographyLinearOperator	build_probe generate_shifts

Wrappers

Wrappers are operators that can be used to adapt existing operators to a new problem.

Table 2 Wrappers

Family	Operators
Multiscale	deepinv.physics.PhysicsMultiScaler deepinv.physics.LinearPhysicsMultiScaler
Padding/Cropping	deepinv.physics.PhysicsCropper

Noise distributions

Noise distributions describe the noise model \(N\), where \(y = N(z)\) with \(z=A(x)\). The noise models can be assigned to any operator in the list above, by setting the set_noise_model attribute at initialization. By default, the noise model is set to ZeroNoise.

Table 3 Noise Distributions and Their Probability Distributions

Noise	\(y|z\)
deepinv.physics.ZeroNoise	\(y=z\)
deepinv.physics.GaussianNoise	\(y\sim \mathcal{N}(z, I\sigma^2)\)
deepinv.physics.PoissonNoise	\(y \sim \mathcal{P}(z/\gamma)\)
deepinv.physics.PoissonGaussianNoise	\(y = \gamma z + \epsilon\), \(z\sim\mathcal{P}(\frac{z}{\gamma})\), \(\epsilon\sim\mathcal{N}(0, I \sigma^2)\)
deepinv.physics.LogPoissonNoise	\(y = \frac{1}{\mu} \log(\frac{\mathcal{P}(\exp(-\mu z) N_0)}{N_0})\)
deepinv.physics.UniformNoise	\(y\sim \mathcal{U}(z-a, z+b)\)
deepinv.physics.SaltPepperNoise	\(y = \begin{cases} 0 & \text{if } z < p\\ x & \text{if } z \in [p, 1-s]\\ 1 & \text{if } z > 1 - s\end{cases}\) with \(z\sim\mathcal{U}(0,1)\)
deepinv.physics.ZeroNoise	\(y = z\)

Mixins

The physics module maximizes code reuse via inheritance. We provide mixin classes to provide specialized methods for certain physics, models, datasets and losses, such as temporal or MRI functionality.

Table 4 Mixins

Mixin	Description
deepinv.physics.MRIMixin	Utility methods for MRI physics.
deepinv.physics.TimeMixin	Methods for expanding and flattening time dimension for dynamic/video data.