Pansharpen

class deepinv.physics.Pansharpen(img_size, filter='bilinear', factor=4, noise_color=GaussianNoise(), noise_gray=GaussianNoise(), device='cpu', padding='circular', **kwargs)[source]

Bases: LinearPhysics

Pansharpening forward operator.

The measurements consist of a high resolution grayscale image and a low resolution RGB image, and are represented using deepinv.utils.TensorList, where the first element is the RGB image and the second element is the grayscale image.

By default, the downsampling is done with a gaussian filter with standard deviation equal to the downsampling, however, the user can provide a custom downsampling filter.

It is possible to assign a different noise model to the RGB and grayscale images.

Parameters:

  • img_size (tuple[int]) – size of the input image.

  • filter (torch.Tensor, str, NoneType) – Downsampling filter. It can be ‘gaussian’, ‘bilinear’ or ‘bicubic’ or a custom torch.Tensor filter. If None, no filtering is applied.

  • factor (int) – downsampling factor.

  • noise_color (torch.nn.Module) – noise model for the RGB image.

  • noise_gray (torch.nn.Module) – noise model for the grayscale image.

  • padding (str) – options are 'valid', 'circular', 'replicate' and 'reflect'. If padding='valid' the blurred output is smaller than the image (no padding) otherwise the blurred output has the same size as the image.

Examples:

Pansharpen operator applied to a random 32x32 image:

>>> from deepinv.physics import Pansharpen
>>> x = torch.randn(1, 3, 32, 32) # Define random 32x32 color image
>>> physics = Pansharpen(img_size=x.shape[1:], device=x.device)
>>> x.shape
torch.Size([1, 3, 32, 32])
>>> y = physics(x)
>>> y[0].shape
torch.Size([1, 3, 8, 8])
>>> y[1].shape
torch.Size([1, 1, 32, 32])
A(x, **kwargs)[source]

Computes forward operator \(y = A(x)\) (without noise and/or sensor non-linearities)

Parameters:

x (torch.Tensor,list[torch.Tensor]) – signal/image

Returns:

(torch.Tensor) clean measurements

A_adjoint(y, **kwargs)[source]

Computes transpose of the forward operator \(\tilde{x} = A^{\top}y\). If \(A\) is linear, it should be the exact transpose of the forward matrix.

Note

If the problem is non-linear, there is not a well-defined transpose operation, but defining one can be useful for some reconstruction networks, such as deepinv.models.ArtifactRemoval.

Parameters:

  • y (torch.Tensor) – measurements.

  • params (None, torch.Tensor) – optional additional parameters for the adjoint operator.

Returns:

(torch.Tensor) linear reconstruction \(\tilde{x} = A^{\top}y\).

forward(x, **kwargs)[source]

Computes forward operator

\[y = N(A(x), \sigma)\]
Parameters:

x (torch.Tensor, list[torch.Tensor]) – signal/image

Returns:

(torch.Tensor) noisy measurements

Examples using Pansharpen:

Stacking and concatenating forward operators.

Stacking and concatenating forward operators.

A tour of forward sensing operators

A tour of forward sensing operators