RAM#
- class deepinv.models.RAM(in_channels=[1, 2, 3], device=None, pretrained=True)[source]#
Bases:
Reconstructor,DenoiserReconstruct Anything Model (RAM) foundation model.
RAM Terris et al.[1] is a convolutional neural network model that has been trained to work on a large variety of linear image reconstruction tasks and datasets (deblurring, inpainting, denoising, tomography, MRI, etc.).
See Inference and fine-tune a foundation model for examples on the performance of RAM and how to fine-tune the foundation model on a specific problem and dataset.
The model works both as a reconstructor or denoiser:
Reconstructor: RAM takes a physics operator
model(y, physics)with an optional noise model defined in the physicsDenoiser: RAM takes optional Gaussian and/or Poisson noise levels (optionally set to 0)
model(y, sigma=sigma, gamma=gamma)
Note
The physics operator should be normalized (i.e. have unit norm) for best results. Use
physics.compute_norm()to check this.- Parameters:
in_channels (list) – Number of input channels. If a list is provided, the model will have separate heads for each channel.
device (str) – Device to which the model should be moved. If None, the model will be created on the default device.
pretrained (bool, str) – If
True, the model will be initialized with pretrained weights. Ifstr, load from file.sigma_threshold (float) – Threshold (minimum value) for the noise level. Default is 1e-3.
- References:
- base_conditioning(x, sigma, gain)[source]#
Stacks the sigma and gain value as additional channel dimensions to the input tensor.
- Parameters:
x (torch.Tensor) – Input tensor
sigma (float) – Gaussian noise level
gain (float) – Poisson noise gain
- Return torch.Tensor:
Input tensor with additional channels for sigma and gain
- constant2map(value, x)[source]#
Converts a constant value to a map of the same size as the input tensor x.
- Parameters:
value (float) – constant value
x (torch.Tensor) – input tensor
- Return torch.Tensor:
a tensor of size (B, 1, W, H) containing constant maps of shapes (W, H) for each value in the batch.
- forward(y, physics=None, sigma=None, gain=None)[source]#
Reconstructs a signal estimate from measurements y
- Parameters:
y (torch.Tensor) – measurements
physics (deepinv.physics.Physics) – forward operator
sigma (float, torch.Tensor) – Gaussian noise level. Ignored if noise_model already specified in physics.
gain (float, torch.Tensor) – Poisson noise level. Ignored if noise_model already specified in physics.
- Returns:
torch.Tensor: reconstructed signal estimate
- forward_unet(x0, sigma=None, gain=None, physics=None, y=None)[source]#
Forward pass of the UNet model.
- Parameters:
x0 (torch.Tensor) – init image
sigma (float) – Gaussian noise level
gamma (float) – Poisson noise gain
physics (deepinv.physics.Physics) – physics measurement operator
y (torch.Tensor) – measurements
- obtain_sigma_gain(physics=None, y=None, sigma=None, gain=None, rescale_val=1.0)[source]#
Defines the sigma and gain values to be used in the model.
If a noise model is specified in the physics, the sigma and gain values will be taken from the noise model. Else, the sigma and gain values will be set to the thresholds defined in the model (if not provided).
- Parameters:
physics (deepinv.physics.Physics) – Physics model
y (torch.Tensor) – Measurements
sigma (float, torch.Tensor) – Gaussian noise level. If None, will be set to the threshold.
gain (float, torch.Tensor) – Poisson noise level. If None, will be set to the threshold.
rescale_val (float) – Rescale value to apply to the sigma and gain values.
- realign_input(x, physics, y, sigma)[source]#
Realign the input x based on the measurements y and the physics model. Applies the proximity operator of the L2 norm with respect to the physics model.
- Parameters:
x (torch.Tensor) – Input tensor
physics (deepinv.physics.Physics) – Physics model
y (torch.Tensor) – Measurements
- Return torch.Tensor:
Realigned input tensor
