Loss

This package contains popular training losses for supervised and self-supervised learning, which are especially designed for inverse problems.

Introduction

All losses inherit from the base class deepinv.loss.Loss(), which is a meth:torch.nn.Module.

deepinv.loss.Loss

Base class for all loss functions.

>>> import torch
>>> import deepinv as dinv
>>> loss = dinv.loss.SureGaussianLoss(.1)
>>> physics = dinv.physics.Denoising()
>>> x = torch.ones(1, 3, 16, 16)
>>> y = physics(x)
>>> model = dinv.models.DnCNN()
>>> x_net = model(y)
>>> l = loss(x_net=x_net, y=y, physics=physics, model=model) # self-supervised loss, doesn't require ground truth x

Supervised Learning

Use a dataset of pairs of signals and measurements (and possibly information about the forward operator), i.e., they can be written as \(\mathcal{L}(x,\inverse{y})\).

deepinv.loss.SupLoss

Standard supervised loss

Self-Supervised Learning

Use a dataset of measurement data alone (and possibly information about the forward operator), i.e., they can be written as \(\mathcal{L}(y,\inverse{y})\) and take into account information about the forward measurement process.

deepinv.loss.MCLoss	Measurement consistency loss
deepinv.loss.EILoss	Equivariant imaging self-supervised loss.
deepinv.loss.MOILoss	Multi-operator imaging loss
deepinv.loss.MOEILoss	Multi-operator equivariant imaging.
deepinv.loss.Neighbor2Neighbor	Neighbor2Neighbor loss.
deepinv.loss.SplittingLoss	Measurement splitting loss.
deepinv.loss.Phase2PhaseLoss	Phase2Phase loss for dynamic data.
deepinv.loss.Artifact2ArtifactLoss	Artifact2Artifact loss for dynamic data.
deepinv.loss.SureGaussianLoss	SURE loss for Gaussian noise
deepinv.loss.SurePoissonLoss	SURE loss for Poisson noise
deepinv.loss.SurePGLoss	SURE loss for Poisson-Gaussian noise
deepinv.loss.TVLoss	Total variation loss (\(\ell_2\) norm).
deepinv.loss.R2RLoss	Recorrupted-to-Recorrupted (R2R) Loss
deepinv.loss.ScoreLoss	Learns score of noise distribution.

Adversarial Learning

Adversarial losses train a generator network by jointly training with an additional discriminator network in a minimax game. We implement various popular (supervised and unsupervised) adversarial training frameworks below. These can be adapted to various flavours of GAN, e.g. WGAN, LSGAN. Generator and discriminator networks are provided in adversarial models. Training is implemented using deepinv.training.AdversarialTrainer which overrides the standard deepinv.Trainer. See Imaging inverse problems with adversarial networks for usage.

deepinv.loss.adversarial.DiscriminatorMetric	Generic GAN discriminator metric building block.
deepinv.loss.adversarial.GeneratorLoss	Base generator adversarial loss.
deepinv.loss.adversarial.DiscriminatorLoss	Base discriminator adversarial loss.
deepinv.loss.adversarial.SupAdversarialGeneratorLoss	Supervised adversarial consistency loss for generator.
deepinv.loss.adversarial.SupAdversarialDiscriminatorLoss	Supervised adversarial consistency loss for discriminator.
deepinv.loss.adversarial.UnsupAdversarialGeneratorLoss	Unsupervised adversarial consistency loss for generator.
deepinv.loss.adversarial.UnsupAdversarialDiscriminatorLoss	Unsupervised adversarial consistency loss for discriminator.
deepinv.loss.adversarial.UAIRGeneratorLoss	Reimplementation of UAIR generator's adversarial loss.

Network Regularization

These losses can be used to regularize the learned function, e.g., controlling its Lipschitz constant.

deepinv.loss.JacobianSpectralNorm	Computes the spectral norm of the Jacobian.
deepinv.loss.FNEJacobianSpectralNorm	Computes the Firm-Nonexpansiveness Jacobian spectral norm.

Loss schedulers

Loss schedulers can be used to control which losses are used when during more advanced training.

deepinv.loss.BaseLossScheduler	Base class for loss schedulers.
deepinv.loss.RandomLossScheduler	Schedule losses at random.
deepinv.loss.InterleavedLossScheduler	Schedule losses sequentially one-by-one.
deepinv.loss.InterleavedEpochLossScheduler	Schedule losses sequentially epoch-by-epoch.
deepinv.loss.StepLossScheduler	Activate losses at specified epoch.

Utils

A set of popular distances that can be used by the supervised and self-supervised losses.

deepinv.loss.metric.LpNorm

\(\ell_p\) metric for \(p>0\).