.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/basics/demo_dip.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_basics_demo_dip.py: Reconstructing an image using the deep image prior. =================================================== This code shows how to reconstruct a noisy and incomplete image using the deep image prior. This method is based on the paper `"Deep Image Prior" by Ulyanov et al. (2018) `_, and reconstructs an image by minimizing the loss function .. math:: \min_{\theta} \|y-Af_{\theta}(z)\|^2 where :math:`z` is a random input and :math:`f_{\theta}` is a convolutional decoder network with parameters :math:`\theta`. The minimization should be stopped early to avoid overfitting. The method uses the Adam optimizer. .. GENERATED FROM PYTHON SOURCE LINES 21-27 .. code-block:: Python import deepinv as dinv from deepinv.utils.plotting import plot import torch from deepinv.utils import load_url_image .. GENERATED FROM PYTHON SOURCE LINES 28-32 Load image from the internet ---------------------------- This example uses an image of Lionel Messi from Wikipedia. .. GENERATED FROM PYTHON SOURCE LINES 32-44 .. code-block:: Python device = dinv.utils.get_freer_gpu() if torch.cuda.is_available() else "cpu" url = ( "https://upload.wikimedia.org/wikipedia/commons/b/b4/" "Lionel-Messi-Argentina-2022-FIFA-World-Cup_%28cropped%29.jpg" ) x = load_url_image(url=url, img_size=32).to(device) # Set the global random seed from pytorch to ensure reproducibility of the example. torch.manual_seed(0) .. rst-class:: sphx-glr-script-out .. code-block:: none .. GENERATED FROM PYTHON SOURCE LINES 45-49 Define forward operator and noise model --------------------------------------- We use image inpainting as the forward operator and Gaussian noise as the noise model. .. GENERATED FROM PYTHON SOURCE LINES 49-54 .. code-block:: Python sigma = 0.1 # noise level physics = dinv.physics.Inpainting(mask=0.5, tensor_size=x.shape[1:], device=device) physics.noise_model = dinv.physics.GaussianNoise(sigma=sigma) .. GENERATED FROM PYTHON SOURCE LINES 55-58 Generate the measurement ------------------------ We apply the forward model to generate the noisy measurement. .. GENERATED FROM PYTHON SOURCE LINES 58-61 .. code-block:: Python y = physics(x) .. GENERATED FROM PYTHON SOURCE LINES 62-77 Define the deep image prior ---------------------------- This method only works with certain convolutional decoder networks. We recommend using the network :class:`deepinv.models.ConvDecoder`. .. note:: The number of iterations and learning rate have been set manually to obtain good results. However, these values may not be optimal for all problems. We recommend experimenting with different values. .. note:: Here we run a small number of iterations to reduce the runtime of the example. However, the results could be improved by running more iterations. .. GENERATED FROM PYTHON SOURCE LINES 77-94 .. code-block:: Python iterations = 100 lr = 1e-2 # learning rate for the optimizer. channels = 64 # number of channels per layer in the decoder. in_size = [2, 2] # size of the input to the decoder. backbone = dinv.models.ConvDecoder( img_shape=x.shape[1:], in_size=in_size, channels=channels ).to(device) f = dinv.models.DeepImagePrior( backbone, learning_rate=lr, iterations=iterations, verbose=True, input_size=[channels] + in_size, ).to(device) .. GENERATED FROM PYTHON SOURCE LINES 95-105 Run DIP algorithm and plot results ---------------------------------- We run the DIP algorithm and plot the results. The good performance of DIP is somewhat surprising, since the network has many parameters and could potentially overfit the noisy measurement data. However, the architecture acts as an implicit regularizer, providing good reconstructions if the optimization is stopped early. While this phenomenon is not yet well understood, there has been some efforts to explain it. For example, see `"The Neural Tangent Link Between CNN Denoisers and Non-Local Filters" `_. .. GENERATED FROM PYTHON SOURCE LINES 105-118 .. code-block:: Python dip = f(y, physics) # compute linear inverse x_lin = physics.A_adjoint(y) # compute PSNR print(f"Linear reconstruction PSNR: {dinv.metric.PSNR()(x, x_lin).item():.2f} dB") print(f"DIP PSNR: {dinv.metric.PSNR()(x, dip).item():.2f} dB") # plot results plot([x_lin, x, dip], titles=["measurement", "ground truth", "DIP"]) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_dip_001.png :alt: measurement, ground truth, DIP :srcset: /auto_examples/basics/images/sphx_glr_demo_dip_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none 0%| | 0/100 [00:00` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: demo_dip.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: demo_dip.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_