.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/basics/demo_physics_tour.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_basics_demo_physics_tour.py: A tour of forward sensing operators =================================================== This example provides a tour of some of the forward operators implemented in DeepInverse. We restrict ourselves to operators where the signal is a 2D image. The full list of operators can be found in `here `_. .. GENERATED FROM PYTHON SOURCE LINES 10-18 .. code-block:: Python import torch import deepinv as dinv from deepinv.utils.plotting import plot from deepinv.utils.demo import load_url_image, get_image_url .. GENERATED FROM PYTHON SOURCE LINES 19-23 Load image from the internet ---------------------------- This example uses an image of the CBSD68 dataset. .. GENERATED FROM PYTHON SOURCE LINES 23-35 .. code-block:: Python device = dinv.utils.get_freer_gpu() if torch.cuda.is_available() else "cpu" url = get_image_url("CBSD_0010.png") x = load_url_image(url, grayscale=False).to(device) x = torch.tensor(x, device=device, dtype=torch.float) x = torch.nn.functional.interpolate(x, size=(64, 64)) img_size = x.shape[1:] # Set the global random seed from pytorch to ensure reproducibility of the example. torch.manual_seed(0) .. rst-class:: sphx-glr-script-out .. code-block:: none /home/runner/work/deepinv/deepinv/examples/basics/demo_physics_tour.py:29: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor). x = torch.tensor(x, device=device, dtype=torch.float) .. GENERATED FROM PYTHON SOURCE LINES 36-41 Denoising --------------------------------------- The denoising class :class:`deepinv.physics.Denoising` is associated with an identity operator. In this example we choose a Poisson noise. .. GENERATED FROM PYTHON SOURCE LINES 41-49 .. code-block:: Python physics = dinv.physics.Denoising(dinv.physics.PoissonNoise(0.1)) y = physics(x) # plot results plot([x, y], titles=["signal", "measurement"]) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_physics_tour_001.png :alt: signal, measurement :srcset: /auto_examples/basics/images/sphx_glr_demo_physics_tour_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 50-56 Inpainting --------------------------------------- The inpainting class :class:`deepinv.physics.Inpainting` is associated with a mask operator. The mask is generated at random (unless an explicit mask is provided as input). We also consider Gaussian noise in this example. .. GENERATED FROM PYTHON SOURCE LINES 56-71 .. code-block:: Python sigma = 0.1 # noise level physics = dinv.physics.Inpainting( mask=0.5, tensor_size=x.shape[1:], noise_model=dinv.physics.GaussianNoise(sigma=sigma), device=device, ) y = physics(x) # plot results plot([x, y], titles=["signal", "measurement"]) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_physics_tour_002.png :alt: signal, measurement :srcset: /auto_examples/basics/images/sphx_glr_demo_physics_tour_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 72-77 Demosaicing --------------------------------------- The demosaicing class :class:`deepinv.physics.Demosaicing` is associated with a Bayer pattern, which is a color filter array used in digital cameras (see `Wikipedia `_). .. GENERATED FROM PYTHON SOURCE LINES 77-85 .. code-block:: Python physics = dinv.physics.Demosaicing(img_size=(64, 64), device=device) y = physics(x) # plot results plot([x, y], titles=["signal", "measurement"]) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_physics_tour_003.png :alt: signal, measurement :srcset: /auto_examples/basics/images/sphx_glr_demo_physics_tour_003.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 86-91 Compressed Sensing --------------------------------------- The compressed sensing class :class:`deepinv.physics.CompressedSensing` is associated with a random Gaussian matrix. Here we take 2048 measurements of an image of size 64x64, which corresponds to a compression ratio of 2. .. GENERATED FROM PYTHON SOURCE LINES 91-106 .. code-block:: Python physics = dinv.physics.CompressedSensing( m=2048, fast=False, channelwise=True, img_shape=img_size, compute_inverse=True, device=device, ) y = physics(x) # plot results plot([x, physics.A_dagger(y)], titles=["signal", "linear inverse"]) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_physics_tour_004.png :alt: signal, linear inverse :srcset: /auto_examples/basics/images/sphx_glr_demo_physics_tour_004.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 107-112 Computed Tomography --------------------------------------- The class :class:`deepinv.physics.Tomography` is associated with the sparse Radon transform. Here we take 20 views of an image of size 64x64, and consider mixed Poisson-Gaussian noise. .. GENERATED FROM PYTHON SOURCE LINES 112-128 .. code-block:: Python physics = dinv.physics.Tomography( img_width=img_size[-1], angles=20, device=device, noise_model=dinv.physics.PoissonGaussianNoise(gain=0.1, sigma=0.05), ) y = physics(x) # plot results plot( [x, (y - y.min()) / y.max(), physics.A_dagger(y)], titles=["signal", "sinogram", "filtered backprojection"], ) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_physics_tour_005.png :alt: signal, sinogram, filtered backprojection :srcset: /auto_examples/basics/images/sphx_glr_demo_physics_tour_005.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 129-139 MRI --------------------------------------- The class :class:`deepinv.physics.MRI` is associated with a subsampling of the Fourier transform. The mask indicates which Fourier coefficients are measured. Here we use a random Cartesian mask, which corresponds to a compression ratio of approximately 4. .. note:: The signal must be complex-valued for this operator, where the first channel corresponds to the real part and the second channel to the imaginary part. In this example, we set the imaginary part to zero. .. GENERATED FROM PYTHON SOURCE LINES 139-159 .. code-block:: Python mask = torch.rand((1, img_size[-1]), device=device) > 0.75 mask = torch.ones((img_size[-2], 1), device=device) * mask mask[:, int(img_size[-1] / 2) - 2 : int(img_size[-1] / 2) + 2] = 1 physics = dinv.physics.MRI( mask=mask, device=device, noise_model=dinv.physics.GaussianNoise(sigma=0.05) ) x2 = torch.cat( [x[:, 0, :, :].unsqueeze(1), torch.zeros_like(x[:, 0, :, :].unsqueeze(1))], dim=1 ) y = physics(x2) # plot results plot( [x2, mask.unsqueeze(0).unsqueeze(0), physics.A_adjoint(y)], titles=["signal", "k-space mask", "linear inverse"], ) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_physics_tour_006.png :alt: signal, k-space mask, linear inverse :srcset: /auto_examples/basics/images/sphx_glr_demo_physics_tour_006.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 160-162 We also provide physics generators for various accelerated MRI masks. These are Cartesian sampling strategies and can be used for static (k) and dynamic (k-t) undersampling: .. GENERATED FROM PYTHON SOURCE LINES 163-184 .. code-block:: Python from deepinv.physics.generator import ( GaussianMaskGenerator, RandomMaskGenerator, EquispacedMaskGenerator, ) # shape (C, T, H, W) mask_gaussian = GaussianMaskGenerator((2, 8, 64, 50), acceleration=4).step()["mask"] mask_uniform = EquispacedMaskGenerator((2, 8, 64, 50), acceleration=4).step()["mask"] mask_random = RandomMaskGenerator((2, 8, 64, 50), acceleration=4).step()["mask"] plot( [ mask_gaussian[:, :, 0, ...], mask_uniform[:, :, 0, ...], mask_random[:, :, 0, ...], ], titles=["Gaussian", "Uniform", "Random uniform"], ) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_physics_tour_007.png :alt: Gaussian, Uniform, Random uniform :srcset: /auto_examples/basics/images/sphx_glr_demo_physics_tour_007.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 185-190 Decolorize --------------------------------------- The class :class:`deepinv.physics.Decolorize` is associated with a simple color-to-gray operator. .. GENERATED FROM PYTHON SOURCE LINES 190-198 .. code-block:: Python physics = dinv.physics.Decolorize(device=device) y = physics(x) # plot results plot([x, y], titles=["signal", "measurement"]) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_physics_tour_008.png :alt: signal, measurement :srcset: /auto_examples/basics/images/sphx_glr_demo_physics_tour_008.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 199-204 Pan-sharpening --------------------------------------- The class :class:`deepinv.physics.Pansharpen` obtains measurements which consist of a high-resolution grayscale image and a low-resolution RGB image. .. GENERATED FROM PYTHON SOURCE LINES 204-212 .. code-block:: Python physics = dinv.physics.Pansharpen(img_size=img_size, device=device) y = physics(x) # plot results plot([x, y[0], y[1]], titles=["signal", "low res rgb", "high res gray"]) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_physics_tour_009.png :alt: signal, low res rgb, high res gray :srcset: /auto_examples/basics/images/sphx_glr_demo_physics_tour_009.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 213-218 Single-Pixel Camera --------------------------------------- The single-pixel camera class :class:`deepinv.physics.SinglePixelCamera` is associated with ``m`` binary patterns. When ``fast=True``, the patterns are generated using a fast Hadamard transform. .. GENERATED FROM PYTHON SOURCE LINES 218-229 .. code-block:: Python physics = dinv.physics.SinglePixelCamera( m=256, fast=True, img_shape=img_size, device=device ) y = physics(x) # plot results plot([x, physics.A_adjoint(y)], titles=["signal", "linear inverse"]) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_physics_tour_010.png :alt: signal, linear inverse :srcset: /auto_examples/basics/images/sphx_glr_demo_physics_tour_010.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 230-235 Blur --------------------------------------- The class :class:`deepinv.physics.Blur` blurs the input image with a specified kernel. Here we use a Gaussian blur with a standard deviation of 2 pixels and an angle of 45 degrees. .. GENERATED FROM PYTHON SOURCE LINES 235-246 .. code-block:: Python physics = dinv.physics.Blur( dinv.physics.blur.gaussian_blur(sigma=(2, 0.1), angle=45.0), device=device ) y = physics(x) # plot results plot([x, y], titles=["signal", "measurement"]) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_physics_tour_011.png :alt: signal, measurement :srcset: /auto_examples/basics/images/sphx_glr_demo_physics_tour_011.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 247-251 Super-Resolution --------------------------------------- The downsampling class :class:`deepinv.physics.Downsampling` is associated with a downsampling operator. .. GENERATED FROM PYTHON SOURCE LINES 251-260 .. code-block:: Python physics = dinv.physics.Downsampling(img_size=img_size, factor=2, device=device) y = physics(x) # plot results plot([x, y], titles=["signal", "measurement"]) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_physics_tour_012.png :alt: signal, measurement :srcset: /auto_examples/basics/images/sphx_glr_demo_physics_tour_012.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 4.539 seconds) .. _sphx_glr_download_auto_examples_basics_demo_physics_tour.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: demo_physics_tour.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: demo_physics_tour.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: demo_physics_tour.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_