.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/physics/demo_blur_tour.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note New to DeepInverse? Get started with the basics with the :ref:`5 minute quickstart tutorial `. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_physics_demo_blur_tour.py: Tour of blur operators =================================================== This example provides a tour of 2D blur operators in DeepInverse. In particular, we show how to use DiffractionBlurs (Fresnel diffraction), motion blurs and space varying blurs. .. GENERATED FROM PYTHON SOURCE LINES 9-17 .. code-block:: Python import torch import deepinv as dinv from deepinv.utils.plotting import plot from deepinv.utils.demo import load_example .. GENERATED FROM PYTHON SOURCE LINES 18-21 ---------------- First, let's load some test images. .. GENERATED FROM PYTHON SOURCE LINES 21-38 .. code-block:: Python dtype = torch.float32 device = "cpu" img_size = (173, 125) x_rgb = load_example( "CBSD_0010.png", grayscale=False, device=device, dtype=dtype, img_size=img_size ) x_gray = load_example( "barbara.jpeg", grayscale=True, device=device, dtype=dtype, img_size=img_size ) # Next, set the global random seed from pytorch to ensure reproducibility of the example. torch.manual_seed(0) torch.cuda.manual_seed(0) .. GENERATED FROM PYTHON SOURCE LINES 39-47 We are now ready to explore the different blur operators. Convolution Basics ------------------ The class :class:`deepinv.physics.Blur` implements convolution operations with kernels. For instance, here is the convolution of a grayscale image with a grayscale filter: .. GENERATED FROM PYTHON SOURCE LINES 47-56 .. code-block:: Python filter_0 = dinv.physics.blur.gaussian_blur(sigma=(2, 0.1), angle=0.0) physics = dinv.physics.Blur(filter_0, device=device) y = physics(x_gray) plot( [x_gray, filter_0, y], titles=["signal", "filter", "measurement"], suptitle="Grayscale convolution", ) .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_001.png :alt: Grayscale convolution, signal, filter, measurement :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none /home/runner/work/deepinv/deepinv/deepinv/utils/plotting.py:320: UserWarning: This figure was using a layout engine that is incompatible with subplots_adjust and/or tight_layout; not calling subplots_adjust. fig.subplots_adjust(top=0.75) .. GENERATED FROM PYTHON SOURCE LINES 57-59 When a single channel filter is used, all channels are convolved with the same filter: .. GENERATED FROM PYTHON SOURCE LINES 59-68 .. code-block:: Python physics = dinv.physics.Blur(filter_0, device=device) y = physics(x_rgb) plot( [x_rgb, filter_0, y], titles=["signal", "filter", "measurement"], suptitle="RGB image + grayscale filter convolution", ) .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_002.png :alt: RGB image + grayscale filter convolution, signal, filter, measurement :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 69-71 By default, the boundary conditions are ``'valid'``, but other options among (``'circular'``, ``'reflect'``, ``'replicate'``) are possible: .. GENERATED FROM PYTHON SOURCE LINES 71-80 .. code-block:: Python physics = dinv.physics.Blur(filter_0, padding="reflect", device=device) y = physics(x_rgb) plot( [x_rgb, filter_0, y], titles=["signal", "filter", "measurement"], suptitle="Reflection boundary conditions", ) .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_003.png :alt: Reflection boundary conditions, signal, filter, measurement :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_003.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 81-84 For circular boundary conditions, an FFT implementation is also available. It is slower that :class:`deepinv.physics.Blur`, but inherits from :class:`deepinv.physics.DecomposablePhysics`, so that the pseudo-inverse and regularized inverse are computed faster and more accurately. .. GENERATED FROM PYTHON SOURCE LINES 84-92 .. code-block:: Python physics = dinv.physics.BlurFFT(img_size=x_rgb[0].shape, filter=filter_0, device=device) y = physics(x_rgb) plot( [x_rgb, filter_0, y], titles=["signal", "filter", "measurement"], suptitle="FFT convolution with circular boundary conditions", ) .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_004.png :alt: FFT convolution with circular boundary conditions, signal, filter, measurement :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_004.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 93-94 One can also change the blur filter in the forward pass as follows: .. GENERATED FROM PYTHON SOURCE LINES 94-104 .. code-block:: Python filter_90 = dinv.physics.blur.gaussian_blur(sigma=(2, 0.1), angle=90.0).to( device=device, dtype=dtype ) y = physics(x_rgb, filter=filter_90) plot( [x_rgb, filter_90, y], titles=["signal", "filter", "measurement"], suptitle="Changing the filter on the fly", ) .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_005.png :alt: Changing the filter on the fly, signal, filter, measurement :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_005.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 105-106 When applied to a new image, the last filter is used: .. GENERATED FROM PYTHON SOURCE LINES 106-113 .. code-block:: Python y = physics(x_gray, filter=filter_90) plot( [x_gray, filter_90, y], titles=["signal", "filter", "measurement"], suptitle="Effect of on the fly change is persistent", ) .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_006.png :alt: Effect of on the fly change is persistent, signal, filter, measurement :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_006.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 114-115 We can also define color filters. In that situation, each channel is convolved with the corresponding channel of the filter: .. GENERATED FROM PYTHON SOURCE LINES 115-129 .. code-block:: Python psf_size = 9 filter_rgb = torch.zeros((1, 3, psf_size, psf_size), device=device, dtype=dtype) filter_rgb[:, 0, :, psf_size // 2 : psf_size // 2 + 1] = 1.0 / psf_size filter_rgb[:, 1, psf_size // 2 : psf_size // 2 + 1, :] = 1.0 / psf_size filter_rgb[:, 2, ...] = ( torch.diag(torch.ones(psf_size, device=device, dtype=dtype)) / psf_size ) y = physics(x_rgb, filter=filter_rgb) plot( [x_rgb, filter_rgb, y], titles=["signal", "Colour filter", "measurement"], suptitle="Color image + color filter convolution", ) .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_007.png :alt: Color image + color filter convolution, signal, Colour filter, measurement :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_007.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 130-134 Blur generators ---------------------- More advanced kernel generation methods are provided with the toolbox thanks to the :class:`deepinv.physics.generator.PSFGenerator`. In particular, motion blurs generators are implemented. .. GENERATED FROM PYTHON SOURCE LINES 136-138 Motion blur generators ~~~~~~~~~~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 138-140 .. code-block:: Python from deepinv.physics.generator import MotionBlurGenerator .. GENERATED FROM PYTHON SOURCE LINES 141-145 In order to generate motion blur kernels, we just need to instantiate a generator with specific the psf size. In turn, motion blurs can be generated on the fly by calling the ``step()`` method. Let's illustrate this now and generate 3 motion blurs. First, we instantiate the generator: .. GENERATED FROM PYTHON SOURCE LINES 145-147 .. code-block:: Python psf_size = 31 motion_generator = MotionBlurGenerator((psf_size, psf_size), device=device, dtype=dtype) .. GENERATED FROM PYTHON SOURCE LINES 148-149 To generate new filters, we call the step() function: .. GENERATED FROM PYTHON SOURCE LINES 149-157 .. code-block:: Python filters = motion_generator.step(batch_size=3) # the `step()` function returns a dictionary: print(filters.keys()) plot( [f for f in filters["filter"]], suptitle="Examples of randomly generated motion blurs", ) .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_008.png :alt: Examples of randomly generated motion blurs :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_008.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none dict_keys(['filter']) .. GENERATED FROM PYTHON SOURCE LINES 158-159 Other options, such as the regularity and length of the blur trajectory can also be specified: .. GENERATED FROM PYTHON SOURCE LINES 159-165 .. code-block:: Python motion_generator = MotionBlurGenerator( (psf_size, psf_size), l=0.6, sigma=1, device=device, dtype=dtype ) filters = motion_generator.step(batch_size=3) plot([f for f in filters["filter"]], suptitle="Different length and regularity") .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_009.png :alt: Different length and regularity :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_009.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 166-171 Diffraction blur generators ~~~~~~~~~~~~~~~~~~~~~~~~~~~ We also implemented diffraction blurs obtained through Fresnel theory and definition of the psf through the pupil plane expanded in Zernike polynomials .. GENERATED FROM PYTHON SOURCE LINES 171-178 .. code-block:: Python from deepinv.physics.generator import DiffractionBlurGenerator diffraction_generator = DiffractionBlurGenerator( (psf_size, psf_size), device=device, dtype=dtype ) .. GENERATED FROM PYTHON SOURCE LINES 179-180 Then, to generate new filters, it suffices to call the step() function as follows: .. GENERATED FROM PYTHON SOURCE LINES 180-183 .. code-block:: Python filters = diffraction_generator.step(batch_size=3) .. GENERATED FROM PYTHON SOURCE LINES 184-186 In this case, the `step()` function returns a dictionary containing the filters, their pupil function and Zernike coefficients: .. GENERATED FROM PYTHON SOURCE LINES 186-204 .. code-block:: Python print(filters.keys()) # Note that we use **0.2 to increase the image dynamics plot( [f for f in filters["filter"] ** 0.5], suptitle="Examples of randomly generated diffraction blurs", ) plot( [ f for f in torch.angle(filters["pupil"][:, None]) * torch.abs(filters["pupil"][:, None]) ], suptitle="Corresponding pupil phases", ) print("Coefficients of the decomposition on Zernike polynomials") print(filters["coeff"]) .. rst-class:: sphx-glr-horizontal * .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_010.png :alt: Examples of randomly generated diffraction blurs :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_010.png :class: sphx-glr-multi-img * .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_011.png :alt: Corresponding pupil phases :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_011.png :class: sphx-glr-multi-img .. rst-class:: sphx-glr-script-out .. code-block:: none dict_keys(['filter', 'coeff', 'pupil']) Coefficients of the decomposition on Zernike polynomials tensor([[-0.0349, 0.0559, -0.0247, -0.0145, 0.0431, -0.0064, -0.0642, 0.0707], [ 0.0322, -0.0109, -0.0623, -0.0014, -0.0501, -0.0519, -0.0204, 0.0022], [ 0.0318, 0.0568, 0.0515, 0.0732, 0.0089, -0.0096, -0.0559, -0.0335]]) .. GENERATED FROM PYTHON SOURCE LINES 205-206 We can change the cutoff frequency (below 1/4 to respect Shannon's sampling theorem) .. GENERATED FROM PYTHON SOURCE LINES 206-215 .. code-block:: Python diffraction_generator = DiffractionBlurGenerator( (psf_size, psf_size), fc=1 / 8, device=device, dtype=dtype ) filters = diffraction_generator.step(batch_size=3) plot( [f for f in filters["filter"] ** 0.5], suptitle="A different cutoff frequency", ) .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_012.png :alt: A different cutoff frequency :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_012.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 216-218 It is also possible to directly specify the Zernike decomposition. For instance, if the pupil is null, the PSF is the Airy pattern .. GENERATED FROM PYTHON SOURCE LINES 218-227 .. code-block:: Python n_zernike = len( diffraction_generator.list_param ) # number of Zernike coefficients in the decomposition filters = diffraction_generator.step(coeff=torch.zeros(3, n_zernike)) plot( [f for f in filters["filter"][:, None] ** 0.3], suptitle="Airy pattern", ) .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_013.png :alt: Airy pattern :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_013.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 228-230 Finally, notice that you can activate the aberrations you want in the ANSI nomenclature https://en.wikipedia.org/wiki/Zernike_polynomials#OSA/ANSI_standard_indices .. GENERATED FROM PYTHON SOURCE LINES 230-239 .. code-block:: Python diffraction_generator = DiffractionBlurGenerator( (psf_size, psf_size), fc=1 / 8, list_param=["Z5", "Z6"], device=device, dtype=dtype ) filters = diffraction_generator.step(batch_size=3) plot( [f for f in filters["filter"] ** 0.5], suptitle="PSF obtained with astigmatism only", ) .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_014.png :alt: PSF obtained with astigmatism only :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_014.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 240-245 Generator Mixture ~~~~~~~~~~~~~~~~~ During training, it's more robust to train on multiple family of operators. This can be done seamlessly with the :class:`deepinv.physics.generator.GeneratorMixture`. .. GENERATED FROM PYTHON SOURCE LINES 245-261 .. code-block:: Python from deepinv.physics.generator import GeneratorMixture torch.cuda.manual_seed(4) torch.manual_seed(6) generator = GeneratorMixture( ([motion_generator, diffraction_generator]), probs=[0.5, 0.5] ) for i in range(4): filters = generator.step(batch_size=3) plot( [f for f in filters["filter"]], suptitle=f"Random PSF generated at step {i + 1}", ) .. rst-class:: sphx-glr-horizontal * .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_015.png :alt: Random PSF generated at step 1 :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_015.png :class: sphx-glr-multi-img * .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_016.png :alt: Random PSF generated at step 2 :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_016.png :class: sphx-glr-multi-img * .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_017.png :alt: Random PSF generated at step 3 :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_017.png :class: sphx-glr-multi-img * .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_018.png :alt: Random PSF generated at step 4 :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_018.png :class: sphx-glr-multi-img .. GENERATED FROM PYTHON SOURCE LINES 262-266 Space varying blurs -------------------- Space varying blurs are also available using :class:`deepinv.physics.SpaceVaryingBlur` .. GENERATED FROM PYTHON SOURCE LINES 266-301 .. code-block:: Python from deepinv.physics.generator import ( DiffractionBlurGenerator, ProductConvolutionBlurGenerator, ) from deepinv.physics.blur import SpaceVaryingBlur psf_size = 32 img_size = (256, 256) n_eigenpsf = 10 spacing = (64, 64) padding = "valid" batch_size = 1 delta = 16 # We first instantiate a psf generator psf_generator = DiffractionBlurGenerator( (psf_size, psf_size), device=device, dtype=dtype ) # Now, scattered random psfs are synthesized and interpolated spatially pc_generator = ProductConvolutionBlurGenerator( psf_generator=psf_generator, img_size=img_size, n_eigen_psf=n_eigenpsf, spacing=spacing, padding=padding, ) params_pc = pc_generator.step(batch_size) physics = SpaceVaryingBlur(**params_pc) dirac_comb = torch.zeros(img_size)[None, None] dirac_comb[0, 0, ::delta, ::delta] = 1 psf_grid = physics(dirac_comb) plot(psf_grid, titles="Space varying impulse responses") .. image-sg:: /auto_examples/physics/images/sphx_glr_demo_blur_tour_019.png :alt: Space varying impulse responses :srcset: /auto_examples/physics/images/sphx_glr_demo_blur_tour_019.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 2.050 seconds) .. _sphx_glr_download_auto_examples_physics_demo_blur_tour.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: demo_blur_tour.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: demo_blur_tour.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: demo_blur_tour.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_