.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/basics/demo_blur_tour.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_basics_demo_blur_tour.py: A tour of blur operators =================================================== This example provides a tour of 2D blur operators in DeepInv. In particular, we show how to use DiffractionBlurs (Fresnel diffraction), motion blurs and space varying blurs. .. GENERATED FROM PYTHON SOURCE LINES 9-17 .. code-block:: Python import torch import deepinv as dinv from deepinv.utils.plotting import plot from deepinv.utils.demo import load_url_image, get_image_url .. GENERATED FROM PYTHON SOURCE LINES 18-21 ---------------- First, let's load some test images. .. GENERATED FROM PYTHON SOURCE LINES 21-40 .. code-block:: Python dtype = torch.float32 device = "cpu" img_size = (173, 125) url = get_image_url("CBSD_0010.png") x_rgb = load_url_image( url, grayscale=False, device=device, dtype=dtype, img_size=img_size ) url = get_image_url("barbara.jpeg") x_gray = load_url_image( url, grayscale=True, device=device, dtype=dtype, img_size=img_size ) # Next, set the global random seed from pytorch to ensure reproducibility of the example. torch.manual_seed(0) torch.cuda.manual_seed(0) .. GENERATED FROM PYTHON SOURCE LINES 41-49 We are now ready to explore the different blur operators. Convolution Basics ------------------ The class :class:`deepinv.physics.Blur` implements convolution operations with kernels. For instance, here is the convolution of a grayscale image with a grayscale filter: .. GENERATED FROM PYTHON SOURCE LINES 49-58 .. code-block:: Python filter_0 = dinv.physics.blur.gaussian_blur(sigma=(2, 0.1), angle=0.0) physics = dinv.physics.Blur(filter_0, device=device) y = physics(x_gray) plot( [x_gray, filter_0, y], titles=["signal", "filter", "measurement"], suptitle="Grayscale convolution", ) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_001.png :alt: Grayscale convolution, signal, filter, measurement :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 59-61 When a single channel filter is used, all channels are convolved with the same filter: .. GENERATED FROM PYTHON SOURCE LINES 61-70 .. code-block:: Python physics = dinv.physics.Blur(filter_0, device=device) y = physics(x_rgb) plot( [x_rgb, filter_0, y], titles=["signal", "filter", "measurement"], suptitle="RGB image + grayscale filter convolution", ) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_002.png :alt: RGB image + grayscale filter convolution, signal, filter, measurement :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 71-73 By default, the boundary conditions are ``'valid'``, but other options among (``'circular'``, ``'reflect'``, ``'replicate'``) are possible: .. GENERATED FROM PYTHON SOURCE LINES 73-82 .. code-block:: Python physics = dinv.physics.Blur(filter_0, padding="reflect", device=device) y = physics(x_rgb) plot( [x_rgb, filter_0, y], titles=["signal", "filter", "measurement"], suptitle="Reflection boundary conditions", ) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_003.png :alt: Reflection boundary conditions, signal, filter, measurement :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_003.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 83-86 For circular boundary conditions, an FFT implementation is also available. It is slower that :class:`deepinv.physics.Blur`, but inherits from :class:`deepinv.physics.DecomposablePhysics`, so that the pseudo-inverse and regularized inverse are computed faster and more accurately. .. GENERATED FROM PYTHON SOURCE LINES 86-94 .. code-block:: Python physics = dinv.physics.BlurFFT(img_size=x_rgb[0].shape, filter=filter_0, device=device) y = physics(x_rgb) plot( [x_rgb, filter_0, y], titles=["signal", "filter", "measurement"], suptitle="FFT convolution with circular boundary conditions", ) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_004.png :alt: FFT convolution with circular boundary conditions, signal, filter, measurement :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_004.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 95-96 One can also change the blur filter in the forward pass as follows: .. GENERATED FROM PYTHON SOURCE LINES 96-106 .. code-block:: Python filter_90 = dinv.physics.blur.gaussian_blur(sigma=(2, 0.1), angle=90.0).to( device=device, dtype=dtype ) y = physics(x_rgb, filter=filter_90) plot( [x_rgb, filter_90, y], titles=["signal", "filter", "measurement"], suptitle="Changing the filter on the fly", ) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_005.png :alt: Changing the filter on the fly, signal, filter, measurement :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_005.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 107-108 When applied to a new image, the last filter is used: .. GENERATED FROM PYTHON SOURCE LINES 108-115 .. code-block:: Python y = physics(x_gray, filter=filter_90) plot( [x_gray, filter_90, y], titles=["signal", "filter", "measurement"], suptitle="Effect of on the fly change is persistent", ) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_006.png :alt: Effect of on the fly change is persistent, signal, filter, measurement :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_006.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 116-117 We can also define color filters. In that situation, each channel is convolved with the corresponding channel of the filter: .. GENERATED FROM PYTHON SOURCE LINES 117-131 .. code-block:: Python psf_size = 9 filter_rgb = torch.zeros((1, 3, psf_size, psf_size), device=device, dtype=dtype) filter_rgb[:, 0, :, psf_size // 2 : psf_size // 2 + 1] = 1.0 / psf_size filter_rgb[:, 1, psf_size // 2 : psf_size // 2 + 1, :] = 1.0 / psf_size filter_rgb[:, 2, ...] = ( torch.diag(torch.ones(psf_size, device=device, dtype=dtype)) / psf_size ) y = physics(x_rgb, filter=filter_rgb) plot( [x_rgb, filter_rgb, y], titles=["signal", "Colour filter", "measurement"], suptitle="Color image + color filter convolution", ) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_007.png :alt: Color image + color filter convolution, signal, Colour filter, measurement :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_007.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 132-136 Blur generators ---------------------- More advanced kernel generation methods are provided with the toolbox thanks to the :class:`deepinv.physics.generator.PSFGenerator`. In particular, motion blurs generators are implemented. .. GENERATED FROM PYTHON SOURCE LINES 138-140 Motion blur generators ~~~~~~~~~~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 140-142 .. code-block:: Python from deepinv.physics.generator import MotionBlurGenerator .. GENERATED FROM PYTHON SOURCE LINES 143-147 In order to generate motion blur kernels, we just need to instantiate a generator with specific the psf size. In turn, motion blurs can be generated on the fly by calling the ``step()`` method. Let's illustrate this now and generate 3 motion blurs. First, we instantiate the generator: .. GENERATED FROM PYTHON SOURCE LINES 147-149 .. code-block:: Python psf_size = 31 motion_generator = MotionBlurGenerator((psf_size, psf_size), device=device, dtype=dtype) .. GENERATED FROM PYTHON SOURCE LINES 150-151 To generate new filters, we call the step() function: .. GENERATED FROM PYTHON SOURCE LINES 151-159 .. code-block:: Python filters = motion_generator.step(batch_size=3) # the `step()` function returns a dictionary: print(filters.keys()) plot( [f for f in filters["filter"]], suptitle="Examples of randomly generated motion blurs", ) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_008.png :alt: Examples of randomly generated motion blurs :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_008.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none dict_keys(['filter']) .. GENERATED FROM PYTHON SOURCE LINES 160-161 Other options, such as the regularity and length of the blur trajectory can also be specified: .. GENERATED FROM PYTHON SOURCE LINES 161-167 .. code-block:: Python motion_generator = MotionBlurGenerator( (psf_size, psf_size), l=0.6, sigma=1, device=device, dtype=dtype ) filters = motion_generator.step(batch_size=3) plot([f for f in filters["filter"]], suptitle="Different length and regularity") .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_009.png :alt: Different length and regularity :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_009.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 168-173 Diffraction blur generators ~~~~~~~~~~~~~~~~~~~~~~~~~~~ We also implemented diffraction blurs obtained through Fresnel theory and definition of the psf through the pupil plane expanded in Zernike polynomials .. GENERATED FROM PYTHON SOURCE LINES 173-180 .. code-block:: Python from deepinv.physics.generator import DiffractionBlurGenerator diffraction_generator = DiffractionBlurGenerator( (psf_size, psf_size), device=device, dtype=dtype ) .. GENERATED FROM PYTHON SOURCE LINES 181-182 Then, to generate new filters, it suffices to call the step() function as follows: .. GENERATED FROM PYTHON SOURCE LINES 182-185 .. code-block:: Python filters = diffraction_generator.step(batch_size=3) .. GENERATED FROM PYTHON SOURCE LINES 186-188 In this case, the `step()` function returns a dictionary containing the filters, their pupil function and Zernike coefficients: .. GENERATED FROM PYTHON SOURCE LINES 188-206 .. code-block:: Python print(filters.keys()) # Note that we use **0.2 to increase the image dynamics plot( [f for f in filters["filter"] ** 0.5], suptitle="Examples of randomly generated diffraction blurs", ) plot( [ f for f in torch.angle(filters["pupil"][:, None]) * torch.abs(filters["pupil"][:, None]) ], suptitle="Corresponding pupil phases", ) print("Coefficients of the decomposition on Zernike polynomials") print(filters["coeff"]) .. rst-class:: sphx-glr-horizontal * .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_010.png :alt: Examples of randomly generated diffraction blurs :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_010.png :class: sphx-glr-multi-img * .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_011.png :alt: Corresponding pupil phases :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_011.png :class: sphx-glr-multi-img .. rst-class:: sphx-glr-script-out .. code-block:: none dict_keys(['filter', 'coeff', 'pupil']) Coefficients of the decomposition on Zernike polynomials tensor([[-0.0349, 0.0559, -0.0247, -0.0145, 0.0431, -0.0064, -0.0642, 0.0707], [ 0.0322, -0.0109, -0.0623, -0.0014, -0.0501, -0.0519, -0.0204, 0.0022], [ 0.0318, 0.0568, 0.0515, 0.0732, 0.0089, -0.0096, -0.0559, -0.0335]]) .. GENERATED FROM PYTHON SOURCE LINES 207-208 We can change the cutoff frequency (below 1/4 to respect Shannon's sampling theorem) .. GENERATED FROM PYTHON SOURCE LINES 208-217 .. code-block:: Python diffraction_generator = DiffractionBlurGenerator( (psf_size, psf_size), fc=1 / 8, device=device, dtype=dtype ) filters = diffraction_generator.step(batch_size=3) plot( [f for f in filters["filter"] ** 0.5], suptitle="A different cutoff frequency", ) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_012.png :alt: A different cutoff frequency :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_012.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 218-220 It is also possible to directly specify the Zernike decomposition. For instance, if the pupil is null, the PSF is the Airy pattern .. GENERATED FROM PYTHON SOURCE LINES 220-229 .. code-block:: Python n_zernike = len( diffraction_generator.list_param ) # number of Zernike coefficients in the decomposition filters = diffraction_generator.step(coeff=torch.zeros(3, n_zernike)) plot( [f for f in filters["filter"][:, None] ** 0.3], suptitle="Airy pattern", ) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_013.png :alt: Airy pattern :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_013.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 230-232 Finally, notice that you can activate the aberrations you want in the ANSI nomenclature https://en.wikipedia.org/wiki/Zernike_polynomials#OSA/ANSI_standard_indices .. GENERATED FROM PYTHON SOURCE LINES 232-241 .. code-block:: Python diffraction_generator = DiffractionBlurGenerator( (psf_size, psf_size), fc=1 / 8, list_param=["Z5", "Z6"], device=device, dtype=dtype ) filters = diffraction_generator.step(batch_size=3) plot( [f for f in filters["filter"] ** 0.5], suptitle="PSF obtained with astigmatism only", ) .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_014.png :alt: PSF obtained with astigmatism only :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_014.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 242-247 Generator Mixture ~~~~~~~~~~~~~~~~~ During training, it's more robust to train on multiple family of operators. This can be done seamlessly with the :class:`deepinv.physics.generator.GeneratorMixture`. .. GENERATED FROM PYTHON SOURCE LINES 247-263 .. code-block:: Python from deepinv.physics.generator import GeneratorMixture torch.cuda.manual_seed(4) torch.manual_seed(6) generator = GeneratorMixture( ([motion_generator, diffraction_generator]), probs=[0.5, 0.5] ) for i in range(4): filters = generator.step(batch_size=3) plot( [f for f in filters["filter"]], suptitle=f"Random PSF generated at step {i + 1}", ) .. rst-class:: sphx-glr-horizontal * .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_015.png :alt: Random PSF generated at step 1 :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_015.png :class: sphx-glr-multi-img * .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_016.png :alt: Random PSF generated at step 2 :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_016.png :class: sphx-glr-multi-img * .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_017.png :alt: Random PSF generated at step 3 :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_017.png :class: sphx-glr-multi-img * .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_018.png :alt: Random PSF generated at step 4 :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_018.png :class: sphx-glr-multi-img .. GENERATED FROM PYTHON SOURCE LINES 264-268 Space varying blurs -------------------- Space varying blurs are also available using :class:`deepinv.physics.SpaceVaryingBlur` .. GENERATED FROM PYTHON SOURCE LINES 268-303 .. code-block:: Python from deepinv.physics.generator import ( DiffractionBlurGenerator, ProductConvolutionBlurGenerator, ) from deepinv.physics.blur import SpaceVaryingBlur psf_size = 32 img_size = (256, 256) n_eigenpsf = 10 spacing = (64, 64) padding = "valid" batch_size = 1 delta = 16 # We first instantiate a psf generator psf_generator = DiffractionBlurGenerator( (psf_size, psf_size), device=device, dtype=dtype ) # Now, scattered random psfs are synthesized and interpolated spatially pc_generator = ProductConvolutionBlurGenerator( psf_generator=psf_generator, img_size=img_size, n_eigen_psf=n_eigenpsf, spacing=spacing, padding=padding, ) params_pc = pc_generator.step(batch_size) physics = SpaceVaryingBlur(method="product_convolution2d", **params_pc) dirac_comb = torch.zeros(img_size)[None, None] dirac_comb[0, 0, ::delta, ::delta] = 1 psf_grid = physics(dirac_comb) plot(psf_grid, titles="Space varying impulse responses") .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_blur_tour_019.png :alt: Space varying impulse responses :srcset: /auto_examples/basics/images/sphx_glr_demo_blur_tour_019.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 1.499 seconds) .. _sphx_glr_download_auto_examples_basics_demo_blur_tour.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: demo_blur_tour.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: demo_blur_tour.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: demo_blur_tour.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_