.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/sampling/demo_diffusers.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note New to DeepInverse? Get started with the basics with the :ref:`5 minute quickstart tutorial `. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_sampling_demo_diffusers.py: Using state-of-the-art diffusion models from HuggingFace Diffusers with DeepInverse ====================================================================================== This demo shows you how to use our wrapper :class:`deepinv.models.DiffusersDenoiserWrapper` to turn any SOTA models from the HuggingFace Hub to an image denoiser. It also can be used to perform unconditional image generation or for posterior sampling. See more about the `diffusers pipeline `_ and our posterior sampling `user guide `_. .. note:: This example requires the `diffusers` and `transformers` package. You can install it via `pip install diffusers transformers`. .. GENERATED FROM PYTHON SOURCE LINES 15-31 .. code-block:: Python import torch import deepinv as dinv from deepinv.models.wrapper import DiffusersDenoiserWrapper device = "cuda" if torch.cuda.is_available() else "cpu" dtype = torch.float32 figsize = 2.5 from deepinv.sampling import ( PosteriorDiffusion, EulerSolver, VarianceExplodingDiffusion, VariancePreservingDiffusion, ) from deepinv.optim import ZeroFidelity .. GENERATED FROM PYTHON SOURCE LINES 32-36 ---------------------------------------------------- Let us first load a pretrained diffusion model from the HuggingFace Hub. Here, we use the `google/ddpm-ema-celebahq-256` model. This model is trained on 256x256 of CelebA dataset using the DDPM scheduler. .. GENERATED FROM PYTHON SOURCE LINES 36-61 .. code-block:: Python # We can wrap any diffusers model as a DeepInv denoiser using one line of code: denoiser = DiffusersDenoiserWrapper( mode_id="google/ddpm-ema-celebahq-256", device=device ) # Load an example image x = dinv.utils.load_example( "celeba_example2.jpg", img_size=256, resize_mode="resize", ).to(device) # Add noise and test the denoiser sigma = 0.1 x_noisy = x + sigma * torch.randn_like(x) with torch.no_grad(): x_denoised = denoiser(x_noisy, sigma=sigma) dinv.utils.plot( [x, x_noisy, x_denoised], figsize=(figsize * 3, figsize), titles=["Original image", "Noisy image", "Denoised image"], ) .. image-sg:: /auto_examples/sampling/images/sphx_glr_demo_diffusers_001.png :alt: Original image, Noisy image, Denoised image :srcset: /auto_examples/sampling/images/sphx_glr_demo_diffusers_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none Fetching 4 files: 0%| | 0/4 [00:00` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: demo_diffusers.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: demo_diffusers.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_