.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/basics/demo_dataset.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_basics_demo_dataset.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_basics_demo_dataset.py:


Creating your own dataset
====================================================================================================

This example shows how to create your own dataset for deep image inverse problems from a base dataset
of images. Here we use Set3C as a base dataset of natural images. This base dataset contains 3 images.

.. GENERATED FROM PYTHON SOURCE LINES 9-17

.. code-block:: Python


    import deepinv as dinv
    from pathlib import Path
    import torch
    from torch.utils.data import DataLoader
    from torchvision import transforms
    from deepinv.utils.demo import load_dataset








.. GENERATED FROM PYTHON SOURCE LINES 18-21

Setup paths for data loading and results.
----------------------------------------------------------------------------------------


.. GENERATED FROM PYTHON SOURCE LINES 21-26

.. code-block:: Python


    BASE_DIR = Path(".")
    DATA_DIR = BASE_DIR / "measurements"









.. GENERATED FROM PYTHON SOURCE LINES 27-33

Load base image datasets
----------------------------------------------------------------------------------------

We download the Set3 dataset which is a :class:`torchvision.datasets.ImageFolder` dataset.
You can use any other dataset as long as it is a :class:`torch.utils.data.Dataset`.


.. GENERATED FROM PYTHON SOURCE LINES 33-55

.. code-block:: Python


    # Set the global random seed from pytorch to ensure reproducibility of the example.
    torch.manual_seed(0)

    device = dinv.utils.get_freer_gpu() if torch.cuda.is_available() else "cpu"

    # Set up the variable to fetch dataset and operators.
    dataset_name = "set3c"

    img_size = 256 if torch.cuda.is_available() else 32

    val_transform = transforms.Compose(
        [transforms.CenterCrop(img_size), transforms.ToTensor()]
    )

    # add batch and channel dimensions
    dataset = load_dataset(dataset_name, transform=val_transform)

    # display an image from the base dataset
    dinv.utils.plot(dataset[0][0].unsqueeze(0))





.. image-sg:: /auto_examples/basics/images/sphx_glr_demo_dataset_001.png
   :alt: demo dataset
   :srcset: /auto_examples/basics/images/sphx_glr_demo_dataset_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Downloading datasets/set3c.zip

      0%|          | 0.00/385k [00:00<?, ?iB/s]
    100%|██████████| 385k/385k [00:00<00:00, 47.6MiB/s]
    set3c dataset downloaded in datasets




.. GENERATED FROM PYTHON SOURCE LINES 56-63

Generate a dataset of degraded images and load it.
--------------------------------------------------------------------------------
We use a simple denoising forward operator with Gaussian noise.

.. note::
     :func:`deepinv.datasets.generate_dataset` will ignore other attributes than the image,
     e.g. the class labels if there are any.

.. GENERATED FROM PYTHON SOURCE LINES 63-89

.. code-block:: Python



    n_channels = 3  # 3 for color images, 1 for gray-scale images
    physics = dinv.physics.Denoising(dinv.physics.GaussianNoise(0.2))

    # Use parallel dataloader if using a GPU to fasten training,
    # otherwise, as all computes are on CPU, use synchronous data loading.
    num_workers = 4 if torch.cuda.is_available() else 0

    operation = "denoise"
    measurement_dir = DATA_DIR / dataset_name / operation
    dinv_dataset_path = dinv.datasets.generate_dataset(
        train_dataset=dataset,
        test_dataset=None,
        physics=physics,
        device=device,
        save_dir=measurement_dir,
        num_workers=num_workers,
    )

    dataset = dinv.datasets.HDF5Dataset(path=dinv_dataset_path, train=True)

    # display an image from the base dataset
    x, y = dataset[0]
    dinv.utils.plot([x.unsqueeze(0), y.unsqueeze(0)])




.. image-sg:: /auto_examples/basics/images/sphx_glr_demo_dataset_002.png
   :alt: demo dataset
   :srcset: /auto_examples/basics/images/sphx_glr_demo_dataset_002.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Dataset has been saved at measurements/set3c/denoise/dinv_dataset0.h5




.. GENERATED FROM PYTHON SOURCE LINES 90-99

Create a dataloader
--------------------------------------------------------------------

We iterate over the dataset using a dataloader, which will return a batches of pairs of signals and measurements.

.. note::

    You can adapt this code to build your custom train function, in case :func:`deepinv.train`
    doesn't meet your needs.

.. GENERATED FROM PYTHON SOURCE LINES 99-107

.. code-block:: Python


    batch_size = 2
    dataloader = DataLoader(
        dataset, batch_size=batch_size, num_workers=num_workers, shuffle=False
    )

    for x, y in dataloader:
        dinv.utils.plot([x, y])



.. rst-class:: sphx-glr-horizontal


    *

      .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_dataset_003.png
         :alt: demo dataset
         :srcset: /auto_examples/basics/images/sphx_glr_demo_dataset_003.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/basics/images/sphx_glr_demo_dataset_004.png
         :alt: demo dataset
         :srcset: /auto_examples/basics/images/sphx_glr_demo_dataset_004.png
         :class: sphx-glr-multi-img






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.786 seconds)


.. _sphx_glr_download_auto_examples_basics_demo_dataset.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: demo_dataset.ipynb <demo_dataset.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: demo_dataset.py <demo_dataset.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: demo_dataset.zip <demo_dataset.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_