.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/self-supervised-learning/demo_lowfieldmri.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note New to DeepInverse? Get started with the basics with the :ref:`5 minute quickstart tutorial `.. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_self-supervised-learning_demo_lowfieldmri.py: Low-field MRI denoising without ground truth ============================================ We demonstrate self-supervised (blind) denoising of a low-field MRI scan without ground truth data. In low-field MRI, images have high noise due to the fixed permanent magnet, and acquiring clean reference images is often impossible. One could average the images over multiple repetitions, but if there's any patient motion, the image would be blurry. Here, we fine-tune the Reconstruct Anything Model (:class:`deepinv.models.RAM`) :footcite:p:`terris2025reconstruct` on a single noisy scan using the self-supervised :class:`Recorrupted2Recorrupted loss ` :footcite:p:`pang2021recorrupted`. Play around with different self-supervised denoising losses (see :ref:`self-supervised-losses`) and models! Note that, if more data is available, better results can be obtained by fine-tuning on more samples. .. GENERATED FROM PYTHON SOURCE LINES 15-21 .. code-block:: Python import torch import deepinv as dinv device = dinv.utils.get_device() .. rst-class:: sphx-glr-script-out .. code-block:: none Selected GPU 0 with 4064.25 MiB free memory .. GENERATED FROM PYTHON SOURCE LINES 22-28 Load data --------- We load a single low-field T1 MRI scan (Oper 0.3T) from the M4Raw dataset :footcite:p:`lyu2023m4raw`, which contains multiple repetitions of the same scan with inter-scan motion. We use the first repetition as our noisy measurement, and average all three repetitions to create a "reference" (which is still blurry due to motion). .. GENERATED FROM PYTHON SOURCE LINES 28-62 .. code-block:: Python def open_m4raw(fname: str) -> torch.Tensor: """Load M4Raw slice""" x = dinv.io.load_ismrmd(fname, data_slice=8).unsqueeze( 0 ) # Load middle slice, shape 12NHW x = dinv.utils.MRIMixin().kspace_to_im(x) # Convert to image space x = dinv.utils.MRIMixin().rss( x, multicoil=True ) # Root-sum-square following original paper, shape 11HW x = dinv.utils.normalize_signal(x, mode="min_max") # Normalise to 0-1 return x DATA_DIR = dinv.utils.get_data_home() / "m4raw" / "motion" DATA_DIR.mkdir(parents=True, exist_ok=True) dinv.utils.download_example("demo_m4raw_inter-scan_motion_0.h5", DATA_DIR) dinv.utils.download_example("demo_m4raw_inter-scan_motion_1.h5", DATA_DIR) dinv.utils.download_example("demo_m4raw_inter-scan_motion_2.h5", DATA_DIR) y = open_m4raw(DATA_DIR / "demo_m4raw_inter-scan_motion_0.h5") x = torch.cat( [ open_m4raw(DATA_DIR / "demo_m4raw_inter-scan_motion_0.h5"), open_m4raw(DATA_DIR / "demo_m4raw_inter-scan_motion_1.h5"), open_m4raw(DATA_DIR / "demo_m4raw_inter-scan_motion_2.h5"), ] ).mean( dim=0, keepdim=True ) # Average 3 repetitions .. GENERATED FROM PYTHON SOURCE LINES 63-65 Since the data is raw, we estimate the noise level in both the single scan and the averaged scan using patch covariance. Note that the averaged scan has lower noise as expected, but observe the motion blurring! .. GENERATED FROM PYTHON SOURCE LINES 65-76 .. code-block:: Python noise_estimator = dinv.models.PatchCovarianceNoiseEstimator() dinv.utils.plot( {"Noisy scan": y, "3x rep, averaged": x}, subtitles=[ f"sigma: {noise_estimator(y).item():.4f}", f"sigma: {noise_estimator(x).item():.4f}", ], ) .. image-sg:: /auto_examples/self-supervised-learning/images/sphx_glr_demo_lowfieldmri_001.png :alt: Noisy scan, 3x rep, averaged :srcset: /auto_examples/self-supervised-learning/images/sphx_glr_demo_lowfieldmri_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 77-80 Physics ------- We define a simple denoising physics with Gaussian noise matching the estimated noise level. .. GENERATED FROM PYTHON SOURCE LINES 80-85 .. code-block:: Python with torch.no_grad(): sigma = noise_estimator(y.to(device)) physics = dinv.physics.Denoising(dinv.physics.GaussianNoise(sigma=sigma), device=device) .. GENERATED FROM PYTHON SOURCE LINES 86-89 Zero-shot reconstruction ------------------------ First, we apply the pre-trained RAM model as a zero-shot denoiser without any training. .. GENERATED FROM PYTHON SOURCE LINES 89-94 .. code-block:: Python model = dinv.models.RAM(device=device) with torch.no_grad(): x_net = model(y.to(device), physics) .. GENERATED FROM PYTHON SOURCE LINES 95-102 Self-supervised fine-tuning ---------------------------- We create a dataset from the single noisy scan and fine-tune RAM using the R2R loss. See the :ref:`dedicated R2R example ` for more. .. note:: We train for 50 epochs on GPU. For faster execution on CPU, set epochs to a smaller value. .. GENERATED FROM PYTHON SOURCE LINES 102-119 .. code-block:: Python dataset = dinv.datasets.TensorDataset(y=y) trainer = dinv.Trainer( model=model, physics=physics, train_dataloader=torch.utils.data.DataLoader(dataset, batch_size=1), optimizer=torch.optim.AdamW(model.parameters(), lr=1e-5), epochs=1 if str(device) == "cpu" else 50, losses=dinv.loss.R2RLoss(noise_model=None), metrics=None, device=device, save_path=None, ) model = trainer.train() model = model.eval() .. rst-class:: sphx-glr-script-out .. code-block:: none /local/jtachell/deepinv/deepinv/deepinv/training/trainer.py:1354: UserWarning: non_blocking_transfers=True but DataLoader.pin_memory=False; set pin_memory=True to overlap host-device copies with compute. self.setup_train() The model has 35618813 trainable parameters /local/jtachell/deepinv/deepinv/deepinv/training/trainer.py:549: UserWarning: Update progress bar frequency of 1 may slow down training on GPU. Consider setting freq_update_progress_bar > 1. warnings.warn( 0%| | 0/1 [00:00` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: demo_lowfieldmri.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: demo_lowfieldmri.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_