Skip to content
Snippets Groups Projects
Select Git revision
  • f5cd9b7cda0a780da45f1faefe270e8ac172df35
  • main default protected
2 results

mupix

Fabrice Daian's avatar
Fabrice Daian authored
f5cd9b7c
History

Code, Models & Data Repository

μPIX : Leveraging Generative AI for Enhanced, Personalized and Sustainable Microscopy

Gabriel Bon, Daniel Sapede, Cédric Matthews, and Fabrice Daian

Preprint BioRXiv DOI:10.1101/2024.10.25.620201

Dataset & pre-trained µPIX Models

Dataset pre-trained µPIX Models
csbdeep download download
rejuvenation download download
metrology download download

Demo Notebooks on Google Colab

  • Use Case n°1: Use a pre-trained µPIX model to denoise an image dataset (metrology dataset) Open In Colab
  • Use Case n°2: Train a µPIX model from scratch using a custom dataset (metrology dataset) Open In Colab

Reproductibility

  • Code supporting Tables and Figures: Open In Colab

Install µPIX Environment and Sources (Tutorial VIDEO)

Click to expand
Requirements
Python 3.11
Tensorflow 2.14
CUDA 12.2
OS Linux

If you don't have already miniconda installed on your system, please refere to the officical miniconda documentation and then :

Create a miniconda environment and source it:

conda create -n mupix_env python=3.11
source ~/.bashrc

Finally activate the environment:

conda activate mupix_env

Clone this repository to download µPIX sources :

git clone https://gitlab.lis-lab.fr/sicomp/mupix

and then proceed to the installation of required Python packages:

pip install -r mupix/requirements.txt

Use Case n°1 - Use a pre-trained µPIX model to denoise an image dataset (Tutorial VIDEO)

Click to expand

µPIX comes with 3 ptre-trained models along with their respective datasets:

Dataset pre-trained µPIX Models
csbdeep download download
gastruloids download download
metrology download download

For this example, we will use a pre-trained µPIX model on the metrology dataset and we will use it to denoise the metrology test dataset.

First, move to the mupix directory:

cd mupix

1 - Download and unzip the pre-trained µPIX metrology model

To download the metrology model, type in your Terminal:

curl -o metrology_model.zip https://sync.lis-lab.fr/index.php/s/degZsCxN7ZXxeB6/download -q
unzip -qq metrology_model.zip -d metrology_model

The model is now stored inside ./metrology_model folder.

2 - Download and unzip the metrology dataset

To download the metrology dataset, type in your Terminal:

curl -o metrology.zip https://sync.lis-lab.fr/index.php/s/mYDRTeAQxMxNPPJ/download -q

and

unzip -qq metrology.zip

This will create a metrology directory containing two subdirectories: train and test.

  • train (Training data) directory contains two folders:
    • GT folder contains clean images (Ground Truth)
    • low folder contains the corresponding noisy images
  • test (Testing data) directory contains all the images that will be used for testing/inference the model.

For this example, we will only use the test images.

3 - Create a new µPIX experiment

The new_experiment.py script allows you to set up a new experiment by specifying the required paths and parameters to the dataset.

Usage : python new_experiment.py --experiment_name EXPERIMENT_NAME --experiment_path EXPERIMENT_PATH --clean_data_path CLEAN_DATA_PATH --noisy_data_path NOISY_DATA_PATH [--test_data_path TEST_DATA_PATH]

Argument Description
--experiment_name Name of the µPIX experiment.
--experiment_path Path where the µPIX experiment will be saved.
--clean_data_path Path to the clean dataset (Train).
--noisy_data_path Path to the noisy dataset (Train).
--test_data_path (optional) Path to the test dataset (if available).

In our case, as we want just to use the pre-trained µPIX metrology model in inference (no training), we don't need to provide the clean_data_path and noisy_data_path parameters so we left it empty. Nevertheless, we have to indicate the path that contains the test data image we want to denoise using µPIX using the test_data_path parameter, and the experiment_name we want to create as well as the location where the experiment will be stored using the experiment_path parameter.

Type the following to create experiment:

python new_experiment.py --experiment_name "metrology_experiment" --experiment_path "./experiments" --clean_data_path "" --noisy_data_path "" --test_data_path "./metrology/test"

If everything works correctly you should see:

Experiment 'metrology_experiment' created successfully at ./experiments/metrology_experiment/

4 - Transfer the metrology model to the experiment path

Before starting the denoising process, we need to transfer the model weights to the experiment paths:

 mv metrology_model/metrology/Generator/ experiments/metrology_experiment/results/networks/

4 - Denoising

The mupixinfer.py script allows you to use a pre-trained µPIX model to denoise a dataset located in the test directory inside the experiment path.

Usage : python mupixinfer.py --experiment_path EXPERIMENT_PATH

For our example, here is the command you need to type to start the inference using the pre-trained µPIX model for the metrology test dataset:

python mupixinfer.py --experiment_path "./experiments/metrology_experiment/"

Once finished, the denoised images are stored inside ./experiments/metrology_experiment/predictions/ directory.

Use Case n°2 - Train a µPIX model from scratch using a custom dataset

Click to expand

Move to the mupix directory:

cd mupix

As an example, we will train a µPIX model from scratch using the metrology dataset.

2.1 - Download the dataset

First download the dataset manually (here), and unzip it.

Alternatively, if you have curl installed on your system, you can download and extract the dataset using:

curl -o metrology.zip https://sync.lis-lab.fr/index.php/s/mYDRTeAQxMxNPPJ/download -q

and

unzip -qq metrology.zip

This will create a metrology directory containing two subdirectories: train and test.

  • train (Training data) directory contains two folders:
    • GT folder contains clean images (Ground Truth)
    • low folder contains the corresponding noisy images

Important: Each image in the GT/ folder must have a corresponding noisy image in low/ with the same filename, both in TIF format.

Example: if it exist an image called image0.tif in the GT folder, it must have the corresponding image0.tif noisy image in the low folder.

  • test (Testing data) directory contains all the images that will be used for testing/inference the model.

2.2 - Create a µPIX experiment

The new_experiment.py script allows you to set up a new experiment by specifying the required paths and parameters to the dataset.

Usage

python new_experiment.py --experiment_name EXPERIMENT_NAME 
                         --experiment_path EXPERIMENT_PATH
                         --clean_data_path CLEAN_DATA_PATH
                         --noisy_data_path NOISY_DATA_PATH 
                         [--test_data_path TEST_DATA_PATH]
Argument Description
--experiment_name Name of the µPIX experiment.
--experiment_path Path where the µPIX experiment will be saved.
--clean_data_path Path to the clean dataset (Train).
--noisy_data_path Path to the noisy dataset (Train).
--test_data_path (optional) Path to the test dataset (if available).

For our example, here is the command you need to type to create a µPIX experiment for the metrology dataset:

python new_experiment.py --experiment_name "metrology_experiment" --experiment_path "./experiments" --clean_data_path "./metrology/train/GT" --noisy_data_path "./metrology/train/low" --test_data_path "./metrology/test"

If everything works correctly you should see:

Experiment 'metrology_experiment' created successfully at ./experiments/metrology_experiment/

This command has created a new experiments/metrology_experiment directory containing:

  • hyperparameters.json: default parameters used by µPIX
  • results directory: will contains training logs you can monitor during the training (logs.txt), model checkpoints (networks), image generated during the training (images) of a the µPIX model.
  • predictions directory: will contains the generated image on the test dataset.

For information, here is the hyperparameters.json file created by default by µPIX for our example experiment that could be freely modifiable

{
    "learning_rate_generator": 0.0001,
    "learning_rate_discriminator": 0.0001,
    "batch_size": 16,
    "num_epochs": 100,
    "loss_weight": 10,
    "tile_size": 256,
    "patience": 20,
    "valid_size": 0.1,
    "seed": 42,
    "data_paths": {
        "clean": "./data/metrology/GT/",
        "noisy": "./data/metrology/low",
        "test": null
    }
}
Hyperparameter Type Description
learning_rate_generator float Learning rate for the generator.
learning_rate_discriminator float Learning rate for the discriminator.
batch_size int Number of samples per batch.
num_epochs int Total number of training epochs.
loss_weight int Weight factor for µPIX loss calculation.
tile_size int Size of image tiles used for training.
patience int Number of epochs to wait before triggering µPIX early stopping if no improvement.
valid_size float Proportion of the dataset used for validation.
seed int Random seed for reproducibility.
data_paths.clean str Path to the clean dataset.
data_paths.noisy str Path to the noisy dataset.
data_paths.test str or null Path to the test dataset (if available).

2.3 - Training

The mupixtraining.py script allows you to train a µPIX model by specifying the required path to an existing experiment.

Usage

python mupixtraining.py --experiment_path EXPERIMENT_PATH [--retrain]

Details

Argument Description
--experiment_path Path to the previously created experiment.
--retrain (optional) Use this flag to continue training an existing µPIX model located inside the experiment path.

For our example, here is the command you need to type to start the training of a µPIX model for the metrology dataset:

python mupixtraining.py --experiment_path "./experiments/metrology_experiment"

You should see the training starting:

Segmentation Models: using `tf.keras` framework.
Input Path: ('./dataset_metrology/train/low', './dataset_metrology/train/GT')
Output Path: ./experiments/metrology/results
Retraining: False
Data loading...
85 images loaded
2212 tiles extracted for training
303 tiles extracted for validation
Data loaded !
Training from scratch...
2025-02-27 12:27:50.608620: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:894] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2025-02-27 12:27:50.608906: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2211] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
/home/fabrice/miniconda3/envs/mupix_env/lib/python3.11/site-packages/keras/src/initializers/initializers.py:120: UserWarning: The initializer RandomNormal is unseeded and being called multiple times, which will return identical values each time (even if the initializer is unseeded). Please update your code to provide a seed to the initializer, or avoid using the same initializer instance more than once.
  warnings.warn(
3645.2395 4581.112
10/10 [==============================] - 6s 509ms/step
Epoch 1
Val> mse[3.827e+08], ssim[-1.143e-04]
1/1 [==============================] - 0s 399ms/step
Step>1, Generator loss : 1.265e+05
1/1 [==============================] - 0s 330ms/step
Step>2, Generator loss : 1.883e+05
1/1 [==============================] - 0s 354ms/step
Step>3, Generator loss : 1.941e+05
1/1 [==============================] - 0s 363ms/step
...
...
...

By default, the model is trained for 100 epochs (see hyperparameters.json) but contains an EarlyStopping mechanisms (See the paper methods section pour details) governed by the patience parameters (see hyperparameters.json).

You can stop the training anytime. The best checkpoints of your model is available inside experiments/metrology_experiment/results/networks/

If the training stops because it reached the maximum number of epochs defined inside the hyperparameters.json configuration file, and you want to continue to train your model for more epochs, you can use the --retrain parameters to resume the training where it stops:

python mupixtraining.py --experiment_path "./experiments/metrology_experiment" --retrain

2.4 - Inference on the test dataset

The mupixinfer.py script allows you to use a pre-trained µPIX model to denoise a dataset located in the test directory inside the experiment path.

Usage

python mupixinfer.py --experiment_path EXPERIMENT_PATH

For our example, here is the command you need to type to start the inference using the newly trained µPIX model best (checkpoint) for the metrology test dataset:

python mupixinfer.py --experiment_path "./experiments/metrology_experiment/"

Once done you can see that the predictions has been stored inside ./experiments/metrology_experiment/predictions/ directory:

Saved weights to ./experiments/metrology_experiment/results/networks/Generator.h5
1/1 [==============================] - 7s 7s/step
Saved prediction: ./experiments/metrology_experiment/predictions/X_StageData0001.tif
All predictions saved successfully!
Data predicted at ./experiments/metrology_experiment/predictions !

Cite This Work

If you would like to cite this work, please use the following BibTeX entry:

@article{mupix_preprint_2024,
  author = {Gabriel Bon, Daniel Sapede, Cédric Matthews and Fabrice Daian},
  title = {µPIX: Leveraging Generative AI for Enhanced, Personalized and Sustainable Microscopy},
  journal = {bioRxiv},
  year = {2024},
  doi = {10.1101/2024.10.25.620201},
  url = {https://doi.org/10.1101/2024.10.25.620201}
}


## License
This code repository is release under the [CC BY-NS-SA 4.0](https://gitlab.lis-lab.fr/sicomp/mupix/-/blob/main/LICENSE?ref_type=heads)