Code & Data Repository

μPIX : Leveraging Generative AI for Enhanced, Personalized and Sustainable Microscopy

Gabriel Bon, Daniel Sapede, Cédric Matthews, and Fabrice Daian

	Paper
Preprint
Paper	To be published

Demo Notebooks
Inference on Metrology dataset using pre-trained µPIX model

Install required µPIX Environment and Sources

If you don't have alread miniconda installed on your system, please refere to the officical miniconda documentation and then :

Create a miniconda environment and source it:

conda create -n mupix_env python=3.11

source ~/.bashrc

Finally activate the environment:

conda activate mupix_env

Clone this repository to download µPIX sources :

git clone https://gitlab.lis-lab.fr/sicomp/mupix

and then proceed to the installation of required Python packages:

pip install -r mupix/requirements.txt

2 - Train a µPIX model from scratch using a custom dataset

Once the environment is installed, you can start using µPIX scripts.

Move to the mupix directory:

cd mupix

As an example, we will train a µPIX model from scratch using the Metrology dataset.

2.1 - Download/Prepare the dataset

First download the dataset manually (here), and unzip it.

Alternatively, if you have curl installed on your system, you can download and extract the dataset using:

curl -o metrology.zip https://sync.lis-lab.fr/index.php/s/mYDRTeAQxMxNPPJ/download -q

and

unzip -qq metrology.zip

This will create a metrology directory containing two subdirectories: train and test.

• train (Training data) directory contains two folders:
• • GT folder contains clean images (Ground Truth)
• • low folder contains the corresponding noisy images

Important: Each image in the GT/ folder must have a corresponding noisy image in low/ with the same filename, both in TIF format.

Example: if it exist an image called image0.tif in the GT folder, it must have the corresponding image0.tif noisy image in the low folder.

• test (Testing data) directory contains all the images that will be used for testing/inference the model.

2.2 - Create a µPIX experiment

The new_experiment.py script allows you to set up a new experiment by specifying the required paths and parameters to the dataset.

Usage

python new_experiment.py --experiment_name EXPERIMENT_NAME 
                         --experiment_path EXPERIMENT_PATH
                         --clean_data_path CLEAN_DATA_PATH
                         --noisy_data_path NOISY_DATA_PATH 
                         [--test_data_path TEST_DATA_PATH]

For our example, here is the command you need to type to create a µPIX experiment for the metrology dataset:

python new_experiment.py --experiment_name metrology_experiment 
                         --experiment_path ./experiments
                         --clean_data_path ./metrology/train/GT
                         --noisy_data_path ./metrology/train/low
                         --test_data_path ./metrology/test

If everything works correctly you should see:

Experiment 'metrology_experiment' created successfully at ./experiments/metrology_experiment/

This command has created a new experiments/metrology_experiment directory containing:

• hyperparameters.json: default parameters used by µPIX
• results directory: will contains training logs you can monitor during the training (logs.txt), model checkpoints (networks), image generated during the training (images) of a the µPIX model.
• predictions directory: will contains the generated image on the test dataset.

For information, here is the hyperparameters.json file created by default by µPIX for our example experiment that could be freely modifiable

{
    "learning_rate_generator": 0.0001,
    "learning_rate_discriminator": 0.0001,
    "batch_size": 16,
    "num_epochs": 100,
    "loss_weight": 10,
    "tile_size": 256,
    "patience": 20,
    "valid_size": 0.1,
    "seed": 42,
    "data_paths": {
        "clean": "./data/metrology/GT/",
        "noisy": "./data/metrology/low",
        "test": null
    }
}

2.3 - Training

The mupixtraining.py script allows you to train a µPIX model by specifying the required path to an existing experiment.

Usage

python mupixtraining.py --experiment_path EXPERIMENT_PATH [--retrain]

Details

For our example, here is the command you need to type to start the training of a µPIX model for the metrology dataset:

python mupixtraining.py --experiment_path ./experiments/metrology_experiment

You should see the training starting:

Segmentation Models: using `tf.keras` framework.
Input Path: ('./dataset_metrology/train/low', './dataset_metrology/train/GT')
Output Path: ./experiments/metrology/results
Retraining: False
Data loading...
85 images loaded
2212 tiles extracted for training
303 tiles extracted for validation
Data loaded !
Training from scratch...
2025-02-27 12:27:50.608620: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:894] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2025-02-27 12:27:50.608906: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2211] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
/home/fabrice/miniconda3/envs/mupix_env/lib/python3.11/site-packages/keras/src/initializers/initializers.py:120: UserWarning: The initializer RandomNormal is unseeded and being called multiple times, which will return identical values each time (even if the initializer is unseeded). Please update your code to provide a seed to the initializer, or avoid using the same initializer instance more than once.
  warnings.warn(
3645.2395 4581.112
10/10 [==============================] - 6s 509ms/step
Epoch 1
Val> mse[3.827e+08], ssim[-1.143e-04]
1/1 [==============================] - 0s 399ms/step
Step>1, Generator loss : 1.265e+05
1/1 [==============================] - 0s 330ms/step
Step>2, Generator loss : 1.883e+05
1/1 [==============================] - 0s 354ms/step
Step>3, Generator loss : 1.941e+05
1/1 [==============================] - 0s 363ms/step
...
...
...

By default, the model is trained for 100 epochs (see hyperparameters.json) but contains an EarlyStopping mechanisms (See the paper methods section pour details) governed by the patience parameters (see hyperparameters.json). The number of steps per epochs is calculated by dividing the number of training tiles divided by the batch_size.

You can stop the training anytime. The best checkpoints of your model is available inside experiments/metrology_experiment/results/networks/

2.4 - Inference on the test dataset

The mupixinfer.py script allows you to use a pre-trained µPIX model to denoise a dataset located in the test directory inside the experiment path.

Usage

python mupixinfer.py --experiment_path EXPERIMENT_PATH

For our example, here is the command you need to type to start the inference using the newly trained µPIX model best (checkpoint) for the metrology test dataset:

python mupixinfer.py --experiment_path ./experiments/metrology_experiment/

1 - Train a µPIX model from scratch

2 - Use a pre-trained µPIX model to denoise images

3 - Fine-Tune a pre-trained µPIX model on a new dataset

2 - Create a µPIX experiments

Before running µPIX for training or inference, you need to create a µPIX experiment.

The new_experiment.py script allows you to set up a new experiment by specifying the required paths and parameters.

Usage

python new_experiment.py --experiment_name EXPERIMENT_NAME --experiment_path EXPERIMENT_PATH --clean_data_path CLEAN_DATA_PATH --noisy_data_path NOISY_DATA_PATH [--test_data_path TEST_DATA_PATH]

Example

To create an experiment with the provided Metrology dataset:

python new_experiment.py --experiment_name "metrology_exp" --experiment_path "./data/metrology" --clean_data_path "./data/metrology/GT/" --noisy_data_path "./data/metrology/low"

Feel free to adapt to your own dataset keeping in minds that, as µPIX use a supervised learnign apporach, for one images inside the clean_data_path and the noisy_data_path must have the same name: example image0.tif inside the clean_data_path must corresponds to the image0.tif image inside the noisy_data_path.

This command will create a new metrology_exp experiment directory containing:

• hyperparameters.json: default parameters used by µPIX
• results directory: will contains training logs (logs.txt), model checkpoints (networks), image generated during the training (images) using mupixtraining.py.
• predictions directory: will contains the generated image on the test dataset using the script mupixinfer.py

For information, here is the hyperparameters.json file created by default by µPIX for our example experiment that could be freely modifiable

{
    "learning_rate_generator": 0.0001,
    "learning_rate_discriminator": 0.0001,
    "batch_size": 16,
    "num_epochs": 100,
    "loss_weight": 10,
    "tile_size": 256,
    "patience": 20,
    "valid_size": 0.1,
    "seed": 42,
    "data_paths": {
        "clean": "./data/metrology/GT/",
        "noisy": "./data/metrology/low",
        "test": null
    }
}

3- Train a µPIX

The mupixtraining.py script allows you to train a µPIX model by specifying the required path to an existing experiment.

Usage

python mupixtraining.py --experiment_path EXPERIMENT_PATH [--retrain]

Example

Train a new µPIX model:

python mupixtraining.py --experiment_path "./data/metrology"

Or Continue to train a µPIX model:

python mupixtraining.py --experiment_path "./data/metrology" --retrain

4 - Inference on New Images Using a Trained µPIX Model

The mupixinfer.py script allows you to use a trained µPIX model to denoise a dataset located in the test directory inside the experiment path.

Usage

python mupixinfer.py --experiment_path EXPERIMENT_PATH

Example

python mupixinfer.py --experiment_path "./data/metrology"

Runnin the code using our Docker image

You can retrieve the image from docker hub with the following command:

docker pull gabrielbon/mupix:latest

Create a new experiment

docker run -it --entrypoint bin/bash -v /path/to/your/file/mupix:/workspace gabrielbon/mupix:latest 

In case your data are located outside of the mupix folder, docker will need their location:

docker run -it --entrypoint bin/bash -v /path/to/your/file/mupix:/workspace -v /path/to/your/data:/workspace/data gabrielbon/mupix:latest 

Given you are working in the container, you will have to specify the path do data differently

3 - License

This code repository is release under the CC BY-NS-SA 4.0

b) Using the µPIX Docker container

If you don't have Docker already installed on your computer, please use our help to guide through the installation of Docker on your machine.

We have developped a "ready-to-use" Docker image (repo on dockerhub) containing a ready-to-use µPIX environment.

To get the image, open a Terminal and type:

docker pull gabrielbon/mupix:latest

Finally, you can clone this repository to install µPIX scripts and datasets:

git clone https://gitlab.lis-lab.fr/sicomp/mupix