Skip to content
Snippets Groups Projects
Commit 34ca0e1d authored by Fabrice Daian's avatar Fabrice Daian
Browse files

modifs

parent 433a6041
No related branches found
No related tags found
No related merge requests found
......@@ -59,18 +59,119 @@ and then proceed to the installation of required Python packages:
pip install -r mupix/requirements.txt
```
## 2 - Train a µPIX model from scratch using a custom dataset
Once the environment is installed, you can start using µPIX scripts.
## Use Case n°1 - Use a pre-trained µPIX model to denoise an image dataset
µPIX comes with 3 ptre-trained models along with their respective datasets:
| | Dataset | pre-trained µPIX Models |
|--------------------------|--------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------|
| ```csbeep``` | [download](https://sync.lis-lab.fr/index.php/s/sy3SrGgqNafbP5X/download) | [download](https://sync.lis-lab.fr/index.php/s/9SxkR2QH3Z79Bbc/download) |
| ```gastruloids``` | [download](https://sync.lis-lab.fr/index.php/s/kDyLKjCt48BfNMK/download) | [download](https://sync.lis-lab.fr/index.php/s/n2oJAw4LobTCeNZ/download) |
| ```metrology``` | [download](https://sync.lis-lab.fr/index.php/s/mYDRTeAQxMxNPPJ/download) | [download](https://sync.lis-lab.fr/index.php/s/degZsCxN7ZXxeB6/download) |
For this example, we will use a pre-trained µPIX model on the ```metrology``` dataset and we will use it to denoise the ```metrology``` test dataset.
First, move to the `mupix` directory:
```bash
cd mupix
```
### 1 - Download and unzip the pre-trained µPIX ```metrology``` model
To download the ```metrology``` model, type in your Terminal:
```bash
curl -o metrology_model.zip https://sync.lis-lab.fr/index.php/s/degZsCxN7ZXxeB6/download -q
```
```bash
unzip -qq metrology_model.zip -d metrology_model
```
The model is now stored inside ```./metrology_model``` folder.
### 2 - Download and unzip the ```metrology``` dataset
To download the ```metrology``` dataset, type in your Terminal:
```bash
curl -o metrology.zip https://sync.lis-lab.fr/index.php/s/mYDRTeAQxMxNPPJ/download -q
```
and
```bash
unzip -qq metrology.zip
```
This will create a ```metrology``` directory containing two subdirectories: ```train``` and ```test```.
- ```train``` (Training data) directory contains two folders:
- - ```GT``` folder contains clean images (Ground Truth)
- - ```low``` folder contains the corresponding noisy images
- ```test``` (Testing data) directory contains all the images that will be used for testing/inference the model.
For this example, we will only use the ```test``` images.
### 3 - Create a new µPIX experiment
The ```new_experiment.py``` script allows you to set up a new experiment by specifying the required paths and parameters to the dataset.
#### Usage : ``` python new_experiment.py --experiment_name EXPERIMENT_NAME --experiment_path EXPERIMENT_PATH --clean_data_path CLEAN_DATA_PATH --noisy_data_path NOISY_DATA_PATH [--test_data_path TEST_DATA_PATH]```
| Argument | Description |
|--------------------------|-------------|
| `--experiment_name` | Name of the µPIX experiment. |
| `--experiment_path` | Path where the µPIX experiment will be saved. |
| `--clean_data_path` | Path to the clean dataset (Train). |
| `--noisy_data_path` | Path to the noisy dataset (Train). |
| `--test_data_path` *(optional)* | Path to the test dataset (if available). |
In our case, as we want just to use the pre-trained µPIX ```metrology``` model in inference (no training), we don't need to provide the ```clean_data_path``` and ```noisy_data_path``` parameters so we left it empty.
Nevertheless, we have to indicate the path that contains the test data image we want to denoise using µPIX using the ```test_data_path``` parameter, and the ```experiment_name``` we want to create as well as the location where the experiment will be stored using the ```experiment_path``` parameter.
Type the following to create experiment:
```bash
python new_experiment.py --experiment_name "metrology_experiment" --experiment_path "./experiments" --clean_data_path "" --noisy_data_path "" --test_data_path "./metrology/test"
```
If everything works correctly you should see:
```
Experiment 'metrology_experiment' created successfully at ./experiments/metrology_experiment/
```
### 4 - Denoising
The `mupixinfer.py` script allows you to use a pre-trained µPIX model to denoise a dataset located in the `test` directory inside the experiment path.
#### Usage : ```python mupixinfer.py --experiment_path EXPERIMENT_PATH```
For our example, here is the command you need to type to start the inference using the pre-trained µPIX model for the ```metrology``` test dataset:
```bash
python mupixinfer.py --experiment_path "./experiments/metrology_experiment/"
```
Once finished, the denoised images are stored inside ```./experiments/metrology_experiment/predictions/``` directory.
## Use case n°2 - Train a µPIX model from scratch using a custom dataset
Move to the `mupix` directory:
```bash
cd mupix
```
As an example, we will train a µPIX model from scratch using the Metrology dataset.
As an example, we will train a µPIX model from scratch using the ```metrology``` dataset.
### 2.1 - Download/Prepare the dataset
### 2.1 - Download the dataset
First download the dataset manually ([here](https://sync.lis-lab.fr/index.php/s/mYDRTeAQxMxNPPJ/download)), and unzip it.
Alternatively, if you have ```curl``` installed on your system, you can download and extract the dataset using:
......@@ -224,11 +325,11 @@ Step>3, Generator loss : 1.941e+05
...
...
```
By default, the model is trained for 100 epochs (see ```hyperparameters.json```) but contains an ```EarlyStopping``` mechanisms (See the paper methods section pour details) governed by the ```patience``` parameters (see ```hyperparameters.json```). The number of steps per epochs is calculated by dividing the number of training tiles divided by the ```batch_size```.
By default, the model is trained for 100 epochs (see ```hyperparameters.json```) but contains an ```EarlyStopping``` mechanisms (See the paper methods section pour details) governed by the ```patience``` parameters (see ```hyperparameters.json```).
You can stop the training anytime. The best checkpoints of your model is available inside ```experiments/metrology_experiment/results/networks/```
If the training stops because it reached the maximum number of epochs defined inside the ```hyperparameters.json``` configuration file, nd you want to continue to train your model for more epochs, you can use the ```--retrain``` parameters to resume the training:
If the training stops because it reached the maximum number of epochs defined inside the ```hyperparameters.json``` configuration file, and you want to continue to train your model for more epochs, you can use the ```--retrain``` parameters to resume the training where it stops:
```bash
python mupixtraining.py --experiment_path "./experiments/metrology_experiment" --retrain
......@@ -252,31 +353,13 @@ For our example, here is the command you need to type to start the inference usi
python mupixinfer.py --experiment_path "./experiments/metrology_experiment/"
```
## 3 - Use a pre-trained µPIX model to denoise an image dataset
For this example, we will use a pre-trained µPIX model on the ```metrology``` dataset and we will use it to denoise the ```metrology``` test dataset.
### 3.1 - Download and unzip the pre-trained µPIX ```metrology``` model
```bash
curl -o metrology_model.zip https://sync.lis-lab.fr/index.php/s/degZsCxN7ZXxeB6/download -q
```
```bash
unzip -qq metrology_model.zip -d metrology_model
Once done you can see that the predictions has been stored inside ```./experiments/metrology_experiment/predictions/``` directory:
```
The model is now stored inside ```./metrology_model```
### 3.2 - Create a new experiment containing the image dataset
As we are using the model in inference only (no training), we can skip the ```---clean_data_path``` and ```--noisy_data_path``` parameters of the ```new_experiment.py``` script.
If you don't already have donwload the ```metrology``` dataset, please download it (section 2.1: Create a µPIX experiments)
```bash
python new_experiment.py --experiment_name metrology_inferecence --experiment_path "./experiments" --clean_data_path "" --noisy_data_path "" --test_data_path "./metrology/test"
Saved weights to ./experiments/metrology_experiment/results/networks/Generator.h5
1/1 [==============================] - 7s 7s/step
Saved prediction: ./experiments/metrology_experiment/predictions/X_StageData0001.tif
All predictions saved successfully!
Data predicted at ./experiments/metrology_experiment/predictions !
```
......@@ -285,6 +368,5 @@ python new_experiment.py --experiment_name metrology_inferecence --experiment_pa
## License
This code repository is release under the [CC BY-NS-SA 4.0](https://gitlab.lis-lab.fr/sicomp/mupix/-/blob/main/LICENSE?ref_type=heads)
......@@ -65,8 +65,8 @@ def main():
# Organize the hyperparameters into a dictionary
hyperparameters = {
'learning_rate_generator': 1e-4,
'learning_rate_discriminator': 1e-4,
'learning_rate_generator': 1e-3,
'learning_rate_discriminator': 1e-3,
'batch_size': 16,
'num_epochs': 100,
'loss_weight': 10,
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment