Skip to content
Snippets Groups Projects
Commit 433a6041 authored by Fabrice Daian's avatar Fabrice Daian
Browse files

update

parent 46b9bf41
No related branches found
No related tags found
No related merge requests found
...@@ -118,11 +118,7 @@ python new_experiment.py --experiment_name EXPERIMENT_NAME ...@@ -118,11 +118,7 @@ python new_experiment.py --experiment_name EXPERIMENT_NAME
For our example, here is the command you need to type to create a µPIX experiment for the ```metrology``` dataset: For our example, here is the command you need to type to create a µPIX experiment for the ```metrology``` dataset:
```bash ```bash
python new_experiment.py --experiment_name metrology_experiment python new_experiment.py --experiment_name "metrology_experiment" --experiment_path "./experiments" --clean_data_path "./metrology/train/GT" --noisy_data_path "./metrology/train/low" --test_data_path "./metrology/test"
--experiment_path ./experiments
--clean_data_path ./metrology/train/GT
--noisy_data_path ./metrology/train/low
--test_data_path ./metrology/test
``` ```
If everything works correctly you should see: If everything works correctly you should see:
``` ```
...@@ -191,8 +187,8 @@ python mupixtraining.py --experiment_path EXPERIMENT_PATH [--retrain] ...@@ -191,8 +187,8 @@ python mupixtraining.py --experiment_path EXPERIMENT_PATH [--retrain]
For our example, here is the command you need to type to start the training of a µPIX model for the ```metrology``` dataset: For our example, here is the command you need to type to start the training of a µPIX model for the ```metrology``` dataset:
``` ```bash
python mupixtraining.py --experiment_path ./experiments/metrology_experiment python mupixtraining.py --experiment_path "./experiments/metrology_experiment"
``` ```
You should see the training starting: You should see the training starting:
...@@ -232,145 +228,17 @@ By default, the model is trained for 100 epochs (see ```hyperparameters.json```) ...@@ -232,145 +228,17 @@ By default, the model is trained for 100 epochs (see ```hyperparameters.json```)
You can stop the training anytime. The best checkpoints of your model is available inside ```experiments/metrology_experiment/results/networks/``` You can stop the training anytime. The best checkpoints of your model is available inside ```experiments/metrology_experiment/results/networks/```
If the training stops because it reached the maximum number of epochs defined inside the ```hyperparameters.json``` configuration file, nd you want to continue to train your model for more epochs, you can use the ```--retrain``` parameters to resume the training:
```bash
### 2.4 - Inference on the test dataset python mupixtraining.py --experiment_path "./experiments/metrology_experiment" --retrain
The `mupixinfer.py` script allows you to use a pre-trained µPIX model to denoise a dataset located in the `test` directory inside the experiment path.
#### Usage
```
python mupixinfer.py --experiment_path EXPERIMENT_PATH
```
For our example, here is the command you need to type to start the inference using the newly trained µPIX model best (checkpoint) for the ```metrology``` test dataset:
```
python mupixinfer.py --experiment_path ./experiments/metrology_experiment/
```
1 - Train a µPIX model from scratch
2 - Use a pre-trained µPIX model to denoise images
3 - Fine-Tune a pre-trained µPIX model on a new dataset
## 2 - Create a µPIX experiments
Before running µPIX for training or inference, you need to create a *µPIX experiment*.
The `new_experiment.py` script allows you to set up a new experiment by specifying the required paths and parameters.
#### Usage
```
python new_experiment.py --experiment_name EXPERIMENT_NAME --experiment_path EXPERIMENT_PATH --clean_data_path CLEAN_DATA_PATH --noisy_data_path NOISY_DATA_PATH [--test_data_path TEST_DATA_PATH]
```
| Argument | Description |
|--------------------------|-------------|
| `--experiment_name` | Name of the experiment. |
| `--experiment_path` | Path where the experiment will be saved. |
| `--clean_data_path` | Path to the clean dataset. |
| `--noisy_data_path` | Path to the noisy dataset. |
| `--test_data_path` *(optional)* | Path to the test dataset (if available). |
#### Example
To create an experiment with the provided *Metrology* dataset:
```
python new_experiment.py --experiment_name "metrology_exp" --experiment_path "./data/metrology" --clean_data_path "./data/metrology/GT/" --noisy_data_path "./data/metrology/low"
```
Feel free to adapt to your own dataset keeping in minds that, as µPIX use a supervised learnign apporach, for one images inside the clean_data_path and the noisy_data_path must have the same name: example `image0.tif` inside the `clean_data_path` must corresponds to the `image0.tif` image inside the `noisy_data_path`.
This command will create a new `metrology_exp` experiment directory containing:
- `hyperparameters.json`: default parameters used by µPIX
- `results` directory: will contains training logs (```logs.txt```), model checkpoints (```networks```), image generated during the training (```images```) using ```mupixtraining.py```.
- `predictions` directory: will contains the generated image on the test dataset using the script ```mupixinfer.py```
For information, here is the ```hyperparameters.json``` file created by default by µPIX for our example experiment that could be freely modifiable
```json
{
"learning_rate_generator": 0.0001,
"learning_rate_discriminator": 0.0001,
"batch_size": 16,
"num_epochs": 100,
"loss_weight": 10,
"tile_size": 256,
"patience": 20,
"valid_size": 0.1,
"seed": 42,
"data_paths": {
"clean": "./data/metrology/GT/",
"noisy": "./data/metrology/low",
"test": null
}
}
```
| Hyperparameter | Type | Description |
|--------------------------------|---------|-------------|
| `learning_rate_generator` | `float` | Learning rate for the generator. |
| `learning_rate_discriminator` | `float` | Learning rate for the discriminator. |
| `batch_size` | `int` | Number of samples per batch. |
| `num_epochs` | `int` | Total number of training epochs. |
| `loss_weight` | `int` | Weight factor for µPIX loss calculation. |
| `tile_size` | `int` | Size of image tiles used for training. |
| `patience` | `int` | Number of epochs to wait before triggering µPIX early stopping if no improvement. |
| `valid_size` | `float` | Proportion of the dataset used for validation. |
| `seed` | `int` | Random seed for reproducibility. |
| `data_paths.clean` | `str` | Path to the clean dataset. |
| `data_paths.noisy` | `str` | Path to the noisy dataset. |
| `data_paths.test` | `str or null` | Path to the test dataset (if available). |
## 3- Train a µPIX
The `mupixtraining.py` script allows you to train a µPIX model by specifying the required path to an existing experiment.
#### Usage
```
python mupixtraining.py --experiment_path EXPERIMENT_PATH [--retrain]
```
| Argument | Description |
|--------------------------|-------------|
| `--experiment_path` | Path to the previously created experiment. |
| `--retrain` *(optional)* | Use this flag to continue training an existing µPIX model located inside the experiment path. |
#### Example
Train a new µPIX model:
```
python mupixtraining.py --experiment_path "./data/metrology"
``` ```
Or Continue to train a µPIX model:
```
python mupixtraining.py --experiment_path "./data/metrology" --retrain
```
## 4 - Inference on New Images Using a Trained µPIX Model ### 2.4 - Inference on the test dataset
The `mupixinfer.py` script allows you to use a trained µPIX model to denoise a dataset located in the `test` directory inside the experiment path. The `mupixinfer.py` script allows you to use a pre-trained µPIX model to denoise a dataset located in the `test` directory inside the experiment path.
#### Usage #### Usage
...@@ -378,61 +246,45 @@ The `mupixinfer.py` script allows you to use a trained µPIX model to denoise a ...@@ -378,61 +246,45 @@ The `mupixinfer.py` script allows you to use a trained µPIX model to denoise a
python mupixinfer.py --experiment_path EXPERIMENT_PATH python mupixinfer.py --experiment_path EXPERIMENT_PATH
``` ```
#### Example For our example, here is the command you need to type to start the inference using the newly trained µPIX model best (checkpoint) for the ```metrology``` test dataset:
``` ```bash
python mupixinfer.py --experiment_path "./data/metrology" python mupixinfer.py --experiment_path "./experiments/metrology_experiment/"
``` ```
## 3 - Use a pre-trained µPIX model to denoise an image dataset
For this example, we will use a pre-trained µPIX model on the ```metrology``` dataset and we will use it to denoise the ```metrology``` test dataset.
### 3.1 - Download and unzip the pre-trained µPIX ```metrology``` model
## Runnin the code using our Docker image
You can retrieve the image from docker hub with the following command:
```bash ```bash
docker pull gabrielbon/mupix:latest curl -o metrology_model.zip https://sync.lis-lab.fr/index.php/s/degZsCxN7ZXxeB6/download -q
``` ```
Create a new experiment
```bash ```bash
docker run -it --entrypoint bin/bash -v /path/to/your/file/mupix:/workspace gabrielbon/mupix:latest unzip -qq metrology_model.zip -d metrology_model
``` ```
The model is now stored inside ```./metrology_model```
### 3.2 - Create a new experiment containing the image dataset
As we are using the model in inference only (no training), we can skip the ```---clean_data_path``` and ```--noisy_data_path``` parameters of the ```new_experiment.py``` script.
If you don't already have donwload the ```metrology``` dataset, please download it (section 2.1: Create a µPIX experiments)
In case your data are located outside of the mupix folder, docker will need their location:
```bash ```bash
docker run -it --entrypoint bin/bash -v /path/to/your/file/mupix:/workspace -v /path/to/your/data:/workspace/data gabrielbon/mupix:latest python new_experiment.py --experiment_name metrology_inferecence --experiment_path "./experiments" --clean_data_path "" --noisy_data_path "" --test_data_path "./metrology/test"
``` ```
Given you are working in the container, you will have to specify the path do data differently
## 3 - License
This code repository is release under the [CC BY-NS-SA 4.0](https://gitlab.lis-lab.fr/sicomp/mupix/-/blob/main/LICENSE?ref_type=heads)
### b) Using the µPIX Docker container
If you don't have Docker already installed on your computer, please use our help to guide through the [installation of Docker on your machine](./docker_install.md).
We have developped a "ready-to-use" Docker image ([repo on dockerhub](https://hub.docker.com/r/gabrielbon/mupix)) containing a ready-to-use µPIX environment.
To get the image, open a Terminal and type: ## License
This code repository is release under the [CC BY-NS-SA 4.0](https://gitlab.lis-lab.fr/sicomp/mupix/-/blob/main/LICENSE?ref_type=heads)
```bash
docker pull gabrielbon/mupix:latest
```
Finally, you can clone this repository to install µPIX scripts and datasets:
```bash
git clone https://gitlab.lis-lab.fr/sicomp/mupix
```
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment