Fix omp_wo_weights when coupled with normalize_D. Fix missing selected trees saving for omp and random. Update compute results (not done yet). Fix omp_distillation.

Fix flw_pairs loading. Prepare all new exps: omp_distillation, preds coherence, preds correlation, normalize_D when OMP, n_jobs=-1 in SOTA. In exps script, test both train+dev,train+dev and train,dev

Handle similarity_similarities and similarity_predictions in the pipeline and set lfw_pairs to binary classif (todo: change the labels for omp)

Update fix of normalize_D in the case of wo weights. Add density plot in compute results for all stages and add some plots in results.

Integrate Paolo's code of method 'Ensemble selection from libraries of models' by Rich Caruana et al

Fix parallelization and estimator default hyperparams in kmeans and similarity methods. Fix on resume mode in train.py. Fix stage5 saving (tmp) in compute_results.py

- Add new results for stage4.

- Add stage4 (results and experiments);
- Do not save model object.

- Remove extracted_forest_sizes_number parameter from compute_results.py and retreive the value instead;
- Add almost all remaining experiment config files of stages 1, 2 and 3;
- Add almost all remaining result plots of stages 1, 2 and 3;
- Add some temporary scripts to run all stages experiments.

- Add command lines example for stage 3;
- Add experiment_id option that is useful sometimes;
- Fix subsets_used param;
- Remove experiment_id in config experiment file names;
- Add config experiment files for stages 2 and 3;
- Add results for stages 2 and 3 (california_housing).

- Fix some variable names;
- Add exp files of stage1 for california housing

- Reduce the extracted forest sizes upper bound and number because OMP seems to converge only with small forest sizes;
- Add extraction_strategy parameter in order to save base forest and the forests trained with the same size as the extracted forest sizes used in the experiment that used OMP.

POC of possible wrong way to compute best hyperparams. Are there the best only before the application of OMP extraction?

- Definitely use the correct forest size (either the one from best hyperparameters or the one specified in parameter);
- Use a number of extracted forest sizes proportional as the forest size instead of fixed forest size;
- Add an option to save the current command line name instead of using the unamed directory;
- Add new california housing dataset best hyperparameters, and convert all value types that are number from string to int/float in other best hyperparameter files;
- Remove useless code from compute_results.py in prevision of the changes;
- Before best hyperparameters saving, save number as int or float instead of string;
- Add job_number option for parallelisation in both train.py and compute_hyperparameters.py scripts;
- Clean-up TODO list.

Introduce the notion of "stage" in the experiments (see TODO comment in compute_results.py file. Add example of experiment configuration file tree in experiments/boston/stage3 for stage 3 (unoptimized parameters due to missing stage 1 and 2 results).

- Add a TODO for the subset in train
- Remove an useless new line

- Fix bug of resolve_experiment_id after 10 exp ids;
- Display the new experiment id at the beginning of the training;
- For now there's only a simple losses plot in compute_results.

- In compute_results, add loadenv, load raw results, and update experiment_ids so that it's possible to specify a list of experiments ids. The default behavior is to load all exp ids;
- Fix normalization option in train.py. By default it normalizes D, and it doesn't when specify wo_normalization option;
- Fix logger.warn to logger.warning in train.py
- Replace the dumping of result in trainer.py by a dedicated class to save and load the trained model and training metadatas: model_raw_results.py;
- Rename too long func DatasetLoader.load_from_name to DatasetLoader.load;
- Add loading functions in dataset_parameters and model_parameters;
- Set console logging level to INFO to summarize the most important console logs;
- Add a load function in model_factory;
- In omp_forest_regressor, move private funcs to the bottom of the file.

TODO: compute the plot from the loaded raw results in compute_results file.