- Fix possible issues for extracted forest sizes computation: around to reduce possible zeroes and remove duplicates;
- Create output experiment stage dir if not exists;
- Add base_score_metric to model raw results class;
- Add best params for lfw_pairs (maybe try with a larger number of random seeds since the score is not that high).

- Remove useless getter in Dataset class.

- Even if hyperparameters file is ignore with skip_best_hyperparams option, still use the same forest_size to be comparable;
- Update experiment files for stage1 wo_param experiments (using the same forest size as the with_params experiments);
- In compute_results: remove useless folder creation; temporary add extracted_forest_sizes_number option to specify the extracted forest sizes number; temporary not plotting train and dev losses in stage1 loss values figure;
- In plotter, clean-up stage1 figure generation;
- Add first unbiased losses plot (stage1: best params vs default params in california housing dataset).

- Fix some variable names;
- Add exp files of stage1 for california housing

- Reduce the extracted forest sizes upper bound and number because OMP seems to converge only with small forest sizes;
- Add extraction_strategy parameter in order to save base forest and the forests trained with the same size as the extracted forest sizes used in the experiment that used OMP.

POC of possible wrong way to compute best hyperparams. Are there the best only before the application of OMP extraction?

- Definitely use the correct forest size (either the one from best hyperparameters or the one specified in parameter);
- Use a number of extracted forest sizes proportional as the forest size instead of fixed forest size;
- Add an option to save the current command line name instead of using the unamed directory;
- Add new california housing dataset best hyperparameters, and convert all value types that are number from string to int/float in other best hyperparameter files;
- Remove useless code from compute_results.py in prevision of the changes;
- Before best hyperparameters saving, save number as int or float instead of string;
- Add job_number option for parallelisation in both train.py and compute_hyperparameters.py scripts;
- Clean-up TODO list.

- Add new best params for 7 datasets.

- Ignore unamed experiment configuration file backups;
- Factorize default dataset loading parameters;
- Add missing return_X_y in basic dataset loaders.

Finish to fix sklearn dataset fetchers. Keep 20newsgroups_vectorized instead of non vectorized version that is the preprocessed version of this text dataset. Remove kddcup99 dataset.

Fix most of the missing dataset fetchers. Remainings: 20newsgroups, 20newsgroups_vectorized, kddcup99

- Update TODO list.

- Move bolsonaro imports on top of the file in compute_hyperparam file;
- Move change_binary_func_load to utils file.

When training, look if there is bayesian search results, if yes use this. Exception: forest_size use the one given by parser if applicable

Big changes: Create intermediate classes OMPForest and SingleOmpForest for code factoring: share code between OmpForestRegressor and OmpForestBinaryClassifer. Remove set_wweights and set_forest which are not relevant anymore. load function from model_factory isn't trustfull now: raises an error. TODO: multiclass classifier

Introduce the notion of "stage" in the experiments (see TODO comment in compute_results.py file. Add example of experiment configuration file tree in experiments/boston/stage3 for stage 3 (unoptimized parameters due to missing stage 1 and 2 results).

- Add experiment_configuration parameter to run an experiment from a json configuration file. If the experiment configuration are commnig from the arguments, save it to a file to keep trace of it;
- Add few comments in train.py.

- Use as much CPU as possible when training a random forest regressor.