- Add stage4 (results and experiments);
- Do not save model object.

- Add diamonds in experiment scripts and add factorize run_compute_results.sh script;
- Add best params for diamonds dataset.

Add quick working version of diamonds dataset to test that the improvement of OMP are consistent in other regression tasks.

- Remove extracted_forest_sizes_number parameter from compute_results.py and retreive the value instead;
- Add almost all remaining experiment config files of stages 1, 2 and 3;
- Add almost all remaining result plots of stages 1, 2 and 3;
- Add some temporary scripts to run all stages experiments.

- Add command lines example for stage 3;
- Add experiment_id option that is useful sometimes;
- Fix subsets_used param;
- Remove experiment_id in config experiment file names;
- Add config experiment files for stages 2 and 3;
- Add results for stages 2 and 3 (california_housing).

- Fix possible issues for extracted forest sizes computation: around to reduce possible zeroes and remove duplicates;
- Create output experiment stage dir if not exists;
- Add base_score_metric to model raw results class;
- Add best params for lfw_pairs (maybe try with a larger number of random seeds since the score is not that high).

- Remove useless getter in Dataset class.

- Even if hyperparameters file is ignore with skip_best_hyperparams option, still use the same forest_size to be comparable;
- Update experiment files for stage1 wo_param experiments (using the same forest size as the with_params experiments);
- In compute_results: remove useless folder creation; temporary add extracted_forest_sizes_number option to specify the extracted forest sizes number; temporary not plotting train and dev losses in stage1 loss values figure;
- In plotter, clean-up stage1 figure generation;
- Add first unbiased losses plot (stage1: best params vs default params in california housing dataset).

- Fix some variable names;
- Add exp files of stage1 for california housing

- Reduce the extracted forest sizes upper bound and number because OMP seems to converge only with small forest sizes;
- Add extraction_strategy parameter in order to save base forest and the forests trained with the same size as the extracted forest sizes used in the experiment that used OMP.

POC of possible wrong way to compute best hyperparams. Are there the best only before the application of OMP extraction?

- Definitely use the correct forest size (either the one from best hyperparameters or the one specified in parameter);
- Use a number of extracted forest sizes proportional as the forest size instead of fixed forest size;
- Add an option to save the current command line name instead of using the unamed directory;
- Add new california housing dataset best hyperparameters, and convert all value types that are number from string to int/float in other best hyperparameter files;
- Remove useless code from compute_results.py in prevision of the changes;
- Before best hyperparameters saving, save number as int or float instead of string;
- Add job_number option for parallelisation in both train.py and compute_hyperparameters.py scripts;
- Clean-up TODO list.

- Add new best params for 7 datasets.

- Ignore unamed experiment configuration file backups;
- Factorize default dataset loading parameters;
- Add missing return_X_y in basic dataset loaders.

Finish to fix sklearn dataset fetchers. Keep 20newsgroups_vectorized instead of non vectorized version that is the preprocessed version of this text dataset. Remove kddcup99 dataset.

Fix most of the missing dataset fetchers. Remainings: 20newsgroups, 20newsgroups_vectorized, kddcup99

Farah notation and related work

See merge request !7

# Conflicts:
#   .gitignore

clean scripts

See merge request !3

(