Skip to content
Snippets Groups Projects

Resolve "Experiment pipeline"

Merged Resolve "Experiment pipeline"
12-experiment-pipeline into master
Merged Charly Lamothe requested to merge 12-experiment-pipeline into master
Viewing commit 51ba8a0e
Prev Next
Show latest version
5 files
+ 22
− 22
Compare changes
  • Side-by-side
  • Inline
Files
5
  • 51ba8a0e
    Charly Lamothe
    - Unignore results; · 51ba8a0e
    Charly Lamothe authored 
    - Even if hyperparameters file is ignore with skip_best_hyperparams option, still use the same forest_size to be comparable;
- Update experiment files for stage1 wo_param experiments (using the same forest size as the with_params experiments);
- In compute_results: remove useless folder creation; temporary add extracted_forest_sizes_number option to specify the extracted forest sizes number; temporary not plotting train and dev losses in stage1 loss values figure;
- In plotter, clean-up stage1 figure generation;
- Add first unbiased losses plot (stage1: best params vs default params in california housing dataset).
code/train.py
+ 7
− 3
@@ -107,7 +107,7 @@ python code/train.py --dataset_name=california_housing --seeds 1 2 3 --save_expe
python code/train.py --dataset_name=california_housing --seeds 1 2 3 --extraction_strategy=none --skip_best_hyperparams --save_experiment_configuration 1 none_wo_params
python code/train.py --dataset_name=california_housing --seeds 1 2 3 --extraction_strategy=random --skip_best_hyperparams --save_experiment_configuration 1 random_wo_params
python code/train.py --dataset_name=california_housing --seeds 1 2 3 --skip_best_hyperparams --save_experiment_configuration 1 omp_wo_params
python code/compute_results.py --stage 1 --experiment_ids 1 2 3 4 5 6
python code/compute_results.py --stage 1 --experiment_ids 1 2 3 4 5 6 --dataset_name=california_housing --extracted_forest_sizes_number=5
"""
if __name__ == "__main__":
load_dotenv(find_dotenv('.env'))
@@ -160,10 +160,14 @@ if __name__ == "__main__":
logger = LoggerFactory.create(LOG_PATH, os.path.basename(__file__))
hyperparameters_path = os.path.join('experiments', args.dataset_name, 'stage1', 'params.json')
if os.path.exists(hyperparameters_path) and not args.skip_best_hyperparams:
if os.path.exists(hyperparameters_path):
logger.info("Hyperparameters found for this dataset at '{}'".format(hyperparameters_path))
with open(hyperparameters_path, 'r+') as file_hyperparameter:
hyperparameters = json.load(file_hyperparameter)['best_parameters']
loaded_hyperparameters = json.load(file_hyperparameter)['best_parameters']
if args.skip_best_hyperparams:
hyperparameters = {'n_estimators': loaded_hyperparameters['n_estimators']}
else:
hyperparameters = loaded_hyperparameters
else:
hyperparameters = {}