example2.rst

Example 2 : Understanding the hyper-parameter optimization

If you are not familiar with hyper-parameter optimization, see :base_doc:`Hyper-parameters 101 <tutorials/hps_theory.html>`

Hands-on experience

In order to understand the process and it's usefulness, let's run some configurations and analyze the results.

This example will focus only on some lines of the configuration file :

• split:, controlling the ration of size between the testing set and the training set,
• hps_type:, controlling the type of hyper-parameter search,
• hps_args:, controlling the parameters of the hyper-parameters search method,
• nb_folds:, controlling the number of folds in the cross-validation process.

Example 2.1 : No hyper-parameter optimization, impact of split size

For this example, we only used a subset of the available classifiers in |platf|, to reduce the computation time and the complexity of the results.

Each classifier will first be learned on the default hyper-parameters (as in :base_doc:`Example 1 <tutorials/example1.html>`)

The monoview classifiers that will be used are Adaboost and a decision tree, and the multivew classifier is a late fusion majority vote. In order to use only a subset of the available classifiers, three lines in the configuration file are useful :

• type: (:base_source:`l45 <multiview_platform/examples/config_files/config_example_2_1_1.yml#L45>`) in which one has to specify which type of algorithms are needed, here we used type: ["monoview","multiview"],
• algos_monoview: (:base_source:`l47 <multiview_platform/examples/config_files/config_example_2_1_1.yml#L45>`) in which one specifies the names of the monoview algorithms to run, here we used : algos_monoview: ["decision_tree", "adaboost", ]
• algos_multiview: (:base_source:`l49 <multiview_platform/examples/config_files/config_example_2_1_1.yml#L45>`) is the same but with multiview algorithms, here we used : algos_multiview: ["majority_voting_fusion", ]

In order for the platform to understand the names, the user has to give the name of the python module in which the classifier is implemented in the platform.

In the config file, the default values for Adaboost's hyper-parameters are :

adaboost:
  n_estimators: 50
  base_estimator: "DecisionTreeClassifier"

(see adaboost's sklearn's page for more information)

For the decision tree :

decision_tree:
  max_depth: 3
  criterion: "gini"
  splitter: "best"

(sklearn's decision tree)