Let $ X \in\mathbb{R}^{n \times d}$ be the matrix data and $Y \in\mathbb{R}^{n}$ be the labels vector associated to the matrix $X$, where for each $i$, $\textbf{x}_i \in\cal{X}\subseteq\mathbb{R}^d$ and $y_i \in{\cal Y}\subseteq\mathbb{R}$. \\
Let $ X \in\mathbb{R}^{n \times d}$\todo{est-ce-que le non-gras majuscule est fréquent pour les matrices?}be the matrix data and $Y \in\mathbb{R}^{n}$ be the labels vector associated to the matrix $X$, where for each $i$, $\textbf{x}_i \in\mathcal{X}\subseteq\mathbb{R}^{d}$ and $y_i \in{\mathcal Y}\subseteq\mathbb{R}$. \\
A random forest $F_{t_1, \dots, t_l}$ is a classifier made of a collection of $l$ trees ${t_1, \dots, t_l}$. This forest can be seen as a function, and noted as:
A random forest $F_{t_1, \dots, t_l}$\todo[inline]{confusion possible notation: majuscule non gras: fonction ou matrice? les deux, à éclaircir}is a classifier made of a collection of $l$ trees ${t_1, \dots, t_l}$. This forest can be seen as a function, and noted as:
where $f$ is a function which depend on the task. In a regression setup, where ${\cal Y}=\mathbb{R}$, this function can be defined as:
where $f$ is a function which depend on the task\todo{f unclear: why to introduce it?}. In a regression setup, where ${\cal Y}=\mathbb{R}$\todo{I don't think it is usefull}, this function can be defined as:
We will need to define the vector prediction of a forest for all the data matrix: $F_{t_1, \dots, t_l}(X)=\begin{pmatrix}
\todo{$\mathds{1}$ not defined}We \todo{no we}will need to define the vector prediction of a forest for all the data matrix: $F_{t_1, \dots, t_l}(X)=\begin{pmatrix}
F_{t_1, \dots, t_l}(x_1)\\
\dots\\
F_{t_1, \dots, t_l}(x_n)
...
...
@@ -57,7 +60,10 @@ We will need to define the vector prediction of a forest for all the data matrix
%
%
All these notations can be summarized in the following table:\\
\begin{tabular}{l c}%\caption{Notation table}
\begin{table}
\begin{tabular}{l c}
%\hline
\textbf{x}& the vector x \\
$k$& the desired (pruned) forest size \\
...
...
@@ -71,13 +77,18 @@ All these notations can be summarized in the following table:\\
$F_{t_1, \dots, t_l}(\textbf{x})\in{\cal Y}$& the predicted label of \textbf{x} by the forest $F_{t_1, \dots, t_l}$\\
$F_{t_1, \dots, t_l}(X)\in{\cal Y}^n$& the predicted label of all the data of $X$ by the forest $F_{t_1, \dots, t_l}$\\
%\hline
\end{tabular}
\end{tabular}
\caption{Notations}
\end{table}\todo[inline]{ajouter les codifications des notations: bold minuscule: vecteur; non-bold majuscule: matrix, etc..}
Given a matrix $D =[d_1, \dots , d_l]\in\mathbb{R}^n\times l$ (also called a dictionary) and a signal $\textbf{y}\in\mathbb{R}^n$, finding a $k$-sparse vector $\textbf{w}\in\mathbb{R}^l$ (i.e. $|| \textbf{w} ||_0\leq k$) that minimize $|| X\textbf{w}-\textbf{y}||$ is an NP-hard problem (ref).
Given a matrix $D =[d_1, \dots , d_l]\in\mathbb{R}^{n\times l}$\todo{l undefined in text} (also called a dictionary) and a signal $\textbf{y}\in\mathbb{R}^n$, finding a $k$-sparse vector $\textbf{w}\in\mathbb{R}^l$ (i.e. $|| \textbf{w} ||_0\leq k$) that minimize $|| X\textbf{w}-\textbf{y}||$ is an NP-hard problem \todo{(ref np-hardness)}.
The Orthogonal Matching Pursuit (OMP) algorithm is a greedy algorithm that aim to give an approximate solution of this problem.
The approximation of $\textbf{y}$ is build one term at a time. Noting $\textbf{y}_k$ the current
approximation and $r_k =\textbf{y}_k-\textbf{y}_k$ the so-called residual, we select at each time step the atom (i.e. the column of $X$) which has the largest inner product with $r_k$, and update the approximation.
The approximation of $\textbf{y}$ is built one term at a time. Noting $\textbf{y}_k$ the current
approximation and $r_k =\textbf{y}-\textbf{y}_k$ the so-called residual, we select at each time step the atom (i.e. the column of $X$) which has the largest inner product with $r_k$, and update the approximation.
This step is repeated until a satisfactory approximation. This can be summarized in Algorithm \ref{algo: OMP}
...
...
@@ -93,19 +104,20 @@ This step is repeated until a satisfactory approximation. This can be summarized
In general, the OMP algorithm can be seen as a algorithm that 'summarize' the most useful column of the dictionary for expressing the signal \textbf{y}.
In general, the OMP algorithm can be seen as an algorithm that 'summarize' \todo{summarize: unclear}the most useful column of the dictionary for expressing the signal \textbf{y}.
In this paper, we use this algorithm to reduce the forest's size by selecting the most informative trees in the forest (see Section \ref{sec: forest pruning} for more details).
\todo[inline]{réduire: sélectionner l'essentiel, regrouper en catégorie d'algorithmes et se positionner (forward vs backward selection, etc.)}
\begin{itemize}
\item\cite{Yang2012}: once the forest $(F = t_1, \dots, t_n)$ is built, he gives each tree a score (which measures the importance of the tree in the forest). The tree with the lowest score is removed from the forest. To eliminate the next tree, all the scores are recomputed, and the tree with the lowest score is removed...\\
They present in this paper 4 different tree's score. For each tree $t_i$, we compute:
\item\cite{Yang2012}: once the forest $(F = t_1, \dots, t_n)$\todo{inconsistent notations} is built, he \todo{who?}gives each tree a score (which measures the importance of the tree in the forest). The tree with the lowest score is removed from the forest. To eliminate the next tree, all the scores are recomputed, and the tree with the lowest score is removed...\todo{and so forth and so on}\\
They present in this paper 4 \todo{numbers}different tree's score. For each tree $t_i$, we compute:
In this section, we will describe our method for pruning the forest, and thus reduce its size. \\
Consider a forest $F_{t_1, \dots, t_l}$ of $l =100$ trees, trained using the training data set, witch consist of the 60\% of the data. For every $i \in\{1, \dots , l\}$, we note the vector of prediction of the tree $t_i$ in all the $n$ data by:
Consider a forest $F_{t_1, \dots, t_l}$ of $l =100$\todo{trop concret}trees, trained using the training data set, witch consist of the 60\% of the data. For every $i \in\{1, \dots , l\}$, we note the vector of prediction of the tree $t_i$ in all the $n$ data by:
We apply the OMP algorithm to the $Y$ matrix and to the reals labels vector $\textbf{y}$. Thus, we will look for the $k$ most informative trees to predict the true labels.
We apply the OMP algorithm to the $Y$ matrix and to the reals labels vector $\textbf{y}$. Thus, we will look for the $k$ most informative trees to predict the true labels.\todo{détailler plus comment est utilisé OMP}\todo{ajouter l'algorithme}
