Struct DecisionTree

pub struct DecisionTree<F: Float, L: Label> { /* private fields */ }

A fitted decision tree model for classification.

Structure

A decision tree structure is a binary tree where:

• Each internal node specifies a decision, represented by a choice of a feature and a “split value” such that all observations for which feature <= split_value is true fall in the left subtree, while the others fall in the right subtree.

• leaf nodes make predictions, and their prediction is the most popular label in the node

Algorithm

Starting with a single root node, decision trees are trained recursively by applying the following rule to every node considered:

• Find the best split value for each feature of the observations belonging in the node;
• Select the feature (and its best split value) that maximizes the quality of the split;
• If the score of the split is sufficiently larger than the score of the unsplit node, then two child nodes are generated, the left one containing all observations with feature <= split value and the right one containing the rest.
• If no suitable split is found, the node is marked as leaf and its prediction is set to be the most common label in the node;

The quality score used can be specified in the parameters.

Predictions

To predict the label of a sample, the tree is traversed from the root to a leaf, choosing between left and right children according to the values of the features of the sample. The final prediction for the sample is the prediction of the reached leaf.

Additional constraints

In order to avoid overfitting the training data, some additional constraints on the quality/quantity of splits can be added to the tree. A description of these additional rules is provided in the parameters page.

Example

Here is an example on how to train a decision tree from its parameters:

use linfa_trees::DecisionTree;
use linfa::prelude::*;
use linfa_datasets;

// Load the dataset
let dataset = linfa_datasets::iris();
// Fit the tree
let tree = DecisionTree::params().fit(&dataset).unwrap();
// Get accuracy on training set
let accuracy = tree.predict(&dataset).confusion_matrix(&dataset).unwrap().accuracy();

assert!(accuracy > 0.9);

Implementations

impl<F: Float, L: Label> DecisionTree<F, L>

pub fn iter_nodes(&self) -> NodeIter<'_, F, L>

Create a node iterator in level-order (BFT)

pub fn features(&self) -> Vec<usize>

Return features_idx of this tree (BFT)

pub fn mean_impurity_decrease(&self) -> Vec<F>

Return the mean impurity decrease for each feature

pub fn relative_impurity_decrease(&self) -> Vec<F>

Return the relative impurity decrease for each feature

pub fn feature_importance(&self) -> Vec<F>

Return the feature importance, i.e. the relative impurity decrease, for each feature

pub fn root_node(&self) -> &TreeNode<F, L>

Return root node of the tree

pub fn max_depth(&self) -> usize

Return max depth of the tree

pub fn num_leaves(&self) -> usize

Return the number of leaves in this tree

pub fn export_to_tikz(&self) -> Tikz<'_, F, L>

Generates a Tikz structure to print the fitted tree in Tex using tikz and forest, with the following default parameters:

• legend=false
• complete=true

impl<F: Float, L: Label> DecisionTree<F, L>

pub fn params() -> DecisionTreeParams<F, L>

Defaults are provided if the optional parameters are not specified:

• split_quality = SplitQuality::Gini
• max_depth = None
• min_weight_split = 2.0
• min_weight_leaf = 1.0
• min_impurity_decrease = 0.00001

Trait Implementations

impl<F: Clone + Float, L: Clone + Label> Clone for DecisionTree<F, L>

fn clone(&self) -> DecisionTree<F, L>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<F: Debug + Float, L: Debug + Label> Debug for DecisionTree<F, L>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<F: PartialEq + Float, L: PartialEq + Label> PartialEq for DecisionTree<F, L>

fn eq(&self, other: &DecisionTree<F, L>) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<F: Float, L: Label + Default, D: Data<Elem = F>> PredictInplace<ArrayBase<D, Dim<[usize; 2]>>, ArrayBase<OwnedRepr<L>, Dim<[usize; 1]>>> for DecisionTree<F, L>

fn predict_inplace(&self, x: &ArrayBase<D, Ix2>, y: &mut Array1<L>)

Make predictions for each row of a matrix of features x.

fn default_target(&self, x: &ArrayBase<D, Ix2>) -> Array1<L>

Create targets that predict_inplace works with.

impl<F: Float, L: Label> StructuralPartialEq for DecisionTree<F, L>