SVR-Tree improves classification trees for imbalanced and sparse data.
problem Classification difficulties in imbalanced and sparse data.
method Proposes SVR-Tree, penalizing the Surface-to-Volume Ratio of decision sets.
result SVR-Tree improves generalization error compared to other imbalance algorithms.
Optimal decision trees learned via dynamic programming and search.
problem Finding the best classification trees that are both accurate and compact.
method Dynamic programming and search techniques.
result Our algorithm learns optimal classification trees significantly faster and scales better than existing methods.
Proposes a new tree-based algorithm for class-imbalanced data.
problem Classification of imbalanced data.
method Precision-Recall Curve (PRC) classification tree algorithm.
result The PRC classification tree outperforms classic algorithms for class-imbalanced data.
In this paper, we present a novel sequential paradigm for classification in crowdsourcing systems. Considering that workers are unreliable and they perform the tests with errors, we study the construction of decision trees so as to minimize the probability of mis-classification. By exploiting the connection between pro…
A new randomized tree classifier outperforms traditional CARTs.
problem Traditional greedy CARTs are slow and inaccurate.
method Optimal decision trees using continuous optimization.
result Randomized tree classifier outperforms traditional CARTs.
Quantum circuits represent binary classification trees with binary features.
problem Classifying data using binary classification trees with binary features.
method Quantum circuits and probabilistic approach for traversing decision trees.
result First realization of a decision tree classifier on a quantum device.
Max-Cut decision tree improves classification accuracy and reduces computation time.
problem Improving decision tree accuracy and efficiency for complex classification tasks.
method Alternative splitting metric (max cut) and PCA-based feature selection at each node.
result 49% improvement in accuracy with 94% reduction in CPU time on CIFAR-100 data.
New method builds robust trees from noisy data.
problem Building accurate classification trees from noisy labeled data.
method Combines SVM-like splitting rules and label noise detection.
result Effective in detecting and mitigating label noise.
A new classification method based on Minimum Spanning Trees
problem Improving classification in supervised learning
method Proposing a classification algorithm based on Minimum Spanning Trees
result The proposed method is effective and computationally efficient
AF improves classification models by adaptively weighting trees.
problem Improving classification model performance.
method AF combines OP2T for input-dependent weights and MIO for dynamic refinement.
result AF consistently outperforms RF, XGBoost, and other weighted RF.
Enhanced tree-based classifiers use derivatives and geometry for better function classification.
problem Improving classification of high-dimensional time series data.
method Integrates Functional Data Analysis with tree-based ensemble techniques, leveraging derivative and geometric features.
result Significant improvements over traditional approaches in function classification.
New groups defined that act on trees without repeating.
problem Understanding groups acting on trees without repeating.
method Developed a new concept of acylindrically arboreal groups and classified specific groups.
result Provided examples of groups with tree actions but no non-elementary acylindrical actions.
Two new methods improve monotonic constraint enforcement in regression and classification trees.
problem Improving monotonic constraints in regression and classification trees.
method Proposed two new methods: one yields better results than LightGBM, the other yields even better results but is slower.
result The best method consistently beats the current implementation of LightGBM, achieving up to 1% loss reduction.
Tree ensembles like RF and GBT can be seen as kernels, improving regression and classification performance.
problem Improving kernel methods for tree ensemble based models.
method Investigation of RF and GBT kernels in simulation and real data.
result RF and GBT kernels are competitive to their respective ensembles in higher dimensions, particularly with noisy features.
DTE uses tree leaf means to embed data, balancing accuracy and speed.
problem High variance in decision tree splits and computational inefficiency of ensembles.
method DTE constructs an interpretable feature representation using leaf means of a trained tree.
result DTE strikes a balance between accuracy and computational efficiency, outperforming ensembles.
New algorithm reduces discrimination in predictions.
problem Tackles potential discrimination in AI predictions.
method Integrates fairness adjustments into tree-building process.
result Reduces discriminatory predictions without significant loss in accuracy.
Simultaneous Latent Budget Trees for stratified classification
problem Classification with stratification factors
method Probabilistic machine learning framework
result Interpretation of latent components and conditional split rule
Several classification methods assume that the underlying distributions follow tree-structured graphical models. Indeed, trees capture statistical dependencies between pairs of variables, which may be crucial to attain low classification errors. The resulting classifier is linear in the log-transformed univariate and b…
We present an algorithm for learning decision trees using stochastic gradient information as the source of supervision. In contrast to previous approaches to gradient-based tree learning, our method operates in the incremental learning setting rather than the batch learning setting, and does not make use of soft splits…
Tree-based algorithm for functional data analysis reduces generalization error.
problem Classification and regression problems with functional data.
method Constrained convex optimization for weighted functional L2 space, multiple splitting rules, and weighted integral features. result Reduces generalization error while maintaining interpretability.
A new supervised tree-Wasserstein distance improves document classification.
problem Measuring document similarity efficiently and accurately.
method Rewriting Wasserstein distance on tree metric, using contrastive loss for optimization.
result The Supervised Tree-Wasserstein (STW) distance improves document classification accuracy.
New framework optimizes classification trees with logistic loss and ℓ1 regularization.
problem Improving interpretability and generalization of classification trees.
method Developed a generalized framework for CTs, incorporating logistic loss and ℓ1 regularization. result Optimal Logistic Tree model outperforms state-of-the-art MIP-based approaches in terms of interpretability and generalization.
ODTLearn learns optimal decision trees for predictive and prescriptive tasks.
problem Learning optimal decision trees for high-stakes predictive and prescriptive tasks.
method Mixed-integer optimization framework and object-oriented design.
result Implementation of optimal decision trees for various tasks.
A new decision tree induction method using MIP for faster optimization.
problem Optimizing decision tree split rules for better classification performance.
method Mixed-integer programming (MIP) for Gini reduction maximization, efficient search algorithm.
result bsnsing trees outperform other tree models in new case discrimination.
Hollow-tree Super resolves feature importance in large datasets.
problem Lack of effective scaling for large feature numbers in boosted tree models.
method Hollow-tree Super (HOTS) methodology for feature importance visualization.
result HOTS effectively resolves feature importance and directionality in high-dimensional neuroscientific data.
We propose a novel "tree-averaging" model that utilizes the ensemble of classification and regression trees (CART). Each constituent tree is estimated with a subset of similar data. We treat this grouping of subsets as Bayesian ensemble trees (BET) and model them as an infinite mixture Dirichlet process. We show that B…
We introduce a semiparametric approach to neighbor-based classification. We build off the recently proposed Boundary Trees algorithm by Mathy et al.(2015) which enables fast neighbor-based classification, regression and retrieval in large datasets. While boundary trees use an Euclidean measure of similarity, the Differ…
Forest tree species mapped with high accuracy using satellite data.
problem Classifying dominant tree species in Swedish forests.
method Extreme gradient boosting model with Bayesian optimization, combining Sentinel-1/2 satellite data and field observations.
result Overall accuracy of 85%, F1 score of 0.82, Matthews correlation coefficient of 0.81.
Study optimal rates for multiclass classification, resolving open questions.
problem Optimal rates for multiclass classification with any label space.
method Establishes optimal rates for all hypothesis classes, defining new tree structures.
result Optimal rates for multiclass classification with no infinite DSL trees.
Proposes a deep tree-ensemble model for multi-output prediction.
problem Lack of efficient solutions for multi-output prediction.
method Integrates tree-embeddings into deep tree-ensembles for structured output prediction.
result Superior performance in multi-label classification and multi-target regression tasks.
The paper proposes a method to improve random forest classification accuracy by weighting trees based on their decision path reliability.
problem Random forests' uniform voting fails to correct errors in regions where incorrect tree representations outnumber correct ones.
method The paper introduces using the structural pattern of each tree's decision path as an instance-adaptive reliability signal to identify and weight more reliable trees.
result Using the proposed method yields a statistically significant accuracy improvement over RF on 36 binary classification benchmarks.
A novel gradient-based method optimizes decision trees for complex tasks.
problem Training decision trees with arbitrary differentiable loss functions.
method Gradient-based optimization using first and second derivatives of loss functions.
result Improves accuracy and flexibility in decision tree optimization.
Resource-efficient oblique trees reduce neural signal classification costs.
problem Implementing efficient neural signal classifiers on resource-constrained devices.
method Integrating model compression, probabilistic routing, and cost-aware learning.
result Significant reduction in model size and feature extraction cost compared to state-of-the-art models.
Decision trees are consistent for regression and classification tasks even with many predictors.
problem Consistency of decision trees with many predictors.
method CART and C4.5 methodology, oracle inequality, sparsity constraints.
result Decision trees and random forests are consistent for various types of data.
The paper tackles robust classification trees for distribution shifts, improving accuracy in public health and social work.
problem Learning robust classification trees for high-stakes settings with distribution shifts.
method Mixed-integer robust optimization technology to reformulate as a two-stage linear robust optimization problem.
result Increase of up to 12.48% in worst-case accuracy and 4.85% in average-case accuracy.
OPLTs online train label trees for multi-label and multi-class classification.
problem Online multi-label and multi-class classification challenges.
method Fully online training of label trees without prior knowledge.
result Strong theoretical guarantees and low complexity.
One-class classifiers are trained with target class only samples. Intuitively, their conservative modelling of the class description may benefit classical classification tasks where classes are difficult to separate due to overlapping and data imbalance. In this work, three methods are proposed which leverage on the co…
New method prunes classification trees for biased data.
problem Pruning classification trees in imbalanced training data.
method Optimal pruning procedure for inhomogeneous data.
result First efficient procedure for optimal pruning under covariate shift.
This paper proposes an improved active learning method using classification trees.
problem Reducing the size of training sets while maintaining high accuracy in supervised learning.
method A wrapper active learning method using a classification tree to sub-sample from low-entropy regions.
result The proposed method constructs accurate classification models even with severely restricted labeled data.
We introduce a novel incremental decision tree learning algorithm, Hoeffding Anytime Tree, that is statistically more efficient than the current state-of-the-art, Hoeffding Tree. We demonstrate that an implementation of Hoeffding Anytime Tree---"Extremely Fast Decision Tree", a minor modification to the MOA implementat…
Ensembles of classification and regression trees remain popular machine learning methods because they define flexible non-parametric models that predict well and are computationally efficient both during training and testing. During induction of decision trees one aims to find predicates that are maximally informative …
FoLDTree improves oblique decision trees with ULDA, enhancing accuracy and feature selection.
problem Axis-orthogonal splits limit traditional decision trees' performance on oblique decision boundaries.
method Integrates ULDA into decision tree structure for efficient oblique splits, feature selection, and handling missing values.
result FoLDTree outperforms other methods in accuracy and feature selection, comparable to random forest.
New classifiers account for context-specific independences.
problem Restrictions in generative models for classification.
method Staged tree classifiers that account for context-specific independences.
result Staged tree classifiers achieve competitive classification accuracy.
Automatic classification of trees using remotely sensed data has been a dream of many scientists and land use managers. Recently, Unmanned aerial vehicles (UAV) has been expected to be an easy-to-use, cost-effective tool for remote sensing of forests, and deep learning has attracted attention for its ability concerning…
RST improves environmental time series classification accuracy using randomized B-spline trees.
problem Improving accuracy in classifying complex environmental time series.
method Randomized Spline Trees (RST) integrates randomized functional representations into ensemble learning.
result RST variants outperform standard Random Forests and Gradient Boosting on most environmental time series datasets.
Regression-via-Classification (RvC) is the process of converting a regression problem to a classification one. Current approaches for RvC use ad-hoc discretization strategies and are suboptimal. We propose a neural regression tree model for RvC. In this model, we employ a joint optimization framework where we learn opt…
Tree-based models biased when trained on imbalanced data, requiring new calibration methods.
problem Bias in tree-based models trained on imbalanced datasets.
method Analytical calibration of random forest models, demonstrating bias in decision trees.
result Calibrating tree-based models on imbalanced data negatively impacts predictions, especially for the minority class.
Extreme classification problems are multiclass and multilabel classification problems where the number of outputs is so large that straightforward strategies are neither statistically nor computationally viable. One strategy for dealing with the computational burden is via a tree decomposition of the output space. Whil…