arXiv research

Improves tree model performance by considering future node splits.

Deep forests enhance expressiveness exponentially with depth, not width or tree size.

Decision tree learning heuristics fail even in smoothed analysis for complex targets.

The paper improves bounds on skein tree depth and delta-crossing numbers for knots and links.

ForestPrune optimizes tree ensemble pruning for compactness and speed.

Recent advances in bandit tools and techniques for sequential learning are steadily enabling new applications and are promising the resolution of a range of challenging related problems. We study the game tree search problem, where the goal is to quickly identify the optimal move in a given game tree by sequentially sa…

Decision trees perform well in complex interactions, even when interactions are not fully accounted for.

Transformer learns to search through reinforcement learning, mimicking DFS.

The paper improves bounds on the complexity of computing link polynomials.

New methods improve prediction performance and reduce computation time in boosting and random forest models.

We consider the problem of estimating the conditional probability of a label in time O(log n), where n is the number of possible labels. We analyze a natural reduction of this problem to a set of binary regression problems organized in a tree structure, proving a regret bound that scales with the depth of the tree. Mot…

The depth of a link measures the minimum height of a resolving tree for the link whose leaves are all unlinks. We show that the depth of the closure of a strictly positive braid word is the length of the word minus the number of distinct letters.

Paper analyzes soft tree ensembles using NTK, finding only leaf count matters.

In the context of tree-search stochastic planning algorithms where a generative model is available, we consider on-line planning algorithms building trees in order to recommend an action. We investigate the question of avoiding re-planning in subsequent decision steps by directly using sub-trees as action recommender. …

Enhanced ODT with Feature Concatenation boosts learning efficiency.

Regression Trees analyze stock returns, revealing market excess return as the most informative factor.

A new statistical concept, lepto-variance, is defined for stock returns using Regression Trees.

This paper presents a detailed comparison of a recently proposed algorithm for optimizing decision trees, tree alternating optimization (TAO), with other popular, established algorithms. We compare their performance on a number of classification and regression datasets of various complexity, different size and dimensio…

Improved isolation forest for better outlier detection.

This paper addresses questions of quasi-isometric rigidity and classification for fundamental groups of finite graphs of groups, under the assumption that the Bass-Serre tree of the graph of groups has finite depth. The main example of a finite depth graph of groups is one whose vertex and edge groups are coarse Poinca…

In this paper we analyze, evaluate, and improve the performance of training Random Forest (RF) models on modern CPU architectures. An exact, state-of-the-art binary decision tree building algorithm is used as the basis of this study. Firstly, we investigate the trade-offs between using different tree building algorithm…

We consider multi-label classification where the goal is to annotate each data point with the most relevant $\textit{subset}$ of labels from an extremely large label set. Efficient annotation can be achieved with balanced tree predictors, i.e. trees with logarithmic-depth in the label complexity, whose leaves correspon…

Smooth fractal trees via analytic generators, preserving combinatorial and geometric properties.

In an attempt to gather a deeper understanding of how convolutional neural networks (CNNs) reason about human-understandable concepts, we present a method to infer labeled concept data from hidden layer activations and interpret the concepts through a shallow decision tree. The decision tree can provide information abo…

This study argues for pruning trees in random forests to improve performance in low signal-to-noise scenarios.

Gromov-Wasserstein (GW) is a powerful tool to compare probability measures whose supports are in different metric spaces. GW suffers however from a computational drawback since it requires to solve a complex non-convex quadratic program. We consider in this work a specific family of cost metrics, namely \textit{tree me…

This paper shows that scientific discovery can be efficiently learned via compositional function trees, reducing the sample complexity.

New framework detects model weaknesses in decision tree ensembles.

Deep imagination optimizes decision-making in large trees with limited resources.

For any positive integer $k$, there exist neural networks with $Θ(k^3)$ layers, $Θ(1)$ nodes per layer, and $Θ(1)$ distinct parameters which can not be approximated by networks with $\mathcal{O}(k)$ layers unless they are exponentially large --- they must possess $Ω(2^k)$ nodes. This result is proved here for a class o…

A study on the depth of graph neural networks on sparse graphs, revealing a dichotomy based on the Kesten-Stigum ratio.

Quantum circuits represent binary classification trees with binary features.

While many recent advances in deep reinforcement learning (RL) rely on model-free methods, model-based approaches remain an alluring prospect for their potential to exploit unsupervised data to learn environment model. In this work, we provide an extensive study on the design of deep generative models for RL environmen…

Boosting meta-trees improve decision tree performance.

A new algorithm, Regular Tree Search, tackles non-convex simulation optimization problems.

Many data are naturally modeled by an unobserved hierarchical structure. In this paper we propose a flexible nonparametric prior over unknown data hierarchies. The approach uses nested stick-breaking processes to allow for trees of unbounded width and depth, where data can live at any node and are infinitely exchangeab…

The paper proposes a new probability distribution for rooted trees.

Inferring a decision tree from a given dataset is one of the classic problems in machine learning. This problem consists of buildings, from a labelled dataset, a tree such that each node corresponds to a class and a path between the tree root and a leaf corresponds to a conjunction of features to be satisfied in this c…

Tree Index evaluates cluster quality by creating decision trees from data.

Tree ensembles are flexible predictive models that can capture relevant variables and to some extent their interactions in a compact and interpretable manner. Most algorithms for obtaining tree ensembles are based on versions of boosting or Random Forest. Previous work showed that boosting algorithms exhibit a cyclic b…

Prediction suffix trees (PST) provide an effective tool for sequence modelling and prediction. Current prediction techniques for PSTs rely on exact matching between the suffix of the current sequence and the previously observed sequence. We present a provably correct algorithm for learning a PST with approximate suffix…

Two new methods reduce random forest latency and improve accuracy.

A tree-based dictionary learning model is developed for joint analysis of imagery and associated text. The dictionary learning may be applied directly to the imagery from patches, or to general feature vectors extracted from patches or superpixels (using any existing method for image feature extraction). Each image is …

Optimal decision trees learned via dynamic programming and search.

Decision trees can be biased towards minority class, contrary to belief.

Recent advances in machine learning and artificial intelligence are now being considered in safety-critical autonomous systems where software defects may cause severe harm to humans and the environment. Design organizations in these domains are currently unable to provide convincing arguments that their systems are saf…

The theory of tunnel number 1 knots detailed in our previous paper, The tree of knot tunnels, provides a non-negative integer invariant called the depth of the tunnel. We give various results related to the depth invariant. Noting that it equals the minimum number of Goda-Scharlemann-Thompson tunnel moves needed to con…

Paper proposes a VB method for TS-SBP mixture models with reduced computational cost.