arXiv research

Paper improves tree probability estimation using stochastic optimization and variance reduction.

Reduces conjecture to tree-based Artin groups.

A new method, tree-SNE, solves the scale problem in t-SNE.

Decision trees with binary splits are popularly constructed using Classification and Regression Trees (CART) methodology. For binary classification and regression models, this approach recursively divides the data into two near-homogenous daughter nodes according to a split point that maximizes the reduction in sum of …

Paper explains how tree ensembles improve predictions by smoothing and regulating smoothness.

A new decision tree induction method using MIP for faster optimization.

Novel loss functions improve decision tree learning from noisy data.

Machine learning-based applications are increasingly prevalent in IoT devices. The power and storage constraints of these devices make it particularly challenging to run modern neural networks, limiting the number of new applications that can be deployed on an IoT system. A number of compression techniques have been pr…

We consider the problem of estimating the conditional probability of a label in time O(log n), where n is the number of possible labels. We analyze a natural reduction of this problem to a set of binary regression problems organized in a tree structure, proving a regret bound that scales with the depth of the tree. Mot…

TreeDOX predicts chaotic systems without hyperparameter tuning.

We introduce a multiscale supervised dimension reduction method for SPatial Interaction Network (SPIN) data, which consist of a collection of spatially coordinated interactions. This type of predictor arises when the sampling unit of data is composed of a collection of primitive variables, each of them being essentiall…

We introduce a semiparametric approach to neighbor-based classification. We build off the recently proposed Boundary Trees algorithm by Mathy et al.(2015) which enables fast neighbor-based classification, regression and retrieval in large datasets. While boundary trees use an Euclidean measure of similarity, the Differ…

Approximates large Random Forest models to save space.

Neural networks and tree ensembles are state-of-the-art learners, each with its unique statistical and computational advantages. We aim to combine these advantages by introducing a new layer for neural networks, composed of an ensemble of differentiable decision trees (a.k.a. soft trees). While differentiable trees dem…

New test improves tree ensemble pruning for better model performance.

Graph regularized autoencoder improves anomaly detection performance.

A novel tree algorithm improves time series forecasting accuracy.

New method wraps black-box classifiers to reduce bias.

We adopt data structure in the form of cover trees and iteratively apply approximate nearest neighbour (ANN) searches for fast compressed sensing reconstruction of signals living on discrete smooth manifolds. Levering on the recent stability results for the inexact Iterative Projected Gradient (IPG) algorithm and by us…

We study very small trees from the point of view of reducing systems of free factors, which are analogues of reducing systems of curves for a surface lamination; a non-trivial, proper free factor $F \leq \FN$ reduces $T$ if and only if $F$ acts on some subtree of $T$ with dense orbits. We characterize those trees, call…

Two new methods improve monotonic constraint enforcement in regression and classification trees.

Recombinant binomial trees are binary trees where each non-leaf node has two child nodes, but adjacent parents share a common child node. Such trees arise in finance when pricing an option. For example, valuation of a European option can be carried out by evaluating the expected value of asset payoffs with respect to r…

The paper proposes a method to improve random forest classification accuracy by weighting trees based on their decision path reliability.

The paper studies statistical properties of CART regression trees.

Defines super stable maps and proves quotient superorbifolds for genus zero.

We introduce a new algorithm, called CDER, for supervised machine learning that merges the multi-scale geometric properties of Cover Trees with the information-theoretic properties of entropy. CDER applies to a training set of labeled pointclouds embedded in a common Euclidean space. If typical pointclouds correspondin…

Let S be a compact connected oriented surface with one boundary component, and let P be the fundamental group of S. The Johnson filtration is a decreasing sequence of subgroups of the Torelli group of S, whose k-th term consists of the self-homeomorphisms of S that act trivially at the level of the k-th nilpotent quoti…

New optimizer MARS-M combines variance reduction with Muon for faster LLM training.

As nilpotent studies in knot theory, we focus on invariants of Milnor, Orr, and Kontsevich. We show that the Orr invariant of degree $k$ is equivalent to the tree reduction of the Kontsevich invariant of degree $< 2k$. Furthermore, we will see a close relation between the Orr invariant and the Milnor invariant, and …

Scientists in many fields have the common and basic need of dimensionality reduction: visualizing the underlying structure of the massive multivariate data in a low-dimensional space. However, many dimensionality reduction methods confront the so-called "crowding problem" that clusters tend to overlap with each other i…

Let $Σ$ be a compact connected oriented surface with one boundary component and let $\mathcal{M}$ denote the mapping class group of $Σ$. By considering the action of $\mathcal{M}$ on the fundamental group of $Σ$ it is possible to define different filtrations of $\mathcal{M}$ together with some homomorphisms on each ter…

DR-MCTS improves decision quality and sample efficiency in complex environments.

New tree and forest methods use oblique splits for better risk bounds.

LILI clustering reduces bias in causal inference by grouping similar counterfactual outcomes.

The Hitchin component is a connected component of the character variety of reductive group homomorphisms from the fundamental group of a closed surface S of genus greater than 1 to the Lie group PSL_m(R). The Teichmuller space of S naturally embeds into the Hitchin component. The limit points in the Thurston compactifi…

Reduces conjecture for Artin groups to simpler cases.

Let M denote the mapping class group of S, a compact connected oriented surface with one boundary component. The action of M on the nilpotent quotients of the fundamental group of S allows to define the so-called Johnson filtration and the Johnson homomorphisms. J. Levine introduced a new filtration of M, called the La…

Regularizes decision trees to reduce inference time by up to 4x with minimal accuracy loss.

This paper describes structuring data and constructing plots to explore forest classification models interactively. A forest classifier is an example of an ensemble, produced by bagging multiple trees. The process of bagging and combining results from multiple trees, produces numerous diagnostics which, with interactiv…

We seek decision rules for prediction-time cost reduction, where complete data is available for training, but during prediction-time, each feature can only be acquired for an additional cost. We propose a novel random forest algorithm to minimize prediction error for a user-specified {\it average} feature acquisition b…

Standard methods for anomaly detection assume that all features are observed at both learning time and prediction time. Such methods cannot process data containing missing values. This paper studies five strategies for handling missing values in test queries: (a) mean imputation, (b) MAP imputation, (c) reduction (redu…

FIGS improves prediction performance while maintaining interpretability, especially in medical domains.

Hippo optimizes deep learning hyper-parameters by reducing redundant trials.

We design and study a Contextual Memory Tree (CMT), a learning memory controller that inserts new memories into an experience store of unbounded size. It is designed to efficiently query for memories from that store, supporting logarithmic time insertion and retrieval operations. Hence CMT can be integrated into existi…

RLF uses Riemann-Lebesgue cutting for better regression.

Many biological characteristics of evolutionary interest are not scalar variables but continuous functions. Here we use phylogenetic Gaussian process regression to model the evolution of simulated function-valued traits. Given function-valued data only from the tips of an evolutionary tree and utilising independent pri…

Study of wild mapping class groups and their cabled braids.

SX-GeoTree improves spatially coherent explanations in geospatial regression trees.