arXiv research

Flexible tree ensemble learning framework supports arbitrary loss functions and multi-task learning.

Attention augments forest for tabular data accuracy.

Regression trees learn gradients of differentiable functions.

Paper introduces algorithms for private decision tree learning.

Neural networks and tree ensembles are state-of-the-art learners, each with its unique statistical and computational advantages. We aim to combine these advantages by introducing a new layer for neural networks, composed of an ensemble of differentiable decision trees (a.k.a. soft trees). While differentiable trees dem…

Differentiable optimization bridges arbitrary metrics to tree metrics.

We introduce a semiparametric approach to neighbor-based classification. We build off the recently proposed Boundary Trees algorithm by Mathy et al.(2015) which enables fast neighbor-based classification, regression and retrieval in large datasets. While boundary trees use an Euclidean measure of similarity, the Differ…

New method learns binary decision trees efficiently.

The paper proposes a method to infer differentiation trees from RNA velocity data.

A novel gradient-based method optimizes decision trees for complex tasks.

Decision trees are ubiquitous in machine learning for their ease of use and interpretability. Yet, these models are not typically employed in reinforcement learning as they cannot be updated online via stochastic gradient descent. We overcome this limitation by allowing for a gradient update over the entire tree that i…

A new approach to Morse theory using folded ribbon trees.

GeoPhy uses geometric gradients to efficiently infer phylogenetic trees from molecular data.

New spanning tree model connects knot homology, s-invariant, and exotic discs.

Paper uses CMAB to improve NAS efficiency and accuracy.

DiPriMe forests use private medians to create balanced tree splits for privacy-protected data.

TREX explains tree ensembles by identifying key training examples.

The Gradient Boosting Decision Tree (GBDT) is a popular machine learning model for various tasks in recent years. In this paper, we study how to improve model accuracy of GBDT while preserving the strong guarantee of differential privacy. Sensitivity and privacy budget are two key design aspects for the effectiveness o…

The theme in this paper is the recombining binomial tree to price American put option when the underlying stock follows constant elasticity of variance(CEV) process. Recombining nodes of binomial tree are decided from finite difference scheme to emulate CEV process and the tree has a linear complexity. Also it is deriv…

The Jones polynomial can be expressed in terms of spanning trees of the graph obtained by checkerboard coloring a knot diagram. We show there exists a complex generated by these spanning trees whose homology is the reduced Khovanov homology. The spanning trees provide a filtration on the reduced Khovanov complex and a …

Bayesian GBMs improve predictive uncertainty calibration for tabular data.

TF Boosted Trees (TFBT) is a new open-sourced frame-work for the distributed training of gradient boosted trees. It is based on TensorFlow, and its distinguishing features include a novel architecture, automatic loss differentiation, layer-by-layer boosting that results in smaller ensembles and faster prediction, princ…

New KD-tree based method for private synthetic data generation.

We study the limits of holonomy representations of complex projective structures on a compact Riemann surface in the Morgan-Shalen compactification of the character variety. We show that the dual R-trees of the quadratic differentials associated to a divergent sequence of projective structures determine the Morgan-Shal…

This paper introduces TNTK to study infinite soft tree ensembles.

Until recently, transcriptomics was limited to bulk RNA sequencing, obscuring the underlying expression patterns of individual cells in favor of a global average. Thanks to technological advances, we can now profile gene expression across thousands or millions of individual cells in parallel. This new type of data has …

We study the combinatorial geometry of "lattice" Jenkins--Strebel differentials with simple zeroes and simple poles on $\mathbb{C}P^1$ and of the corresponding counting functions. Developing the results of M. Kontsevich we evaluate the leading term of the symmetric polynomial counting the number of such "lattice" Jenki…

The isoresidual fibration maps Riemann sphere strata to resonance arrangements.

Single tree outperforms random forest in testing accuracy.

Most decision tree induction algorithms are based on a greedy top-down recursive partitioning strategy for tree growth. In this paper, we propose several methods for induction of decision trees and their ensembles based on evolutionary algorithms. The main difference of our approach is using real-valued vector represen…

Minimal diffeomorphisms extend uniquely with $L^1$ Hopf differential.

Paper finds how Steklov eigenvalues change on graphs and trees.

Improved DP KDE with better privacy and efficiency.

Develops a variational method for ultrametric phylogenetic trees.

Ensembles of classification and regression trees remain popular machine learning methods because they define flexible non-parametric models that predict well and are computationally efficient both during training and testing. During induction of decision trees one aims to find predicates that are maximally informative …

Combining deep model-free reinforcement learning with on-line planning is a promising approach to building on the successes of deep RL. On-line planning with look-ahead trees has proven successful in environments where transition models are known a priori. However, in complex environments where transition models need t…

We consider certain groups of tree automorphisms as so-called diffeological groups. The notion of diffeology, due to Souriau, allows to endow non-manifold topological spaces, such as regular trees that we look at, with a kind of a differentiable structure that in many ways is close to that of a smooth manifold; a suita…

We present a general architecture of deep differentiable forest and its sparse attention mechanism. The differentiable forest has the advantages of both trees and neural networks. Its structure is a simple binary tree, easy to use and understand. It has full differentiability and all variables are learnable parameters.…

Study of non-Archimedean Hitchin map for SL2(F) characters.

We study compositional generalization, viz., the problem of zero-shot generalization to novel compositions of concepts in a domain. Standard neural networks fail to a large extent on compositional learning. We propose Tree Stack Memory Units (Tree-SMU) to enable strong compositional generalization. Tree-SMU is a recurs…

The Mondrian process represents an elegant and powerful approach for space partition modelling. However, as it restricts the partitions to be axis-aligned, its modelling flexibility is limited. In this work, we propose a self-consistent Binary Space Partitioning (BSP)-Tree process to generalize the Mondrian process. Th…

We iterate Manolescu's unoriented skein exact triangle in knot Floer homology with coefficients in the field of rational functions over $\mathbb{Z}/2\mathbb{Z}$. The result is a spectral sequence which converges to a stabilized version of delta-graded knot Floer homology. The $(E_2,d_2)$ page of this spectral sequence …

AD-HOC simplifies high-order derivative calculations in C++.

Derives Black-Scholes model without stochastic calculus or PDEs.

sGBM speeds up gradient boosting by parallelizing and adapting base learners.

Infinitesimal gradient boosting is a new algorithm derived from gradient boosting.

Since its inception in the 1980s, ID3 has become one of the most successful and widely used algorithms for learning decision trees. However, its theoretical properties remain poorly understood. In this work, we introduce a novel metric of a decision tree algorithm's performance, called mean iteration statistical consis…

The paper proposes a method for interpretable mixture density estimation using a tree structure.