A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Partition Tree estimates conditional densities for mixed continuous and categorical variables.
The Mondrian process represents an elegant and powerful approach for space partition modelling. However, as it restricts the partitions to be axis-aligned, its modelling flexibility is limited. In this work, we propose a self-consistent Binary Space Partitioning (BSP)-Tree process to generalize the Mondrian process. Th…
Recent theory work has found that a special type of spatial partition tree - called a random projection tree - is adaptive to the intrinsic dimension of the data from which it is built. Here we examine this same question, with a combination of theory and experiments, for a broader class of trees that includes k-d trees…
Oblique BART improves tree-based predictions.
Online BSP-Forest improves space partitioning for large-scale classification and regression.
flexBART improves BART for categorical predictors by creating flexible tree partitions.
We introduce a method for creating a special type of tree, called a tree position, from a weighted graph. Leaves of the tree correspond to vertices of the original graph, and the tree edges contain information which can be used to partition these vertices. By repeatedly applying reducing operations to the tree position…
A new algorithm, Regular Tree Search, tackles non-convex simulation optimization problems.
In this paper, we investigate adaptive nonlinear regression and introduce tree based piecewise linear regression algorithms that are highly efficient and provide significantly improved performance with guaranteed upper bounds in an individual sequence manner. We use a tree notion in order to partition the space of regr…
SBT model uses randomized sharding and sub-models to improve Bayesian Additive Regression Trees.
Causal trees struggle with accuracy in estimating treatment effects.
A new approach uses partial likelihood to improve tree-based density estimation and inference.
We present a new way of constructing an ensemble classifier, named the Guided Random Forest (GRAF) in the sequel. GRAF extends the idea of building oblique decision trees with localized partitioning to obtain a global partitioning. We show that global partitioning bridges the gap between decision trees and boosting alg…
We propose a new outline for adaptive dictionary learning methods for sparse encoding based on a hierarchical clustering of the training data. Through recursive application of a clustering method, the data is organized into a binary partition tree representing a multiscale structure. The dictionary atoms are defined ad…
Space partitioning methods such as random forests and the Mondrian process are powerful machine learning methods for multi-dimensional and relational data, and are based on recursively cutting a domain. The flexibility of these methods is often limited by the requirement that the cuts be axis aligned. The Ostomachion p…
The paper challenges the use of decision trees for pointwise inference due to slow convergence rates.
The study analyzes decision trees on real and categorical features, deriving bounds on their VC dimension and proposing improved pruning algorithms.
The Binary Space Partitioning~(BSP)-Tree process is proposed to produce flexible 2-D partition structures which are originally used as a Bayesian nonparametric prior for relational modelling. It can hardly be applied to other learning tasks such as regression trees because extending the BSP-Tree process to a higher dim…
DTE uses tree leaf means to embed data, balancing accuracy and speed.
This paper introduces the Partition Tree Weighting technique, an efficient meta-algorithm for piecewise stationary sources. The technique works by performing Bayesian model averaging over a large class of possible partitions of the data into locally stationary segments. It uses a prior, closely related to the Context T…
We present the first tree-based regressor whose convergence rate depends only on the intrinsic dimension of the data, namely its Assouad dimension. The regressor uses the RPtree partitioning procedure, a simple randomized variant of k-d trees.
Survey of Bayesian nonparametric space partition models and their applications.
This work proposes a robust ensemble method for decision trees that resists adversarial attacks.
Improved Random Forests detect pure interactions better.
LA-MCTS learns search space partition for black-box optimization using Monte Carlo Tree Search.
Proposes a new method for subgroup analysis using optimal trees with parameter fusion.
Undirected graphical models encode in a graph the dependency structure of a random vector . In many applications, it is of interest to model given another random vector as input. We refer to the problem of estimating the graph of conditioned on as ``graph-valued regression.'' In this pap…
Paper introduces XBART for nonlinear regression, outperforming XGBoost.
Paper recovers lattice signal partitions efficiently.
Develops a new method to recover large latent tree models efficiently.
This research designs a data-driven partition to test independence between continuous variables.
Decision trees are a popular technique in statistical data classification. They recursively partition the feature space into disjoint sub-regions until each sub-region becomes homogeneous with respect to a particular class. The basic Classification and Regression Tree (CART) algorithm partitions the feature space using…
We introduce a novel approach to feed-forward neural network interpretation based on partitioning the space of sequences of neuron activations. In line with this approach, we propose a model-specific interpretation method, called YASENN. Our method inherits many advantages of model-agnostic distillation, such as an abi…
We introduce inference trees (ITs), a new class of inference methods that build on ideas from Monte Carlo tree search to perform adaptive sampling in a manner that balances exploration with exploitation, ensures consistency, and alleviates pathologies in existing adaptive methods. ITs adaptively sample from hierarchica…
Space partitions of underlie a vast and important class of fast nearest neighbor search (NNS) algorithms. Inspired by recent theoretical work on NNS for general metric spaces [Andoni, Naor, Nikolov, Razenshteyn, Waingarten STOC 2018, FOCS 2018], we develop a new framework for building space partitions re…
MSTs provide a fast and meaningful clustering method in low-dimensional data.
The modern scale of data has brought new challenges to Bayesian inference. In particular, conventional MCMC algorithms are computationally very expensive for large data sets. A promising approach to solve this problem is embarrassingly parallel MCMC (EP-MCMC), which first partitions the data into multiple subsets and r…
In this work, we propose a simple but effective method to interpret black-box machine learning models globally. That is, we use a compact binary tree, the interpretation tree, to explicitly represent the most important decision rules that are implicitly contained in the black-box machine learning models. This tree is l…
Decision Machines embeds decision trees into vector spaces for improved optimization.
The problem of community detection in networks is usually formulated as finding a single partition of the network into some "correct" number of communities. We argue that it is more interpretable and in some regimes more accurate to construct a hierarchical tree of communities instead. This can be done with a simple to…
A post-hoc framework improves model performance by calibrating different feature spaces.
State-of-the-art clustering algorithms use heuristics to partition the feature space and provide little insight into the rationale for cluster membership, limiting their interpretability. In healthcare applications, the latter poses a barrier to the adoption of these methods since medical researchers are required to pr…
We propose a multiresolution Gaussian process to capture long-range, non-Markovian dependencies while allowing for abrupt changes. The multiresolution GP hierarchically couples a collection of smooth GPs, each defined over an element of a random nested partition. Long-range dependencies are captured by the top-level GP…
Hierarchical structure is ubiquitous in data across many domains. There are many hierarchical clustering methods, frequently used by domain experts, which strive to discover this structure. However, most of these methods limit discoverable hierarchies to those with binary branching structure. This limitation, while com…
Evolutionary algorithms improve decision tree ensembles.
Bounds on the log partition function are important in a variety of contexts, including approximate inference, model fitting, decision theory, and large deviations analysis. We introduce a new class of upper bounds on the log partition function, based on convex combinations of distributions in the exponential domain, th…
Binomial tree methods (BTM) and explicit difference schemes (EDS) for the variational inequality model of American options with time dependent coefficients are studied. When volatility is time dependent, it is not reasonable to assume that the dynamics of the underlying asset's price forms a binomial tree if a partitio…
Both supervised and unsupervised machine learning algorithms have been used to learn partition-based index structures for approximate nearest neighbor (ANN) search. Existing supervised algorithms formulate the learning task as finding a partition in which the nearest neighbors of a training set point belong to the same…