arXiv research

FastForest boosts Random Forest speed by 24%.

The paper studies statistical properties of CART regression trees.

It is widely believed that the prediction accuracy of decision tree models is invariant under any strictly monotone transformation of the individual predictor variables. However, this statement may be false when predicting new observations with values that were not seen in the training-set and are close to the location…

Paper improves anomaly detection by using non-uniform random choices in isolation forests.

Gradient Boosting Decision Tree (GBDT) are popular machine learning algorithms with implementations such as LightGBM and in popular machine learning toolkits like Scikit-Learn. Many implementations can only produce trees in an offline manner and in a greedy manner. We explore ways to convert existing GBDT implementatio…

New Thompson sampling algorithm for stochastic partial monitoring achieves logarithmic regret.

Paper shows FB and FC are equally hard up to logarithmic factors.

We propose a new algorithm called PLUTO for building logistic regression trees to binary response data. PLUTO can capture the nonlinear and interaction patterns in messy data by recursively partitioning the sample space. It fits a simple or a multiple linear logistic regression model in each partition. PLUTO employs th…

The most popular approach for analyzing survival data is the Cox regression model. The Cox model may, however, be misspecified, and its proportionality assumption may not always be fulfilled. An alternative approach for survival prediction is random forests for survival outcomes. The standard split criterion for random…

A new statistical concept, lepto-variance, is defined for stock returns using Regression Trees.

New batched Langevin Thompson Sampling reduces communication costs for sequential decision making.

The best-known and most commonly used distribution-property estimation technique uses a plug-in estimator, with empirical frequency replacing the underlying distribution. We present novel linear-time-computable estimators that significantly "amplify" the effective amount of data available. For a large variety of distri…

Logarithmic network width suffices for robust memorization.

Improved sampling for high-dimensional posteriors with underdamped Langevin.

New algorithms achieve logarithmic regret in KL-regularized Markov games.

The Dantzig selector has received popularity for many applications such as compressed sensing and sparse modeling, thanks to its computational efficiency as a linear programming problem and its nice sampling properties. Existing results show that it can recover sparse signals mimicking the accuracy of the ideal procedu…

New Thompson Sampling for partially observed context bandits reduces regret logarithmically with time.

Improved statistical inference for adaptive Thompson Sampling.

Optimal unimodal fitting for linear loss functions in a sequential, efficient manner.

Survival trees exhibit end-cut preference, leading to biased splits.

Machine learning algorithms aim at minimizing the number of false decisions and increasing the accuracy of predictions. However, the high predictive power of advanced algorithms comes at the costs of transparency. State-of-the-art methods, such as neural networks and ensemble methods, often result in highly complex mod…

The paper shows how to learn causal representations with few environments and finite samples.

Study hypothesis testing under quantized samples with communication constraints, achieving near-optimal sample complexity.

A new method for estimating probabilities and risks using Markov processes.

Improved model-free RL algorithm with reduced sample complexity.

We study multi-armed bandit problems with graph feedback, in which the decision maker is allowed to observe the neighboring actions of the chosen action, in a setting where the graph may vary over time and is never fully revealed to the decision maker. We show that when the feedback graphs are undirected, the original …

Boosting improves accuracy by combining weak learners into a voting classifier.

Study examines sample complexity for RL with safety constraints.

A decentralized policy achieves logarithmic regret for multi-agent MAB problems with communication constraints.

The study approximates option prices using Hermite polynomials without assuming a specific distribution.

New bounds for Bayesian bandits show prior improves performance.

We propose a new algorithmic framework for sequential hypothesis testing with i.i.d. data, which includes A/B testing, nonparametric two-sample testing, and independence testing as special cases. It is novel in several ways: (a) it takes linear time and constant space to compute on the fly, (b) it has the same power gu…

Study shows sample complexity for learning optimal policies in SSP with generative model.

We consider reinforcement learning in parameterized Markov Decision Processes (MDPs), where the parameterization may induce correlation across transition probabilities or rewards. Consequently, observing a particular state transition might yield useful information about other, unobserved, parts of the MDP. We present a…

We study the decades-old problem of online portfolio management and propose the first algorithm with logarithmic regret that is not based on Cover's Universal Portfolio algorithm and admits much faster implementation. Specifically Universal Portfolio enjoys optimal regret $\mathcal{O}(N\ln T)$ for $N$ financial instrum…

Improved uniform convergence bound with fat-shattering dimension reduces sample complexity gap.

A robust bandit algorithm uses Dirichlet sampling to minimize regret under various distributional assumptions.

Algorithm identifies Pareto front using multiple context directions and reuses exploration samples.

This paper improves system identification by reducing sample complexity for high-dimensional linear dynamical systems.

We introduce canonical correlation forests (CCFs), a new decision tree ensemble method for classification and regression. Individual canonical correlation trees are binary decision trees with hyperplane splits based on local canonical correlation coefficients calculated during training. Unlike axis-aligned alternatives…

Study sparsity benefits in infinite feature contextual bandits.

Optimal ReLU networks can memorize any separable set of points with a small number of parameters.

Gaussian Graphical Models (GGMs) have wide-ranging applications in machine learning and the natural and social sciences. In most of the settings in which they are applied, the number of observed samples is much smaller than the dimension and they are assumed to be sparse. While there are a variety of algorithms (e.g. G…

In this paper, we introduce the notions of logarithmic Poisson structure and logarithmic principal Poisson structure; we prove that the latter induces a representation by logarithmic derivation of the module of logarithmic Kahler differentials; therefore, it induces a differential complex from which we derive the notio…

Causal discovery from empirical data is a fundamental problem in many scientific domains. Observational data allows for identifiability only up to Markov equivalence class. In this paper we first propose a polynomial time algorithm for learning the exact correctly-oriented structure of the transitive reduction of any c…

Transformers capture combinatorial tasks with bounded error and logarithmic sample dependence.

The paper analyzes the statistical cost of tuning kernel hyperparameters in robust regression.

We give a new, very general, formulation of the compressed sensing problem in terms of coordinate projections of an analytic variety, and derive sufficient sampling rates for signal reconstruction. Our bounds are linear in the coherence of the signal space, a geometric parameter independent of the specific signal and m…