arXiv research

ECV method optimizes ensemble parameters for randomized ensembles.

New hyperparameter ensembles boost neural network performance and uncertainty.

A new ensemble method using random projections for kNN classification.

Develops RKHS framework for analyzing tree ensembles.

Extends spectral number variance convergence to random matrix ensembles for twisted Laplacians.

In this work a novel method to quantify spectral ergodicity for random matrices is presented. The new methodology combines approaches rooted in the metrics of Thirumalai-Mountain (TM) and Kullbach-Leibler (KL) divergence. The method is applied to a general study of deep and recurrent neural networks via the analysis of…

Randomized gradient-based ensemble improves prediction accuracy.

HD algorithm simulates dynamics on random matrix ensembles without generating full matrices.

The paper verifies the robustness of classifier ensembles against randomized attacks.

Random matrix ensembles yield uniform distributions on manifolds.

Random Hyperboxes is a simple yet effective ensemble classifier.

Develops an empirical likelihood framework for random forests and ensembles.

Ensembles dynamic models using random feature approximations.

An ensemble of randomized NNs improves time series forecasting accuracy.

Tree ensembles such as random forests and boosted trees are accurate but difficult to understand, debug and deploy. In this work, we provide the inTrees (interpretable trees) framework that extracts, measures, prunes and selects rules from a tree ensemble, and calculates frequent variable interactions. An rule-based le…

Although the methods of bagging and random forests are some of the most widely used prediction methods, relatively little is known about their algorithmic convergence. In particular, there are not many theoretical guarantees for deciding when an ensemble is "large enough" --- so that its accuracy is close to that of an…

BoostForest combines multiple BoostTree models for improved accuracy.

Ensembles of random-feature models can't outperform a single large model.

RAFs Ensemble improves neural network uncertainty quantification.

Consider a binary decision making process where a single machine learning classifier replaces a multitude of humans. We raise questions about the resulting loss of diversity in the decision making process. We study the potential benefits of using random classifier ensembles instead of a single classifier in the context…

K-nearest neighbor (kNN) search has wide applications in many areas, including data mining, machine learning, statistics and many applied domains. Inspired by the success of ensemble methods and the flexibility of tree-based methodology, we propose random projection forests (rpForests), for kNN search. rpForests finds …

We propose a new randomized ensemble technique with a provable security guarantee against black-box transfer attacks. Our proof constructs a new security problem for random binary classifiers which is easier to empirically verify and a reduction from the security of this new model to the security of the ensemble classi…

SPQR improves Q-ensemble diversity in reinforcement learning.

Paper studies ensemble probabilistic regression trees for smooth approximations.

New ensemble SVM model reduces prediction error without choosing best kernel.

SWEEN improves certified robustness via weighted ensembling of smoothed classifiers.

Recently, the binary expansion testing framework was introduced to test the independence of two continuous random variables by utilizing symmetry statistics that are complete sufficient statistics for dependence. We develop a new test based on an ensemble approach that uses the sum of squared symmetry statistics and di…

Sharp threshold found for Frechet mean of inhomogeneous graphs.

Recent studies have revealed the vulnerability of deep neural networks: A small adversarial perturbation that is imperceptible to human can easily make a well-trained deep neural network misclassify. This makes it unsafe to apply neural networks in security-critical applications. In this paper, we propose a new defense…

Random forests reduce bias and variance, especially in low SNR settings.

When randomized ensemble methods such as bagging and random forests are implemented, a basic question arises: Is the ensemble large enough? In particular, the practitioner desires a rigorous guarantee that a given ensemble will perform nearly as well as an ideal infinite ensemble (trained on the same data). The purpose…

We generalize the recently discovered relationship between JT gravity and double-scaled random matrix theory to the case that the boundary theory may have time-reversal symmetry and may have fermions with or without supersymmetry. The matching between variants of JT gravity and matrix ensembles depends on the assumed s…

Overparameterized ensembles don't offer generalization benefits over single large models.

RST improves environmental time series classification accuracy using randomized B-spline trees.

Optimal survival trees ensemble reduces tree count and improves predictive performance.

RaSE ensemble framework improves sparse classification accuracy.

Improved regret bound for linear ensemble sampling.

New matrix ensembles better match deep neural network spectral densities.

The paper develops a stationary-distribution theory for Random Forest ensemble size selection.

Unified framework for cost-sensitive ensemble learning.

Deep ensembles have been empirically shown to be a promising approach for improving accuracy, uncertainty and out-of-distribution robustness of deep learning models. While deep ensembles were theoretically motivated by the bootstrap, non-bootstrap ensembles trained with just random initialization also perform well in p…

When randomized ensembles such as bagging or random forests are used for binary classification, the prediction error of the ensemble tends to decrease and stabilize as the number of classifiers increases. However, the precise relationship between prediction error and ensemble size is unknown in practice. In the standar…

We develop ensemble Convolutional Neural Networks (CNNs) to classify the transportation mode of trip data collected as part of a large-scale smartphone travel survey in Montreal, Canada. Our proposed ensemble library is composed of a series of CNN models with different hyper-parameter values and CNN architectures. In o…

In ensemble methods, the outputs of a collection of diverse classifiers are combined in the expectation that the global prediction be more accurate than the individual ones. Heterogeneous ensembles consist of predictors of different types, which are likely to have different biases. If these biases are complementary, th…

Improved neural network regression uncertainty estimation.

This study shows how social insects and machine learning methods share a common mathematical framework.

Study finds Calabi-Yau models' operator spectra match random matrix theory.

Tree ensembles, such as random forest and boosted trees, are renowned for their high prediction performance, whereas their interpretability is critically limited. In this paper, we propose a post processing method that improves the model interpretability of tree ensembles. After learning a complex tree ensembles in a s…