Recently, the binary expansion testing framework was introduced to test the independence of two continuous random variables by utilizing symmetry statistics that are complete sufficient statistics for dependence. We develop a new test based on an ensemble approach that uses the sum of squared symmetry statistics and di…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “randomized ensemble test”
Random Hyperboxescs.LG
Random Hyperboxes is a simple yet effective ensemble classifier.
problem Improving classification accuracy using ensemble methods.
method Random subsets of sample and feature spaces are used to train individual hyperbox-based classifiers, which are then combined into an ensemble.
result The proposed classifier outperforms other fuzzy min-max neural networks and ensemble methods on 20 datasets.
Ensembles of random-feature models can't outperform a single large model.
problem Finding the optimal balance between model size and ensemble size.
method Deterministic equivalent risk estimates and scaling laws analysis.
result Ensembles of random-feature models achieve near-optimal performance only under specific conditions.
MULTIFIT tests independence between two random vectors using multiscale Fisher's test.
problem Detecting local dependence between two random vectors.
method MULTIFIT uses a resampling-free approach to test independence.
result MULTIFIT can easily handle large sample sizes and interpret dependency nature.
A Random Projection k Nearest Neighbours Ensemble for Classification via Extended Neighbourhood Rulestat.ML
A new ensemble method using random projections for kNN classification.
problem Improving kNN classification accuracy through ensemble methods.
method Random projection of bootstrap samples into lower dimensions, using extended neighbourhood rule for base learners.
result Enhanced classification accuracy compared to traditional kNN and other ensembles.
Study loop corrections in random feature models affecting training and test errors.
problem Analyzing loop corrections in random feature models to understand training and test errors.
method Statistical physics and effective field theory approach to study loop corrections.
result Derived loop corrections to training error, test error, and generalization gap.
A Randomized Permutation Whole-Model Test Heuristic for Self-Validated Ensemble Models (SVEM)stat.ME
A new test validates ensemble models against the null hypothesis.
problem Validating ensemble models against the null hypothesis of a constant response.
method Randomized permutation test on SVEM model predictions.
result The test maintains Type I error rate even with more parameters than observations.
New method tests tree models without causing computational pressure.
problem Incompatibility of adversarial robustness testing with tree ensembles.
method Smooths tree ensembles with sigmoid functions and uses gradient descent.
result Successfully reveals adversarial vulnerability of tree ensemble models.
Study on ensemble, distillation, and self-distillation in deep learning models.
problem Improving test accuracy in deep learning models using ensemble and distillation methods.
method Formal study of ensemble and distillation, considering multi-view data structure.
result Proven that ensemble and distillation can improve test accuracy in deep learning models, and the superior performance can be distilled into a single model.
While statistical learning methods have proved powerful tools for predictive modeling, the black-box nature of the models they produce can severely limit their interpretability and the ability to conduct formal inference. However, the natural structure of ensemble learners like bagged trees and random forests has been …
SPQR improves Q-ensemble diversity in reinforcement learning.
problem Overestimation bias in Q-learning for complex tasks.
method Introduces SPQR for Q-ensemble independence regularization.
result SPQR outperforms baseline algorithms in online and offline RL benchmarks.
Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothersstat.ML
Paper explains how tree ensembles improve predictions by smoothing and regulating smoothness.
problem Understanding why tree ensembles perform well despite their complexity.
method Interpreting tree ensembles as adaptive and self-regularizing smoothers.
result Ensemble trees make more smooth predictions than individual trees and adjust smoothness based on input dissimilarity.
Extends Infinitesimal Jackknife for model covariance, enhancing ensemble model analysis.
problem Estimating uncertainty in combinations of models.
method Extends Infinitesimal Jackknife to estimate covariance between models.
result Theoretical consistency of Infinitesimal Jackknife covariance estimate demonstrated.
Optimal survival trees ensemble reduces tree count and improves predictive performance.
problem Improving predictive performance in survival analysis.
method Grows a forest of optimal survival trees by ranking and selecting the best trees based on out-of-bag error.
result Reduces the number of trees in the ensemble while improving predictive performance.
CreDRO learns credal ensembles via distributionally robust optimization, improving EU quantification.
problem Quantifying predictive epistemic uncertainty in credal models.
method Distributionally robust optimization to capture EU from training randomness and potential distribution shifts.
result Empirically, CreDRO outperforms existing credal methods on various tasks.
A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or "spike") is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughou…
Ensemble learning has had many successes in supervised learning, but it has been rare in unsupervised learning and dimensionality reduction. This study explores dimensionality reduction ensembles, using principal component analysis and manifold learning techniques to capture linear, nonlinear, local, and global feature…
Ensemble learning is a powerful paradigm that has been usedin the top state-of-the-art machine learning methods like Random Forestsand XGBoost. Inspired by the success of such methods, we have devel-oped a new Genetic Programming method called Ensemble GP. The evo-lutionary cycle of Ensemble GP follows the same steps a…
HRF enhances tree diversity in random forests to improve performance.
problem Selection bias and lack of diversity in random forests.
method Introducing heterogeneity during tree construction by assigning lower weights to features used in previous trees.
result HRF outperforms other ensemble methods in accuracy across 52 datasets.
The continually increasing number of complex datasets each year necessitates ever improving machine learning methods for robust and accurate categorization of these data. This paper introduces Random Multimodel Deep Learning (RMDL): a new ensemble, deep learning approach for classification. Deep learning models have ac…
The paper analyzes fluctuations in ensemble models in high-dimensional settings.
problem Understanding statistical fluctuations in ensemble models in high-dimensional settings.
method Develops a rigorous theory for the study of fluctuations in ensemble of generalised linear models.
result Provides a complete description of the asymptotic joint distribution of the empirical risk minimizer for convex losses in high-dimensional settings.
Enhances robustness for time series classification using self-ensemble method.
problem Limited adversarial robustness in time series classification.
method Proposes a self-ensemble method to improve Randomized Smoothing's robustness certification.
result Demonstrates superior robustness compared to baseline approaches.
Ensembles of randomized decision trees, usually referred to as random forests, are widely used for classification and regression tasks in machine learning and statistics. Random forests achieve competitive predictive performance and are computationally efficient to train and test, making them excellent candidates for r…
In complex systems, crucial parameters are often subject to unpredictable changes in time. Climate, biological evolution and networks provide numerous examples for such non-stationarities. In many cases, improved statistical models are urgently called for. In a general setting, we study systems of correlated quantities…
This paper examines the use of a residual bootstrap for bias correction in machine learning regression methods. Accounting for bias is an important obstacle in recent efforts to develop statistical inference for machine learning methods. We demonstrate empirically that the proposed bootstrap bias correction can lead to…
A new ensemble method improves kNN performance by extending the neighborhood rule.
problem Traditional kNN's limitations when test points are outside the spherical region and ensemble's high errors.
method Determines neighbors in k steps, using bootstrap samples and optimal models selection.
result The proposed ensemble method outperforms state-of-the-art methods on 17 benchmark datasets.
Tree ensembles are flexible predictive models that can capture relevant variables and to some extent their interactions in a compact and interpretable manner. Most algorithms for obtaining tree ensembles are based on versions of boosting or Random Forest. Previous work showed that boosting algorithms exhibit a cyclic b…
DASH improves ensemble generalizability by encouraging diverse, flat loss landscapes.
problem Improving generalization and robustness of deep ensembles.
method DASH promotes diversity and flatness in deep ensembles by encouraging base learners to move towards low-loss regions of minimal sharpness.
result DASH improves ensemble generalizability, as demonstrated by extensive empirical evidence.
Decentralized Gaussian processes for multi-agent systems.
problem Scalable and flexible learning solutions for multi-agent systems.
method Asymptotically exact decentralized solution to Gaussian processes, with online Bayesian model averaging for hyperparameter selection.
result Asymptotically exact decentralized Gaussian process approximation and online Bayesian model averaging.
ECV method optimizes ensemble parameters for randomized ensembles.
problem Efficient tuning of ensemble parameters in randomized ensembles.
method ECV (Extrapolated Cross-Validation) method for tuning ensemble and subsample sizes.
result ECV yields δ-optimal ensembles for squared prediction risk.
New hyperparameter ensembles boost neural network performance and uncertainty.
problem Improving neural network robustness and uncertainty quantification.
method Designing ensembles over both weights and hyperparameters, stratified across random initializations.
result Hyper-deep and hyper-batch ensembles outperform deep and batch ensembles on various architectures.
This work develops formal statistical inference procedures for machine learning ensemble methods. Ensemble methods based on bootstrapping, such as bagging and random forests, have improved the predictive accuracy of individual trees, but fail to provide a framework in which distributional results can be easily determin…
Study compares single vs ensemble feature selection for cancer diagnosis.
problem Identifying relevant variables for cancer diagnosis and prognosis.
method Comparison of single feature selection algorithms and ensemble of diverse algorithms.
result Ensemble approach did not improve predictive performance over individual algorithms.
Develops RKHS framework for analyzing tree ensembles.
problem Analyzing the theoretical properties of tree ensembles.
method Reproducing Kernel Hilbert Spaces (RKHS) for tree ensembles.
result Characterizes Random Forest predictor as minimizer of a penalized empirical risk functional in RKHS.
This paper analyzes how randomizing rewards in MBRL can improve performance without being overly optimistic.
problem The gap between theoretical worst-case regret analysis and empirical performance in MBRL.
method Reward randomization in model-based reinforcement learning (MBRL) with kernelized linear regulator (KNR) model.
result Reward randomization guarantees partial optimism and near-optimal worst-case regret.
Extends spectral number variance convergence to random matrix ensembles for twisted Laplacians.
problem Spectral number variance convergence for twisted Laplacians and Dirac operators.
method Extends Rudnick's approach to Gaussian ensembles for twisted Laplacians and Dirac operators.
result Convergence to Gaussian ensembles for twisted Laplacians and Dirac operators.
In this work a novel method to quantify spectral ergodicity for random matrices is presented. The new methodology combines approaches rooted in the metrics of Thirumalai-Mountain (TM) and Kullbach-Leibler (KL) divergence. The method is applied to a general study of deep and recurrent neural networks via the analysis of…
Randomized gradient-based ensemble improves prediction accuracy.
problem Improving prediction accuracy in machine learning.
method Randomization and gradient-based aggregation of weakly-correlated estimators.
result The method outperforms existing techniques in terms of increased accuracy.
HD algorithm simulates dynamics on random matrix ensembles without generating full matrices.
problem Simulating dynamics on dense random matrix ensembles with high space and time complexity.
method Householder reflectors for adaptive and recursive construction, deferring decisions.
result Significant reductions in runtime and memory footprint for practical .
The paper verifies the robustness of classifier ensembles against randomized attacks.
problem Ensuring classifier ensembles are robust against arbitrary randomized attacks.
method Formal verification procedure using SMT and MILP encodings to assess robustness.
result Proves the NP-hardness of the robustness-checking problem and provides upper bounds on attack sets.
Eigen, singular, cosine-sine, and Autonne--Takagi vectors distributions of random matrix ensemblesmath.PR
Random matrix ensembles yield uniform distributions on manifolds.
problem Understanding distributions of vectors in random matrix ensembles.
method Analyzing eigenvalues, singular values, and Autonne-Takagi vectors of various random matrix ensembles.
result Uniform distributions on specific manifolds for different types of random matrix ensembles.
Paper uses Random Matrix Theory for optimal training-testing data split.
problem Finding ideal training-testing data split for linear regression.
method Random Matrix Theory applied to Gaussian multivariate data.
result Ideal training and test sizes derived for any model.
FastForest boosts Random Forest speed by 24%.
problem Efficiency in processing speed for Random Forest.
method Subsample Aggregating, Logarithmic Split-Point Sampling, Dynamic Restricted Subspacing.
result Average 24% increase in processing speed with accuracy maintained.
Develops an empirical likelihood framework for random forests and ensembles.
problem Quantifying the statistical uncertainty of random forests and ensembles.
method Empirical likelihood framework exploiting the incomplete -statistic structure of ensemble predictions.
result Modified empirical likelihood statistic achieves accurate coverage and practical reliability.
Ensembles dynamic models using random feature approximations.
problem Online scalable Bayesian learning with dynamic models and ensembling.
method Random feature approximations and dynamic models using random walks.
result Better performance with alternative basis expansions like Hilbert space Gaussian processes.
Survey of machine learning methods for time series forecasting.
problem Improving accuracy of time series predictions.
method Linear and nonlinear machine learning models, including neural networks and ensemble methods.
result Demonstrates superior predictive ability of certain machine learning models.
Novel analysis improves weighted majority vote in multiclass classification.
problem Improving the performance of weighted majority vote in multiclass classification.
method Analyzes expected risk of weighted majority vote, considering prediction correlations and provides a bound for efficient minimization.
result Minimization of the new bound typically does not degrade the test error of the ensemble.
An ensemble of randomized NNs improves time series forecasting accuracy.
problem Forecasting time series with multiple seasonality and nonstationarity.
method Randomized neural networks with pattern-based time series representation and diversity control strategies.
result Outperforms statistical and machine learning models in forecasting accuracy.