Variable selection in sparse regression models is an important task as applications ranging from biomedical research to econometrics have shown. Especially for higher dimensional regression problems, for which the link function between response and covariates cannot be directly detected, the selection of informative va…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Permutation Importance Measure”
A new method reduces computational costs for testing RF variable importance measures.
problem Testing variable importance measures from random forests is computationally expensive and challenging.
method Sequential permutation testing and sequential p-value estimation to reduce computational costs.
result Theoretical properties of sequential tests are confirmed, maintaining type-I error and high power.
Paper proves a Central Limit Theorem for Random Forest Permutation Importance Measure.
problem Lack of theoretical analysis of Random Forest Permutation Importance Measure (RFPIM).
method Formal proof using U-Statistics theory, deviating from conventional Random Forest model.
result Established a Central Limit Theorem for RFPIM.
The paper derives theoretical foundations for two common machine learning variable importance measures.
problem Understanding variable importance in machine learning problems.
method The paper derives closed-form expressions for Permute-and-Predict (PaP) and Leave-One-Covariate-Out (LOCO) methods.
result Theoretical derivations explain the behavior of PaP and LOCO under collinearity, linking them to coefficients and predictor variability.
A new method improves feature importance and model stress-testing reliability.
problem Estimating feature contributions in machine learning models for trust and transparency.
method Replacing multiple random permutations with a single, deterministic, and optimal permutation.
result Improved bias-variance tradeoffs and accuracy in challenging scenarios.
Two new methods assess feature importance for fairness in machine learning models.
problem Understanding how features influence fairness in machine learning models.
method Two model-agnostic approaches: permutation and occlusion.
result Simple, scalable, and interpretable methods to quantify feature importance for fairness.
Paper introduces new importance metrics for machine learning models, linking them to CATE.
problem Interpreting black-box models' importance metrics due to data dependence and non-parametric nature.
method Introduces MVIM and CVIM, proposing permutation-based estimation and bias-variance decomposition.
result MVIM and CVIM have a quadratic relationship with CATE, addressing bias in correlated predictors.
Proposes a simple solution to Gini importance bias in random forests.
problem Gini importance measure in random forests is biased and unreliable.
method Computes loss reduction on out-of-bag samples instead of in-bag.
result Solves the misleading/untrustworthy Gini importance issue.
CPI overcomes limitations of permutation importance by providing accurate variable selection.
problem Misidentification of unimportant variables in complex models due to covariate correlations.
method Developed a model agnostic and computationally lean Conditional Permutation Importance (CPI) approach.
result CPI provides accurate type-I error control and more parsimonious variable selection.
New methods reduce extrapolation errors in feature importance.
problem Flawed feature importance methods using unrestricted permutations lead to extrapolation errors.
method Three new approaches: conditional model reliance, Knockoffs with Gaussian transformation, and restricted ALE plot designs.
result Theoretical and numerical results show our strategies reduce/eliminate extrapolation.
TRIP detects unreliable feature importance scores in random forests.
problem Unreliable feature importance scores in random forests due to model extrapolation.
method Develops TRIP (Test for Reliable Interpretation via Permutation) to detect unreliable permutation feature importance scores.
result TRIP reliably detects unreliable permutation feature importance scores in high-dimensional settings.
Interpretation of machine learning predictions for patient outcomes in electronic health recordscs.CY
Electronic health records are an increasingly important resource for understanding the interactions between patient health, environment, and clinical decisions. In this paper we report an empirical study of predictive modeling of several patient outcomes using three state-of-the-art machine learning methods. Our primar…
Proposes a new method for variable importance using targeted learning.
problem Uncertainty quantification in variable importance metrics.
method Employing the targeted learning (TL) framework for conditional permutation variable importance.
result Improved accuracy in finite sample contexts compared to traditional methods.
This paper reviews and advocates against the use of permute-and-predict (PaP) methods for interpreting black box functions. Methods such as the variable importance measures proposed for random forests, partial dependence plots, and individual conditional expectation plots remain popular because they are both model-agno…
Study examines challenges in variable importance ranking due to feature correlation.
problem Challenges in variable importance ranking under correlation.
method Simulation study and theoretical analysis of feature knockoffs and conditional predictive impact (CPI).
result Highly correlated features increase the correlation of knockoff variables, posing a limitation for CPI.
New method corrects correlation bias in feature importance.
problem Correlations between features in high-dimensional data bias statistical and machine learning models.
method Pairwise Permutation Algorithm (PPA)
result PPA corrects correlation bias, identifying biological relevant biomarkers.
Improves full conformal prediction for stochastic non-conformity measures.
problem Inability of existing conditions to guarantee full conformal prediction validity under stochastic settings.
method Introduces a new sufficient condition: Conditional Independence & Permutation Invariance in Distribution.
result Corrects the insufficient condition and provides a new sufficient condition for full conformal prediction validity.
Proposes CLIQUE for improved local variable importance in multi-class classification.
problem Lack of methods to characterize local structure in model loss space.
method CLIQUE (Conditional Local Importance by Quantile Expectations)
result CLIQUE emphasizes locally dependent information and captures interaction behavior.
Permutation invariant network learns Wasserstein metrics.
problem Understanding the space of probability measures and comparing distributions.
method Permutation invariant network mapping samples to a low-dimensional space.
result Network can generalize to compute distances between unseen densities and learn moments.
A new method resolves permutation issues in shuffled linear regression for large-scale applications.
problem Estimating latent features through linear transformation with unknown permutations.
method Spectral matching method to align spectral components of measurement and feature covariances.
result Achieves accurate estimates in shuffled LS and LASSO settings with sufficient samples.
Graph neural networks improve systemic risk measures for financial networks.
problem Computing systemic risk measures for graph-structured financial networks.
method Extended permutation equivariant neural networks (X-PENNs) for numerical approximation.
result Graph neural networks outperform other methods in approximating optimal allocations.
Unlabeled sensing is a linear inverse problem where the measurements are scrambled under an unknown permutation leading to loss of correspondence between the measurements and the rows of the sensing matrix. Motivated by practical tasks such as mobile sensor networks, target tracking and the pose and correspondence esti…
TIME explains temporal models by analyzing feature importance.
problem Existing methods struggle with temporal models and feature importance.
method Model-agnostic permutation-based approach, temporal feature importance, hypothesis testing.
result TIME provides statistical rigor for explaining temporal models.
Proposes Population Difference Criterion for visually observed subpopulation differences.
problem Statistical significance of visually observed subpopulation differences in high-dimensional and high-signal contexts.
method Balanced permutation approach and bootstrap confidence interval for quantifying uncertainty.
result Balanced permutation approach is more powerful in high-signal contexts.
In "Unlabeled Sensing", one observes a set of linear measurements of an underlying signal with incomplete or missing information about their ordering, which can be modeled in terms of an unknown permutation. Previous work on the case of a single noisy measurement vector has exposed two main challenges: 1) a high requir…
New link topology connects permutation discrepancies to Diaconis-Graham inequalities.
problem Characterize permutations for which Diaconis-Graham inequalities hold with equality.
method Relate permutation discrepancies to the Euler characteristic of their associated links.
result Permutation discrepancies are directly related to the Euler characteristic of their associated links.
Improves modeling of sets with permutation invariant densities.
problem Challenges in calculating trace limit practicality of current methods.
method Proposes an alternative approach to define permutation equivariant transformations with closed form trace.
result Improves both training and final performance.
New tests detect high-order interactions without permutations.
problem Scalability issues in kernel-based tests for high-order interactions.
method Permutation-free high-order tests using V-statistics and cross-centring.
result Tests yield standard normal distribution under null hypothesis.
Bayesian optimization method for permutations accelerates combinatorial search.
problem Optimizing expensive-to-evaluate objectives on permutation problems.
method LAW2ORDER, a batch Bayesian optimization method based on the acquisition weighted kernel.
result LAW2ORDER achieves sublinear batch cumulative regret, demonstrating accelerated search.
The paper examines properties of GW optimal transport plans, showing they can be sparse and permutation-supported.
problem Properties of Gromov-Wasserstein optimal transport plans.
method Exploration of sparsity, permutation support, and cyclical monotonicity properties.
result GW optimal plans can be sparse and permutation-supported under certain conditions.
Sample efficiency and scalability to a large number of agents are two important goals for multi-agent reinforcement learning systems. Recent works got us closer to those goals, addressing non-stationarity of the environment from a single agent's perspective by utilizing a deep net critic which depends on all observatio…
Relative Feature Importancestat.ML
Introduces RFI for assessing feature importance relative to any subset of features.
problem Lack of nuanced feature importance computation.
method Generalizes PFI and CFI to assess relative feature importance.
result Derives general interpretation rules for RFI.
Neural networks learn from ensemble forecasts without considering their order.
problem Improving reliability of probabilistic weather forecasts.
method Permutation-invariant neural networks for postprocessing ensemble forecasts.
result Models achieve state-of-the-art prediction quality in surface temperature and wind gust forecasts.
New metric captures individual neuron tuning across neural networks.
problem Need a metric that respects individual neuron tuning across different neural networks.
method Derived a 'soft' permutation-based metric using optimal transport theory.
result Metric avoids counter-intuitive outcomes and captures geometric insights.
Permutation-equivariant neural networks improve auction mechanisms by reducing regret and sample complexity.
problem Designing optimal auction mechanisms that balance revenue and bidders' regret.
method Introduced permutation-equivariant neural networks to auction mechanisms.
result Permutation-equivariant neural networks decrease expected ex-post regret and improve model generalizability.
This paper evaluates and improves metrics for identifying important features in machine learning models.
problem Evaluation metrics for explainable AI are limited by multicollinearity and model accuracy.
method Proposes Expected Accuracy Interval (EAI) to predict model accuracy with multicollinearity.
result EAI is a useful metric for identifying important features in models with multicollinearity.
Permutation-equivariant and Proximity-aware Graph Neural Networks with Stochastic Message Passingcs.LG
SMP model preserves proximity and permutation in graph neural networks.
problem Challenges in graph mining, such as community and leader finding.
method Stochastic Message Passing (SMP) model that maintains proximity and permutation-equivariance.
result SMP model effectively preserves node proximities and permutation-equivariance.
Researchers formalize PD and PFI to relate them to data generating process.
problem Lack of theory linking PD and PFI to data generating process.
method Formalize PD and PFI as estimators of ground truth estimands, account for model variance with learner-PD and learner-PFI.
result PD and PFI estimates deviate from ground truth due to statistical biases, model variance, and Monte Carlo approximation errors.
New sampling methods improve Shapley value estimation for machine learning models.
problem Approximating Shapley values for non-trivial models is computationally challenging.
method Investigates new quadrature techniques and quasi-Monte Carlo methods for permutation sampling.
result Significant improvements in Shapley value estimates over existing methods.
C-MinHash reduces the number of permutations needed for MinHash from thousands to just two.
problem Approximating Jaccard similarity in large binary datasets using many permutations.
method Initial permutation followed by circulant shifting of a second permutation to generate hashes.
result C-MinHash achieves unbiased Jaccard similarity estimation with uniformly smaller variance.
Resolving Schwartz's quadratic meander number conjecture
problem Meander number of cyclic permutations
method Constructing families of cyclic permutations
result Meander number is bounded above and below quadratically in n
VarPro selects features without model dependence, achieving balanced performance.
problem Finding a small set of features with high explanatory power.
method Rule-based variable priority approach, avoiding model-specific methods and artificial data.
result VarPro has a consistent filtering property for noise variables and achieves balanced performance.
Surrogate Monte Carloq-fin.CP
Proposes a simple algorithm to generate data similar to real series.
problem Generating data similar to real series with constraints.
method Random permutation of Monte Carlo generated numbers, accepted if objective function is minimized.
result Demonstrated by generating S\&P 500 log-returns.
Enhances functional classifier performance with new tree-based methods and unbiased feature importance assessment.
problem Challenges of high-dimensional functional data and biased feature importance assessment.
method Augmented functional classification trees and random forests with ad-hoc conditional permutations for unbiased feature importance.
result Significant enhancement in predictive power of functional classifiers through new feature importance assessment.
New method for accurate permutation inference in CCA.
problem Inaccurate permutation inference in CCA.
method Proposed solutions for permutation inference in CCA, including transforming residuals and stepwise estimation.
result Valid permutation tests for CCA with and without nuisance variables.
This study compares feature importance and explainability in quantum vs classical ML models.
problem Lack of transparency in ML models, especially in sensitive fields.
method Comparison of classical ML (SVM, Random Forest) and hybrid quantum ML (VQC, QSVC) models using feature importance and explainability methods.
result Quantum ML models provide insights similar to classical models but with unique quantum features.
Given two distinct datasets, an important question is if they have arisen from the the same data generating function or alternatively how their data generating functions diverge from one another. In this paper, we introduce an approach for measuring the distance between two datasets with high dimensionality using varia…
A new nonparametric test measures dependence between variables using decision trees.
problem Measuring statistical dependence between two variables robustly and efficiently.
method An ensemble of decision trees discriminates between observed and permuted samples without generating the latter.
result The method effectively detects complex relationships from noisy data.