Paper proposes a statistical test for feature selection pipelines using selective inference.
problem Assessing the significance of feature selection pipelines in data analysis.
method Selective inference technique applied to feature selection pipelines composed of various algorithms.
result The proposed statistical test controls false positive feature selection probabilities.
Discovering statistically significant patterns from databases is an important challenging problem. The main obstacle of this problem is in the difficulty of taking into account the selection bias, i.e., the bias arising from the fact that patterns are selected from extremely large number of candidates in databases. In …
Laplace kernel feature selection offers statistical guarantees for nonparametric models with few samples.
problem Statistical guarantees for kernel-based feature selection in nonconvex optimization problems.
method Sharp characterization of the gradient of the objective function for Laplace kernel feature selection.
result Model-selection consistency for Laplace kernel-based feature selection in nonparametric settings with n∼logp samples. Statistical guarantees for hyperparameter selection
problem Hyperparameter selection in AI systems
method Learn-then-test framework
result Provable reliability and safety
Paper uses SLT to improve model selection for SHM.
problem Model selection for SHM using data-based systems.
method Utilizes Statistical Learning Theory to rigorously estimate generalisation.
result Incorporating domain knowledge improves model generalisation.
The study explores whether model selection guarantees apply to contextual bandits.
problem Applying model selection guarantees to contextual bandits.
method Investigates whether similar guarantees for model selection in statistical learning can be extended to contextual bandit learning.
result Initial findings suggest that model selection guarantees may not directly apply to contextual bandits.
aLTT selects hyperparameters efficiently with statistical guarantees.
problem Statistical validity and efficiency in hyperparameter selection.
method Sequential data-dependent multiple hypothesis testing with early termination.
result Reduces testing rounds while maintaining statistical validity.
Enhances selective inference for generalized lasso using parametric programming.
problem Low statistical power in selective inference for generalized lasso.
method Parametric programming to compute solution paths and identify model selection events.
result Improves selective inference power and practicality for various problems.
Unified statistical framework for LSTM model selection.
problem Model selection and hyperparameter tuning in LSTM networks is heuristic and computationally expensive.
method Proposes a statistical framework extending classical model selection ideas to LSTM networks.
result Improved performance of the proposed framework demonstrated on biomedical data.
Approximate Bayesian computation is an established and popular method for likelihood-free inference with applications in many disciplines. The effectiveness of the method depends critically on the availability of well performing summary statistics. Summary statistic selection relies heavily on domain knowledge and care…
Private variable selection method controls FDR with simulations showing reasonable power.
problem Performing variable selection with privacy constraints.
method Private knockoff filter using Gaussian and Laplace mechanisms.
result Achieves controlled false discovery rate (FDR) in variable selection.
A new knockoff statistic using conditional prediction function improves variable selection in complex models.
problem Controlling false discovery rate in complex models with nonlinear relationships.
method Introducing a knockoff statistic based on the conditional prediction function for use with machine learning models.
result The CPF statistics provide superior power in detecting prognostic variables over existing knockoff statistics.
Two statistical tasks are shown to have equivalent sample complexity.
problem Determining if a function depends on only a few variables and identifying those variables.
method Proved statistical equivalence of feature selection and junta testing through sample complexity analysis.
result Brute-force algorithm is sample-optimal for both tasks with optimal sample size.
Quantum statistical models with singularities are studied for state estimation and model selection.
problem Understanding statistical properties of quantum singular models.
method Classical singular learning theory extended to quantum state estimation and model selection using algebraic geometrical methods.
result Asymptotically unbiased estimator (QWAIC) for quantum generalization loss constructed.
A method to select validation data from a dataset using statistical criteria.
problem Selecting a validation basis from a full dataset for machine learning model validation.
method Adopting a 'design of experiments' point of view and using statistical criteria, particularly Maximum Mean Discrepancy criteria.
result The 'support points' concept is particularly relevant for selecting validation data.
Paper introduces PTL-SI for statistical inference in TL-HDR, controlling FPR.
problem Quantifying statistical significance in TL-HDR with limited data.
method PTL-SI framework for valid p-values in TL-HDR feature selection. result Valid p-values and controlled FPR in TL-HDR feature selection. Study proposes a statistical testing framework for evaluating clustering pipelines.
problem Quantifying the statistical reliability of clustering results from data analysis pipelines.
method Selective inference-based statistical testing framework for clustering pipelines.
result The proposed test controls the type I error rate and is effective in validating clustering results.
Forward regression is a statistical model selection and estimation procedure which inductively selects covariates that add predictive power into a working statistical regression model. Once a model is selected, unknown regression parameters are estimated by least squares. This paper analyzes forward regression in high-…
Measuring divergence between two distributions is essential in machine learning and statistics and has various applications including binary classification, change point detection, and two-sample test. Furthermore, in the era of big data, designing divergence measure that is interpretable and can handle high-dimensiona…
New method estimates selection event for black-box models.
problem Infeasibility of conditional selective inference due to unavailable selection event.
method Bootstrapping to estimate selection event and conditional distribution.
result Feasibility of conditional selective inference for models without exact selection event.
Two new deterministic offspring selection methods reduce statistical distance in SMC and pMCMC.
problem Improving the performance of resampling in SMC methods.
method Proposes two deterministic offspring selection methods to minimize KL divergence and TV distance.
result Our methods outperform or match state-of-the-art resampling schemes on benchmarks.
SFS-DA method statistically tests FS reliability under domain adaptation.
problem Feature selection reliability under domain adaptation with limited target data.
method Selective Inference framework to control false positive rate and enhance true positive rate.
result SFS-DA method controls FPR below a pre-specified level α (e.g., 0.05) while maximizing true positive rate. Bayesian approach controls FDR in high-dimensional models.
problem High-dimensional variable selection and inference.
method Adapted Mirror Statistic to Bayesian framework for FDR control.
result Effective FDR control without data splitting.
This work analyzes tree-based methods from a ranking perspective, providing insights and new statistics.
problem Understanding the effectiveness of tree-based methods in finite-sample settings, especially symbolic feature selection.
method Local ranking perspective, finite-sample analysis, oracle bounds, posterior contraction results, concordant divergence statistics.
result New insights and statistics for evaluating symbolic feature mappings.
The paper proposes a method to test features selected by SeqFS-DA with controlled FPR.
problem Ensuring reliability of feature selection after domain adaptation in high-dimensional regression.
method Proposes a novel method to test features selected by SeqFS-DA with controlled FPR.
result The proposed method controls FPR below a significance level α (e.g., 0.05) and enhances statistical power. Active learning method for ABC statistics selection reduces expert work and improves posterior estimates.
problem Handling intractable likelihood functions in models with domain knowledge.
method Active learning method for selecting summary statistics in ABC.
result Better posterior estimates than existing methods, especially with limited simulation budget.
We present an Automatic Relevance Determination prior Bayesian Neural Network(BNN-ARD) weight l2-norm measure as a feature importance statistic for the model-x knockoff filter. We show on both simulated data and the Norwegian wind farm dataset that the proposed feature importance statistic yields statistically signific…
R package for multi-objective model selection in statistics.
problem Model selection challenges in statistics, especially for penalized models.
method Multi-objective optimization using Gaussian process-based optimization.
result Identification of hyperparameter values that represent desirable trade-offs.
In the era of big data, analysts usually explore various statistical models or machine learning methods for observed data in order to facilitate scientific discoveries or gain predictive power. Whatever data and fitting procedures are employed, a crucial step is to select the most appropriate model or method from a set…
The most popular approach for analyzing survival data is the Cox regression model. The Cox model may, however, be misspecified, and its proportionality assumption may not always be fulfilled. An alternative approach for survival prediction is random forests for survival outcomes. The standard split criterion for random…
Financial asset markets are sociotechnical systems whose constituent agents are subject to evolutionary pressure as unprofitable agents exit the marketplace and more profitable agents continue to trade assets. Using a population of evolving zero-intelligence agents and a frequent batch auction price-discovery mechanism…
The paper explores effective data selection methods for weakly supervised learning.
problem Efficiently selecting a subset of unlabeled data for weakly supervised learning.
method Using a surrogate model to predict labels and selecting a subset of samples for training.
result Data selection can significantly improve model performance over training on the full dataset.
Proposes a modified Morgan-Pitman test for evaluating variances in machine learning models.
problem Limited ability to account for sampling variability in model selection.
method Enhances the classic Morgan-Pitman test for robustness in non-linear models with heavy-tailed distributions or outliers.
result Demonstrates the test's effectiveness and practical utility in model evaluation and selection.
Study provides selective inference method for latent block models.
problem Challenges in constructing a test on a block structure selected by clustering algorithms.
method Developed a selective inference method for latent block models using squared residue minimization and simulated annealing.
result Proposed tests effectively handle selective bias in block structures compared to naive tests.
Gaussian graphical model is a graphical representation of the dependence structure for a Gaussian random vector. It is recognized as a powerful tool in different applied fields such as bioinformatics, error-control codes, speech language, information retrieval and others. Gaussian graphical model selection is a statist…
We develop a statistical framework to benchmark and select large language models based on their risks.
problem Benchmarking and selecting large language models based on their associated risks.
method A distributional framework using first and second order stochastic dominance, linked to mean-risk models in finance.
result Formalizes a risk-aware approach for model selection, balancing risk and utility.
A novel feature selection method using noise-based hypothesis testing improves feature selection accuracy.
problem Challenges in feature selection for complex, high-dimensional datasets.
method Introduces multiple random noise features and evaluates feature importance against noise feature maxima using non-parametric bootstrap-based hypothesis testing.
result Outperforms existing methods in simulated and real-world datasets.
New method corrects selection bias in complex models.
problem Selection bias in statistical studies leading to systematic distortions.
method Amortized Bayesian inference with neural posterior estimation.
result Recover well-calibrated posterior distributions across diverse selection mechanisms.
A DP method selects best sparse models in high dimensions efficiently.
problem Model selection in high-dimensional sparse linear regression under privacy constraints.
method Differential privacy (DP) with exponential mechanism and Metropolis-Hastings algorithm.
result The method identifies active features quickly under privacy constraints.
Statistical boosting algorithms have triggered a lot of research during the last decade. They combine a powerful machine-learning approach with classical statistical modelling, offering various practical advantages like automated variable selection and implicit regularization of effect estimates. They are extremely fle…
The paper analyzes ℓ1-LinR for Ising model selection using statistical mechanics.
problem Model selection consistency of ℓ1-LinR for Ising models. method Replica method from statistical mechanics, ℓ1-regularized linear regression (ℓ1-LinR). result Model selection consistency with sample complexity $M=\mathcal{O}\left(\log N
ight)$.
Paper evaluates and improves private feature selection methods.
problem Feature selection in high-dimensional datasets with privacy constraints.
method Correlations-based order statistic privatized for feature selection.
result Our method significantly outperforms established baseline for private feature selection.
Bayesian method selects important covariates in modal regression.
problem Bayesian modal regression with heavy-tailed responses.
method Expectation-maximization algorithm for parameter estimation; test statistic for variable selection.
result Efficacy of the proposed method in identifying important covariates.
We develop the necessary theory in computational algebraic geometry to place Bayesian networks into the realm of algebraic statistics. We present an algebra{statistics dictionary focused on statistical modeling. In particular, we link the notion of effiective dimension of a Bayesian network with the notion of algebraic…
A neural network approach unifies Lasso for variable selection.
problem Combining statistical and machine learning techniques for variable selection.
method Representing Lasso through a neural network and developing a new optimization algorithm.
result The new optimization algorithm achieves better performance than previous methods.
Refining one's hypotheses in the light of data is a common scientific practice; however, the dependency on the data introduces selection bias and can lead to specious statistical analysis. An approach for addressing this is via conditioning on the selection procedure to account for how we have used the data to generate…
Extends model-x framework to handle missing data.
problem Inability to control false selections in missing data settings.
method Posterior sampled imputation, univariate imputation, joint imputation and sampling knockoffs.
result Preserves theoretical guarantees of model-x framework in missing data setting.
si4onnx enables selective inference on deep learning models.
problem Establishing the reliability of AI systems through statistical significance of identified regions.
method Selective inference techniques implemented through a Python package.
result Controlled type I error rates for hypothesis testing on deep learning models.