arXiv research

Paper proposes a statistical test for feature selection pipelines using selective inference.

The paper introduces a statistical distance matrix for better feature representation and clustering.

Laplace kernel feature selection offers statistical guarantees for nonparametric models with few samples.

New test assesses reliability of auto-generated features.

iLOCO measures feature interactions without assumptions, providing statistical inference.

The goal of feature selection is to identify important features that are relevant to explain an outcome variable. Most of the work in this domain has focused on identifying globally relevant features, which are features that are related to the outcome using evidence across the entire dataset. We study a more fine-grain…

An important problem in machine learning and statistics is to identify features that causally affect the outcome. This is often impossible to do from purely observational data, and a natural relaxation is to identify features that are correlated with the outcome even conditioned on all other observed features. For exam…

A new framework learns system design using neural features in function space.

Paper introduces PTL-SI for statistical inference in TL-HDR, controlling FPR.

We present an Automatic Relevance Determination prior Bayesian Neural Network(BNN-ARD) weight l2-norm measure as a feature importance statistic for the model-x knockoff filter. We show on both simulated data and the Norwegian wind farm dataset that the proposed feature importance statistic yields statistically signific…

We develop a theory of higher-order feature attribution for complex models.

Finding statistically significant high-order interaction features in predictive modeling is important but challenging task. The difficulty lies in the fact that, for a recent applications with high-dimensional covariates, the number of possible high-order interaction features would be extremely large. Identifying stati…

The Ising model replicates financial asset statistical features.

Two statistical tasks are shown to have equivalent sample complexity.

Spofe bridges statistical rigor and interpretability in feature extraction from tabular data.

AICO tests feature significance in machine learning models.

This work analyzes tree-based methods from a ranking perspective, providing insights and new statistics.

Approximate Bayesian computation is an established and popular method for likelihood-free inference with applications in many disciplines. The effectiveness of the method depends critically on the availability of well performing summary statistics. Summary statistic selection relies heavily on domain knowledge and care…

Kernel methods are powerful learning methodologies that allow to perform non-linear data analysis. Despite their popularity, they suffer from poor scalability in big data scenarios. Various approximation methods, including random feature approximation, have been proposed to alleviate the problem. However, the statistic…

Unsupervised domain adaptation methods aim to alleviate performance degradation caused by domain-shift by learning domain-invariant representations. Existing deep domain adaptation methods focus on holistic feature alignment by matching source and target holistic feature distributions, without considering local feature…

A novel feature selection method using noise-based hypothesis testing improves feature selection accuracy.

Paper extracts features from time series to improve forecasting accuracy.

Paper tackles class-incremental time series classification with dual-stream feature extraction.

Proposes $φ$-table for statistical SHAP explanations in regression models.

New method distinguishes feature relevance in non-linear contexts.

Identifying statistical dependence between the features and the label is a fundamental problem in supervised learning. This paper presents a framework for estimating dependence between numerical features and a categorical label using generalized Gini distance, an energy distance in reproducing kernel Hilbert spaces (RK…

Study shows topological features improve time series classification.

An algorithm reduces breast cancer detection data complexity using effect sizes.

Feature selection with high-dimensional data and a very small proportion of relevant features poses a severe challenge to standard statistical methods. We have developed a new approach (HARVEST) that is straightforward to apply, albeit somewhat computer-intensive. This algorithm can be used to pre-screen a large number…

VEST automates feature engineering for time series forecasting.

Detects which features have shifted in data distributions.

We study the relationship between social media output and National Football League (NFL) games, using a dataset containing messages from Twitter and NFL game statistics. Specifically, we consider tweets pertaining to specific teams and games in the NFL season and use them alongside statistical game data to build predic…

Extraneous variables are variables that are irrelevant for a certain task, but heavily affect the distribution of the available data. In this work, we show that the presence of such variables can degrade the performance of deep-learning models. We study three datasets where there is a strong influence of known extraneo…

DiffKnock improves feature selection in neural networks with complex dependencies and non-linear associations.

Method identifies causal drivers from background features.

We study the generalization properties of ridge regression with random features in the statistical learning framework. We show for the first time that $O(1/\sqrt{n})$ learning bounds can be achieved with only $O(\sqrt{n}\log n)$ random features rather than $O({n})$ as suggested by previous results. Further, we prove fa…

The paper proposes a method to test features selected by SeqFS-DA with controlled FPR.

New statistical methods improve explainability of boosting models.

Measuring divergence between two distributions is essential in machine learning and statistics and has various applications including binary classification, change point detection, and two-sample test. Furthermore, in the era of big data, designing divergence measure that is interpretable and can handle high-dimensiona…

The search for higher-order feature interactions that are statistically significantly associated with a class variable is of high relevance in fields such as Genetics or Healthcare, but the combinatorial explosion of the candidate space makes this problem extremely challenging in terms of computational efficiency and p…

Random feature maps are ubiquitous in modern statistical machine learning, where they generalize random projections by means of powerful, yet often difficult to analyze nonlinear operators. In this paper, we leverage the "concentration" phenomenon induced by random matrix theory to perform a spectral analysis on the Gr…

Proposes a framework for automated radiation therapy treatment planning with uncertainty quantification.

Develops methods to analyze feature-outcome associations in subpopulations.

Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifica…

Statistical mechanics explains learning in large neural networks near interpolation.

theft package simplifies feature extraction for time series analysis in R.

Infinite mixture models are commonly used for clustering. One can sample from the posterior of mixture assignments by Monte Carlo methods or find its maximum a posteriori solution by optimization. However, in some problems the posterior is diffuse and it is hard to interpret the sampled partitionings. In this paper, we…

New model optimizes feature alignment, improving statistical and computational efficiency.