Proposes a new method to find features affecting treatment effect distribution.
problem Existing methods fail to detect differences in treatment effect distribution parameters other than the mean.
method Formulates and estimates a feature importance measure that quantifies feature influence on potential outcome distribution discrepancies. Develops a feature selection algorithm to control type I error rate.
result Successfully discovers important features and outperforms existing mean-based methods.
A new method for feature selection robust to noise and design variability.
problem Feature selection in high-dimensional regression under sampling variability and measurement error.
method Injects controlled additive noise into the design matrix, fits a base selector, and aggregates selection frequencies.
result Improved robustness compared to Stability Selection and standard base selectors.
We propose an iterative scheme for feature-based positioning using a new weighted dissimilarity measure with the goal of reducing the impact of large errors among the measured or modeled features. The weights are computed from the location-dependent standard deviations of the features and stored as part of the referenc…
Supporting evidence for adaptive feature program across diverse models.
problem Analyzing feature learning in neural networks.
method Over-parameterized sequence models and feature error measure (FEM).
result FEM is decreasing during training of adaptive feature models.
We present an autoencoder that leverages learned representations to better measure similarities in data space. By combining a variational autoencoder with a generative adversarial network we can use learned feature representations in the GAN discriminator as basis for the VAE reconstruction objective. Thereby, we repla…
Efficient classifier error estimation without re-training.
problem Estimating classifier error without re-training.
method Generalized resubstitution based on empirical measures.
result Consistent and asymptotically unbiased error estimation.
New method for mixed data FI controls type I error and achieves high power.
problem Statistical inadequacy of feature importance measures for mixed data.
method Combining CPI framework with sequential knockoffs for mixed data.
result Our method controls type I error and achieves high power for mixed data.
There is no known efficient method for selecting k Gaussian features from n which achieve the lowest Bayesian classification error. We show an example of how greedy algorithms faced with this task are led to give results that are not optimal. This motivates us to propose a more robust approach. We present a Branch and …
Sparse feature selection improves batch RL efficiency.
problem High-dimensional batch RL with many features.
method Sparse linear function approximation, Lasso, group Lasso, fitted Q-evaluation, fitted Q-iteration.
result Sparse feature selection makes batch RL more sample efficient.
Random Fourier features model reconstructs wind fields from sparse measurements.
problem Reconstructing wind fields from limited data.
method Random Fourier features approximating velocity field with adaptive sampling.
result Random Fourier features model outperforms benchmarks.
Automated feature selection is important for text categorization to reduce the feature size and to speed up the learning process of classifiers. In this paper, we present a novel and efficient feature selection framework based on the Information Theory, which aims to rank the features with their discriminative capacity…
Method learns feature map between source and target domains for high-dimensional regression with missing features.
problem High-dimensional regression with differing feature sets in target and source domains.
method First learns a feature map between missing and observed features using source data, then imputes missing features in target domain, and performs two-step transfer learning for penalized regression.
result Developed upper bounds on estimation and prediction errors for HTL, showing dependence on model complexity, sample size, feature map quality, and domain differences.
In this paper, we propose a new wrapper feature selection approach with partially labeled training examples where unlabeled observations are pseudo-labeled using the predictions of an initial classifier trained on the labeled training set. The wrapper is composed of a genetic algorithm for proposing new feature subsets…
NGMs create mirrored features to assess neural network feature importance.
problem Lack of feature relevance information in DNNs limits their applicability.
method Structured perturbation and kernel-based conditional dependence measure for feature importance evaluation.
result Controls feature selection error rate and maintains high selection power with correlated features.
Paper introduces a novel measure to analyze excess error in classification under covariate shift.
problem Analyzing excess error in classification under covariate shift.
method Utilizes vicinity information to characterize excess error.
result Faster or competitive convergence rates compared to previous techniques.
The paper analyzes how data augmentation affects the test error in regression models.
problem Understanding the impact of data augmentation on the test error in regression models.
method Characterizes the test error in terms of population quantities and augmentation statistics.
result Provides a tight characterization of the test error in mean squared error.
The paper provides theoretical guarantees for optimized sampling in compressed sensing, showing error vanishes with more measurements.
problem Theoretical and practical improvements in compressed sensing with optimized sampling schemes.
method Theoretical analysis and empirical experiments with optimized sampling schemes for subsampled unitary matrices.
result The error caused by measurement noise vanishes with an increasing number of measurements for optimized sampling schemes, assuming Gaussian noise.
Pattern recognition is a central topic in Learning Theory with numerous applications such as voice and text recognition, image analysis, computer diagnosis. The statistical set-up in classification is the following: we are given an i.i.d. training set (X1,Y1),...(Xn,Yn) where Xi represents a feature…
LatentNN corrects neural network attenuation bias in astronomical data.
problem Neural networks underestimate extreme values due to measurement errors.
method Jointly optimizes network parameters and latent input values.
result LatentNN reduces attenuation bias across various signal-to-noise ratios.
New Fourier features improve high-precision approximation in large-scale problems.
problem Designing scalable, high-precision Fourier features for large-scale kernel methods.
method Introducing a new family of quadrature rules that accurately approximate the Gaussian measure in higher dimensions.
result Improved approximation bounds with new Fourier features.
Identifying statistical dependence between the features and the label is a fundamental problem in supervised learning. This paper presents a framework for estimating dependence between numerical features and a categorical label using generalized Gini distance, an energy distance in reproducing kernel Hilbert spaces (RK…
Develops UKP for comparing feature representations in multitask learning.
problem Comparing feature representations learned by different models without access to test data.
method Uniform Kernel Prober (UKP) for comparing representations in kernel ridge regression tasks.
result UKP provides a uniform measure of prediction error on test data without access to test data.
Motivated by value function estimation in reinforcement learning, we study statistical linear inverse problems, i.e., problems where the coefficients of a linear system to be solved are observed in noise. We consider penalized estimators, where performance is evaluated using a matrix-weighted two-norm of the defect of …
One of the challenging questions in time series forecasting is how to find the best algorithm. In recent years, a recommender system scheme has been developed for time series analysis using a meta-learning approach. This system selects the best forecasting method with consideration of the time series characteristics. I…
Estimates the generalization error of deep neural networks without relying on capacity measures.
problem Understanding how generalization error scales with training data for deep neural networks.
method Derives estimates of generalization error for deep networks based on two assumptions: zero training error and error probability proportional to distance to nearest training point.
result Estimates the generalization error of DNNs as O(1/(δN^{1/d})), matching experimental behavior.
To realize efficient computational fluid dynamics (CFD) prediction of two-phase flow, a multi-scale framework was proposed in this paper by applying a physics-guided data-driven approach. Instrumental to this framework, Feature Similarity Measurement (FSM) technique was developed for error estimation in two-phase flow …
Kernel methods linked to feature subspaces and maximal correlation kernels.
problem Understanding kernel methods and their relationship to feature extraction.
method Established a correspondence between feature subspaces and kernels, introduced maximal correlation kernels, and demonstrated their optimality.
result Kernel SVM on maximal correlation kernel achieves minimum prediction error.
In this paper we measured the stability of stochastic gradient method (SGM) for learning an approximated Fourier primal support vector machine. The stability of an algorithm is considered by measuring the generalization error in terms of the absolute difference between the test and the training error. Our problem is to…
Develops bounds predicting deep learning generalization using optimal transport.
problem Discrepancy between theoretical error bounds and empirical observations in deep learning.
method Margin-based generalization bounds with optimal transport costs.
result Theoretical bounds robustly predict generalization error on large datasets.
This paper extends neural collapse to regression problems, revealing key features and structures.
problem Understanding the structure learned by deep neural networks in regression tasks.
method Established Neural Regression Collapse (NRC) across different models, analyzing feature and weight alignments.
result Deep neural regression models exhibit a collapsed feature space, aligning with target dimensions and covariances.
New method reveals why GNNs perform well on certain datasets.
problem Understanding why GNNs perform differently on similar datasets.
method Deriving exact generalization error for various GNN architectures.
result Benchmark datasets favor architectures that rely on graph structure.
Diamond method controls FDR for trustworthy feature interaction discovery in ML models.
problem Limited interpretability of ML models due to black box nature.
method Diamond method integrates model-X knockoffs framework to control FDR for non-additive interactions.
result Diamond method ensures accurate discovery of feature interactions with FDR control.
Study online linear regression with paid noise reduction.
problem Online linear regression with noisy features and the ability to pay for reduced noise.
method Analyzes regret against optimal predictor, uses matrix martingale concentration.
result Optimal regret rates for known and unknown noise covariance.
Paper proposes a new method to learn features from error representations.
problem Learning from error representations in machine learning.
method Inverse feature learning (IFL) based on deep clustering.
result IFL leads to improved performance in classification and clustering.
Study finds simple model-agreement scores perform well in various error estimation scenarios.
problem Evaluating model performance on unseen distributions using disparate scoring functions.
method Rigorously studied popular scoring functions (confidence, local manifold smoothness, model agreement) independently of mechanism choice.
result Simple model-agreement scores outperform confidence- and smoothness-based scores in realistic settings with compromised training data.
A non-vector-based dissimilarity measure is proposed by combining vector-based distance metrics and set operations. This proposed compound dissimilarity measure (CDM) is applicable to quantify similarity of collections of attribute/feature pairs where not all attributes are present in all collections. This is a typical…
Nonlinear kernels can be approximated using finite-dimensional feature maps for efficient risk minimization. Due to the inherent trade-off between the dimension of the (mapped) feature space and the approximation accuracy, the key problem is to identify promising (explicit) features leading to a satisfactory out-of-sam…
Current system thermal-hydraulic codes have limited credibility in simulating real plant conditions, especially when the geometry and boundary conditions are extrapolated beyond the range of test facilities. This paper proposes a data-driven approach, Feature Similarity Measurement FFSM), to establish a technical basis…
We find a deterministic equivalent for random feature regression's test error, independent of feature map dimension.
problem Understanding the generalization performance of random feature ridge regression.
method We derive a deterministic equivalent for the test error of RFRR under a concentration property, showing it can be approximated by a closed-form expression dependent on feature map eigenvalues.
result Our approximation guarantee is non-asymptotic, multiplicative, and independent of the feature map dimension, providing a tight result for the smallest number of features achieving optimal minimax error rate.
This paper analyzes errors in Shapley value-based model explanations.
problem Biased or unreliable explanations from existing SVA methods.
method Error theoretical analysis framework decomposing errors into observation and structural biases.
result Trade-off between observation and structural biases in SVA explanations.
This paper introduces a new feature learning technique based on error representation.
problem Learning high-level features for classification from diverse and imbalanced data.
method Inverse feature learning using error representation approach.
result Significantly better performance compared to state-of-the-art techniques.
The performance of deep neural networks is often attributed to their automated, task-related feature construction. It remains an open question, though, why this leads to solutions with good generalization, even in cases where the number of parameters is larger than the number of samples. Back in the 90s, Hochreiter and…
Bayesian framework tackles measurement error in covariates.
problem Misleading inference due to corrupted covariates.
method Bayesian Nonparametric Learning framework robust to misspecification.
result General framework for Classical and Berkson error models.
Robust variable selection for high-dimensional data with missing and measurement errors.
problem Missing data and measurement errors confound data distribution.
method Exponential loss function with inverse probability weighting and additive error models.
result The Atan punishment method improves robust variable selection.
Accelerometer measurements are the prime type of sensor information most think of when seeking to measure physical activity. On the market, there are many fitness measuring devices which aim to track calories burned and steps counted through the use of accelerometers. These measurements, though good enough for the aver…
Study loop corrections in random feature models affecting training and test errors.
problem Analyzing loop corrections in random feature models to understand training and test errors.
method Statistical physics and effective field theory approach to study loop corrections.
result Derived loop corrections to training error, test error, and generalization gap.
The study examines methods to correct measurement error in nutritional epidemiology studies.
problem Measurement error in nutritional studies leads to biased and underconfident estimates.
method The article reviews various bias-correction models for exposure variables in nutritional epidemiology.
result Bias-correction methods are essential for accurate inference in nutritional studies.
New stability measures for similar features improve feature selection accuracy.
problem Existing stability measures fail to distinguish similar features in highly correlated datasets.
method Introduce new adjusted stability measures that consider feature similarities.
result One new stability measure considers highly similar features as interchangeable.