Compact bilinear pooling approximates covariance features for faster training.
problem Efficiently approximating covariance features for faster training.
method Compact bilinear pooling extended to polynomial approximations of covariance features.
result The proposed method achieves comparable accuracy with fewer dimensions.
Predict covariance from features using convex optimization.
problem Predicting the covariance of a Gaussian vector from another feature vector.
method A generalized linear model with convex optimization for fitting parameters.
result Predicted covariance matrices are symmetric positive definite.
NeurT-FDR controls FDR by incorporating feature hierarchy.
problem Controlling FDR in complex, large-scale hypothesis testing problems.
method NeurT-FDR uses a neural network to parametrize test-level covariates and a regression framework to adjust feature hierarchy.
result NeurT-FDR makes substantially more discoveries than competitive baselines.
We propose a method for feature selection that employs kernel-based measures of independence to find a subset of covariates that is maximally predictive of the response. Building on past work in kernel dimension reduction, we show how to perform feature selection via a constrained optimization problem involving the tra…
Paper proposes Gini distance statistics for estimating feature-label dependence.
problem Identifying statistical dependence between features and categorical labels.
method Generalized Gini distance in RKHS for feature-label dependence estimation.
result Gini distance statistics converge faster and have tighter error bounds than distance covariance.
We introduce new families of Integral Probability Metrics (IPM) for training Generative Adversarial Networks (GAN). Our IPMs are based on matching statistics of distributions embedded in a finite dimensional feature space. Mean and covariance feature matching IPMs allow for stable training of GANs, which we will call M…
Method uses random forest with distance covariance for transfer learning in healthcare.
problem Transfer learning in random forests with sparse differences between source and target.
method Distance covariance-based feature weights in residual random forest.
result Upper bound on mean square error rate for transfer learning in RF.
New insights into how high-dimensional models handle covariate shifts.
problem Covariate shift in high-dimensional random feature regression.
method Exact high-dimensional asymptotics of random feature regression under covariate shift.
result Overparameterized models exhibit enhanced robustness to covariate shift.
Method estimates sparse inverse covariance and partial correlation matrices efficiently.
problem Sparse high-dimensional inverse covariance and partial correlation matrix estimation.
method Two-stage estimation method using partial regression with positive semi-definiteness.
result Efficient estimation of inverse covariance and partial correlation matrices with derived non-asymptotic rates.
This paper explores how effective sample size, dimensionality, and model performance are related in covariate shift adaptation.
problem Understanding the relationship between effective sample size, dimensionality, and generalization in covariate shift adaptation.
method Building a unified theory connecting effective sample size, data dimensionality, and generalization in the context of covariate shift adaptation.
result Dimensionality reduction or feature selection can increase effective sample size, supporting the practice of reducing dimensionality before covariate shift adaptation.
The paper proposes a method to analyze categorical feature interactions in large datasets using graph covariance and LLMs.
problem Analyzing complex datasets with numerous categorical features and timestamps.
method Binarization of categorical features using one-hot encoding, computation of graph covariance, identifying significant feature pairs, and using LLMs to generate explanations.
result The method identifies meaningful feature pairs and potential data stories underlying categorical feature interactions.
EiGLasso speeds up sparse Kronecker-sum covariance estimation.
problem Sparse Kronecker-sum inverse covariance estimation challenges in scalability and parameter identification.
method Newton's method combined with eigendecomposition of sample and feature graphs, approximating Hessian for speed.
result Two to three orders-of-magnitude speed-up on simulated and real-world data.
Proposes an L1-regularized functional SVM for binary classification with functional covariates.
problem Binary classification with multivariate functional covariates.
method L1-regularized functional support vector machine (SVM) with an accompanying algorithm.
result The proposed classifier performs well in prediction and feature selection.
Sparse models for high-dimensional linear regression and machine learning have received substantial attention over the past two decades. Model selection, or determining which features or covariates are the best explanatory variables, is critical to the interpretability of a learned model. Much of the current literature…
Characterizes test error in learning with deep, structured feature maps.
problem Characterizing test error in learning with deep, structured feature maps.
method Asymptotic analysis of feature covariance and population covariance.
result Closed-form formula for feature covariance in Gaussian rainbow neural networks.
Safe-DRFS selects features robust to covariate shifts for reliable performance.
problem Feature selection fails in diverse deployment environments.
method Safe-DRFS extends safe screening to distributionally robust settings under covariate shift.
result Safe-DRFS identifies a feature subset encompassing optimal subsets across distribution shifts.
Paper compares feature selection methods using GCM and LOCO, showing GCM methods generally outperform LOCO.
problem Feature selection and importance estimation in model-agnostic settings.
method Comparison of feature selection methods related to GCM and LOCO under three model settings.
result GCM-related methods generally outperform LOCO under suitable regularity conditions, as shown by theoretical and empirical results.
Meta learns low-rank covariance factors for better uncertainty estimation.
problem Sub-optimal covariance matrices in multi-task settings.
method Meta learns diagonal or diagonal plus low-rank factors using an attentive set encoder.
result Efficiently constructed task-specific covariance matrices improve uncertainty estimation.
DRSS method identifies unnecessary samples and features in DR covariate shift.
problem Identifying unnecessary samples and features in DR covariate shift.
method Combines DR learning and safe screening techniques.
result DRSS method provides reliable identification of unnecessary samples and features under specified distribution uncertainty.
Covariate shift relaxes the widely-employed independent and identically distributed (IID) assumption by allowing different training and testing input distributions. Unfortunately, common methods for addressing covariate shift by trying to remove the bias between training and testing distributions using importance weigh…
New asymmetric kernel methods improve feature learning.
problem Improving feature learning with asymmetric kernels.
method Coupled covariance eigenproblem and Nyström method.
result Empirical evaluations show benefits of KSVD.
DPERC efficiently estimates covariance matrices for mixed data with missing values.
problem Estimating covariance matrices for datasets with missing values and mixed features.
method Direct Parameter Estimation for Randomly Missing Data with Categorical Features (DPERC).
result DPERC outperforms other methods in estimating covariance matrices for mixed data with missing values.
Adaptive classifier optimizes high-dimensional data with spiked covariance structure.
problem Classification of high-dimensional data with spiked covariance structure.
method Adaptive classifier that whitens data, screens features, and applies Fisher linear discriminant.
result The classifier is Bayes optimal under certain conditions and performs well on real and synthetic data.
New model optimizes feature alignment, improving statistical and computational efficiency.
problem Statistical and computational challenges in feature alignment.
method Covariance alignment model, Gromov-Wasserstein algorithm.
result Gromov-Wasserstein algorithm is minimax optimal and practical for covariance alignment.
New methods test correlation between network structure and node features.
problem Assessing correlation between network structure and node-level covariates.
method Four novel methods based on linear models and canonical correlation analysis.
result Theoretical guarantees and computational efficiency for testing network dependency.
Study addresses covariate mismatch in federated learning, improving model accuracy.
problem Learning from clients with different feature sets in federated learning.
method Developed two approaches for linear prediction under covariate mismatch: plug-in estimator and impute-then-regress strategy.
result Proposed methods provide asymptotic and finite-sample learning rates, improving model accuracy.
New tree and forest methods use oblique splits for better risk bounds.
problem Improving risk bounds for regression algorithms.
method Randomized decision trees and forests with oblique splits.
result Oblique splits lead to better risk bounds for multi-index models.
Compared with global average pooling in existing deep convolutional neural networks (CNNs), global covariance pooling can capture richer statistics of deep features, having potential for improving representation and generalization abilities of deep CNNs. However, integration of global covariance pooling into deep CNNs …
Covariance in physics and CNNs share similarities, with simple assumptions uniquely determining convolution forms.
problem Understanding the similarities between physics and CNNs using covariance.
method Examined similarities and differences, and demonstrated that simple assumptions lead to unique convolution forms.
result Simple assumptions of covariance, locality, linearity, and weight sharing uniquely determine convolution forms.
Efficient knockoffs for large-scale feature selection.
problem Large-scale feature selection problems.
method Gaussian model-X knockoffs with efficient methods for solving semidefinite programs.
result Efficient knockoffs can be generated with linear complexity in the dimension.
Inference and Estimation in Missing Information (MI) scenarios are important topics in Statistical Learning Theory and Machine Learning (ML). In ML literature, attempts have been made to enhance prediction through precise feature selection methods. In sparse linear models, LASSO is well-known in extracting the desired …
Many features of dimensional reduction schemes are determined by the breaking of higher dimensional general covariance associated with the selection of a particular subset of coordinates. By investigating residual covariance we introduce lower dimensional tensors --generalizing to one side Kaluza-Klein gauge fields and…
The paper proves a new method to improve generalization in covariate-shift scenarios.
problem Improving performance on test distributions that differ from training distributions.
method Independence-driven importance weighting algorithms for feature selection.
result Theoretical proof that these algorithms can identify optimal variables for covariate-shift generalization.
The paper proves asymptotic normality for multinomial logistic regression on null covariates.
problem Classical asymptotic normality results fail in high-dimensional multinomial logistic models.
method Developed asymptotic normality and chi-square results for multinomial logistic MLE on null covariates.
result Validated new methodology to test feature significance in high-dimensional classification problems.
New algorithm reduces sample complexity for sparse linear regression.
problem Sparse linear regression with correlated covariates and approximate dependencies.
method Polynomial-time algorithm that adapts the Lasso to tolerate approximate dependencies.
result Achieves near-optimal sample complexity for constant sparsity and ill-conditioned covariates.
Current methods for covariate-shift adaptation use unlabelled data to compute importance weights or domain-invariant features, while the final model is trained on labelled data only. Here, we consider a particular case of covariate shift which allows us also to learn from unlabelled data, that is, combining adaptation …
Two new regularization methods improve neural network performance and complexity control.
problem Improving neural network performance and complexity control with correlated or high-dimensional features.
method Two regularization strategies: covariance-aware ridge and covariance-aware lasso.
result Improves predictive performance and complexity control over standard penalties.
The interpretation of complex high-dimensional data typically requires the use of dimensionality reduction techniques to extract explanatory low-dimensional representations. However, in many real-world problems these representations may not be sufficient to aid interpretation on their own, and it would be desirable to …
GANs mode collapse solved with Bures distance.
problem GANs mode collapse or mode dropping.
method Use Bures distance to match real and fake batch diversity in feature space.
result Diversity matching reduces mode collapse and improves sample quality.
Taking into account high-order interactions among covariates is valuable in many practical regression problems. This is, however, computationally challenging task because the number of high-order interaction features to be considered would be extremely large unless the number of covariates is sufficiently small. In thi…
DFI maps covariates to latent representations for feature importance.
problem Feature importance when predictors are statistically dependent.
method Disentangled Feature Importance (DFI) using entropic optimal transport.
result DFI yields stable, interpretable, uncertainty-quantified attributions of shared predictive signal.
A new classification rule for FDA improves classification performance by accounting for unequal covariance matrices.
problem Unequal covariance matrices in practical situations affect the performance of FDA and its variants.
method Proposes a novel classification rule for FDA that accounts for unequal covariance matrices, applicable to many FDA variants.
result The new classification rule improves classification performance compared to original FDA and variants.
A new clustering method handles uncertain covariates efficiently.
problem Clustering with uncertain covariates in datasets.
method Greedy and optimistic clustering algorithm using non-linear transformation and empirical uncertainty sets.
result Improved performance in finding sibling stars.
NeurT-FDR controls FDR by incorporating auxiliary covariates in deep learning.
problem Controlling FDR in complex large-scale problems with indirect relations among covariates.
method NeurT-FDR uses a deep Black-Box framework that parametrizes test-level covariates as a neural network and adjusts auxiliary covariates through a regression framework.
result NeurT-FDR makes substantially more discoveries in real datasets compared to competitive baselines.
This work explores how feature decorrelation improves self-supervised learning.
problem Complete and dimensional collapse issues in self-supervised learning.
method Study of a concise framework and connection to feature decorrelation.
result Feature decorrelation improves self-supervised learning.
The paper compares LOCO and Shapley values for feature importance, highlighting their limitations and suggesting improvements.
problem Quantifying feature importance in the presence of feature correlation.
method LOCO and Shapley Values, critiquing their axioms and proposing new measures.
result Shapley values do not eliminate feature correlation, and a modified LOCO is recommended.
The expressive power of Gaussian processes depends heavily on the choice of kernel. In this work we propose the novel harmonizable mixture kernel (HMK), a family of expressive, interpretable, non-stationary kernels derived from mixture models on the generalized spectral representation. As a theoretically sound treatmen…
This paper presents new algorithms to solve the feature-sparsity constrained PCA problem (FSPCA), which performs feature selection and PCA simultaneously. Existing optimization methods for FSPCA require data distribution assumptions and are lack of global convergence guarantee. Though the general FSPCA problem is NP-ha…