Proposes a new method to find features affecting treatment effect distribution.
problem Existing methods fail to detect differences in treatment effect distribution parameters other than the mean.
method Formulates and estimates a feature importance measure that quantifies feature influence on potential outcome distribution discrepancies. Develops a feature selection algorithm to control type I error rate.
result Successfully discovers important features and outperforms existing mean-based methods.
One-shot algorithm for feature-distributed kernel PCA reduces communication costs.
problem Efficiently perform kernel PCA in distributed computing environments.
method Inspired by dual relationship between sample-distributed and feature-distributed scenarios, proposes a one-shot algorithm for feature-distributed kernel PCA.
result The algorithm provides high-quality results with low communication costs, especially when eigenvalues decay fast.
Develops method to assess feature importance in black-box models for unconditional distribution.
problem Lack of methods to analyze feature importance in black-box models for unconditional distribution.
method Approximation method to compute feature importance curves for unconditional distribution.
result Produces sparse and faithful results, computationally efficient.
PFDL improves deep learning models' OOD generalization by decorrelating feature embeddings.
problem Out-of-distribution generalization in deep learning models.
method PFDL algorithm that optimizes feature decomposition network and image classification model.
result PFDL improves the accuracy of image classification models on OOD datasets.
New adversarial attack method based on deep feature distributions.
problem Adversarial attacks on CNN classifiers using output layer information.
method Modeling and exploiting class-wise and layer-wise deep feature distributions.
result Achieves state-of-the-art transfer-based attack results for undefended ImageNet models.
New distributions allow greedy arm selection in sparse bandit problems.
problem Sparse contextual bandit problem with sparse parameters and feature distributions.
method Introduced new distribution classes and demonstrated that mixtures of these distributions are also greedy-applicable.
result Greedy algorithm applicable to a wider range of arm feature distributions, including those with origin-asymmetric support.
GOTEX synthesizes textures by optimizing feature distributions using optimal transport.
problem Texture synthesis with control over feature distributions.
method Optimal transport for statistical feature distribution comparison, semi-dual formulation, minimax optimization.
result GOTEX produces high-quality textures with various features, outperforming state-of-the-art methods.
Distributed sensors compress and send features to a fusion center for linear regression.
problem Efficiently compress and transmit features from distributed sensors to a fusion center under varying communication constraints.
method Designs a distributed and adaptive feature compression scheme using optimal quantizers and simple adaptive strategies.
result Demonstrates improved inference performance through simulated experiments.
CIBP models feature abundance in latent feature models.
problem Modeling feature abundance in latent feature models.
method Proposes a new Bayesian nonparametric prior, the CIBP, for latent feature models.
result The expected number of features is bounded even as the number of objects increases.
New framework learns sufficient invariant features robustly across distribution shifts.
problem Learning robust models under distribution shifts between training and test datasets.
method Sufficient Invariant Learning (SIL) framework and Adaptive Sharpness-aware Group Distributionally Robust Optimization (ASGDRO) algorithm.
result Empirical evaluations confirm ASGDRO's robustness against distribution shifts.
RoSHAP stabilizes feature attribution in machine learning models.
problem Stochastic variation in feature attribution measures.
method Modeling feature attribution score distribution and estimating it through bootstrap resampling and kernel density estimation.
result RoSHAP provides stable feature rankings and improves model performance.
Accuracy on in-distribution data correlates with out-of-distribution data when data is noisy or contains nuisance features.
problem Correlation between in-distribution and out-of-distribution accuracy in noisy or feature-rich data.
method Analyzes the impact of noise and nuisance features on model performance.
result Accuracy on in-distribution and out-of-distribution data can become negatively correlated in noisy or feature-rich data.
Two semimetrics on probability distributions are proposed, given as the sum of differences of expectations of analytic functions evaluated at spatial or frequency locations (i.e, features). The features are chosen so as to maximize the distinguishability of the distributions, by optimizing a lower bound on test power f…
Two novel methods identify influential features in CMABs for better reward distribution.
problem Suboptimal features degrade rewards, interpretability, and efficiency in CMABs.
method Heterogeneous Incremental Effect (HIE) and Heterogeneous Distribution Divergence (HDD) methods.
result Consistent ability to identify influential HTE features, enhancing CMAB performance.
TSRGA scales multivariate linear regression for feature-distributed data.
problem Multivariate linear regression for feature-distributed data with high dimensions and many computing nodes.
method Two-stage relaxed greedy algorithm (TSRGA) for multivariate linear regression.
result TSRGA is highly scalable and can yield low-rank coefficient estimates.
KD2M unifies feature matching in neural networks.
problem Matching neural network activations for knowledge transfer.
method Formalizes distribution matching strategy.
result New theoretical results for KD.
Paper quantifies label shift with robustness guarantees using distribution feature matching.
problem Estimating target label distribution under label shift.
method Distribution feature matching (DFM) framework and robustness analysis.
result General performance bound and robustness analysis in misspecified settings.
We present a principled approach for detecting out-of-distribution (OOD) and adversarial samples in deep neural networks. Our approach consists in modeling the outputs of the various layers (deep features) with parametric probability distributions once training is completed. At inference, the likelihoods of the deep fe…
It has been shown that instead of learning actual object features, deep networks tend to exploit non-robust (spurious) discriminative features that are shared between training and test sets. Therefore, while they achieve state of the art performance on such test sets, they achieve poor generalization on out of distribu…
Proposes FSM-IRL to learn invariant network representations considering feature and structural shifts.
problem Spatial heterogeneity and temporal dynamics lead to OOD generalization issues in geographic networks.
method Introduces FSM-IRL model that accounts for feature and structural distribution shifts using causal attention and reweighting.
result Demonstrates strong learning capabilities on geographic and social network datasets in OOD scenarios.
Energy distance measures feature heterogeneity in federated learning.
problem Heterogeneity across data sources hinders model aggregation in federated learning.
method Introduced Taylor approximations of energy distance for efficient computation.
result Taylor approximations accurately capture feature discrepancies, improving convergence.
SFP prunes ID features to improve OOD generalization without domain data.
problem Improving out-of-distribution (OOD) generalization in biased models.
method Spurious Feature-targeted model Pruning (SFP) framework.
result SFP achieves optimal OOD generalization by pruning ID features.
Hermite polynomials improve private data generation by reducing feature count.
problem Infinite-dimensional features in kernel mean embedding are impractical for private data generation.
method Replace random features with Hermite polynomial features, leveraging their ordered nature.
result Hermite polynomial features yield a more accurate approximation of kernel mean embedding with fewer features.
BIF assesses feature importance using Dirichlet distribution and Bayesian inference.
problem Quantitative feature importance assessment in statistical models.
method Utilizes Dirichlet distribution for probabilistic feature importance assessment via approximate Bayesian inference.
result Learned importance provides relative significance and confidence quantification of features.
FeAT improves OOD generalization by learning richer features.
problem Improving feature learning for out-of-distribution (OOD) generalization.
method Feature Augmented Training (FeAT) iteratively augments and retains features from different subsets of training data.
result FeAT effectively learns richer features, boosting OOD performance.
As the size of modern data sets exceeds the disk and memory capacities of a single computer, machine learning practitioners have resorted to parallel and distributed computing. Given that optimization is one of the pillars of machine learning and predictive modeling, distributed optimization methods have recently garne…
Distributions over exchangeable matrices with infinitely many columns, such as the Indian buffet process, are useful in constructing nonparametric latent variable models. However, the distribution implied by such models over the number of features exhibited by each data point may be poorly- suited for many modeling tas…
Hierarchical VAEs detect out-of-distribution data by identifying low-level in-distribution features.
problem Out-of-distribution data often has in-distribution low-level features, leading to misleading likelihood estimates in deep generative models.
method Developed a fast, scalable, unsupervised likelihood-ratio score for out-of-distribution detection based on hierarchical variational autoencoders.
result Achieved state-of-the-art results on out-of-distribution detection across various data and model combinations.
Genome-wide association studies (GWAS) have achieved great success in the genetic study of Alzheimer's disease (AD). Collaborative imaging genetics studies across different research institutions show the effectiveness of detecting genetic risk factors. However, the high dimensionality of GWAS data poses significant cha…
A novel method for feature selection using a reparameterized logitNormal distribution.
problem Feature selection for reconstruction in high-dimensional data.
method Introducing a reparameterization of the logitNormal distribution to address differentiability and covariance issues.
result The method provides an effective exploration scheme and efficient feature selection for reconstruction.
Feature selection problems have been extensively studied for linear estimation, for instance, Lasso, but less emphasis has been placed on feature selection for non-linear functions. In this study, we propose a method for feature selection in high-dimensional non-linear function estimation problems. The new procedure is…
A nonparametric approach for policy learning for POMDPs is proposed. The approach represents distributions over the states, observations, and actions as embeddings in feature spaces, which are reproducing kernel Hilbert spaces. Distributions over states given the observations are obtained by applying the kernel Bayes' …
Unsupervised domain adaptation methods aim to alleviate performance degradation caused by domain-shift by learning domain-invariant representations. Existing deep domain adaptation methods focus on holistic feature alignment by matching source and target holistic feature distributions, without considering local feature…
FIT evaluates time series model feature importance quantifying distributional shift.
problem Lack of explanations for time series models in high-stakes applications.
method FIT framework quantifies feature importance based on distributional shift using KL-divergence.
result FIT identifies important time points and observations superiorly compared to baselines.
Distributed learning with random features and gradient descent improves performance and reduces memory usage.
problem Improving generalization in decentralized learning with limited memory.
method Distributed Gradient Descent with Random Features and Implicit Regularization.
result High probability bounds on predictive performance with optimal statistical rates.
We introduce new families of Integral Probability Metrics (IPM) for training Generative Adversarial Networks (GAN). Our IPMs are based on matching statistics of distributions embedded in a finite dimensional feature space. Mean and covariance feature matching IPMs allow for stable training of GANs, which we will call M…
Linear classification has been widely used in many high-dimensional applications like text classification. To perform linear classification for large-scale tasks, we often need to design distributed learning methods on a cluster of multiple machines. In this paper, we propose a new distributed learning method, called f…
A new algorithm detects out-of-distribution samples by concentrating them in feature space.
problem Building safe AI systems requires effective out-of-distribution detection.
method The paper proposes a novel algorithm based on the observation that OoD samples concentrate in feature space.
result The algorithm achieves state-of-the-art performance on various OoD detection benchmarks.
While discriminative classifiers often yield strong predictive performance, missing feature values at prediction time can still be a challenge. Classifiers may not behave as expected under certain ways of substituting the missing values, since they inherently make assumptions about the data distribution they were train…
AFA evaluates AI feature acquisition strategies in domains with high costs.
problem Evaluate AI feature acquisition strategies in domains with high costs.
method Apply missing data methods and offline reinforcement learning under NDE and NUC assumptions.
result Propose a novel semi-offline reinforcement learning framework with three new estimators.
This work improves transferability by considering conditional distributions in feature representations.
problem Improving transferability across multiple domains by considering conditional distributions.
method Introducing von Neumann conditional divergence to quantify the functional dependence between features and desired response.
result Favorable performance in terms of smaller generalization error and less catastrophic forgetting.
Dida learns meta-features invariant to feature permutations.
problem Learning invariant meta-features from dataset samples.
method Distribution-based deep architecture inheriting universal approximation and robustness.
result Dida outperforms state-of-the-art methods on dataset-level tasks.
Detects which features have shifted in data distributions.
problem Identifying which specific features have caused a distribution shift.
method Formalizes the problem as multiple conditional distribution hypothesis tests, proposes non-parametric and parametric statistical tests, and uses a test statistic based on the density model score function.
result Demonstrates methods for identifying when and where a shift occurs in multivariate time-series data.
Random features provide a practical framework for large-scale kernel approximation and supervised learning. It has been shown that data-dependent sampling of random features using leverage scores can significantly reduce the number of features required to achieve optimal learning bounds. Leverage scores introduce an op…
Proposes DWMD for better matching of hidden representations across domains.
problem Measuring data distribution discrepancy between semantically related domains for feature representation matching.
method DWMD, a moment-based probability distribution metric that explicitly orders and weights higher-order moments.
result DWMD is error-free and can strictly reflect distribution differences without feature distribution assumptions.
New methods improve anomaly detection in deep networks by leveraging hierarchical likelihoods and multi-scale features.
problem Challenges in detecting anomalies in high-level features due to model bias and domain prior.
method Two methods: 1) Log likelihood ratios between in-distribution and general distribution models, 2) Multi-scale likelihood contribution.
result Strong anomaly detection performance in unsupervised setting, slightly underperforming supervised methods.
SQFA learns features maximizing Fisher-Rao distance for better classification.
problem Improving classification accuracy through feature learning.
method SQFA learns linear features maximizing Fisher-Rao distance between class-conditional distributions.
result SQFA-H features achieve the best classification accuracy.
OOD detection methods often misidentify OOD points, leading to ineffective safety improvements.
problem Improving model safety through OOD detection methods often leads to incorrect identification of out-of-distribution points.
method Re-examine popular OOD detection procedures based on predictive uncertainty or features of supervised models trained on in-distribution data.
result Popular OOD detection methods incorrectly conflate high uncertainty and far feature-space distance with being out-of-distribution.