New findings clarify the link between distributional closeness and representational similarity.
problem When and why do different neural network representations become similar?
method Identifiability theory, focusing on model families including autoregressive language models.
result Small Kullback-Leibler divergence does not guarantee similar representations.
DBGAN learns graph node representations by balancing distribution consistency.
problem Graph representation learning overfits due to ignoring data distribution.
method DBGAN uses a structure-aware prior distribution and bidirectional adversarial learning.
result DBGAN achieves better trade-off between robustness and dimensionality.
Enhances DIM to match learned representations to a specific distribution.
problem Learning representations conforming to a specific distribution.
method Injecting noise into normalized outputs of the encoder while keeping the InfoMax training objective.
result Learning uniformly and normally distributed representations, as well as representations of other absolutely continuous distributions.
Better data representations can simplify learning tasks by aligning model distributions with true data distributions.
problem Learning complexity influenced by the alignment of model distributions with true data distributions.
method Analyzed the effect of data representations on learning complexity using a task complexity score and information coding length.
result Better representations can simplify learning tasks by aligning model distributions with true data distributions, improving learning outcomes.
Proposes FSM-IRL to learn invariant network representations considering feature and structural shifts.
problem Spatial heterogeneity and temporal dynamics lead to OOD generalization issues in geographic networks.
method Introduces FSM-IRL model that accounts for feature and structural distribution shifts using causal attention and reweighting.
result Demonstrates strong learning capabilities on geographic and social network datasets in OOD scenarios.
Unsupervised learning representations generalize better than supervised learning under distribution shifts.
problem Robustness of unsupervised representations to distribution shift.
method Extensive evaluation on synthetic and realistic datasets, including controllable domain generalization datasets.
result Unsupervised representations learned from SSL and AE generalize better than supervised learning under various distribution shifts.
This paper classifies tweets into positive and negative sentiments using distributed word and sentence representations.
problem Classifying tweets into positive and negative sentiments.
method Used distributed representations of words and sentences, and LSTM and CNN networks for classification.
result Achieved accuracies as high as 81%.
Unpaired multi-domain causal representation learning is possible with sufficient conditions.
problem Learning shared causal representation from unpaired data across domains.
method Identify sufficient conditions for joint distribution and shared causal graph recovery.
result Practical method to recover shared latent causal graph from marginal distributions.
New method uses small perturbations to improve representation learning from few labels.
problem Stability issues and label scarcity in representation learning.
method Introduces small-perturbation ideology on representation probability distribution models.
result Proposed models show better performance in clustering compared to baseline methods.
IRM learns representations invariant to training distributions for better generalization.
problem Learning generalizable models across different training distributions.
method IRM learns a data representation that remains consistent across multiple training distributions, ensuring an optimal classifier matches across them.
result IRM enables out-of-distribution generalization by learning invariant correlations.
Researchers identify valid auxiliary functions for extreme value distributions and their max-domains of attraction.
problem Characterize valid auxiliary functions for extreme value distributions and their max-domains of attraction.
method Introduced 'universal' auxiliary functions valid for both VR and vMR representations, identified sets of valid auxiliary functions, and proposed a method for finding appropriate auxiliary functions.
result Characterized valid auxiliary functions for both VR and vMR representations for the entire MDA distribution families.
We present a layered Boltzmann machine (BM) that can better exploit the advantages of a distributed representation. It is widely believed that deep BMs (DBMs) have far greater representational power than its shallow counterpart, restricted Boltzmann machines (RBMs). However, this expectation on the supremacy of DBMs ov…
Improves NF for complex data distributions with multiple modes.
problem Difficulty in handling data distributions with multiple isolated modes.
method Proposes a new framework using variational latent representation to improve NF.
result Significantly more powerful for generating data distributions with multiple modes.
This paper tackles sequential distribution shifts in representation learning.
problem Learning meaningful representations in a sequence of distribution shifts.
method Nonlinear Independent Component Analysis (ICA) framework for continual causal representation learning.
result The method achieves performance comparable to joint training on multiple offline distributions and shows no benefit from the incoming new distribution on all latent variables.
This study examines how neural network latent representations correlate with model uncertainty.
problem Detecting model uncertainty in neural networks.
method Empirical verification and analysis of latent representations' distribution and conditional output.
result Deep layers in neural networks can infer uncertainty similar to more computationally expensive methods.
Geo2DR learns graph representations using substructure patterns.
problem Learning distributed representations of graphs efficiently.
method Unsupervised learning with discrete substructure patterns and neural language models.
result Geo2DR achieves high reproducibility and interoperability in graph classification.
Paper improves forest representation learning by optimizing margin distribution.
problem Improving generalization gap in forest representation learning.
method Reformulated as an additive model, optimizing margin distribution ratio λ.
result Substantially improved upper bound of generalization gap from O(√(ln m / m)) to O((ln m) / m).
Empower efficient representation of distributions through moment-preserving methods.
problem Representing high-dimensional probability measures efficiently and accurately.
method Empower efficient representation of distributions through moment-preserving methods.
result Empowers efficient and accurate representation of high-dimensional probability measures.
Geospatial analysis lacks methods like the word vector representations and pre-trained networks that significantly boost performance across a wide range of natural language and computer vision tasks. To fill this gap, we introduce Tile2Vec, an unsupervised representation learning algorithm that extends the distribution…
Generative Distribution Embeddings learn multiscale representations of distributions.
problem Learning representations of entire distributions for multiscale reasoning.
method Introducing GDE framework that lifts autoencoders to the space of distributions, using conditional generative models and distributional invariance.
result GDEs learn predictive sufficient statistics embedded in Wasserstein space, recovering distances and trajectories for Gaussian and Gaussian mixture distributions.
Study evaluates scalability and real-world impact of disentangled representations.
problem Scalability and real-world impact of disentangled representations.
method New high-resolution dataset and architectures for disentangled representation learning.
result Disentanglement predicts out-of-distribution task performance.
The paper tackles fair representation learning by smoothing feature mappings.
problem Legal liability for discriminatory use of data by organizations.
method Mapping features to a fair representation space, certifying fairness through chi-squared mutual information.
result Smoothing representation distribution provides generalization guarantees of fairness and maintains accuracy for downstream tasks.
Proposes a new approach for domain adaptation using latent representations.
problem Handling distribution shifts between source and target domains in high-dimensional data.
method Learn compact latent representations based on the label's Markov blanket, partitioning into parents, children, and spouses.
result General domain adaptation can be achieved by learning representations of the label's parents, children, and spouses.
Estimates model performance under distribution shift using domain-invariant predictors.
problem Poor performance of models on test distributions different from training distributions.
method Uses domain-invariant predictors as a proxy for unknown target labels.
result Shows that the complexity of latent representations influences target risk.
Representations based on random walks can exploit discrete data distributions for clustering and classification. We extend such representations from discrete to continuous distributions. Transition probabilities are now calculated using a diffusion equation with a diffusion coefficient that inversely depends on the dat…
New framework uses geometry of embeddings to predict robustness.
problem Monitoring robustness in models without OOD labels.
method Constructs graphs from embeddings, measures spectral complexity and curvature.
result Representation geometry predicts robustness reliably.
Sparse representations improve reinforcement learning control policies.
problem Sparse representations are underused in reinforcement learning control.
method Incremental learning with sparse representations from neural networks, using distributional regularizers.
result Sparse representations avoid catastrophic interference and provide stable values for reinforcement learning.
p-DkNN uses deep representations to detect out-of-distribution data with statistical tests.
problem Lack of reliable confidence estimates in neural networks for safety-critical applications.
method Statistical testing of deep neural network's intermediate hidden representations.
result p-DkNN enables more accurate and reliable predictions by abstaining from incorrect predictions.
Investigates statistical properties of perturb-softmax and perturb-argmax distributions.
problem Underexplored statistical properties of Gumbel-Softmax and Gumbel-Argmax distributions.
method Investigates convexity and differentiability to determine completeness and minimality of these distributions.
result Identifies parameters that admit complete and minimal representation of probability distributions.
A simple method flags images as out-of-distribution based on their distance to nearest neighbors.
problem Detecting images not aligned with a trained model's in-distribution data.
method Flag images as OOD if their average distance to K nearest neighbors is large in the classifier's representation space.
result Simple methods can outperform more complex ones when considering learned representations.
Reduces data leakage in distributed deep learning models.
problem Prevents reconstruction of sensitive raw data patterns during client communications.
method Reduces distance correlation between raw data and learned representations.
result Resilient to reconstruction attacks while maintaining model accuracy.
NURD improves model performance by distilling representations independent of nuisance variables.
problem Models trained under spurious correlations may fail on data with different nuisance-label relationships.
method Developed Nuisance-Randomized Distillation (NURD) to find representations independent of nuisance variables.
result NURD finds representations that perform better regardless of nuisance-label relationships.
New method identifies stable latent variables across different domains using weak distributional invariances.
problem Learning causal representations for multi-domain datasets.
method Autoencoders incorporating weak distributional invariances.
result Autoencoders can identify stable latent variables across different domains.
Study reveals limitations of fair representation learning methods and cautions against their use in performance-sensitive tasks.
problem Limitations of fair representation learning methods in performance-sensitive tasks.
method Using causal reasoning, the study defines and formalizes different sources of dataset bias and examines the performance of fair representation learning under distribution shifts.
result Fundamental limitations on fair representation learning when evaluation data is drawn from the same distribution as training data.
Reduces GAN image priors' representation error using a Deep Decoder.
problem Representation error in GAN priors for in-distribution and out-of-distribution images.
method Hybrid model combining GAN prior and Deep Decoder.
result Consistently higher PSNRs on in-distribution and out-of-distribution images.
Paper introduces a new distributional successor measure for reinforcement learning.
problem Learning the distributional consequences of behavior in reinforcement learning.
method Formulates distributional successor measure as a distribution over distributions, proposes algorithm to learn it from data.
result Demonstrates zero-shot risk-sensitive policy evaluation.
Proposes methods to identify and estimate counterfactual distributions with confounding.
problem Estimating counterfactual distributions in the presence of confounding.
method Nonparametric identification and semiparametric estimation using conditional copulas and machine learning.
result Valid inference for individual-level effects and nonparametric identifiability of latent confounding subspace.
Proposes DWMD for better matching of hidden representations across domains.
problem Measuring data distribution discrepancy between semantically related domains for feature representation matching.
method DWMD, a moment-based probability distribution metric that explicitly orders and weights higher-order moments.
result DWMD is error-free and can strictly reflect distribution differences without feature distribution assumptions.
A model learns successor representations in uncertain environments.
problem Learning effective strategies in partially observable, noisy environments.
method Neurally plausible model using distributional successor features.
result Distributional successor features support reinforcement learning in noisy environments.
Study shows how deep network representations can be transferred between datasets and tasks.
problem Transferability of deep network representations across datasets and tasks.
method Examined layer-wise transferability of representations in deep networks across multiple datasets and tasks.
result Interesting empirical observations on layer-wise transferability of representations.
MetaVAE learns transferable latent representations across related distributions.
problem Generative models struggle to adapt to new distributions.
method Doubly-amortized variational inference sharing computation across related models.
result MetaVAE significantly outperforms baselines on image classification tasks.
A new method improves cross-domain sentiment analysis by learning weighted domain-invariant representations.
problem Label distribution changes across domains harm domain adaptation in DIRL.
method Proposes WDIRL, a modification to DIRL that learns weighted domain-invariant representations.
result Empirical studies show the effectiveness of WDIRL in cross-domain sentiment analysis.
Estimates prediction uncertainty in neural networks using density estimation in representation space.
problem Incorrect predictions with high confidence from models trained on limited data.
method Estimates training data density in representation space and uses it to predict model uncertainty.
result Detects out-of-distribution data without prior exposure, improving model reliability.
Paper introduces a simple method to assign uncertainty in contrastive learning models.
problem Contrastive learning models lack uncertainty measures.
method Trains a deep network to assign uncertainty based on representation variance.
result Deep uncertainty model improves anomaly detection and out-of-distribution classification.
New method learns disentangled discrete representations using categorical variational autoencoders.
problem Learning disentangled representations from discrete latent spaces.
method Replaced standard Gaussian VAE with a categorical VAE to mitigate rotational invariance.
result Categorical distributions improve learning of disentangled representations.
dpVAEs improve VAEs by decoupling representation and generation.
problem VAEs struggle with both representation learning and sample generation.
method Introduce decoupled priors (dpVAEs) that separate representation and generation spaces.
result dpVAEs enable regularization without compromising sample generation.
EIGAN learns private representations without centralized data, outperforming state-of-the-art.
problem Private representation learning with multiple ally and adversary attributes.
method Exclusion-Inclusion Generative Adversarial Network (EIGAN) and Distributed EIGAN (D-EIGAN).
result EIGAN and D-EIGAN outperform state-of-the-art methods in accuracy and scalability.
Introduces a probabilistic view of deep learning for better understanding and explaining neural networks.
problem Explaining the behavior and properties of deep neural networks.
method Introduces a probabilistic representation of deep learning, linking neurons, hidden layers, and the whole architecture to Gibbs distributions and Bayesian neural networks.
result Demonstrates the hierarchy and generalization properties of deep learning through a probabilistic lens.