Method learns representations invariant to task-irrelevant details in reinforcement learning tasks.
problem Learning representations that are invariant to task-irrelevant details in reinforcement learning.
method Uses bisimulation metrics to learn robust latent representations that encode only task-relevant information.
result Demonstrates SOTA performance in modified visual MuJoCo tasks and a first-person driving task.
LLMs are vulnerable to task-irrelevant data changes, limiting their use for data fitting.
problem LLMs' sensitivity to task-irrelevant variations in data representation.
method Analysis of LLMs' performance and attention patterns under various data manipulations.
result LLMs are sensitive to task-irrelevant variations, leading to significant prediction errors.
The paper develops a method to create robust control policies for robots using information bottlenecks.
problem Robotic control policies are sensitive to task-irrelevant state and sensor changes.
method Derives a policy gradient algorithm that creates an information bottleneck between states and task-relevant representations.
result Task-driven policies are more robust to sensor noise and environmental changes.
This paper analyzes self-supervised learning from a multi-view perspective.
problem Understanding and optimizing self-supervised learning from multi-view data.
method Information-theoretical framework to understand and design self-supervised learning objectives.
result Self-supervised representations can extract task-relevant information and discard task-irrelevant information.
Paper tackles robust domain adaptation without target domain data.
problem Learning domain invariant representations without target domain data.
method Integrates deep autoencoder and causal structure learning into a unified model.
result CAE learns causal representations using only source domain data.
Study how models represent features in naturalistic learning problems.
problem Understanding which features models use and ignore in naturalistic tasks.
method Synthetic datasets with controlled task-relevance of features, training models to recognize both easy and hard features.
result Models preferentially represent task-relevant features and suppress task-irrelevant ones over training.
We show that a critical vulnerability in adversarial imitation is the tendency of discriminator networks to learn spurious associations between visual features and expert labels. When the discriminator focuses on task-irrelevant features, it does not provide an informative reward signal, leading to poor task performanc…
The paper explains knowledge distillation by analyzing visual concepts in DNNs.
problem Understanding how knowledge distillation affects the learning of visual concepts in deep neural networks.
method The paper proposes three hypotheses and designs mathematical metrics to evaluate feature representations of DNNs.
result The hypotheses were verified through experiments on various DNNs.
Despite their impressive performance, deep neural networks exhibit striking failures on out-of-distribution inputs. One core idea of adversarial example research is to reveal neural network errors under such distribution shifts. We decompose these errors into two complementary sources: sensitivity and invariance. We sh…
Improves unsupervised domain adaptation by enforcing feature extractor to focus on task-relevant information.
problem Leveraging label information from source domain for accurate target domain models without labels.
method Variational Information Bottleneck (VBDA) method that explicitly enforces feature extractor to ignore irrelevant task factors.
result Significantly outperforms state-of-the-art methods across three domain adaptation benchmark datasets.
SYNC learns time-aware causal representations to improve model generalization in evolving domains.
problem Spurious correlations and shortcut learning in existing EDG methods hinder model generalization.
method SYNC integrates dynamic causal factors and causal mechanism drifts into a sequential VAE framework.
result SYNC achieves superior temporal generalization performance on synthetic and real-world datasets.
New method uses observational data to improve trial design efficiency.
problem Scarce randomized controlled trials; inefficiency of using observational data.
method Active Residual Learning, R-Design framework, R-EPIG criterion.
result Efficiently estimating residuals to correct observational bias improves trial design.
RényiCL uses Rényi divergence for robust contrastive learning with stronger data augmentations.
problem Learning useful representations from multiple data views with hard augmentations.
method RényiCL employs Rényi divergence for contrastive learning, using a novel variational objective to manage hard negative sampling.
result RényiCL achieves better performance with stronger augmentations compared to other methods.
This work enhances collaborative inference privacy by minimizing conditional entropy and boosting robustness against model inversion attacks.
problem Privacy leakage in collaborative inference systems via model inversion attacks.
method Theoretical proof and derivation of a differentiable measure for bounding conditional entropy, followed by a CEM algorithm to maximize it.
result Theoretical proof and experimental validation show that CEM consistently boosts inversion robustness without compromising feature utility or efficiency.
Novel framework improves graph learning for out-of-distribution generalization.
problem Graph out-of-distribution generalization challenges in neural networks.
method Invariant Graph Learning based on Information bottleneck theory (InfoIGL).
result Achieves state-of-the-art performance in graph classification tasks under OOD generalization.
This paper proposes an alternative to E2E training for deep networks, reducing memory footprint.
problem High GPUs memory footprint in end-to-end training of deep networks.
method Locally supervised learning with information propagation loss to avoid information collapse.
result The proposed method achieves competitive performance with less than 40% memory footprint compared to E2E training.
ES-MLP combines Graph-MLP with edge splitting for node classification on both homophilic and heterophilic graphs.
problem Node classification on graphs with mixed homophilic and heterophilic properties.
method Combines Graph-MLP with edge splitting mechanism from ES-GNN to learn two adjacency matrices based on relevant and irrelevant feature pairs.
result ES-MLP achieves performance comparable to homophilic and heterophilic models without using edges during inference.
ANPyC combats forgetting by pruning and consolidating neural parameters.
problem Catastrophic forgetting in neural networks, especially with long-term tasks.
method Adversarial Neural Pruning and Synaptic Consolidation (ANPyC) to balance task-relevant and irrelevant parameters.
result ANPyC prevents forgetting while enabling efficient learning of multiple tasks.
SGATs learn sparse attention coefficients to improve graph learning tasks on large, noisy graphs.
problem Overfitting and noisy edges in GNNs on large, noisy graphs.
method Sparse Graph Attention Networks (SGATs) learn sparse attention coefficients under L0-norm regularization. result SGATs can remove 50%-80% edges from large graphs while maintaining similar classification accuracies.