Dual representations for robust risk measures and uncertainty sets.
problem Characterizing continuity of robust risk measures and their uncertainty sets.
method Develop dual representations for robust risk measures and uncertainty sets based on distinct geometric assumptions.
result Two dual frameworks for consolidated uncertainty sets are complementary, not interchangeable.
The paper tackles adversarial robustness by maximizing worst-case mutual information.
problem Training robust machine learning models against adversarial inputs is challenging.
method Develops a notion of representation vulnerability and an unsupervised learning method to maximize worst-case mutual information.
result Proves a lower bound on minimum adversarial risk and supports robustness of representations.
Neural nets learn robust geometric data representations.
problem Ensuring neural networks are robust to adversarial attacks.
method Topological Data Analysis via persistence diagrams, Lipschitz stability.
result Certified ε-robustness on ORBIT5K dataset. This work improves robustness guarantees for neural networks using low rank representations.
problem Certified robustness to adversarial perturbations in neural networks.
method Low rank representations to provide improved robustness guarantees.
result Improved robustness guarantees for ℓ∞ perturbations using natural low rank representations. AAT separates robust and non-robust features without supervision.
problem Adversarial vulnerability and accuracy reduction in machine learning models.
method Adversarial Asymmetric Training (AAT) algorithm.
result Preserves accuracy and achieves better disentanglement than previous methods.
Paper introduces a method to create robust representations against covariate shifts.
problem Distribution shift between training and testing data in machine learning.
method Introduces a variational objective with two components: discriminative representation and invariant support.
result Optimal representations ensure robustness to covariate shifts, improving performance on DomainBed.
An important goal in deep learning is to learn versatile, high-level feature representations of input data. However, standard networks' representations seem to possess shortcomings that, as we illustrate, prevent them from fully realizing this goal. In this work, we show that robust optimization can be re-cast as a too…
It is always demanding to learn robust visual representation for various learning problems; however, this learning and maintenance process usually suffers from noise, incompleteness or knowledge domain mismatch. Thus, robust representation learning by removing noisy features or samples, complementing incomplete data, a…
Unsupervised learning representations generalize better than supervised learning under distribution shifts.
problem Robustness of unsupervised representations to distribution shift.
method Extensive evaluation on synthetic and realistic datasets, including controllable domain generalization datasets.
result Unsupervised representations learned from SSL and AE generalize better than supervised learning under various distribution shifts.
New method improves graph representations against adversarial attacks.
problem Adversarial robustness in graph contrastive learning.
method Adversarial and edge insertion transformations.
result Promising results in preliminary experiments.
Reprogram deep models to resist adversarial attacks without changing parameters.
problem Improving deep learning models' robustness against adversarial and noisy inputs.
method Proposes a non-linear robust pattern matching technique and three reprogramming paradigms.
result Demonstrates effective reprogramming of deep models for robustness without altering parameters.
Paper improves deep learning models for limit order book data.
problem Deep learning models' performance depends on robust input data representation.
method Identified and modified flaws in existing representations.
result Proposed modifications lead to state-of-the-art performance.
Deep networks are well-known to be fragile to adversarial attacks. We conduct an empirical analysis of deep representations under the state-of-the-art attack method called PGD, and find that the attack causes the internal representation to shift closer to the "false" class. Motivated by this observation, we propose to …
Unsupervised learning techniques in computer vision often require learning latent representations, such as low-dimensional linear and non-linear subspaces. Noise and outliers in the data can frustrate these approaches by obscuring the latent spaces. Our main goal is deeper understanding and new development of robust ap…
Paper presents J-RFDL for robust DL in compressed space, improving data representation robustness and accuracy.
problem Improving data representation robustness and accuracy in the presence of noise and outliers.
method Joint Robust Factorization and Projective Dictionary Learning (J-RFDL) in a factorized compressed space.
result Delivers superior performance in data representation and classification over state-of-the-art methods.
New method learns robust representations by modeling environment variation.
problem Learning invariant representations across varying environments.
method Explicitly modeling variation across environments and marginalizing it out.
result Proposed method outperforms invariant-learning methods in various settings.
We investigate the effect of the dimensionality of the representations learned in Deep Neural Networks (DNNs) on their robustness to input perturbations, both adversarial and random. To achieve low dimensionality of learned representations, we propose an easy-to-use, end-to-end trainable, low-rank regularizer (LR) that…
New model improves neural network robustness against input manipulations.
problem Improving neural network robustness against input manipulations.
method Causal view and deep causal manipulation augmented model (deep CAMA) with data augmentation and test-time fine-tuning.
result Deep CAMA shows superior robustness against unseen manipulations compared to traditional models.
Paper introduces RAS for robust MTL with contamination.
problem Representation-based multi-task learning struggles with contamination.
method Robust and Adaptive Spectral (RAS) method.
result RAS prevents negative transfer and performs well with up to 80% contamination.
Proposes a new method for nonlinear models with robustness guarantees.
problem Distributional robustness in nonlinear models with causality.
method Representation learning and identifiable representation learning.
result First causality-inspired robustness method with finite-radius guarantees in nonlinear settings.
DSCF-Net learns deep features for clustering with robustness and locality preservation.
problem Unsupervised deep representation learning for clustering.
method Integrates robust deep concept factorization, deep self-expressive representation, and adaptive locality preserving feature learning.
result Delivers state-of-the-art performance on public databases.
MKA incorporates manifold geometry into kernel alignment for more robust representation comparison.
problem Inadequate accounting for manifold geometry in kernel alignment metrics.
method Derives a theoretical framework for Manifold Approximated Kernel Alignment (MKA).
result MKA provides a more robust foundation for measuring representations.
New framework uses geometry of embeddings to predict robustness.
problem Monitoring robustness in models without OOD labels.
method Constructs graphs from embeddings, measures spectral complexity and curvature.
result Representation geometry predicts robustness reliably.
Lower class selectivity makes networks more robust to natural perturbations but more vulnerable to adversarial attacks.
problem Understanding how class selectivity affects robustness to different types of perturbations in neural networks.
method Investigated the relationship between class selectivity and robustness to natural and adversarial perturbations in neural networks.
result Lower class selectivity increases robustness to natural perturbations but decreases robustness to adversarial attacks.
Paper tackles robust domain adaptation without target domain data.
problem Learning domain invariant representations without target domain data.
method Integrates deep autoencoder and causal structure learning into a unified model.
result CAE learns causal representations using only source domain data.
Study shows how pretraining robustness transfers to downstream tasks.
problem Understanding how robustness is transferred from pretraining to downstream tasks.
method Theoretical analysis and practical validation of robustness constraints.
result Robustness of a linear predictor on downstream tasks can be constrained by the robustness of its underlying representation.
This work explores how neural network architecture affects robustness to noisy labels.
problem The impact of neural network architecture on robustness to noisy labels.
method Formal framework connecting robustness to architecture alignments, measured by predictive power in representations.
result Network robustness to noisy labels improves when its architecture is more aligned with the target function.
Study systemic risk measures adjusted to financial markets.
problem Systemic risk in financial systems with market adjustments.
method Dual representation for convex robust systemic risk measures adjusted to the financial market.
result Relation to no-arbitrage conditions.
AI models aligned with human vision perform well on few data tasks.
problem Few-shot learning performance with limited data.
method Information-theoretic analysis and empirical testing of 491 models.
result Highly aligned models show better robustness to attacks and domain shifts.
Machine learning methods often need a large amount of labeled training data. Since the training data is assumed to be the ground truth, outliers can severely degrade learned representations and performance of trained models. Here we apply concepts from robust statistics to derive a novel variational autoencoder that is…
Most artificial networks today rely on dense representations, whereas biological networks rely on sparse representations. In this paper we show how sparse representations can be more robust to noise and interference, as long as the underlying dimensionality is sufficiently high. A key intuition that we develop is that …
Adversarial training leads to clean data generalization with significant robust overfitting gap.
problem Significant robust generalization gap in adversarial training.
method Two theoretical views: representation complexity and training dynamics.
result ReLU nets with O(ND) extra parameters can achieve CGRO. New algorithm learns invariant representations for robust neural networks.
problem Learning robust neural network representations that are invariant to certain factors.
method Causal perspective and distribution matching approach.
result Empirically, the algorithm achieves state-of-the-art performance on domain generalization.
New method learns robust meta-representations for fast task adaptation.
problem Transferability of meta-representations in fast adaptation.
method Decoupled encoder-decoder approach with contrastive objective.
result Meta-representations improve downstream performance and noise robustness.
This paper aims to develop a new and robust approach to feature representation. Motivated by the success of Auto-Encoders, we first theoretical summarize the general properties of all algorithms that are based on traditional Auto-Encoders: 1) The reconstruction error of the input can not be lower than a lower bound, wh…
DBGAN learns graph node representations by balancing distribution consistency.
problem Graph representation learning overfits due to ignoring data distribution.
method DBGAN uses a structure-aware prior distribution and bidirectional adversarial learning.
result DBGAN achieves better trade-off between robustness and dimensionality.
Replicated and validated Rank-N-Contrast for robust regression.
problem Deep regression models struggle with continuous sample orders.
method Contrastive learning of continuous representations by ranking samples.
result Improved performance and robustness of RNC framework.
Class selectivity affects robustness to corruptions but not to adversarial attacks.
problem Understanding the relationship between class selectivity and robustness in neural networks.
method Investigated the impact of class selectivity on robustness to natural corruptions and adversarial attacks using Tiny ImageNetC and CIFAR10C datasets.
result Decreasing class selectivity increases robustness to both natural corruptions and adversarial attacks.
Study on robustness of unsupervised representation learning in slightly misspecified settings.
problem Identify nonlinear representation learning in slightly misspecified settings.
method Formalize and investigate Independent Component Analysis (ICA) with observations generated by a mixing function close to a local isometry.
result Approximate identifiability of nonlinear ICA with almost isometric mixing functions.
Unified proof of Teichmüller components using robust submanifolds.
problem Existence of higher Teichmüller components in representation spaces.
method Introducing robust families of submanifolds for a linear Lie group.
result Unified short proof of Teichmüller components for specific groups.
Robust reinforcement learning agents generalize well to out-of-distribution settings using pretrained representations.
problem Achieving sample-efficient reinforcement learning agents that generalize to real-world settings.
method Trained 240 representations and 10,000 RL policies on a simulated robotic setup, evaluating different pretrained VAE-based representations' effects on OOD generalization.
result Many reinforcement learning agents are surprisingly robust to realistic distribution shifts, including sim-to-real cases.
We show that there may exist an inherent tension between the goal of adversarial robustness and that of standard generalization. Specifically, training robust models may not only be more resource-consuming, but also lead to a reduction of standard accuracy. We demonstrate that this trade-off between the standard accura…
New examples of embeddings defy Anosov representation limits.
problem Examples of robust quasi-isometric embeddings not approximated by Anosov representations.
method Exhibited non-locally rigid, Zariski dense embeddings in SLm(K). result Higher rank Anosov representation theorems fail for m≥30. Multimodal sentiment analysis is a core research area that studies speaker sentiment expressed from the language, visual, and acoustic modalities. The central challenge in multimodal learning involves inferring joint representations that can process and relate information from these modalities. However, existing work l…
The ability to learn disentangled representations that split underlying sources of variation in high dimensional, unstructured data is important for data efficient and robust use of neural networks. While various approaches aiming towards this goal have been proposed in recent times, a commonly accepted definition and …
New method improves transfer and robustness of supervised contrastive learning.
problem Class collapse in supervised contrastive learning leads to poor representation quality.
method Adding a weighted class-conditional InfoNCE loss and a class-conditional autoencoder.
result Improves transfer and robustness on 5 standard datasets and 3 worst-group robustness datasets.
We propose a novel objective function for learning robust deep representations of data based on information theory. Data is projected into a feature-vector space such that the mutual information of all subsets of features relative to the supervising signal is maximized. This objective function gives rise to robust repr…
We present a sparse and invariant representation with low asymptotic complexity for robust unsupervised transient and onset zone detection in noisy environments. This unsupervised approach is based on wavelet transforms and leverages the scattering network from Mallat et al. by deriving frequency invariance. This frequ…