Proposes a novel framework for unsupervised domain adaptation using causal representations.
problem Transferability of deep model representations across domains is limited.
method Integrates causal inference into deep learning pipeline for domain-invariant feature learning.
result Demonstrates superior performance in unsupervised domain adaptation using causal representations.
Improves domain adaptation by clustering target representations.
problem Learning invariant and discriminative representations for unlabeled target domains.
method Simultaneously learns tightly clustered target representations and assigns each cluster to a unique class from the source.
result Achieves state-of-the-art performance in balanced, imbalanced, and partial domain adaptation.
Domain adaptation aims at generalizing a high-performance learner on a target domain via utilizing the knowledge distilled from a source domain which has a different but related data distribution. One solution to domain adaptation is to learn domain invariant feature representations while the learned representations sh…
A new method learns both global and local features for domain adaptation.
problem Lack of local relationship learning between instances in different domains.
method Dual autoencoders (MDAad and MMDA) for global and local feature learning, leveraging label information.
result Outperforms state-of-the-art methods in domain adaptation tasks.
Proposes a new approach for domain adaptation using latent representations.
problem Handling distribution shifts between source and target domains in high-dimensional data.
method Learn compact latent representations based on the label's Markov blanket, partitioning into parents, children, and spouses.
result General domain adaptation can be achieved by learning representations of the label's parents, children, and spouses.
VTAB benchmarks diverse visual tasks to assess representation learning effectiveness.
problem Lack of a unified evaluation for general visual representations.
method Developed VTAB, a benchmark for diverse visual tasks, and evaluated many representation learning algorithms.
result VTAB revealed insights into the effectiveness of various representation learning methods.
Paper shows deep nets can't guarantee domain adaptation without invariant features.
problem Learning domain-invariant features for successful domain adaptation.
method Proposes a counterexample and a new generalization upper bound.
result Shows conditional shift and characterizes a fundamental tradeoff.
Consider the problem: given the data pair (x,y) drawn from a population with f∗(x)=E[y∣x=x], specify a neural network model and run gradient flow on the weights over time until reaching any stationarity. How does ft, the function computed by the neural network…
Paper tackles robust domain adaptation without target domain data.
problem Learning domain invariant representations without target domain data.
method Integrates deep autoencoder and causal structure learning into a unified model.
result CAE learns causal representations using only source domain data.
FLAP adapts policies quickly to new tasks using shared linear representations.
problem Adapting policies to new tasks efficiently and effectively.
method FLAP uses a shared linear representation and a separate adapter network for quick adaptation.
result FLAP achieves up to 8X faster adaptation and significantly better performance on out-of-distribution tasks.
Contrastive learning adapts to data intrinsic dimensions, learning low-dimensional representations.
problem Learning high-dimensional representations from multi-modal data.
method Multi-modal contrastive learning with temperature optimization.
result Contrastive learning adapts to intrinsic dimensions of data, not specified dimensions.
Improved AutoDML estimator for causal inference using outcome-adapted shared covariate representation.
problem Efficiency in estimating treatment or policy effects in causal inference.
method Outcome-adapted AutoDML estimator that uses a shared covariate representation that is predictive of the outcome but not the Riesz representer.
result Outcome-adapted AutoDML estimator is asymptotically more efficient than baseline AutoDML.
MAML adapts faster with deeper architectures, especially in shallow tasks.
problem Understanding and improving MAML's fast adaptation.
method Empirical and theoretical studies on MAML's properties and optimization.
result MAML adapts better with deep architectures even for shallow tasks.
New approach tackles domain adaptation without assuming domain invariant representations.
problem Learning models on source domains for target domains with labeled and unlabeled data.
method Hidden Covariate Shift hypothesis; learning representation to match joint distributions.
result State-of-the-art performance on Amazon Reviews dataset.
Improves transferability of representations from source to target domains with weights and invariant representations.
problem Label shift between source and target domains in unsupervised domain adaptation.
method Integrates weights and invariant representations to bound the target risk, highlighting the role of inductive bias.
result Empirical evidence shows that weak inductive bias makes adaptation more robust.
New risk decompositions clarify domain adaptation issues.
problem Domain adaptation challenges with different training and test distributions.
method Representation Bayesian Risk Decompositions, hybrid argument.
result Clarifies factors (2) and (3) as reasons for generalization failure.
Fisher loss improves deep domain adaptation by learning discriminative within-class compact and between-class separable representations.
problem Improving deep domain adaptation performance by learning discriminative representations.
method Proposes a Fisher loss to learn discriminative representations that are within-class compact and between-class separable.
result Noticeable improvements in deep domain adaptation performance, e.g., 6.67% absolute improvement in mean accuracy on the Office-Home dataset.
AdaRL adapts quickly to new environments with minimal data.
problem Quickly adapting to new environments in reinforcement learning.
method AdaRL uses a parsimonious graphical representation to encode changes across domains.
result AdaRL can efficiently adapt policies to target domains with few samples.
Paper introduces RAS for robust MTL with contamination.
problem Representation-based multi-task learning struggles with contamination.
method Robust and Adaptive Spectral (RAS) method.
result RAS prevents negative transfer and performs well with up to 80% contamination.
New method learns robust meta-representations for fast task adaptation.
problem Transferability of meta-representations in fast adaptation.
method Decoupled encoder-decoder approach with contrastive objective.
result Meta-representations improve downstream performance and noise robustness.
The paper tackles learning from similar but not identical linear representations, improving performance over single-task learning.
problem Understanding how to learn from tasks with similar but not exactly the same linear representations, especially when dealing with outlier tasks.
method Proposes adaptive and robust penalized empirical risk minimization and spectral methods.
result Both methods outperform single-task learning when representations are similar and perform at least as well otherwise, with minimax optimality demonstrated.
Paper proposes a unified time series forecasting model with adaptive transfer.
problem General forecasting models for diverse time series data.
method Unified representations through Decomposed Frequency Learning and adaptive domain-specific features via Time Series Register.
result State-of-the-art forecasting performance on seven real-world benchmarks.
Spatial Adapter adds structured spatial representation to frozen predictors.
problem Efficiently adding spatial structure to pre-trained models.
method Structured spatial decomposition and closed-form covariance for residual fields.
result Adapter improves spatial prediction and uncertainty quantification.
Meta-GLAR combines global deep representations with local adaptation for improved forecasting accuracy.
problem Joint learning from related time series boosts accuracy but fails for out-of-sample forecasting.
method Meta-GLAR uses a meta-learning approach to adapt RNN representations for each time series.
result Meta-GLAR outperforms state-of-the-art methods in out-of-sample forecasting accuracy.
New method learns low-dimensional representations of AI-generated treatments.
problem Representing AI-generated treatments without losing semantic meaning.
method Double kernel representation learning with alternating minimization.
result Efficiently learned representations guide generative models and facilitate adaptive online experiments.
GmCN adapts GCN feature aggregation to improve graph learning.
problem Fixed neighborhood graph aggregation in GCNs is biased and can be affected by graph structure noises.
method GmCN allows nodes to adaptively select optimal neighbors for feature aggregation.
result GmCN improves graph learning effectiveness through adaptive neighbor selection.
DeepCCG adapts classifiers to representation shifts in one step.
problem Adapting classifiers to shifts in continuous representation.
method Empirical Bayesian approach using class conditional Gaussian classifier and KL divergence for selection.
result DeepCCG reduces performance change due to representation shifts.
New approach shows AI can adapt like toddlers by correcting old knowledge.
problem Lack of adaptability in AI models compared to humans and animals.
method Casting adaptation as posterior correction and using Bayesian Learning Rule.
result AI can learn to adapt quickly by using posterior correction.
Paper proposes ABDR for convex subspace clustering with adaptive block diagonal representation.
problem Subspace clustering with block diagonal structure for noisy data.
method ABDR explicitly pursues block diagonality without sacrificing convexity, using a specially designed convex regularizer.
result Experimental results show ABDR outperforms state-of-the-arts.
Semi-supervised anomaly detection with domain adaptation.
problem Anomaly detection with limited labeled data across domains.
method IRAD method: learns domain-invariant representations using adversarial learning and anomaly detector trained on these representations.
result IRAD outperforms baseline models across various datasets.
Paper investigates robustness to interference as a new training signal for meta-learning.
problem Improving incremental learning through robust representations.
method Directly minimizing catastrophic interference as a training signal.
result Representations learned to minimize interference lead to better incremental learning.
Can we effectively learn a nonlinear representation in time comparable to linear learning? We describe a new algorithm that explicitly and adaptively expands higher-order interaction features over base linear representations. The algorithm is designed for extreme computational efficiency, and an extensive experimental …
ACVAEs improve on CVAEs by learning more flexible latent correlations.
problem Learning latent representations with correlated structure.
method Adaptive prior distribution and belief propagation.
result ACVAEs outperform CVAEs in link prediction and hierarchical clustering.
Deep learning has recently been shown to be instrumental in the problem of domain adaptation, where the goal is to learn a model on a target domain using a similar --but not identical-- source domain. The rationale for coupling both techniques is the possibility of extracting common concepts across domains. Considering…
New bounds for unsupervised domain adaptation account for non-invertibility and support coverage.
problem Theoretical arguments for domain-invariant representations are flawed and do not account for non-invertibility and support coverage.
method Generalization bounds for any representation function acknowledging the cost of non-invertibility and penalizing distance between densities.
result Proposed bounds based on support coverage provide better generalization than current standard practice.
AdaPTS adapts univariate FMs for multivariate time series forecasting.
problem Challenges in managing feature dependencies and uncertainty quantification in multivariate time series forecasting.
method Adapters that transform multivariate inputs into a latent space and apply univariate FMs independently to each dimension.
result AdaPTS enhances forecasting accuracy and uncertainty quantification compared to baseline methods.
New approach improves domain adaptation with label shift assumptions.
problem Improving domain adaptation when label distributions differ between source and target domains.
method Proposes generalized label shift (GLS) and modifies three DA algorithms (JAN, DANN, CDAN) to handle label distribution mismatches. result Modified DA algorithms outperform base versions, especially with large label distribution mismatches.
New findings on representation changes in transfer learning.
problem Understanding representation changes in transfer learning.
method Analysis of pre-trained and fine-tuned representations in transfer learning.
result Representation changes mainly occur in the head, but early layers are also important for cross-domain adaptation.
We consider the problem of non-parametric regression with a potentially large number of covariates. We propose a convex, penalized estimation framework that is particularly well-suited for high-dimensional sparse additive models. The proposed approach combines appealing features of finite basis representation and smoot…
DSCF-Net learns deep features for clustering with robustness and locality preservation.
problem Unsupervised deep representation learning for clustering.
method Integrates robust deep concept factorization, deep self-expressive representation, and adaptive locality preserving feature learning.
result Delivers state-of-the-art performance on public databases.
Domain adaptation leverages the knowledge in one domain - the source domain - to improve learning efficiency in another domain - the target domain. Existing heterogeneous domain adaptation research is relatively well-progressed, but only in situations where the target domain contains at least a few labeled instances. I…
We adapt some of the methods of quantum Teichmüller theory to construct a family of representations of the pure braid group of the sphere.
Proposes using gradients as features for efficient deep learning adaptation.
problem Efficient deep representation learning for different tasks.
method Designs a linear model incorporating gradients and activations of a pre-trained network.
result Shows strong results across various tasks and datasets.
Study on feature learning in Leaky ResNets, explaining bottleneck structure.
problem Understanding feature learning in deep neural networks.
method Lagrangian and Hamiltonian reformulation of representation geodesics.
result Emergence of a bottleneck structure in large effective depth networks.
The paper proposes using a discriminator for both domain adaptation and pseudo labeling confidence.
problem Improving generalization of classifiers trained on labeled source data to unlabeled target data.
method Multi-purposing the discriminator to learn domain-invariant feature representations and generate pseudo labels based on confidence.
result The approach enhances classifier performance by providing confidence measures for pseudo labels.
Paper tackles domain adaptation without labeled target data.
problem Performing well on an unlabeled target domain using only labeled source data.
method Learning self-supervised tasks on both source and target domains simultaneously.
result Successfully generalizes to the unlabeled target domain.
Adaptive tensor modeling preserves continuity in multidimensional data.
problem Discretization of continuous multidimensional data loses important information.
method Functional Tucker decomposition (FTD) with RKHS modeling.
result FTD enables adaptive and expressive tensor modeling.
A tensor model for meta-learning adapts to task-specific features.
problem Learning shared representations for diverse tasks without task-specific observable information.
method Modeling meta-parameters as an order-3 tensor, estimating through tensor regression and method of moments.
result Tensor-based approach improves meta-learning performance with fewer samples.