Paper presents a Transformer model for automatic domain adaptation.
problem Challenges in selecting or designing domain adaptation algorithms.
method Transformer model approximates and selects domain adaptation algorithms.
result Transformers can approximate and automatically select domain adaptation algorithms.
Transformer models show robustness across domains with domain adversarial training.
problem Domain adaptation from multiple sources with no labeled data.
method Domain adversarial training and mixture of experts.
result Domain adversarial training improves representation but not performance.
Learning multiple tasks across heterogeneous domains is a challenging problem since the feature space may not be the same for different tasks. We assume the data in multiple tasks are generated from a latent common domain via sparse domain transforms and propose a latent probit model (LPM) to jointly learn the domain t…
Proves nonemptyness of domains for specific group actions.
problem Nonemptyness of domains of proper discontinuity for Anosov groups of affine Lorentzian transformations.
method Proof of nonemptyness of domains of proper discontinuity.
result Proves nonemptyness of domains for Anosov groups of affine Lorentzian transformations.
This work improves Fourier pricing for multi-asset options using RQMC with domain transformation.
problem Efficiently pricing multi-asset options in high dimensions with Fourier methods.
method Randomized quasi-Monte Carlo (RQMC) with domain transformation to handle singularities.
result RQMC with domain transformation provides accurate and scalable Fourier pricing for multi-asset options.
The characteristics (or numerical patterns) of a feature vector in the transform domain of a perturbation model differ significantly from those of its corresponding feature vector in the input domain. These differences - caused by the perturbation techniques used for the transformation of feature patterns - degrade the…
New method eliminates domain size restrictions for X-ray transform inversion.
problem Injectivity and stability of X-ray transform in convex domains.
method Semiclassical analysis to invert X-ray transform without small domain assumptions.
result Elimination of domain size restrictions for injectivity and stability.
Transformers adapted to spherical geometry using space-filling curves.
problem Generalizing transformers to geometric domains like spheres.
method Attention heads following a space-filling curve.
result Introduction of the Spiroformer on a 2-sphere.
Method uses JPEG transform for faster image classification.
problem Efficient image classification with compressed data.
method Reformulates residual networks for JPEG compressed images.
result Mathematically equivalent to spatial domain networks up to ReLu approximation.
OLinear forecasts time series more efficiently by transforming data orthogonally.
problem Efficiently forecasting time series with entangled dependencies.
method OLinear uses OrthoTrans to transform data orthogonally, then applies NormLin for linear layer.
result OLinear achieves state-of-the-art performance with high efficiency.
Extends SW and GSW to compare heterogeneous joint distributions.
problem Limited applicability of SW and GSW to heterogeneous joint distributions.
method Introduces HHRT and PGRT to extend SW and GSW.
result H2SW distance for heterogeneous joint distributions.
Functions with constant geodesic X-ray transform are restricted to manifolds with specific geometrical properties.
problem Existence of functions with constant geodesic X-ray transform on manifolds.
method Analyzing the geometrical properties of manifolds based on the existence of such functions.
result Functions with constant geodesic X-ray transform impose specific geometrical restrictions on the manifold.
This work tackles OOD generalization by leveraging causal invariance without needing to recover causal features.
problem Learning models that perform well on out-of-distribution (OOD) data.
method Causal invariant transformations to modify non-causal features while preserving causal parts.
result Theoretical and practical methods to learn a minimax optimal model across domains using single domain data.
A new method for estimating sparse inverse covariance matrices.
problem Recovering the connectivity and non-connectivity graph of covariates.
method Adaptive thresholding in a transformed domain of the inverse covariance matrix.
result The proposed method outperforms state-of-the-art methods in accuracy.
Stability is a key aspect of data analysis. In many applications, the natural notion of stability is geometric, as illustrated for example in computer vision. Scattering transforms construct deep convolutional representations which are certified stable to input deformations. This stability to deformations can be interp…
Transforms game optimization dynamics into frequency domain for precise hyperparameter analysis.
problem Analyzing convergence of hyperparameters in game optimization.
method Frequency-domain framework using High-Resolution Differential Equations (HRDEs) and Laplace transforms.
result Derives precise convergence criteria for the Lookahead algorithm.
This research studies affine invariance in continuous-domain convolutional neural networks.
problem Recognizing patterns and features under affine transformations in continuous domains.
method Introduces a new criterion for assessing affine invariance, embeds images into the affine Lie group, and analyzes convolution over this group.
result Extends the scope of geometrical transformations that deep-learning pipelines can handle.
TransCORALNet uses transformer and CORAL for supply chain credit assessment with cold start.
problem Supply chain credit assessment for new borrowers with limited data.
method Two-stream transformer CORAL networks with domain adaptation and LIME.
result TransCORALNet outperforms state-of-the-art models in accuracy.
Data augmentation is a ubiquitous technique for increasing the size of labeled training sets by leveraging task-specific data transformations that preserve class labels. While it is often easy for domain experts to specify individual transformations, constructing and tuning the more sophisticated compositions typically…
Paper proposes linear transformers for efficient in-context learning without context length limitations.
problem Quadratic complexity of softmax transformers limits data processing speed.
method Investigates linear transformers under domain generalization, showing they learn mappings from context distributions to response functions.
result Linear transformers achieve in-context learning with a linear complexity in context length, offering a dimension-independent convergence rate.
Images seen during test time are often not from the same distribution as images used for learning. This problem, known as domain shift, occurs when training classifiers from object-centric internet image databases and trying to apply them directly to scene understanding tasks. The consequence is often severe performanc…
Previous research has shown that computation of convolution in the frequency domain provides a significant speedup versus traditional convolution network implementations. However, this performance increase comes at the expense of repeatedly computing the transform and its inverse in order to apply other network operati…
Non-negative blind source separation (non-negative BSS), which is also referred to as non-negative matrix factorization (NMF), is a very active field in domains as different as astrophysics, audio processing or biomedical signal processing. In this context, the efficient retrieval of the sources requires the use of sig…
MDA learns domain-invariant features for better target domain classification.
problem Improving model performance on unseen target domains using multiple source domains.
method MDA learns a domain-invariant feature transformation with minimal divergence, maximal separability, and compactness.
result MDA achieves better generalization on unseen target domains compared to existing methods.
A multi-step framework tackles online unsupervised domain adaptation with novel mean-target subspace computation.
problem Online unsupervised domain adaptation with unlabelled target data arriving sequentially.
method Multi-step framework with a novel mean-target subspace computation and temporal coherency consideration.
result Improved performance over previous approaches on four datasets.
dAUTOMAP scales AUTOMAP by decomposing domain transformation.
problem Inadequate scalability limits AUTOMAP's practicality.
method Decomposes AUTOMAP's domain transformation for linear scalability.
result dAUTOMAP outperforms AUTOMAP with fewer parameters.
A new MLDS model captures nonlinear tensor time series with improved accuracy and efficiency.
problem Modeling and analyzing nonlinear tensor time series data.
method Transform-based multilinear dynamical system (MLDS) with EM algorithm for parameter estimation.
result Significantly higher prediction accuracy and exponential improvement in training time compared to state-of-the-art models.
Proposes AWS method for precise speech enhancement using DNN.
problem T-F resolution problem in fixed-resolution short-time frequency transforms.
method Incorporates trainable adaptive window switching into speech enhancement procedure.
result Achieved higher signal-to-distortion ratio than conventional methods.
Theory of packing diabolic domains in liquid crystals.
problem Understanding the packing of diabolic domains in liquid crystals.
method Lorentz transformations and geometric analysis.
result Diabolic domains can lower the elastic energy of the system.
New method reduces label and data shifts between domains using optimal transport.
problem Label shift between source and target domains in domain adaptation.
method Developed theory and LDROT method to mitigate label and data shifts.
result Theoretical and experimental validation of LDROT's effectiveness.
URT layer improves few-shot image classification across diverse domains.
problem Few-shot image classification in multi-domain settings.
method Meta-learns to dynamically re-weight and compose domain-specific representations.
result Sets new state-of-the-art on Meta-Dataset.
The large-N limit of Segal-Bargmann transform on spheres is studied.
problem Understanding the behavior of Segal-Bargmann transform on spheres as dimension increases.
method Analyzing the large-N limit of the transform on S N − 1 ( N ) S^{N-1}(\sqrt N) S N − 1 ( N ) , describing geometric models, and showing the transform remains unitary. result The limiting transform is still a unitary map from the limiting domain onto the limiting range.
Paper studies Transformer learning theory for Euclidean and Riemannian domains.
problem Understanding and optimizing Transformer networks for regression tasks.
method Constructive approximation framework using softmax partition of unity and attention mechanism.
result Transformer can achieve uniform ε-approximation error with minimal parameters.
Transformer-based multi-scale model outperforms traditional methods in solving PDEs on irregular domains.
problem Solving partial differential equations on irregular domains using deep learning.
method Introduces Multi-Scale Attention Transformer (\msat{}) for solving PDEs.
result Achieves state-of-the-art generalization on complex geometry problems with significant speedup.
SIG model identifies invariant variables for MSDA with fewer domain constraints.
problem Challenges in enforcing minimal changes across domains for MSDA.
method Subspace identification theory and variational inference.
result SIG model outperforms existing techniques on various benchmark datasets.
Develops a method to learn metrics across multiple domains using heterogeneous transfer learning.
problem Limited labeled data in target domain and heterogeneous data across multiple domains.
method HMTML framework that learns metrics and transformations across multiple domains, maximizing high-order covariance in a common subspace.
result Effective feature transformations and metrics learned across multiple domains, validated by extensive experiments.
Dataset augmentation, the practice of applying a wide array of domain-specific transformations to synthetically expand a training set, is a standard tool in supervised learning. While effective in tasks such as visual recognition, the set of transformations must be carefully designed, implemented, and tested for every …
CATS adapts multivariate time series models by addressing correlation shift.
problem Correlation differences across domains in multivariate time series data.
method CATS introduces correlation shift to measure domain differences, and uses a graph attention module and temporal convolution to align target correlations with source correlations.
result CATS increases over 10% average accuracy compared to vanilla Transformer-based models with minimal additional parameters.
A new iterative low complexity algorithm has been presented for computing the Walsh-Hadamard transform (WHT) of an N N N dimensional signal with a K K K -sparse WHT, where N N N is a power of two and K = O ( N α ) K = O(N^α) K = O ( N α ) , scales sub-linearly in N N N for some 0 < α < 1 0 < α< 1 0 < α < 1 . Assuming a random support model for the non-zero transform domain…
Transformers can learn new tasks from diverse pretraining data but struggle with out-of-domain tasks.
problem Transformer models' ability to learn new tasks in-context is limited by their pretraining data coverage.
method Investigation of transformer models trained on ( x , f ( x ) ) (x, f(x)) ( x , f ( x )) pairs, comparing in-context learning capabilities across different task families. result Transformers can identify and learn within task families in their pretraining data but fail with out-of-domain tasks.
Reconstructing a planar domain from its Dirichlet-to-Neumann data
problem Reconstructing a planar domain from its Dirichlet-to-Neumann data
method Using the Hilbert transform of the boundary curve
result Reconstructing a simply connected planar domain from the DN data
Graph scattering transforms are stable to metric perturbations of network topology.
problem Stability of graph data representations under metric perturbations.
method Extending scattering transforms to network data using multiresolution graph wavelets and graph convolutions.
result Graph scattering transforms are stable to metric perturbations of the underlying network topology.
Transformer architecture struggles with complex tasks due to limitations in function composition.
problem Transformer architecture's limitations in handling complex tasks.
method Used Communication Complexity to prove limitations in composing functions.
result Transformer layer is incapable of handling large domain functions, even when domains are small.
Proposes a measure to predict generalization in non-matching environments.
problem Characterizing and comparing generalization of machine learning models in non-matching environments.
method Neighborhood invariance measure, calculating invariance as the largest fraction of transformed points classified into the same class.
result Strong and robust correlation between neighborhood invariance and actual out-of-domain generalization.
CWAN tackles multi-source heterogeneous domain adaptation with conditional weighting.
problem Learning cross-domain samples from multiple heterogeneous domains.
method CWAN uses a feature transformer, label classifier, and domain discriminator to learn from multiple sources.
result CWAN outperforms state-of-the-art methods on four real-world datasets.
Survey explores methods to adapt deep learning models across multiple labeled domains.
problem Difficulty in obtaining labeled data for deep learning models.
method Multi-source domain adaptation (MDA) to transfer knowledge from labeled to unlabeled or sparsely labeled target domains.
result MDA methods improve performance by minimizing domain shift.
GWIL uses Gromov-Wasserstein distance to align expert and imitation agent states.
problem Cross-domain imitation learning challenges due to different system dimensions and stationary distributions.
method Gromov-Wasserstein Imitation Learning (GWIL) using Gromov-Wasserstein distance.
result GWIL effectively aligns expert and imitation agent states in various continuous control domains.
A multi-task learning model for slot tagging in biomedical domains.
problem Limited labeled data, memory constraints, and domain-specific slot types.
method Multi-task learning using deep bidirectional transformers.
result Outperforms previous state-of-the-art systems in efficiency and effectiveness.