arXiv research

Adaptive networks improve model robustness through conditional normalization.

Enhances physics-informed neural networks with adaptive sampling and weighting.

learn2mix trains neural nets faster by adjusting class proportions dynamically.

Self-training improves gradual domain adaptation with unlabeled data.

AdAdaGrad optimizes batch sizes for deep learning models, reducing the generalization gap.

Self-adaptive training improves deep learning robustness.

Adaptive kernel density estimation improves accuracy in high dimensions.

Paper proposes an adaptive gradient method for federated learning.

Unified framework for adaptive connection sampling in GNNs improves performance and robustness.

Adaptive gradient methods, which adopt historical gradient information to automatically adjust the learning rate, despite the nice property of fast convergence, have been observed to generalize worse than stochastic gradient descent (SGD) with momentum in training deep neural networks. This leaves how to close the gene…

We present ARU, an Adaptive Recurrent Unit for streaming adaptation of deep globally trained time-series forecasting models. The ARU combines the advantages of learning complex data transformations across multiple time series from deep global models, with per-series localization offered by closed-form linear models. Un…

FPGA-based multi-layer equalizer adapts to changing channels.

Prodigy estimates learning rate without tuning, improving convergence.

Adaptive methods optimize machine learning training faster.

Adaptive batch size schedules improve language model training efficiency and generalization.

End-to-end training of automated speech recognition (ASR) systems requires massive data and compute resources. We explore transfer learning based on model adaptation as an approach for training ASR models under constrained GPU memory, throughput and training data. We conduct several systematic experiments adapting a Wa…

New model-based methods adapt pre-trained policies to unseen environments efficiently.

Gradient-based meta-learning has proven to be highly effective at learning model initializations, representations, and update rules that allow fast adaptation from a few samples. The core idea behind these approaches is to use fast adaptation and generalization -- two second-order metrics -- as training signals on a me…

Supporting evidence for adaptive feature program across diverse models.

Adaptive batch sizes improve local gradient methods in distributed training.

The paper improves generalization bounds for domain adaptation.

AdaScale SGD adapts learning rates for large-batch training efficiently.

A probabilistic framework for online test-time adaptation

Stochastic methods with coordinate-wise adaptive stepsize (such as RMSprop and Adam) have been widely used in training deep neural networks. Despite their fast convergence, they can generalize worse than stochastic gradient descent. In this paper, by revisiting the design of Adagrad, we propose to split the network par…

This paper explains why Adam generalizes worse than SGD by analyzing its components.

Discriminative active learning reduces data annotation costs for domain adaptation.

Conformal Bayes under label shift: post-hoc calibration vs. in-training adaptation

Adaptive optimization methods, which perform local optimization with a metric constructed from the history of iterates, are becoming increasingly popular for training deep neural networks. Examples include AdaGrad, RMSProp, and Adam. We show that for simple overparameterized problems, adaptive methods often find drasti…

New method adapts without backprop, faster and better.

Bayesian method improves SOM training for dynamic data.

Paper introduces a new adaptive gradient method with gradient compression for distributed training.

This thesis studies domain adaptation under minimal distribution similarity assumptions using moments.

Domain adaptation is the supervised learning setting in which the training and test data are sampled from different distributions: training data is sampled from a source domain, whilst test data is sampled from a target domain. This paper proposes and studies an approach, called feature-level domain adaptation (FLDA), …

A novel approach of training data augmentation and domain adaptation is presented to support machine learning applications for cognitive radio. Machine learning provides effective tools to automate cognitive radio functionalities by reliably extracting and learning intrinsic spectrum dynamics. However, there are two im…

Local adaptive methods in FL can accelerate convergence but introduce bias, which is corrected.

AMF-VI uses adaptive mixtures of flows for robust VI across diverse distributions.

DIVA generates diverse tasks for complex simulators, enabling adaptive agent training.

We consider unsupervised domain adaptation: given labelled examples from a source domain and unlabelled examples from a related target domain, the goal is to infer the labels of target examples. Under the assumption that features from pre-trained deep neural networks are transferable across related domains, domain adap…

This paper improves neural network generalization by dynamically learning kernel parameters.

Edge devices adapt pre-trained models to local data without backpropagation.

New method trains deep networks robustly without adaptive methods.

We present an adaptive approach to the construction of Gaussian process surrogates for Bayesian inference with expensive-to-evaluate forward models. Our method relies on the fully Bayesian approach to training Gaussian process models and utilizes the expected improvement idea from Bayesian global optimization. We adapt…

New method learns models to adapt to domain shifts at test time.

ADCMs adaptively discretize CMs for efficient training.

Paper tackles adaptive deletion of data points from trained models.

AGMMNs improve learning of copula models by adaptively selecting kernels.

This paper improves volatility forecasting using dynamic subset selection in genetic programming.

DPTA improves CIL by adapting PTMs with dual prototypes.