arXiv research

Adaptive networks improve model robustness through conditional normalization.

Enhances physics-informed neural networks with adaptive sampling and weighting.

learn2mix trains neural nets faster by adjusting class proportions dynamically.

AdAdaGrad optimizes batch sizes for deep learning models, reducing the generalization gap.

Adaptive kernel density estimation improves accuracy in high dimensions.

Paper proposes an adaptive gradient method for federated learning.

Unified framework for adaptive connection sampling in GNNs improves performance and robustness.

We propose self-adaptive training---a new training algorithm that dynamically corrects problematic training labels by model predictions without incurring extra computational cost---to improve generalization of deep learning for potentially corrupted training data. This problem is crucial towards robustly learning from …

Adaptive gradient methods, which adopt historical gradient information to automatically adjust the learning rate, despite the nice property of fast convergence, have been observed to generalize worse than stochastic gradient descent (SGD) with momentum in training deep neural networks. This leaves how to close the gene…

We present ARU, an Adaptive Recurrent Unit for streaming adaptation of deep globally trained time-series forecasting models. The ARU combines the advantages of learning complex data transformations across multiple time series from deep global models, with per-series localization offered by closed-form linear models. Un…

FPGA-based multi-layer equalizer adapts to changing channels.

Prodigy estimates learning rate without tuning, improving convergence.

Adaptive batch size schedules improve language model training efficiency and generalization.

End-to-end training of automated speech recognition (ASR) systems requires massive data and compute resources. We explore transfer learning based on model adaptation as an approach for training ASR models under constrained GPU memory, throughput and training data. We conduct several systematic experiments adapting a Wa…

New model-based methods adapt pre-trained policies to unseen environments efficiently.

Gradient-based meta-learning has proven to be highly effective at learning model initializations, representations, and update rules that allow fast adaptation from a few samples. The core idea behind these approaches is to use fast adaptation and generalization -- two second-order metrics -- as training signals on a me…

Supporting evidence for adaptive feature program across diverse models.

Machine learning systems must adapt to data distributions that evolve over time, in applications ranging from sensor networks and self-driving car perception modules to brain-machine interfaces. We consider gradual domain adaptation, where the goal is to adapt an initial classifier trained on a source domain given only…

Adaptive batch sizes improve local gradient methods in distributed training.

AdaScale SGD adapts learning rates for large-batch training efficiently.

A probabilistic framework for online test-time adaptation

Stochastic methods with coordinate-wise adaptive stepsize (such as RMSprop and Adam) have been widely used in training deep neural networks. Despite their fast convergence, they can generalize worse than stochastic gradient descent. In this paper, by revisiting the design of Adagrad, we propose to split the network par…

This paper explains why Adam generalizes worse than SGD by analyzing its components.

Discriminative active learning reduces data annotation costs for domain adaptation.

Conformal Bayes under label shift: post-hoc calibration vs. in-training adaptation

Adaptive optimization methods, which perform local optimization with a metric constructed from the history of iterates, are becoming increasingly popular for training deep neural networks. Examples include AdaGrad, RMSProp, and Adam. We show that for simple overparameterized problems, adaptive methods often find drasti…

New method adapts without backprop, faster and better.

Bayesian method improves SOM training for dynamic data.

Domain adaptation algorithms are designed to minimize the misclassification risk of a discriminative model for a target domain with little training data by adapting a model from a source domain with a large amount of training data. Standard approaches measure the adaptation discrepancy based on distance measures betwee…

Paper introduces a new adaptive gradient method with gradient compression for distributed training.

This thesis studies domain adaptation under minimal distribution similarity assumptions using moments.

Domain adaptation is the supervised learning setting in which the training and test data are sampled from different distributions: training data is sampled from a source domain, whilst test data is sampled from a target domain. This paper proposes and studies an approach, called feature-level domain adaptation (FLDA), …

A novel approach of training data augmentation and domain adaptation is presented to support machine learning applications for cognitive radio. Machine learning provides effective tools to automate cognitive radio functionalities by reliably extracting and learning intrinsic spectrum dynamics. However, there are two im…

Local adaptive methods in FL can accelerate convergence but introduce bias, which is corrected.

AMF-VI uses adaptive mixtures of flows for robust VI across diverse distributions.

DIVA generates diverse tasks for complex simulators, enabling adaptive agent training.

We consider unsupervised domain adaptation: given labelled examples from a source domain and unlabelled examples from a related target domain, the goal is to infer the labels of target examples. Under the assumption that features from pre-trained deep neural networks are transferable across related domains, domain adap…

This paper improves neural network generalization by dynamically learning kernel parameters.

Edge devices adapt pre-trained models to local data without backpropagation.

New method trains deep networks robustly without adaptive methods.

We present an adaptive approach to the construction of Gaussian process surrogates for Bayesian inference with expensive-to-evaluate forward models. Our method relies on the fully Bayesian approach to training Gaussian process models and utilizes the expected improvement idea from Bayesian global optimization. We adapt…

New method learns models to adapt to domain shifts at test time.

Deep Reinforcement Learning (RL) is proven powerful for decision making in simulated environments. However, training deep RL model is challenging in real world applications such as production-scale health-care or recommender systems because of the expensiveness of interaction and limitation of budget at deployment. One…

ADCMs adaptively discretize CMs for efficient training.

Paper tackles adaptive deletion of data points from trained models.

AGMMNs improve learning of copula models by adaptively selecting kernels.

This paper improves volatility forecasting using dynamic subset selection in genetic programming.

DPTA improves CIL by adapting PTMs with dual prototypes.