arXiv research

A new method for efficient neural network fine-tuning using queryable low-rank update atoms.

Local adaptive methods in FL can accelerate convergence but introduce bias, which is corrected.

A new method improves few-shot image classification by updating top layers.

Adaptive sparse GP model for non-stationary data.

Test-time training adapts a pretrained model to each prompt via parameter updates, improving accuracy under pretraining-to-test distribution shifts.

Meta-Reinforcement learning approaches aim to develop learning procedures that can adapt quickly to a distribution of tasks with the help of a few examples. Developing efficient exploration strategies capable of finding the most useful samples becomes critical in such settings. Existing approaches towards finding effic…

Two algorithms achieve optimal regret with limited adaptivity in multinomial logistic bandits.

New algorithm optimizes MCMC sampling for structural dynamic models.

AdaQuantFL reduces communication in federated learning by adaptively quantizing model updates.

For large scale on-line inference problems the update strategy is critical for performance. We derive an adaptive scan Gibbs sampler that optimizes the update frequency by selecting an optimum mini-batch size. We demonstrate performance of our adaptive batch-size Gibbs sampler by comparing it against the collapsed Gibb…

AMUSE uses reinforcement learning to predict optimal model updates.

We consider 1-qubit mixed quantum state estimation by adaptively updating measurements according to previously obtained outcomes and measurement settings. Updates are determined by the average-variance-optimality (A-optimality) criterion, known in the classical theory of experimental design and applied here to quantum …

Two algorithms address limited adaptivity in generalized linear contextual bandits.

Selective state-adaptive regularization improves offline RL performance.

ADER addresses continual learning in session-based recommendation by periodically replaying exemplars with adaptive distillation.

Diffusion models adapt to low-dimensional data regardless of coefficient choices.

CILF learns adaptive embeddings for class-incremental learning with novel class detection and model update.

Accelerates policy optimization in RL with optimistic and adaptive updates.

CAdam optimizes online learning by adapting to distribution shifts and noise.

A new method improves EEG classification across subjects efficiently.

AdaDPIGU improves privacy in deep learning by adaptively clipping and pruning gradients.

Spectral Adaptive Conformal Prediction for Structured Non-Exchangeable Data

VCoTTA uses variational Bayesian methods to adapt models under continuous domain shifts.

DoubleAdapt improves stock trend forecasting by adapting models to evolving data.

Flora uses random projections to achieve high-rank updates with low memory usage.

Adaptive XGBoost improves accuracy on evolving data streams by updating the ensemble dynamically.

Coordinate descent methods employ random partial updates of decision variables in order to solve huge-scale convex optimization problems. In this work, we introduce new adaptive rules for the random selection of their updates. By adaptive, we mean that our selection rules are based on the dual residual or the primal-du…

The present paper proposes generalized Gaussian kernel adaptive filtering, where the kernel parameters are adaptive and data-driven. The Gaussian kernel is parametrized by a center vector and a symmetric positive definite (SPD) precision matrix, which is regarded as a generalization of the scalar width parameter. These…

AdaptOn achieves logarithmic regret in adaptive control of unknown partially observable linear systems.

We present ARU, an Adaptive Recurrent Unit for streaming adaptation of deep globally trained time-series forecasting models. The ARU combines the advantages of learning complex data transformations across multiple time series from deep global models, with per-series localization offered by closed-form linear models. Un…

Sparse Inverse Covariance Estimation (SICE) is useful in many practical data analyses. Recovering the connectivity, non-connectivity graph of covariates is classified amongst the most important data mining and learning problems. In this paper, we introduce a novel SICE approach using adaptive thresholding. Our method i…

Mini-batch stochastic gradient descent (SGD) and variants thereof approximate the objective function's gradient with a small number of training examples, aka the batch size. Small batch sizes require little computation for each model update but can yield high-variance gradient estimates, which poses some challenges for…

CPATTA uses conformal prediction for efficient test-time adaptation.

NAC-FL optimizes model updates in FL systems by adapting compression to network congestion.

We present a confidence-based single-layer feed-forward learning algorithm SPIRAL (Spike Regularized Adaptive Learning) relying on an encoding of activation spikes. We adaptively update a weight vector relying on confidence estimates and activation offsets relative to previous activity. We regularize updates proportion…

This work analyzes the statistical properties of adaptive gradient methods.

Unified analysis for decentralized SGD across various topologies and updates.

Paper proposes faster adaptation to distribution shifts in online settings.

Step-DAD improves BED by periodically updating a design policy during experiments.

A TTA framework improves forecasting accuracy in non-stationary time series.

M-FISHER detects and adapts to streaming data shifts with statistical validity and stability.

Shampoo achieves higher token efficiency than Muon in language models.

Paper proposes SRA algorithm for online learning robustness and adaptivity.

Method adapts frozen models for few-shot tasks without training.

Optimization algorithms with momentum, e.g., (ADAM), have been widely used for building deep learning models due to the faster convergence rates compared with stochastic gradient descent (SGD). Momentum helps accelerate SGD in the relevant directions in parameter updating, which can minify the oscillations of parameter…

Unseen or out-of-domain data can seriously degrade the performance of a neural network model, indicating the model's failure to generalize to unseen data. Neural net pruning can not only help to reduce a model's size but can improve the model's generalization capacity as well. Pruning approaches look for low-salient ne…

Gradient-EM Bayesian meta-learning accelerates adaptation with reduced computation and improved robustness.

We consider a generic convex-concave saddle point problem with separable structure, a form that covers a wide-ranged machine learning applications. Under this problem structure, we follow the framework of primal-dual updates for saddle point problems, and incorporate stochastic block coordinate descent with adaptive st…