arXiv research

PrIU optimizes machine learning model updates after data cleaning.

Framework for safely updating machine learning models.

A new FL algorithm reduces communication overhead by selectively updating model parameters.

A new method for efficient neural network fine-tuning using queryable low-rank update atoms.

Synthetic data replaces model updates in federated learning.

In several recently proposed stochastic optimization methods (e.g. RMSProp, Adam, Adadelta), parameter updates are scaled by the inverse square roots of exponential moving averages of squared past gradients. Maintaining these per-parameter second-moment estimators requires memory equal to the number of parameters. For …

Two algorithms for linear contextual bandits with rare updates achieve optimal regret and efficiency.

For a long investment time horizon, it is preferable to rebalance the portfolio weights at intermediate times. This necessitates a multi-period market model in which portfolio optimization is usually done through dynamic programming. However, this assumes a known distribution for the parameters of the financial time se…

New PG samplers improve inference in coupled state-space models.

Gradient-based methods introduce noise that penalizes models sensitive to weight perturbations.

Adaptive sparse GP model for non-stationary data.

Test-time training adapts a pretrained model to each prompt via parameter updates, improving accuracy under pretraining-to-test distribution shifts.

Bayesian model updating uses VAEs to approximate likelihood with small data.

FiT combines transfer and meta-learning for efficient few-shot image classification.

Study on rich regime training in deep learning, finding active parameters in bottom layers.

CAdam optimizes online learning by adapting to distribution shifts and noise.

This paper re-examines the problem of parameter estimation in Bayesian networks with missing values and hidden variables from the perspective of recent work in on-line learning [Kivinen & Warmuth, 1994]. We provide a unified framework for parameter estimation that encompasses both on-line learning, where the model is c…

LDP-Fed protects privacy in federated learning with neural networks.

Model analyzes cooccurrence data for recommender systems and item relevance.

The paper deals with learning probability distributions of observed data by artificial neural networks. We suggest a so-called gradient conjugate prior (GCP) update appropriate for neural networks, which is a modification of the classical Bayesian update for conjugate priors. We establish a connection between the gradi…

Transformers can implement reinforcement learning algorithms from data without updates.

New algorithm closes empirical gap in PFSGD performance.

In this paper we revisit the weighted likelihood bootstrap, a method that generates samples from an approximate Bayesian posterior of a parametric model. We show that the same method can be derived, without approximation, under a Bayesian nonparametric model with the parameter of interest defined as minimising an expec…

New optimizer designs respect symmetry, improving deep learning models.

Here, we study different update rules in stochastic gradient descent (SGD) for online forecasting problems. The selection of the learning rate parameter is critical in SGD. However, it may not be feasible to tune this parameter in online learning. Therefore, it is necessary to have an update rule that is not sensitive …

In-Place TTT enhances LLMs with dynamic parameter updates at inference time.

Optimization algorithms with momentum, e.g., (ADAM), have been widely used for building deep learning models due to the faster convergence rates compared with stochastic gradient descent (SGD). Momentum helps accelerate SGD in the relevant directions in parameter updating, which can minify the oscillations of parameter…

Improved HGF networks avoid negative precision errors in volatility updates.

A new update rule for deep reinforcement learning reduces learning variance and variance in reference signals.

Federated learning distributes model training among a multitude of agents, who, guided by privacy concerns, perform training using their local data but share only model parameter updates, for iterative aggregation at the server. In this work, we explore the threat of model poisoning attacks on federated learning initia…

New algorithm uses PSO to optimize DNN training parameters in distributed systems.

Develops efficient method for updating models with small data changes.

This work introduces a fixed-point optimization for variational inference.

Fewer data weight updates lead to faster convergence in machine learning models.

Non-negative tensor factorization models enable predictive analysis on count data. Among them, Bayesian Poisson-Gamma models can derive full posterior distributions of latent factors and are less sensitive to sparse count data. However, current inference methods for these Bayesian models adopt restricted update rules f…

Algorithm learns Sherrington-Kirkpatrick model parameters at low temperatures.

Collaborative learning allows participants to jointly train a model without data sharing. To update the model parameters, the central server broadcasts model parameters to the clients, and the clients send updating directions such as gradients to the server. While data do not leave a client device, the communicated gra…

Study on optimizing model updates in performative prediction.

A framework for federated adversarial learning with convergence analysis.

Geodesic descent optimizes likelihood in dually flat spaces.

Bayesian uncertainty quantification is flawed, according to new research.

We describe $k$-MLE, a fast and efficient local search algorithm for learning finite statistical mixtures of exponential families such as Gaussian mixture models. Mixture models are traditionally learned using the expectation-maximization (EM) soft clustering technique that monotonically increases the incomplete (expec…

Method adapts frozen models for few-shot tasks without training.

DualAdam improves generalization of Adam by integrating its update mechanisms.

Identifying recurring patterns in high-dimensional time series data is an important problem in many scientific domains. A popular model to achieve this is convolutive nonnegative matrix factorization (CNMF), which extends classic nonnegative matrix factorization (NMF) to extract short-lived temporal motifs from a long …

LatentTrack generates model parameters online for nonstationary data.

Shampoo achieves higher token efficiency than Muon in language models.

GENIE balances domain-invariant feature learning and gradient alignment for improved DG performance.