arXiv research

Study on network-valued processes with asynchronous updates, proving consistency in community and changepoint estimation.

In this work we introduce a conditional accelerated lazy stochastic gradient descent algorithm with optimal number of calls to a stochastic first-order oracle and convergence rate $O\left(\frac{1}{\varepsilon^2}\right)$ improving over the projection-free, Online Frank-Wolfe based stochastic gradient descent of Hazan an…

Stochastic principal component analysis (SPCA) has become a popular dimensionality reduction strategy for large, high-dimensional datasets. We derive a simplified algorithm, called Lazy SPCA, which has reduced computational complexity and is better suited for large-scale distributed computation. We prove that SPCA and …

New method converts and optimizes sampling schedules for generative models.

Lazy, perfectly informed investors trade infrequently due to costs.

We discuss the approximation of the value function for infinite-horizon discounted Markov Reward Processes (MRP) with nonlinear functions trained with the Temporal-Difference (TD) learning algorithm. We first consider this problem under a certain scaling of the approximating function, leading to a regime called lazy tr…

Lazy Gradient Descent outperforms existing polytope algorithms in pseudo-regret.

ABC algorithms involve a large number of simulations from the model of interest, which can be very computationally costly. This paper summarises the lazy ABC algorithm of Prangle (2015), which reduces the computational demand by abandoning many unpromising simulations before completion. By using a random stopping decis…

Unified formula for training dynamics of linear networks combining lazy and balanced regimes.

Counterfactual regret minimization (CFR) is the most popular algorithm on solving two-player zero-sum extensive games with imperfect information and achieves state-of-the-art performance in practice. However, the performance of CFR is not fully understood, since empirical results on the regret are much better than the …

LAZO reduces query complexity and variance in ZO methods.

Transformers learn rich in-context dependencies efficiently.

Lazy neural networks are vulnerable to adversarial attacks.

A new method simulates a lazy version of a Markov chain for empirical inference.

Study introduces a new investment strategy model using lazy factor and probability weights.

Gradient descent can find better tensor decompositions than lazy training in over-parameterized settings.

We explore how neural networks train to zero loss, focusing on initial scale.

Theory captures feature learning effects in finite CNNs.

New private algorithms for online learning improve regret in high privacy regimes.

dSMC improves parallel processing of state-space models.

Quantum machine learning faces 'laziness' and 'barren plateaus', but noise can mitigate the latter.

Deep networks prioritize easier examples over harder ones, leading to faster training.

Grokking occurs when neural networks transition from lazy to rich training dynamics, fitting initial features before generalizing.

Study shows optimal model performance at critical level of feature learning.

LASG improves communication efficiency in distributed learning.

Propositionalization is the process of summarizing relational data into a tabular (attribute-value) format. The resulting table can next be used by any propositional learner. This approach makes it possible to apply a wide variety of learning methods to relational data. However, the transformation from relational to pr…

We present an approach towards convex optimization that relies on a novel scheme which converts online adaptive algorithms into offline methods. In the offline optimization setting, our derived methods are shown to obtain favourable adaptive guarantees which depend on the harmonic sum of the queried gradients. We furth…

Clapping reduces memory usage in distributed optimization by reusing data samples.

Study on optimizing model updates in performative prediction.

We propose a framework for solving high-dimensional Bayesian inference problems using \emph{structure-exploiting} low-dimensional transport maps or flows. These maps are confined to a low-dimensional subspace (hence, lazy), and the subspace is identified by minimizing an upper bound on the Kullback--Leibler divergence …

A method learns common bias for multiple low-variance tasks without hyper-parameter tuning.

Study on adversarial robustness in neural networks across initialization and training phases.

We study Bayesian optimal control of a general class of smoothly parameterized Markov decision problems. Since computing the optimal control is computationally expensive, we design an algorithm that trades off performance for computational efficiency. The algorithm is a lazy posterior sampling method that maintains a d…

Analyzes neural networks using linear models to understand their behavior.

Sepsis is a life-threatening host response to infection associated with high mortality, morbidity, and health costs. Its management is highly time-sensitive since each hour of delayed treatment increases mortality due to irreversible organ damage. Meanwhile, despite decades of clinical research, robust biomarkers for s…

A recent line of work has shown that an overparametrized neural network can perfectly fit the training data, an otherwise often intractable nonconvex optimization problem. For (fully-connected) shallow networks, in the best case scenario, the existing theory requires quadratic over-parametrization as a function of the …

The present paper develops a novel aggregated gradient approach for distributed machine learning that adaptively compresses the gradient communication. The key idea is to first quantize the computed gradients, and then skip less informative quantized gradient communications by reusing outdated gradients. Quantizing and…

LazyDINO efficiently solves high-dimensional Bayesian inverse problems with fast and scalable solutions.

Two distinct limits for deep learning have been derived as the network width $h\rightarrow \infty$, depending on how the weights of the last layer scale with $h$. In the Neural Tangent Kernel (NTK) limit, the dynamics becomes linear in the weights and is described by a frozen kernel $Θ$. By contrast, in the Mean-Field …

Study on rich regime training in deep learning, finding active parameters in bottom layers.

Auto-CEI improves LLM reasoning by balancing assertiveness and conservativeness.

Mirror flow in shallow neural networks shows similar implicit bias to gradient flow, with key differences in curvature penalties.

Most machine learning methods require careful selection of hyper-parameters in order to train a high performing model with good generalization abilities. Hence, several automatic selection algorithms have been introduced to overcome tedious manual (try and error) tuning of these parameters. Due to its very high sample …

In this paper, we propose an active perception method for recognizing object categories based on the multimodal hierarchical Dirichlet process (MHDP). The MHDP enables a robot to form object categories using multimodal information, e.g., visual, auditory, and haptic information, which can be observed by performing acti…

Quantum RL algorithm achieves logarithmic regret for exploration.

Study highlights how lazy data practices in fair ML research can unfairly impact minority groups.

New Q-learning method achieves optimal sample complexity for average-reward problems.

Use simplified layerwise linear models to understand neural dynamics.