Bayesian model updating uses VAEs to approximate likelihood with small data.
problem Approximating likelihood for small data sets in structural analysis.
method Uses multimodal VAEs to approximate likelihood, suitable for high-dimensional correlated observations.
result Demonstrates computational efficiency and accuracy compared to original VAE approach.
This work analyzes how often to update the target network in Q-learning.
problem Understanding the optimal frequency of target network updates in Q-learning.
method Formulated target updates as a nested optimization scheme, derived finite-time convergence analysis.
result Optimal target update frequency increases geometrically over time.
Unified framework for analyzing batch updating methods with noisy gradients.
problem Analyzing convergence of batch updating methods with noisy gradients and approximations.
method Unified framework using convergence of stochastic processes.
result Establishes a general theorem for most known convergence results.
We establish decoupled functional CLTs for two-time-scale stochastic approximation.
problem Understanding the asymptotic behavior of two-time-scale stochastic approximation.
method Martingale problem approach and auxiliary sequence.
result The limiting dynamics of two-time-scale SA are independent of each other.
This paper analyzes how periodic and soft target updates stabilize linear Q-learning.
problem Theoretical explanation of stabilization mechanisms for linear Q-learning.
method Exact analysis using switched linear system dynamics and the joint spectral radius.
result Periodic and soft target updates can guarantee convergence to the exact projected Q-Bellman solution under specific conditions.
Paper analyzes ensemble Kalman updates for effective dimension and localization.
problem Why small ensemble sizes work well in inverse problems and data assimilation.
method Non-asymptotic analysis of ensemble Kalman updates, focusing on effective dimension and localization.
result Rigorously explains why a small ensemble size is sufficient when prior covariance has moderate effective dimension.
New method for density estimation without approximating posterior distributions.
problem Challenges in non-smooth data distributions for Bayesian density estimation.
method Autoregressive likelihood decomposition and Gaussian process prior in a quasi-Bayesian framework.
result Achieves state-of-the-art results in small-data regimes.
We investigate finite-time decoupled convergence in nonlinear two-time-scale stochastic approximation.
problem Achieving decoupled convergence in nonlinear two-time-scale stochastic approximation.
method Nested local linearity assumption, suitable step size selection, convergence analysis of matrix cross term, fourth-order moment convergence rates.
result Finite-time decoupled convergence rates can be achieved in nonlinear two-time-scale stochastic approximation with proper step size selection.
Improved HGF networks avoid negative precision errors in volatility updates.
problem Negative posterior precision errors in volatility-coupled nodes of HGF networks.
method Introduced a modified quadratic approximation to variational energy.
result Robust update equations across parameter space that track posterior faithfully.
In this letter, we generalize the convolutional NMF by taking the β-divergence as the contrast function and present the correct multiplicative updates for its factors in closed form. The new updates unify the β-NMF and the convolutional NMF. We state why almost all of the existing updates are inexact and approximat…
New method makes machine learning approximations unbiased and efficient.
problem Efficient sampling of complex probability distributions.
method Uses autoregressive neural networks with cluster updates and physical symmetries.
result Shows unbiased and low-variance approximations for phase transitions.
A well-known problem in data science and machine learning is {\em linear regression}, which is recently extended to dynamic graphs. Existing exact algorithms for updating the solution of dynamic graph regression require at least a linear time (in terms of n: the size of the graph). However, this time complexity might…
Study Whittle index learning algorithms for restless bandits with constant stepsizes.
problem Optimizing decisions in restless multi-armed bandits with constant stepsizes.
method Developed Q-learning algorithms with constant stepsizes for index learning in restless bandits, extending to DQN and function approximations.
result The algorithms learn the Whittle index effectively.
PSI-LinUCB improves scalability for large recommender systems.
problem Efficiently training and inferring for large action spaces in recommender systems.
method Represent inverse design matrix as diagonal + low-rank correction, derive stable rank-1 and batched updates, use projector-splitting integrator.
result Demonstrated effectiveness on recommender system datasets, achieving scalable training and inference.
AuON is a linear-time optimizer that improves upon Muon's performance without approximate orthogonal matrices.
problem High memory and computational costs of orthogonal momentum updates.
method AuON uses normalized nonlinear scaling and a 'emergency brake' to handle exploding attention logits.
result AuON achieves strong performance without approximate orthogonal matrices, preserving structural alignment and reconditioning.
Improves BBVI for high-dimensional Gaussian approximations by using low-rank approximations.
problem Scalability issues with BBVI for high-dimensional multivariate Gaussian approximations.
method Extends BaM framework to handle full covariance matrices by integrating patch step for low-rank parameterization.
result Shows improved efficiency and scalability on synthetic and real-world high-dimensional inference problems.
SnAp approximates RTRL for online training of sparse recurrent networks.
problem Training large sparse recurrent networks online is computationally expensive.
method Sparse n-step Approximation (SnAp) of the RTRL influence matrix.
result SnAp with n=2 remains tractable for highly sparse networks and outperforms backpropagation through time.
New algorithms minimize regret in SSP with optimal sparse updates.
problem Minimizing regret in Stochastic Shortest Path models.
method Implicit finite-horizon approximation for analysis, model-free and model-based algorithms developed.
result Minimax optimal regret for both model-free and model-based algorithms.
Paper introduces a new policy optimization method using importance sampling.
problem Stable and low variance policy learning with small policy updates.
method Derives an alternative objective using importance sampling and introduces an approximation to balance bias and variance.
result The new algorithm improves on-policy policy optimization on continuous control benchmarks.
A new method automatically and dynamically sets learning rates in deep learning.
problem Determining the appropriate learning rate in deep learning tasks is challenging and often subjective.
method Local Quadratic Approximation (LQA) to automatically and dynamically set learning rates.
result The proposed method leads to nearly optimal learning rates in a computationally efficient way.
Proposes variational Gaussian approximations for solving the Kushner equation.
problem Solving the Kushner equation for state estimation with observations.
method Tractable variational Gaussian approximations of proximal losses based on Wasserstein and Fisher metrics.
result The proposed method leads to a Gaussian flow consistent with Kalman-Bucy and Riccati flows.
Improved neural network ensembles using Stein Variational Newton updates.
problem Lack of efficient second-order information in current ensemble methods.
method Proposes a novel approximate Bayesian inference method integrating Stein Variational Newton updates with scalable Hessian approximations.
result Significantly faster convergence and more accurate posterior distribution approximations.
We analyze SA with Markovian data and nonlinear updates, overcoming prior limitations.
problem Analyzing stochastic approximation with Markovian data and nonlinear updates.
method Fine-grained analysis of SA iterates and Markovian data, leveraging smoothness and recurrence properties.
result Established weak convergence and precise asymptotic bias of SA iterates.
A new method for machine learning updates reduces complexity and improves robustness.
problem Stochastic gradient updates are inefficient and sensitive to feature scaling.
method Incremental Gauss-Newton Descent (IGND) reduces the need for matrix operations and improves robustness.
result IGND improves robustness to sensitivity scaling and can be competitive with common stochastic optimizers.
Improves graph-based active learning for non-Gaussian models.
problem Efficiently selecting data points for labeling in graph-based semi-supervised learning.
method Approximates non-Gaussian distributions, introduces rank-one update and model change acquisition function.
result Enhanced active learning for graph-based SSL under non-Gaussian models.
New learning algorithm for real analytic functions without gradient descent.
problem Learning real analytic functions without gradient descent.
method Taylor approximation and sampling data distribution.
result Nonuniform learning result for real analytic functions.
We present a new online boosting algorithm for adapting the weights of a boosted classifier, which yields a closer approximation to Freund and Schapire's AdaBoost algorithm than previous online boosting algorithms. We also contribute a new way of deriving the online algorithm that ties together previous online boosting…
Paper proves SHB convergence with biased gradients and approximate step sizes.
problem Establishing convergence of SHB with biased gradients and approximate step sizes.
method Generalizes SHB convergence conditions for biased gradients, approximate step sizes, and block updating.
result Proves convergence of SHB with new conditions for biased gradients and approximate step sizes.
Improved ADMM for convex distributed learning with differential privacy.
problem Privacy concerns in distributed learning with sensitive data.
method Approximate multi-step ADMM with calibrated noise.
result Higher utility and error bounds asymptotic to state-of-the-art.
This work improves SGD convergence by adaptively adjusting batch sizes.
problem High variance in gradient estimates with small batch sizes.
method Adaptive batch size adjustment based on model training loss.
result Adaptive batch size method requires fewer model updates with same computation.
GANs excel at learning high dimensional distributions, but they can update generator parameters in directions that do not correspond to the steepest descent direction of the objective. Prominent examples of problematic update directions include those used in both Goodfellow's original GAN and the WGAN-GP. To formally d…
Single-timescale actor-critic finds globally optimal policy.
problem Finding globally optimal policy in reinforcement learning.
method Simultaneous actor and critic updates with linear or deep neural network approximations.
result Actor sequence converges to globally optimal policy at O(K−1/2) rate. Multiple-step lookahead policies have demonstrated high empirical competence in Reinforcement Learning, via the use of Monte Carlo Tree Search or Model Predictive Control. In a recent work \cite{efroni2018beyond}, multiple-step greedy policies and their use in vanilla Policy Iteration algorithms were proposed and analy…
Two-Timescale EM Methods improve EM for nonconvex models.
problem Nonconvex latent variable models are challenging for EM.
method Two-stage stochastic updates to handle nonconvex optimization.
result Global convergence for nonconvex objective functions.
Paper presents a rank-1 approximation method for natural policy gradients in deep RL.
problem Computing natural gradients requires inverting the Fisher Information Matrix, which is computationally expensive.
method Develops a rank-1 approximation to the inverse Fisher Information Matrix for efficient natural policy optimization.
result The rank-1 approximation converges faster and has similar sample complexity to stochastic policy gradient methods.
We study a mean-field spike and slab variational Bayes (VB) approximation to Bayesian model selection priors in sparse high-dimensional linear regression. Under compatibility conditions on the design matrix, oracle inequalities are derived for the mean-field VB approximation, implying that it converges to the sparse tr…
In this paper, a Bayesian inference technique based on Taylor series approximation of the logarithm of the likelihood function is presented. The proposed approximation is devised for the case, where the prior distribution belongs to the exponential family of distributions. The logarithm of the likelihood function is li…
In this paper we revisit the weighted likelihood bootstrap, a method that generates samples from an approximate Bayesian posterior of a parametric model. We show that the same method can be derived, without approximation, under a Bayesian nonparametric model with the parameter of interest defined as minimising an expec…
We argue that the existing regret matchings for Nash equilibrium approximation conduct "jumpy" strategy updating when the probabilities of future plays are set to be proportional to positive regret measures. We propose a geometrical regret matching which features "smooth" strategy updating. Our approach is simple, intu…
New EP variants improve inference stability and efficiency.
problem Inference stability and efficiency issues in EP.
method Motivated by natural-gradient optimization, new EP variants are introduced that are robust to Monte Carlo noise and efficient with single samples.
result Improved stability and efficiency in inference tasks.
We develop methods for parameter estimation in settings with large-scale data sets, where traditional methods are no longer tenable. Our methods rely on stochastic approximations, which are computationally efficient as they maintain one iterate as a parameter estimate, and successively update that iterate based on a si…
The paper updates SVD of evolving matrices using projection techniques.
problem Updating the rank-k truncated SVD of evolving matrices.
method Projection viewpoint, building subspaces to approximate singular vectors.
result The proposed algorithm leads to higher accuracy, especially for large singular values.
New method for asynchronous stochastic approximation converges in reinforcement learning.
problem Finding solutions to equations with noisy measurements in reinforcement learning.
method Batch Asynchronous Stochastic Approximation (BASA) with conditions for convergence and rate of convergence.
result Sufficient conditions for convergence and rate of convergence of BASA.
Second-order optimization methods such as natural gradient descent have the potential to speed up training of neural networks by correcting for the curvature of the loss function. Unfortunately, the exact natural gradient is impractical to compute for large models, and most approximations either require an expensive it…
Algorithm learns Nash equilibria in stochastic games using entropy-regularized policies.
problem Learning Nash equilibria in zero-sum stochastic games is computationally expensive.
method Entropy-regularized soft policies for Q-function updates.
result Algorithm converges to Nash equilibrium under certain conditions.
The sum-product or belief propagation (BP) algorithm is a widely used message-passing technique for computing approximate marginals in graphical models. We introduce a new technique, called stochastic orthogonal series message-passing (SOSMP), for computing the BP fixed point in models with continuous random variables.…
A major challenge in current optimization research for deep learning is to automatically find optimal step sizes for each update step. The optimal step size is closely related to the shape of the loss in the update step direction. However, this shape has not yet been examined in detail. This work shows empirically that…
Transformers enable in-context learning with guarantees for a wide range of tasks.
problem How to enable in-context learning with transformers for various tasks.
method Developed a universal approximation theory integrating Barron's function approximation with transformer capabilities.
result Transformers can approximate any target function with vanishingly small risk using a few in-context examples.