arXiv research

The paper analyzes the InfoNCE loss under different temperature schedules using Langevin dynamics.

AdaAnn optimizes annealing for efficient probability density approximation.

The study optimizes simulated annealing's cooling schedule for better performance.

Variational inference (VI) combined with data subsampling enables approximate posterior inference over large data sets, but suffers from poor local optima. We first formulate a deterministic annealing approach for the generic class of conditionally conjugate exponential family models. This approach uses a decreasing te…

We describe an adaptation of the simulated annealing algorithm to nonparametric clustering and related probabilistic models. This new algorithm learns nonparametric latent structure over a growing and constantly churning subsample of training data, where the portion of data subsampled can be interpreted as the inverse …

Study convergence of simulated annealing in continuous and discrete settings.

FAKI improves gradient-free inference for inverse problems.

FlowVAT improves variational inference for multi-modal distributions.

In this paper, we propose a novel uniform generalization bound on the time and inverse temperature for stochastic gradient Langevin dynamics (SGLD) in a non-convex setting. While previous works derive their generalization bounds by uniform stability, we use Rademacher complexity to make our generalization bound indepen…

A new method combines AIS and SMCI for efficient evaluation of Ising models.

Mathematical analysis shows annealing prevents mode collapse in Gaussian mixtures.

CRAFT improves on existing methods for sampling complex distributions.

Quantum annealing improves VB inference, avoiding local minima.

EWFM trains continuous flows with only energy evaluations, improving sample quality with fewer computations.

Proposes a new sampling policy for ranking and selection problems.

Restricted Boltzmann Machine (RBM) is a bipartite graphical model that is used as the building block in energy-based deep generative models. Due to numerical stability and quantifiability of the likelihood, RBM is commonly used with Bernoulli units. Here, we consider an alternative member of exponential family RBM with…

Partition functions of probability distributions are important quantities for model evaluation and comparisons. We present a new method to compute partition functions of complex and multimodal distributions. Such distributions are often sampled using simulated tempering, which augments the target space with an auxiliar…

Finding an energy minimum in the Ising model is an exemplar objective, associated with many combinatorial optimization problems, that is computationally hard in general, but occurs in all areas of modern science. There are several numerical methods, providing solution for the medium size Ising spin systems. However, th…

Stochastic gradient Markov chain Monte Carlo (SG-MCMC) methods are Bayesian analogs to popular stochastic optimization methods; however, this connection is not well studied. We explore this relationship by applying simulated annealing to an SGMCMC algorithm. Furthermore, we extend recent SG-MCMC methods with two key co…

NeuRules learns interpretable rule lists from data without pre-discretization.

We consider the problem of minimizing a convex objective function $F$ when one can only evaluate its noisy approximation $\hat{F}$. Unless one assumes some structure on the noise, $\hat{F}$ may be an arbitrary nonconvex function, making the task of minimizing $F$ intractable. To overcome this, prior work has often focu…

Auto-Compressing Subset Pruning reduces model size for faster inference.

Paper studies convergence of Mean-Field GDA dynamics for MNE of continuous games.

SKT improves EKI for Bayesian inverse problems with non-Gaussian targets.

An algorithmic limit of compressed sensing or related variable-selection problems is analytically evaluated when a design matrix is given by an overcomplete random matrix. The replica method from statistical mechanics is employed to derive the result. The analysis is conducted through evaluation of the entropy, an expo…

We recapitulate the Bayesian formulation of neural network based classifiers and show that, while sampling from the posterior does indeed lead to better generalisation than is obtained by standard optimisation of the cost function, even better performance can in general be achieved by sampling finite temperature ($T$) …

Reverse annealing boosts quantum matrix factorization performance.

New method boosts performance of diffusion models on discrete data like natural language.

Quantum annealing (QA) is a generic method for solving optimization problems using fictitious quantum fluctuation. The current device performing QA involves controlling the transverse field; it is classically simulatable by using the standard technique for mapping the quantum spin systems to the classical ones. In this…

We empirically evaluate a stochastic annealing strategy for Bayesian posterior optimization with variational inference. Variational inference is a deterministic approach to approximate posterior inference in Bayesian models in which a typically non-convex objective function is locally optimized over the parameters of t…

New method uses reinforcement learning to improve Simulated Annealing.

We explore the effects of social influence in a simple market model in which a large number of agents face a binary choice: 'to buy/not to buy' a single unit of a product at a price posted by a single seller (the monopoly case). We consider the case of 'positive externalities': an agent is more willing to buy if the ot…

aMCL uses annealing to improve hypothesis diversity in ambiguous tasks.

This paper presents studies on a deterministic annealing algorithm based on quantum annealing for variational Bayes (QAVB) inference, which can be seen as an extension of the simulated annealing for variational Bayes (SAVB) inference. QAVB is as easy as SAVB to implement. Experiments revealed QAVB finds a better local …

Quantum annealers aim at solving non-convex optimization problems by exploiting cooperative tunneling effects to escape local minima. The underlying idea consists in designing a classical energy function whose ground states are the sought optimal solutions of the original optimization problem and add a controllable qua…

We introduce a novel framework for adversarial training where the target distribution is annealed between the uniform distribution and the data distribution. We posited a conjecture that learning under continuous annealing in the nonparametric regime is stable irrespective of the divergence measures in the objective fu…

Proposes a method to improve SLMC for multimodal distributions.

Simulated annealing improves candidate optimization for multi-objective Bayesian optimization.

New analysis of annealing paths in sampling and estimation.

Improved clustering accuracy with disentangled latent code representation.

We investigate a hybrid quantum-classical solution method to the mean-variance portfolio optimization problems. Starting from real financial data statistics and following the principles of the Modern Portfolio Theory, we generate parametrized samples of portfolio optimization problems that can be related to quadratic b…

Annealed importance sampling (AIS) is a common algorithm to estimate partition functions of useful stochastic models. One important problem for obtaining accurate AIS estimates is the selection of an annealing schedule. Conventionally, an annealing schedule is often determined heuristically or is simply set as a linear…

CR-AIS improves AIS efficiency by constant rate annealing.

Quantum Annealing Enhanced Reinforcement Learning for Accurate RUL Prediction

Study improves sampling from complex distributions using annealed Langevin Monte Carlo.

Researchers analyze a new neural network training method.

We propose a new metaheuristic training scheme that combines Stochastic Gradient Descent (SGD) and Discrete Optimization in an unconventional way. Our idea is to define a discrete neighborhood of the current SGD point containing a number of "potentially good moves" that exploit gradient information, and to search this …

Annealed Entropic Allocation improves ranking and selection by mitigating hard switching and improving finite-budget discrimination.