PA reinterpreted as SB problem, unifying thermodynamics and optimal transport.
problem Optimizing discrete-time paths between probability distributions.
method Schrödinger Bridge theory, solving variational problems.
result PA's reweighting step derived from Schrödinger system.
AFT combines AIS, SMC, and NFs for better Monte Carlo estimates.
problem Estimating normalizing constants of complex probability distributions.
method Annealed Flow Transport (AFT) integrates AIS, SMC, and normalizing flows.
result AFT improves Monte Carlo estimates of normalizing constants and expectations.
We study clustering methods for binary data, first defining aggregation criteria that measure the compactness of clusters. Five new and original methods are introduced, using neighborhoods and population behavior combinatorial optimization metaheuristics: first ones are simulated annealing, threshold accepting and tabu…
Paper proposes an ensemble-based AIS for multimodal sampling.
problem Sampling from multimodal distributions is challenging.
method Combines AIS with population-based Monte Carlo methods.
result Improves efficiency through ensemble interaction.
In this paper, we will provide an introduction to the derivative-free optimization algorithms which can be potentially applied to train deep learning models. Existing deep learning model training is mostly based on the back propagation algorithm, which updates the model variables layers by layers with the gradient desc…
Reverse annealing boosts quantum matrix factorization performance.
problem Improving quantum matrix factorization performance.
method Combining forward and reverse annealing for nonnegative/binary matrix factorization.
result Combination of forward and reverse annealing significantly improves performance.
We empirically evaluate a stochastic annealing strategy for Bayesian posterior optimization with variational inference. Variational inference is a deterministic approach to approximate posterior inference in Bayesian models in which a typically non-convex objective function is locally optimized over the parameters of t…
New method uses reinforcement learning to improve Simulated Annealing.
problem Optimization problems with unknown cost functions.
method Replaces Metropolis engine with Macau Algorithm.
result Effective heuristic for unknown cost functions.
aMCL uses annealing to improve hypothesis diversity in ambiguous tasks.
problem Limitations of Winner-takes-all in predicting plausible hypotheses.
method Combines simulated annealing with Multiple Choice Learning (MCL).
result Enhanced exploration of hypothesis space during training.
This paper presents studies on a deterministic annealing algorithm based on quantum annealing for variational Bayes (QAVB) inference, which can be seen as an extension of the simulated annealing for variational Bayes (SAVB) inference. QAVB is as easy as SAVB to implement. Experiments revealed QAVB finds a better local …
Quantum annealers aim at solving non-convex optimization problems by exploiting cooperative tunneling effects to escape local minima. The underlying idea consists in designing a classical energy function whose ground states are the sought optimal solutions of the original optimization problem and add a controllable qua…
Mathematical analysis shows annealing prevents mode collapse in Gaussian mixtures.
problem Mode collapse in variational inference for multimodal distributions.
method Analyzed annealing strategies for Gaussian mixtures, derived formulas, and tested on neural networks.
result Appropriately chosen annealing schemes can robustly prevent mode collapse.
We introduce a novel framework for adversarial training where the target distribution is annealed between the uniform distribution and the data distribution. We posited a conjecture that learning under continuous annealing in the nonparametric regime is stable irrespective of the divergence measures in the objective fu…
Proposes a method to improve SLMC for multimodal distributions.
problem Difficulty of applying SLMC to multimodal distributions.
method Parallel adaptive annealing with VAE-SLMC.
result Can proficiently obtain accurate samples from multimodal distributions.
Simulated annealing improves candidate optimization for multi-objective Bayesian optimization.
problem Efficient candidate optimization for multi-objective acquisition functions in Bayesian optimization.
method Simulated annealing-based approach for batch acquisition function optimization.
result Simulated annealing outperforms SLSQP in most multi-objective optimization problems, achieving higher hypervolume values and better convergence characteristics.
In this paper, we will provide an introduction to the derivative-free optimization algorithms which can be potentially applied to train deep learning models. Existing deep learning model training is mostly based on the back propagation algorithm, which updates the model variables layers by layers with the gradient desc…
New analysis of annealing paths in sampling and estimation.
problem Sampling from complex distributions and estimating normalization constants.
method Extending known results on Bregman divergence to quasi-arithmetic means under monotonic embedding.
result Analogous result for quasi-arithmetic means, highlighting the interplay between means, parametric families, and divergence functionals.
AdaAnn optimizes annealing for efficient probability density approximation.
problem Efficiently approximating complex probability distributions with multiple modes.
method AdaAnn is an adaptive annealing scheduler that adjusts temperature increments based on KL divergence.
result AdaAnn improves computational efficiency in variational inference and parameter estimation.
CRAFT improves on existing methods for sampling complex distributions.
problem Sampling from complex probability distributions.
method Combines SMC with variational inference using normalizing flows.
result Improves on Annealed Flow Transport Monte Carlo and MCMC-based Stochastic Normalizing Flows.
We investigate a hybrid quantum-classical solution method to the mean-variance portfolio optimization problems. Starting from real financial data statistics and following the principles of the Modern Portfolio Theory, we generate parametrized samples of portfolio optimization problems that can be related to quadratic b…
Annealed importance sampling (AIS) is a common algorithm to estimate partition functions of useful stochastic models. One important problem for obtaining accurate AIS estimates is the selection of an annealing schedule. Conventionally, an annealing schedule is often determined heuristically or is simply set as a linear…
CR-AIS improves AIS efficiency by constant rate annealing.
problem Efficiently sample from intractable distributions.
method Constant rate annealing schedule for AIS.
result CR-AIS outperforms existing Adaptive AIS methods.
Quantum Annealing Enhanced Reinforcement Learning for Accurate RUL Prediction
problem RUL estimation in predictive maintenance
method QAQL framework combining quantum annealing and Q-learning
result Outperforms classical and quantum baselines
The paper analyzes the InfoNCE loss under different temperature schedules using Langevin dynamics.
problem Understanding the dynamics of InfoNCE loss under fixed versus annealed temperature schedules.
method Modeling embedding evolution under Langevin dynamics on a compact Riemannian manifold, with theoretical guarantees for convergence.
result Slow logarithmic inverse-temperature schedules ensure convergence to globally optimal representations, while faster schedules risk suboptimal minima.
Study improves sampling from complex distributions using annealed Langevin Monte Carlo.
problem Sampling from non-log-concave and multimodal distributions.
method Annealed Langevin Monte Carlo algorithm with theoretical guarantees.
result Oracle complexity of O(dβ²A²/ε⁶) for achieving ε² accuracy in Kullback-Leibler divergence.
Researchers analyze a new neural network training method.
problem Training robust configurations in discrete weight neural networks.
method Replicated simulated annealing combining physics and classical simulated annealing.
result Explicit criteria for algorithm convergence and successful sampling.
We describe an adaptation of the simulated annealing algorithm to nonparametric clustering and related probabilistic models. This new algorithm learns nonparametric latent structure over a growing and constantly churning subsample of training data, where the portion of data subsampled can be interpreted as the inverse …
Annealed Entropic Allocation improves ranking and selection by mitigating hard switching and improving finite-budget discrimination.
problem Sequential budget allocation in ranking and selection
method Annealed weighted soft-min framework
result Surrogate converges uniformly to the hard minimum, soft-min weights concentrate on active challengers, and target allocation map is continuous.
Paper analyzes Langevin dynamics for multimodal Gaussian mixtures, controlling errors across dimensions.
problem Challenges in obtaining stable diffusion-based samplers in high- and infinite-dimensional settings.
method Study of preconditioned Annealed Langevin Dynamics (ALD) for Gaussian mixtures, focusing on Euler-Maruyama (EM) and exponential-integrator schemes.
result Proves dimension-uniform KL bounds for the exponential-integrator scheme, allowing arbitrarily small divergence with dimension.
Maximum likelihood estimation (MLE) is one of the most important methods in machine learning, and the expectation-maximization (EM) algorithm is often used to obtain maximum likelihood estimates. However, EM heavily depends on initial configurations and fails to find the global optimum. On the other hand, in the field …
We propose a modified expectation-maximization algorithm by introducing the concept of quantum annealing, which we call the deterministic quantum annealing expectation-maximization (DQAEM) algorithm. The expectation-maximization (EM) algorithm is an established algorithm to compute maximum likelihood estimates and appl…
QAPCA uses quantum annealing for robust PCA.
problem Outliers in data skew L2-norm principal components.
method Quantum annealing for L1-norm optimization.
result QAPCA's reconstruction error is comparable to L1-BF.
KL annealing helps VAEs avoid posterior collapse and overfitting.
problem Posterior collapse and overfitting in VAEs.
method Theoretical analysis of learning dynamics with KL annealing.
result Posterior collapse is inevitable when β exceeds a threshold. Quantum machine learns faster by reverse annealing on AQCs.
problem Training RBMs on AQCs is hard due to low qubit connectivity.
method Embedding RBM nodes to virtual qubits, semantic quantum search, reverse annealing schedule.
result Reverse annealing accelerates RBM training and improves reconstruction scores.
NVA combines variational posteriors, annealing, and natural-gradient learning for multimodal optimization.
problem Finding multiple global and local modes in nonconvex objectives.
method NVA integrates variational posteriors, annealing, and natural-gradient learning.
result NVA outperforms gradient descent and evolution strategies on simulations and real-world problems.
The study optimizes simulated annealing's cooling schedule for better performance.
problem Designing optimal cooling schedules for simulated annealing to improve its performance.
method Analyzed the cooling schedule's impact on simulated annealing's performance and provided sample and simulation complexity results.
result Optimal cooling schedules can be found with a small number of samples, improving the algorithm's runtime or success rate.
Simulated annealing is a popular method for approaching the solution of a global optimization problem. Existing results on its performance apply to discrete combinatorial optimization where the optimization variables can assume only a finite set of possible values. We introduce a new general formulation of simulated an…
Improved SVI with adjustable annealing for better optimization.
problem Improving optimization in stochastic variational inference.
method Tuneable stochastic annealing in SVI with adjustable batch size.
result Approximation to maximum entropy stochastic gradient at desired variance level.
Lossy compression of statistical data using quantum annealing.
problem Efficiently compressing statistical floating-point data.
method Representation learning with binary variables, classical optimization of basis vectors, quantum annealing for coefficients, bias correction.
result Quantum annealing shows promising results with 3.5x better compression than neural-network autoencoders.
New method uses quantum annealing and VAN for better statistical mechanics calculations.
problem Difficulty in computing partition function in statistical mechanics.
method Combines quantum annealing samples with variational autoregressive networks.
result Enhanced accuracy in finite-size Sherrington-Kirkpatrick model.
CoolMomentum combines momentum and Simulated Annealing for deep learning optimization.
problem Global optimization of non-convex functions in deep learning.
method Discretized Langevin dynamics with Simulated Annealing.
result CoolMomentum achieves high accuracy on Resnet-20 on Cifar-10 and Efficientnet-B0 on Imagenet.
We solve a multi-period portfolio optimization problem using D-Wave Systems' quantum annealer. We derive a formulation of the problem, discuss several possible integer encoding schemes, and present numerical examples that show high success rates. The formulation incorporates transaction costs (including permanent and t…
End-to-end portfolio optimization using quantum annealing for financial decision problems.
problem Optimizing financial portfolios with quantum computing constraints.
method Hybrid pipeline combining quantum and classical optimization.
result Quantum-assisted portfolio optimization can achieve competitive returns.
mAIS improves free energy evaluation efficiency.
problem Computational infeasibility of exact free energy evaluation.
method mAIS, a marginalized version of AIS.
result mAIS is more efficient under certain conditions.
Annealing Double-Head calibrates deep neural networks during training.
problem Overestimation or underestimation of predictive confidence in deep neural networks.
method An additional calibration head and Annealing technique to dynamically scale logits.
result State-of-the-art model calibration performance achieved without post-processing.
Kernel-based support vector machines (SVMs) are supervised machine learning algorithms for classification and regression problems. We introduce a method to train SVMs on a D-Wave 2000Q quantum annealer and study its performance in comparison to SVMs trained on conventional computers. The method is applied to both synth…
New method improves performance of Hamiltonian MCMC for log Z estimation.
problem Estimating tight bounds on log Z for unnormalized distributions.
method Uncorrected Hamiltonian Annealing (UHA) using reparameterization gradients.
result Better performance and easier parameter tuning compared to existing methods.
Learning rate annealing helps even in convex problems, improving generalization.
problem Improving generalization in neural networks, especially convex problems.
method Learning rate annealing schedule (large initial, then small learning rate).
result Gradient descent can reach minima with better generalization using learning rate annealing.