Noise helps neural networks escape local optima.
problem Understanding the role of noise in neural network training.
method Perturbed gradient descent and noise annealing.
result Noise guarantees convergence to global optimum in polynomial time.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,341 papers · 148 categories
Trend · papers per month
48 results for “local optimum”
Novel method for high-dimensional BO using CMA to define local regions.
problem Challenges in applying BO to high-dimensional optimization problems.
method CMA strategy to learn search distribution and define local regions.
result Our method outperforms existing techniques on various benchmarks.
Develops an SSBO algorithm for global optimization of expensive models.
problem Global optimization of expensive black-box models.
method Asynchronous hybrid-criterion with interval reduction.
result Improves global search ability and local search efficiency.
Optimum-statistical collaboration improves black-box optimization efficiency.
problem Improving black-box optimization efficiency through better statistical collaboration.
method Introducing optimum-statistical collaboration framework for hierarchical bandits-based optimization.
result Demonstrated improved regret bounds and better performance in experiments.
Global optimization algorithm finds sparse mixed membership matrix factorization's global optimum.
problem Sparse mixed membership matrix factorization problems with local optima.
method Derives a global optimization algorithm for sparse mixed membership matrix factorization.
result Guaranteed -global optimum across random initializations and multiple modes.
Shortcut connections in ResNet help avoid local optima, leading to efficient training.
problem Understanding why shortcut connections in ResNet lead to efficient training.
method Two-layer non-overlapping convolutional ResNet, gradient descent with proper normalization.
result Gradient descent avoids spurious local optima, converging to a global optimum.
New findings on the max margin problem in neural networks.
problem Understanding the max margin problem in neural networks.
method Analyzing gradient flow and max margin problem in linear and ReLU networks.
result The KKT point is not always an optimum of the max margin problem.
Local search algorithms applied to optimization problems often suffer from getting trapped in a local optimum. The common solution for this deficiency is to restart the algorithm when no progress is observed. Alternatively, one can start multiple instances of a local search algorithm, and allocate computational resourc…
Combines global and local search for efficient global optimization with Gaussian processes.
problem Difficulties in building accurate GP models and getting stuck in suboptimal regions.
method Adopting AGLGP model combining global and local GP models, dividing space into regions, and switching between global and local searches.
result Efficiently locates the global optimum with benefits of both global and local search.
The computational asymptotics of Gaussian variational inference and the Laplace approximationstat.CO
New algorithms improve likelihood of finding global optima in Bayesian inference.
problem Finding global optima in Bayesian inference is difficult due to nonconvexity.
method Developed two algorithms: consistent Laplace approximation (CLA) and consistent stochastic variational inference (CSVI).
result Both CSVI and CLA improve likelihood of obtaining global optima compared to standard methods.
Community detection in graphs has been the subject of many algorithms. Recent methods want to optimize a modularity function which shows a maximum of relationships within communities and found a minimum of inter-community relations. these algorithms are applied to unipartite, multipartite and directed graphs. However, …
This paper proves that local maxima in a non-convex optimization problem are close to the SDP optimum.
problem Optimization problems with a large number of local maxima.
method Proving that all local maxima are within a small gap from the SDP optimum for the elliptope problem.
result Local maxima are within a small gap from the SDP optimum, and arbitrarily small relative error can be achieved with a rank constraint of order one.
Contemporary global optimization algorithms are based on local measures of utility, rather than a probability measure over location and value of the optimum. They thus attempt to collect low function values, not to learn about the optimum. The reason for the absence of probabilistic global optimizers is that the corres…
Locally convex sparse learning reduces communication and processing time over networks.
problem Efficiently estimate sparse signals over networked nodes with minimal communication.
method Convex optimization with -norm minimization, incorporating neighbors' estimates.
result Local estimates converge to exact sparse signal under certain conditions.
ASAP learns skills and where to apply them for general task solving.
problem General task solving in complex environments.
method Adaptive Skills, Adaptive Partitions (ASAP) framework.
result ASAP converges to a local optimum under natural conditions and learns where to reuse skills.
Continuous-time distributed mirror descent with integral feedback converges to global optimum.
problem Distributed optimization of a global strongly convex function with local convex components.
method Continuous-time distributed mirror descent with integral feedback.
result Asymptotic convergence to global optimum with constant step-size.
New AI algorithm improves multi-layer optical film design efficiency.
problem Traditional algorithms converge to local optima, limiting global optimal solutions.
method Deep Q-learning for global optimal multi-layer optical film design.
result Deep Q-learning model converges global optimum of optical thin film structure.
Testing that a Local Optimum of the Likelihood is Globally Optimum using Reparameterized Embeddingseess.SP
A test confirms if a local maximum is globally optimal in imaging problems.
problem Confirming if a local maximum is globally optimal in imaging problems.
method Reparameterizing the likelihood function to embed its domain into a higher dimensional parameter space.
result Improved accuracy and reduced computation for camera-blur estimation.
SpectralLeader learns latent variables from streaming data efficiently and globally.
problem Learning latent variable models from a stream of data efficiently and globally.
method SpectralLeader, an online learning algorithm for latent variable models that converges to the global optimum.
result SpectralLeader achieves a sublinear upper bound on -step regret in the bag-of-words model.
Global optimization for low-rank matrix recovery from noisy measurements.
problem Low-rank matrix recovery from noisy measurements.
method Factorized parametrization, curvature bound, stochastic gradient descent.
result Global convergence guarantee for stochastic gradient descent from random initialization.
Langevin algorithm shows metastable behavior near local optima in ERM.
problem Understanding the behavior of discrete Langevin algorithm near local optima in non-convex ERM.
method Analyzing metastability through two-timescale recurrence and escape times.
result The Langevin algorithm can escape from a local optimum within a short recurrence time or stay near it for a long time.
We investigate the role of the initialization for the stability of the k-means clustering algorithm. As opposed to other papers, we consider the actual k-means algorithm and do not ignore its property of getting stuck in local optima. We are interested in the actual clustering, not only in the costs of the solution. We…
Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optimamath.ST
We provide novel theoretical results regarding local optima of regularized -estimators, allowing for nonconvexity in both loss and penalty functions. Under restricted strong convexity on the loss and suitable regularity conditions on the penalty, we prove that \emph{any stationary point} of the composite objective f…
Quantum neural networks generalize better due to flatter parameter space.
problem Generalization in quantum neural networks.
method Mapped feature data to a quantum state, applied unitary evolution, and measured for classification.
result Quantum neural networks have better generalization than classical networks.
Paper studies SGD dynamics in two-layer neural networks using PDE.
problem Understanding why SGD converges to good solutions in neural networks.
method Formulates SGD dynamics as a PDE and proves convergence to networks with ideal generalization.
result SGD dynamics in two-layer neural networks can be described by a PDE, leading to convergence to good solutions.
New method finds global optima in variational inference.
problem Uncertainty in finding global optima in variational inference.
method Deterministic optimization algorithm for variational inference.
result Always converges to globally optimal variational lower bound.
This paper presents studies on a deterministic annealing algorithm based on quantum annealing for variational Bayes (QAVB) inference, which can be seen as an extension of the simulated annealing for variational Bayes (SAVB) inference. QAVB is as easy as SAVB to implement. Experiments revealed QAVB finds a better local …
Ridge regression CV loss may have multiple local optima.
problem Can we globally optimize cross-validation loss in ridge regression?
method Analyzing quasiconvexity of CV loss in ridge regression.
result CV loss may fail to be quasiconvex and have multiple local optima.
SCI-PI solves scale invariant problems efficiently.
problem Solving scale invariant problems in optimization.
method Introduces SCI-PI and proves its convergence.
result SCI-PI achieves local linear convergence.
Bayesian optimization with directionally constrained search improves efficiency within a budget.
problem Optimizing expensive functions with limited computational resources.
method Directionally constrained search to allocate model capability efficiently.
result Our approach outperforms in finding the optimum within a prescribed evaluation budget.
The Baum-Welch algorithm converges to global optima with geometric rate.
problem Estimating HMM from observations with local optima risk.
method Characterization of basin of attraction and geometric convergence guarantees.
result Baum-Welch converges to a small ball around global optima with linear rate.
Improved VAE models avoid posterior collapse in text modeling.
problem Posterior collapse in VAEs leads to poor data manifold parameterization.
method Coupled-VAE couples a VAE with a deterministic autoencoder to improve encoder and decoder parameterizations.
result Coupled-VAE consistently improves results in probability estimation and latent space richness.
New method in Bayesian optimization finds optimal inputs knowing the optimal outputs.
problem Finding optimal inputs when the optimal outputs are known in advance.
method Transform Gaussian process surrogate using known optimum output; propose two acquisition functions.
result Our approaches give quantitatively better performance than standard BO methods.
A new method improves semidefinite programming performance.
problem Structured semidefinite programming with diagonal constraints.
method Low-rank coordinate descent approach called the Mixing method.
result The Mixing method converges to the global optimum almost surely.
Paper proposes efficient method to escape saddle points in non-convex optimization.
problem Hardness of escaping saddle points in non-convex optimization.
method Designs an efficient algorithm using higher order derivatives to converge to third order local optima.
result First efficient algorithm guaranteed to converge to a third order local optimum.
RWR converges to global optimum in certain settings.
problem Proving convergence of RWR to optimal policy.
method Iterative learning with return-weighted log-likelihood.
result RWR converges to global optimum under certain conditions.
OPFython simplifies Optimum-Path Forest for Python users.
problem Lack of diversity and complexity in conventional classification algorithms.
method Develops a Python-based Optimum-Path Forest framework.
result OPFython provides a more friendly and faster prototyping environment.
Estimates river discharge using few measurements at many locations.
problem Accurate river discharge estimation at scale using scarce in-situ measurements.
method Developed a common mechanism regression (CMR) model with local and shared components.
result Found global optimum for non-convex learning objective using spectral initialization and descent methods.
Study uses Bayesian Optimization to analyze noise effects in materials research.
problem Optimizing materials with many variables and experimental noise.
method Batch Bayesian Optimization with synthetic data analysis.
result Noise sensitivity varies by problem landscape, impacting optimization outcomes.
Unified approach to localized kernel learning improves performance.
problem Learning the best kernel for each training example.
method Unified system and new algorithm design.
result Improved performance through localized kernel learning.
Inference problems in graphical models are often approximated by casting them as constrained optimization problems. Message passing algorithms, such as belief propagation, have previously been suggested as methods for solving these optimization problems. However, there are few convergence guarantees for such algorithms…
LES optimizes designs by sampling descent sequences, achieving strong sample efficiency.
problem Optimizing large, complex design spaces is infeasible and unnecessary.
method LES uses Bayesian optimization to target solutions reachable by iterative optimizers.
result LES achieves strong sample efficiency compared to existing methods.
Optimization of very expensive black-box functions requires utilization of maximum information gathered by the process of optimization. Model Guided Sampling Optimization (MGSO) forms a more robust alternative to Jones' Gaussian-process-based EGO algorithm. Instead of EGO's maximizing expected improvement, the MGSO use…
No-regret BO algorithm adapts hyperparameters to optimize unknown functions.
problem Misspecification of hyperparameters in BO leads to poor local optima.
method Adapts hyperparameters online to expand function class and converge to optimum.
result First provably no-regret BO algorithm with unknown hyperparameters.
Stochastic optimization and sparse statistical recovery: An optimal algorithm for high dimensionsstat.ML
We develop and analyze stochastic optimization algorithms for problems in which the expected loss is strongly convex, and the optimum is (approximately) sparse. Previous approaches are able to exploit only one of these two structures, yielding an $\order(\pdim/T)$ convergence rate for strongly convex objectives in $\pd…
Efficiently minimizes regret in non-convex games with gradient-based methods.
problem Computational intractability of standard regret minimization in non-convex games.
method Defining a new notion of regret and using gradient-based optimization methods.
result Achieves optimal regret, leading to convergence to equilibrium.
Bayesian optimisation is improved by incorporating expert prior through space warping.
problem Cold start phase in expensive function optimisation.
method Prior distribution warps the search space around high probability regions of function optimum.
result Improves optimisation performance through acquisition agnostic approach.
AGGLIO optimizes non-convex functions with local convexity guarantees.
problem Optimizing non-convex functions with local convexity.
method Stage-wise, graduated optimization technique for locally convex functions.
result Global convergence to the global optimum for non-convex and locally convex objectives.