A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
ProGO optimizes non-convex functions without gradients, outperforming existing methods.
Multiview representation learning is very popular for latent factor analysis. It naturally arises in many data analysis, machine learning, and information retrieval applications to model dependent structures among multiple data sources. For computational convenience, existing approaches usually formulate the multiview …
Theoretical study explains why federated optimization fails to achieve perfect fitting.
Global optimization finds applications in a wide range of real world problems. The multi-start methods are a popular class of global optimization techniques, which are based on the ideas of conducting local searches at multiple starting points. In this work we propose a new multi-start algorithm where the starting poin…
The paper proves convergence to global optima for a class of distributed algorithms for nonconvex optimization in network-based multi-agent settings. Agents are permitted to communicate over a time-varying undirected graph. Each agent is assumed to possess a local objective function (assumed to be smooth, but possibly …
Improves Bayesian optimization using Gaussian process Thompson sampling.
Physical systems are modelled and investigated within simulation software in an increasing range of applications. In reality an investigation of the system is often performed by empirical test scenarios which are related to typical situations. Our aim is to derive a method which generates diverse test scenarios each re…
This paper addresses problematic global optima in VAEs, proposing a new inference method.
A faster, more stable method for optimizing topological functions.
New perspective on federated learning as posterior inference, improving optimization.
Many modern neural network architectures are trained in an overparameterized regime where the parameters of the model exceed the size of the training dataset. Sufficiently overparameterized neural network architectures in principle have the capacity to fit any set of labels including random noise. However, given the hi…
Many modern learning tasks involve fitting nonlinear models to data which are trained in an overparameterized regime where the parameters of the model exceed the size of the training dataset. Due to this overparameterization, the training loss may have infinitely many global minima and it is critical to understand the …
New findings on the max margin problem in neural networks.
Neural networks have been used prominently in several machine learning and statistics applications. In general, the underlying optimization of neural networks is non-convex which makes their performance analysis challenging. In this paper, we take a novel approach to this problem by asking whether one can constrain neu…
SVH-PSL uses Stein Variational Gradient Descent and Hypernetworks to improve Pareto set learning for expensive MOO.
Gradients help find global optima in complex functions.
Semidefinite programs (SDP) are important in learning and combinatorial optimization with numerous applications. In pursuit of low-rank solutions and low complexity algorithms, we consider the Burer--Monteiro factorization approach for solving SDPs. We show that all approximate local optima are global optima for the pe…
sEM uses optimal transport to improve EM algorithm for better convergence and avoiding local optima.
The Hidden Markov Model (HMM) is one of the mainstays of statistical modeling of discrete time series, with applications including speech recognition, computational biology, computer vision and econometrics. Estimating an HMM from its observation process is often addressed via the Baum-Welch algorithm, which is known t…
GAIL with neural networks converges to global optima and has a known rate.
Postprocessing reduces Bayesian optimization steps for global optima.
Adversarial representation learning is a promising paradigm for obtaining data representations that are invariant to certain sensitive attributes while retaining the information necessary for predicting target attributes. Existing approaches solve this problem through iterative adversarial minimax optimization and lack…
Temporal-difference learning (TD), coupled with neural networks, is among the most fundamental building blocks of deep reinforcement learning. However, due to the nonlinearity in value function approximation, such a coupling leads to nonconvexity and even divergence in optimization. As a result, the global convergence …
Gradient descent finds global optima in ResNets with sufficient parameters.
Ridge regression CV loss may have multiple local optima.
SAM optimizer struggles to converge to global minima or stationary points in practical settings.
We propose a strategy for approximating Pareto optimal sets based on the global analysis framework proposed by Smale (Dynamical systems, New York, 1973, pp. 531-544). The method highlights and exploits the underlying manifold structure of the Pareto sets, approximating Pareto optima by means of simplicial complexes. Th…
Proposes a new approach to generate sparse models from deep networks.
We give a formal and complete characterization of the explicit regularizer induced by dropout in deep linear networks with squared loss. We show that (a) the explicit regularizer is composed of an -path regularizer and other terms that are also re-scaling invariant, (b) the convex envelope of the induced regula…
The paper analyzes the intrinsic exploration terms in policy-gradient algorithms.
This paper proposes and evaluates the k-greedy equivalence search algorithm (KES) for learning Bayesian networks (BNs) from complete data. The main characteristic of KES is that it allows a trade-off between greediness and randomness, thus exploring different good local optima. When greediness is set at maximum, KES co…
We provide novel theoretical results regarding local optima of regularized -estimators, allowing for nonconvexity in both loss and penalty functions. Under restricted strong convexity on the loss and suitable regularity conditions on the penalty, we prove that \emph{any stationary point} of the composite objective f…
Gradient descent with preconditioning finds global optima in overparameterized nonconvex factorization.
GBML with deep nets converges globally and generalizes well.
Non-convex optimization with local search heuristics has been widely used in machine learning, achieving many state-of-art results. It becomes increasingly important to understand why they can work for these NP-hard problems on typical data. The landscape of many objective functions in learning has been conjectured to …
Graph cuts find global optima for Potts models in slight perturbations.
New proof shows how to identify DAGs with weakly increasing errors.
Learning new representations of input observations in machine learning is often tackled using a factorization of the data. For many such problems, including sparse coding and matrix completion, learning these factorizations can be difficult, in terms of efficiency and to guarantee that the solution is a global minimum.…
Bayesian Algorithm Execution uses mutual information to infer properties of black-box functions efficiently.
Gradient descent with noise converges to a unique optimum in nonconvex matrix factorization.
The accuracy of least squares calibration using option premiums and particle filtering of price data to find model parameters is determined. Derivative models using exponential Lévy processes are calibrated using regularized weighted least squares with respect to the minimal entropy martingale measure. Sequential impor…
SAVO actor improves reinforcement learning by avoiding local optima in complex Q-functions.
KSOS improves kernel learning for dynamical systems via global optimization.
Improves EM algorithm for better local optima in mixture models.
For the problem of high-dimensional sparse linear regression, it is known that an -based estimator can achieve a "fast" rate on the prediction error without any conditions on the design matrix, whereas in absence of restrictive conditions on the design matrix, popular polynomial-time methods only guarante…
Our research proves neural collapse in deep ResNets and transformers is globally optimal.
We study modeling and inference with the Elliptical Gamma Distribution (EGD). We consider maximum likelihood (ML) estimation for EGD scatter matrices, a task for which we develop new fixed-point algorithms. Our algorithms are efficient and converge to global optima despite nonconvexity. Moreover, they turn out to be mu…
In many applications of black-box optimization, one can evaluate multiple points simultaneously, e.g. when evaluating the performances of several different neural network architectures in a parallel computing environment. In this paper, we develop a novel batch Bayesian optimization algorithm --- the parallel knowledge…