arXiv research

Improved convergence of fixed-point methods using windowed Anderson acceleration.

Belief propagation (BP) is an iterative method to perform approximate inference on arbitrary graphical models. Whether BP converges and if the solution is a unique fixed point depends on both the structure and the parametrization of the model. To understand this dependence it is interesting to find \emph{all} fixed poi…

Banach's fixed point theorem for contraction maps has been widely used to analyze the convergence of iterative methods in non-convex problems. It is a common experience, however, that iterative maps fail to be globally contracting under the natural metric in their domain, making the applicability of Banach's theorem li…

The principle of convergence stability for geometric flows is the combination of the continuous dependence of the flow on initial conditions, with the stability of fixed points. It implies that if the flow from an initial state $g_0$ exists for all time and converges to a stable fixed point, then the flows of solutions…

Convex message passing algorithms converge to a fixed point.

The Bass model is calibrated to vanilla options using a fixed-point equation.

Developed an efficient iterative algorithm for SVI model.

A recent analysis of a model of iterative neural network in Hilbert spaces established fundamental properties of such networks, such as existence of the fixed points sets, convergence analysis, and Lipschitz continuity. Building on these results, we show that under a single mild condition on the weights of the network,…

The high computational and parameter complexity of neural networks makes their training very slow and difficult to deploy on energy and storage-constrained computing systems. Many network complexity reduction techniques have been proposed including fixed-point implementation. However, a systematic approach for designin…

Develops accelerated fixed-point methods with delayed oracles for scientific computing.

New methods for federated learning reduce communication costs.

With the inflation of the data, clustering analysis, as a branch of unsupervised learning, lacks unified understanding and application of its mathematical law. Based on the view of fixed point, this paper restates the model-based clustering and proposes a unified clustering framework. In order to find fixed points as c…

Interpreting gradient methods as fixed-point iterations, we provide a detailed analysis of those methods for minimizing convex objective functions. Due to their conceptual and algorithmic simplicity, gradient methods are widely used in machine learning for massive data sets (big data). In particular, stochastic gradien…

Study reveals three limiting regimes for neural network functionals.

Let G be a compact Lie group and X be a compact smooth G-manifold with finitely many G-fixed points. We show that if X admits a G-equivariant hyperbolic diffeomorphism having a certain convergence property, there exists an open covering of X indexed by the G-fixed points so that each open set is G-stable and G-equivari…

New method stabilizes DEQ models by regularizing Jacobian of fixed-point equations.

FedSplit improves federated learning by ensuring correct convergence to optimal solutions.

Improved convergence speed of principal component analysis through modified learning rules.

This paper studies a valuation framework for financial contracts subject to reference and counterparty default risks with collateralization requirement. We propose a fixed point approach to analyze the mark-to-market contract value with counterparty risk provision, and show that it is a unique bounded and continuous fi…

A number of problems in statistical physics and computer science can be expressed as the computation of marginal probabilities over a Markov random field. Belief propagation, an iterative message-passing algorithm, computes exactly such marginals when the underlying graph is a tree. But it has gained its popularity as …

Gradient descent with biased rounding errors converges faster under certain conditions.

Faster algorithms for solving multichain MDPs under average-reward criterion.

The paper studies how neural networks evolve representations, finding a unique fixed point for nonlinear activations.

Finding a fixed point to a nonexpansive operator, i.e., $x^*=Tx^*$, abstracts many problems in numerical linear algebra, optimization, and other areas of scientific computing. To solve fixed-point problems, we propose ARock, an algorithmic framework in which multiple agents (machines, processors, or cores) update $x$ i…

EDML is a recently proposed algorithm for learning MAP parameters in Bayesian networks. In this paper, we present a number of new advances and insights on the EDML algorithm. First, we provide the multivalued extension of EDML, originally proposed for Bayesian networks over binary variables. Next, we identify a simplif…

Unified framework for solving fixed-point equations in deterministic and stochastic settings.

This paper extends stability analysis to non-convergent neural network training.

This thesis investigates belief propagation's performance in graphical models with loops.

Quantile Temporal-Difference learning proved convergent with proof.

Novel algorithm accelerates PnP methods for image deblurring and super-resolution.

The maximum a posteriori (MAP) configuration of binary variable models with submodular graph-structured energy functions can be found efficiently and exactly by graph cuts. Max-product belief propagation (MP) has been shown to be suboptimal on this class of energy functions by a canonical counterexample where MP conver…

We study the iterations of a class of curvature image operators $Λ_p^{\varphi}$ introduced by the author in (J. Funct. Anal. 271 (2016) 2133--2165). The fixed points of these operators are the solutions of the $L_p$ Minkowski problems with the positive continuous prescribed data $\varphi$. One of our results states tha…

Motivated by a recent result of Daskalakis et al. 2018, we analyze the population version of Expectation-Maximization (EM) algorithm for the case of \textit{truncated} mixtures of two Gaussians. Truncated samples from a $d$-dimensional mixture of two Gaussians $\frac{1}{2} \mathcal{N}(\vecμ, \vecΣ)+ \frac{1}{2} \mathca…

Value iteration is a fixed point iteration technique utilized to obtain the optimal value function and policy in a discounted reward Markov Decision Process (MDP). Here, a contraction operator is constructed and applied repeatedly to arrive at the optimal solution. Value iteration is a first order method and therefore …

This work studies the contraction coefficients of Schrödinger bridge problems in linear systems.

In this paper, we propose a new primal-dual algorithm for minimizing $f(x) + g(x) + h(Ax)$, where $f$, $g$, and $h$ are proper lower semi-continuous convex functions, $f$ is differentiable with a Lipschitz continuous gradient, and $A$ is a bounded linear operator. The proposed algorithm has some famous primal-dual algo…

The paper finds optimal strategies for hedging in incomplete markets using derivatives.

We characterize the price of an Asian option, a financial contract, as a fixed-point of a non-linear operator. In recent years, there has been interest in incorporating changes of regime into the parameters describing the evolution of the underlying asset price, namely the interest rate and the volatility, to model sud…

Study efficient derivative computation for nondifferentiable maps in machine learning.

The wide adoption of DNNs has given birth to unrelenting computing requirements, forcing datacenter operators to adopt domain-specific accelerators to train them. These accelerators typically employ densely packed full precision floating-point arithmetic to maximize performance per area. Ongoing research efforts seek t…

Gradient-based clustering method for various cost functions.

The paper analyzes when credal sets stabilize under iterative updates in machine learning.

New insights into quantized neural networks reveal learning dynamics and generalization errors.

Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems. This paper proposes and analyzes a generalization of Opper and Winther's expectation consistent (EC) approximate inference method. Th…

Gradient flow converges to a minimal convex structure.

New TD algorithms stabilize RL tasks by reformulating updates into fixed point equations.

We study the Immediate Exchange model, recently introduced by Heinsalu and Patriarca [Eur. Phys. J. B 87: 170 (2014)], who showed by simulations that the wealth distribution in this model converges to a Gamma distribution with shape parameter $2$. Here we justify this conclusion analytically, in the infinite-population…

The paper analyzes neural networks for solving high-dimensional Schrödinger eigenvalue problems.