arXiv research

Kaczmarz++ accelerates convergence for ill-conditioned systems.

New method tackles tensor regression with robust Kaczmarz approach.

Paper analyzes Hit-and-Run's convergence rates and applies similar methods to randomized Kaczmarz.

New algorithms improve machine learning performance with explicit regret bounds.

New study shows faster convergence of SGD and Kaczmarz methods.

A new method for LDA using randomized Kaczmarz improves accuracy for large datasets.

Unified framework for solving linear systems with improved convergence rates.

The paper improves Kaczmarz algorithm with momentum for linear least squares.

Paper provides linear convergence guarantees for KZIHT and KZPT methods.

AI models solved the Kaczmarz algorithm's worst-case complexity.

The Kaczmarz algorithm is popular for iteratively solving an overdetermined system of linear equations. The traditional Kaczmarz algorithm can approximate the solution in few sweeps through the equations but a randomized version of the Kaczmarz algorithm was shown to converge exponentially and independent of number of …

We obtain an improved finite-sample guarantee on the linear convergence of stochastic gradient descent for smooth and strongly convex objectives, improving from a quadratic dependence on the conditioning $(L/μ)^2$ (where $L$ is a bound on the smoothness and $μ$ on the strong convexity) to a linear dependence on $L/μ$. …

This paper is about randomized iterative algorithms for solving a linear system of equations $X β= y$ in different settings. Recent interest in the topic was reignited when Strohmer and Vershynin (2009) proved the linear convergence rate of a Randomized Kaczmarz (RK) algorithm that works on the rows of $X$ (data points…

We revisit the problem of inferring the overall ranking among entities in the framework of Bradley-Terry-Luce (BTL) model, based on available empirical data on pairwise preferences. By a simple transformation, we can cast the problem as that of solving a noisy linear system, for which a ready algorithm is available in …

New algorithm reduces bias and variance in weighted least-squares solutions.

In this paper we present a convergence rate analysis of inexact variants of several randomized iterative methods. Among the methods studied are: stochastic gradient descent, stochastic Newton, stochastic proximal point and stochastic subspace ascent. A common feature of these methods is that in their update rule a cert…

New approach connects stochastic gradient descent to ODE splitting schemes.

We propose a methodology for computing single and multi-asset European option prices, and more generally expectations of scalar functions of (multivariate) random variables. This new approach combines the ability of Monte Carlo simulation to handle high-dimensional problems with the efficiency of function approximation…

New method shows stochastic momentum can converge quickly on optimization problems.

Randomized algorithms that base iteration-level decisions on samples from some pool are ubiquitous in machine learning and optimization. Examples include stochastic gradient descent and randomized coordinate descent. This paper makes progress at theoretically evaluating the difference in performance between sampling wi…

The paper sharpens the analysis of sketch-and-project methods using randomized singular value decomposition.

In recent literature, a general two step procedure has been formulated for solving the problem of phase retrieval. First, a spectral technique is used to obtain a constant-error initial estimate, following which, the estimate is refined to arbitrary precision by first-order optimization of a non-convex loss function. N…

The study explores how Transformers predict the next token in a sequence.

We study the phase retrieval problem, which solves quadratic system of equations, i.e., recovers a vector $\boldsymbol{x}\in \mathbb{R}^n$ from its magnitude measurements $y_i=|\langle \boldsymbol{a}_i, \boldsymbol{x}\rangle|, i=1,..., m$. We develop a gradient-like algorithm (referred to as RWF representing reshaped W…

SGD achieves near optimal convergence rate in smooth interpolation regime.

Improved rates for continual learning using SGD and last-iterate analysis.

Improved iterative methods for risk parity portfolio weights.

We describe a novel optimization method for finite sums (such as empirical risk minimization problems) building on the recently introduced SAGA method. Our method achieves an accelerated convergence rate on strongly convex smooth problems. Our method has only one parameter (a step size), and is radically simpler than o…

A new method combines Laplace and Variational Bayes for scalable inference.

Unified framework for model explanation methods based on feature removal.

This work reviews and evaluates methods for predicting prediction intervals in regression problems.

Derives kernel PCA with Nyström method for scalability.

In this paper, the author considers the numerical computation of CVA for large systems by Mote Carlo methods. He introduces two types of stochastic mesh methods for the computations of CVA. In the first method, stochastic mesh method is used to obtain the future value of the derivative contracts. In the second method, …

New method combines spectral and sparse methods for Gaussian processes.

A comprehensive benchmark of 15 scRNA-seq imputation methods across various datasets and analyses.

New methods using natural gradient for structured optimization.

Recently, {\it stochastic momentum} methods have been widely adopted in training deep neural networks. However, their convergence analysis is still underexplored at the moment, in particular for non-convex optimization. This paper fills the gap between practice and theory by developing a basic convergence analysis of t…

We investigate methods for pricing American options under the variance gamma model. The variance gamma process is a pure jump process which is constructed by replacing the calendar time by the gamma time in a Brownian motion with drift, which makes it a time-changed Brownian motion. In general, the finite difference me…

A new method speeds up deep neural network training.

We propose a new stochastic dual coordinate ascent technique that can be applied to a wide range of regularized learning problems. Our method is based on Alternating Direction Multiplier Method (ADMM) to deal with complex regularization functions such as structured regularizations. Although the original ADMM is a batch…

NCG methods improve shape optimization efficiency.

We propose two localized Radial Basis Function (RBF) methods, the Radial Basis Function Partition of Unity method (RBF-PUM) and the Radial Basis Function generated Finite Differences method (RBF-FD), for solving financial derivative pricing problems arising from market models with multiple stochastic factors. We demons…

Proposes UTC method for stock price prediction with uncertainty quantification.

Survey of spectral, probabilistic, and deep metric learning methods.

Various approaches to gene selection for cancer classification based on microarray data can be found in the literature and they may be grouped into two categories: univariate methods and multivariate methods. Univariate methods look at each gene in the data in isolation from others. They measure the contribution of a p…

A novel weighted feature selection method using fuzzy sets improves classification accuracy and stability.

New method improves accuracy in computing implied volatility.

The paper examines Wiener process for LID estimation methods.