arXiv research

The paper addresses ill-conditioning in large spatial data, proposing solutions for prediction and likelihood estimation.

SAPPHIRE tackles ill-conditioned rERM problems with faster convergence.

This paper analyzes adaptive gradient algorithms for better performance in ill-conditioned problems.

Improved convergence for overparameterized low-rank matrix sensing.

New framework uses score-based priors to solve ill-conditioned polynomial equations, improving signal recovery from noisy data.

Scaled gradient descent improves matrix recovery for ill-conditioned matrices with optimal sampling complexity.

Scalable method completes ill-conditioned matrices from few samples.

ScaledGD accelerates ill-conditioned low-rank estimation.

Improves numerical solution of ill-conditioned linear systems for machine learning.

Paper explores challenges in training PINNs and loss landscape effects.

Preconditioned SGD accelerates convergence for ill-conditioned huge-scale matrix completion.

Kaczmarz++ accelerates convergence for ill-conditioned systems.

Improved subgradient method tackles ill-conditioned composite optimization problems.

Dual-space sampling tackles ill-conditioned inverse problems with Bayesian methods.

This paper accelerates distributed convex optimization by mitigating ill-conditioning issues.

The total variation (TV) penalty, as many other analysis-sparsity problems, does not lead to separable factors or a proximal operatorwith a closed-form expression, such as soft thresholding for the $\ell\_1$ penalty. As a result, in a variational formulation of an inverse problem or statisticallearning estimation, it l…

New algorithms solve nonconvex-nonconcave minimax optimization problems.

We present a machine learning approach to the inversion of Fredholm integrals of the first kind. The approach provides a natural regularization in cases where the inverse of the Fredholm kernel is ill-conditioned. It also provides an efficient and stable treatment of constraints. The key observation is that the stabili…

Proposes an efficient shrinkage path for ridge regression.

ScaledGD improves gradient descent for ill-conditioned low-rank matrix estimation.

Recently, Neural networks have seen a huge surge in its adoption due to their ability to provide high accuracy on various tasks. On the other hand, the existence of adversarial examples have raised suspicions regarding the generalization capabilities of neural networks. In this work, we focus on the weight matrix learn…

We orthogonalize the NSS model to condition and diagnose its ill-conditioned parameters.

Consider Least Squares Monte Carlo (LSM) algorithm, which is proposed by Longstaff and Schwartz (2001) for pricing American style securities. This algorithm is based on the projection of the value of continuation onto a certain set of basis functions via the least squares problem. We analyze the stability of the algori…

In this paper, we study large-scale convex optimization algorithms based on the Newton method applied to regularized generalized self-concordant losses, which include logistic regression and softmax regression. We first prove that our new simple scheme based on a sequence of problems with decreasing regularization para…

In this paper, we extend the geometric descent method recently proposed by Bubeck, Lee and Singh to tackle nonsmooth and strongly convex composite problems. We prove that our proposed algorithm, dubbed geometric proximal gradient method (GeoPG), converges with a linear rate $(1-1/\sqrtκ)$ and thus achieves the optimal …

Paper shows linear convergence of ISTA and FISTA for ill-conditioned images.

New insights into convergence of optimization methods for DAG structure learning.

Several methods have been recently proposed for estimating sparse Gaussian graphical models using $\ell_{1}$ regularization on the inverse covariance matrix. Despite recent advances, contemporary applications require methods that are even faster in order to handle ill-conditioned high dimensional modern day datasets. I…

New algorithm reduces sample complexity for sparse linear regression.

A new method for robust matrix completion overcomes limitations of existing approaches.

Regularized LAEs learn principal components efficiently.

New algorithm improves heteroskedastic PCA performance.

New approach improves deep learning robustness in medical imaging.

A new approach of solving the ill-conditioned inverse problem for analytical continuation is proposed. The root of the problem lies in the fact that even tiny noise of imaginary-time input data has a serious impact on the inferred real-frequency spectra. By means of a modern regularization technique, we eliminate redun…

Bayesian optimisation is a powerful tool to solve expensive black-box problems, but fails when the stationary assumption made on the objective function is strongly violated, which is the case in particular for ill-conditioned or discontinuous objectives. We tackle this problem by proposing a new Bayesian optimisation f…

With the advent of massive data sets much of the computational science and engineering community has moved toward data-intensive approaches in regression and classification. However, these present significant challenges due to increasing size, complexity and dimensionality of the problems. In particular, covariance mat…

HSNLD solves robust Hankel recovery efficiently and robustly.

The natural gradient allows for more efficient gradient descent by removing dependencies and biases inherent in a function's parameterization. Several papers present the topic thoroughly and precisely. It remains a very difficult idea to get your head around however. The intent of this note is to provide simple intuiti…

A new optimization method reduces memory and compute requirements for deep learning.

AI-driven framework optimizes MCMC-based preconditioners for faster linear system solving.

Regularizes 3D inverse scattering with tangent-point energy for better solutions.

We provide a new theoretical analysis framework to investigate online gradient descent in the dynamic environment. Comparing with the previous work, the new framework recovers the state-of-the-art dynamic regret, but does not require extra gradient queries for every iteration. Specifically, when functions are $α$ stron…

Paper introduces a new measure of conditional dependence avoiding matrix inversions.

SONIA optimizes machine learning problems with a novel algorithm.

New method speeds up optimization over probability measures.

New method speeds up Gaussian process training and inference for large datasets.

Gaussian Processes (GPs) are a popular approach to predict the output of a parameterized experiment. They have many applications in the field of Computer Experiments, in particular to perform sensitivity analysis, adaptive design of experiments and global optimization. Nearly all of the applications of GPs require the …

Short-and-sparse deconvolution (SaSD) is the problem of extracting localized, recurring motifs in signals with spatial or temporal structure. Variants of this problem arise in applications such as image deblurring, microscopy, neural spike sorting, and more. The problem is challenging in both theory and practice, as na…