arXiv research

The paper addresses ill-conditioning in large spatial data, proposing solutions for prediction and likelihood estimation.

Scalable method completes ill-conditioned matrices from few samples.

Improves numerical solution of ill-conditioned linear systems for machine learning.

Paper explores challenges in training PINNs and loss landscape effects.

SAPPHIRE tackles ill-conditioned rERM problems with faster convergence.

This paper analyzes adaptive gradient algorithms for better performance in ill-conditioned problems.

Improved convergence for overparameterized low-rank matrix sensing.

Scaled gradient descent improves matrix recovery for ill-conditioned matrices with optimal sampling complexity.

ScaledGD accelerates ill-conditioned low-rank estimation.

New framework uses score-based priors to solve ill-conditioned polynomial equations, improving signal recovery from noisy data.

Preconditioned SGD accelerates convergence for ill-conditioned huge-scale matrix completion.

Proposes an efficient shrinkage path for ridge regression.

Recently, Neural networks have seen a huge surge in its adoption due to their ability to provide high accuracy on various tasks. On the other hand, the existence of adversarial examples have raised suspicions regarding the generalization capabilities of neural networks. In this work, we focus on the weight matrix learn…

Kaczmarz++ accelerates convergence for ill-conditioned systems.

Regularized LAEs learn principal components efficiently.

Improved subgradient method tackles ill-conditioned composite optimization problems.

We orthogonalize the NSS model to condition and diagnose its ill-conditioned parameters.

Dual-space sampling tackles ill-conditioned inverse problems with Bayesian methods.

This paper accelerates distributed convex optimization by mitigating ill-conditioning issues.

Paper shows linear convergence of ISTA and FISTA for ill-conditioned images.

A new optimization method reduces memory and compute requirements for deep learning.

AI-driven framework optimizes MCMC-based preconditioners for faster linear system solving.

New algorithms solve nonconvex-nonconcave minimax optimization problems.

ScaledGD improves gradient descent for ill-conditioned low-rank matrix estimation.

The total variation (TV) penalty, as many other analysis-sparsity problems, does not lead to separable factors or a proximal operatorwith a closed-form expression, such as soft thresholding for the $\ell\_1$ penalty. As a result, in a variational formulation of an inverse problem or statisticallearning estimation, it l…

New method speeds up optimization over probability measures.

New algorithm reduces sample complexity for sparse linear regression.

In this paper, we study large-scale convex optimization algorithms based on the Newton method applied to regularized generalized self-concordant losses, which include logistic regression and softmax regression. We first prove that our new simple scheme based on a sequence of problems with decreasing regularization para…

A new method for robust matrix completion overcomes limitations of existing approaches.

New method speeds up Gaussian process training and inference for large datasets.

We present a machine learning approach to the inversion of Fredholm integrals of the first kind. The approach provides a natural regularization in cases where the inverse of the Fredholm kernel is ill-conditioned. It also provides an efficient and stable treatment of constraints. The key observation is that the stabili…

Gaussian Processes (GPs) are a popular approach to predict the output of a parameterized experiment. They have many applications in the field of Computer Experiments, in particular to perform sensitivity analysis, adaptive design of experiments and global optimization. Nearly all of the applications of GPs require the …

New insights into convergence of optimization methods for DAG structure learning.

Developed shrinkage methods for Poisson regression models with experts to handle multicollinearity.

New algorithm improves heteroskedastic PCA performance.

New insights into adversarial vulnerability linked to manifold separability issues.

Consider Least Squares Monte Carlo (LSM) algorithm, which is proposed by Longstaff and Schwartz (2001) for pricing American style securities. This algorithm is based on the projection of the value of continuation onto a certain set of basis functions via the least squares problem. We analyze the stability of the algori…

Several methods have been recently proposed for estimating sparse Gaussian graphical models using $\ell_{1}$ regularization on the inverse covariance matrix. Despite recent advances, contemporary applications require methods that are even faster in order to handle ill-conditioned high dimensional modern day datasets. I…

In this paper, we extend the geometric descent method recently proposed by Bubeck, Lee and Singh to tackle nonsmooth and strongly convex composite problems. We prove that our proposed algorithm, dubbed geometric proximal gradient method (GeoPG), converges with a linear rate $(1-1/\sqrtκ)$ and thus achieves the optimal …

Deep-learning-based methods for different applications have been shown vulnerable to adversarial examples. These examples make deployment of such models in safety-critical tasks questionable. Use of deep neural networks as inverse problem solvers has generated much excitement for medical imaging including CT and MRI, b…

New scalable methods for unbalanced optimal transport improve efficiency and applicability.

The natural gradient allows for more efficient gradient descent by removing dependencies and biases inherent in a function's parameterization. Several papers present the topic thoroughly and precisely. It remains a very difficult idea to get your head around however. The intent of this note is to provide simple intuiti…

The natural gradient method has been used effectively in conjugate Gaussian process models, but the non-conjugate case has been largely unexplored. We examine how natural gradients can be used in non-conjugate stochastic settings, together with hyperparameter learning. We conclude that the natural gradient can signific…

With the advent of massive data sets much of the computational science and engineering community has moved toward data-intensive approaches in regression and classification. However, these present significant challenges due to increasing size, complexity and dimensionality of the problems. In particular, covariance mat…

A new approach of solving the ill-conditioned inverse problem for analytical continuation is proposed. The root of the problem lies in the fact that even tiny noise of imaginary-time input data has a serious impact on the inferred real-frequency spectra. By means of a modern regularization technique, we eliminate redun…

HSNLD solves robust Hankel recovery efficiently and robustly.

We provide a new theoretical analysis framework to investigate online gradient descent in the dynamic environment. Comparing with the previous work, the new framework recovers the state-of-the-art dynamic regret, but does not require extra gradient queries for every iteration. Specifically, when functions are $α$ stron…

SLMC improves sampling efficiency for high-dimensional distributions.