arXiv research

Bayesian optimisation framework considers only variable orderings to handle ill-conditioned objectives.

SAPPHIRE tackles ill-conditioned rERM problems with faster convergence.

ScaledGD accelerates ill-conditioned low-rank estimation.

This paper analyzes adaptive gradient algorithms for better performance in ill-conditioned problems.

The paper addresses ill-conditioning in large spatial data, proposing solutions for prediction and likelihood estimation.

Paper shows linear convergence of ISTA and FISTA for ill-conditioned images.

New method speeds up optimization over probability measures.

Scalable method completes ill-conditioned matrices from few samples.

Improved subgradient method tackles ill-conditioned composite optimization problems.

Improves numerical solution of ill-conditioned linear systems for machine learning.

Paper explores challenges in training PINNs and loss landscape effects.

Improved convergence for overparameterized low-rank matrix sensing.

This paper investigates the control of an ML component within the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) devoted to black-box optimization. The known CMA-ES weakness is its sample complexity, the number of evaluations of the objective function needed to approximate the global optimum. This weakness is…

Scaled gradient descent improves matrix recovery for ill-conditioned matrices with optimal sampling complexity.

We provide a new theoretical analysis framework to investigate online gradient descent in the dynamic environment. Comparing with the previous work, the new framework recovers the state-of-the-art dynamic regret, but does not require extra gradient queries for every iteration. Specifically, when functions are $α$ stron…

New framework uses score-based priors to solve ill-conditioned polynomial equations, improving signal recovery from noisy data.

Preconditioned SGD accelerates convergence for ill-conditioned huge-scale matrix completion.

RQMC improves optimization in variational Bayes problems.

Proposes an efficient shrinkage path for ridge regression.

Recently, Neural networks have seen a huge surge in its adoption due to their ability to provide high accuracy on various tasks. On the other hand, the existence of adversarial examples have raised suspicions regarding the generalization capabilities of neural networks. In this work, we focus on the weight matrix learn…

Kaczmarz++ accelerates convergence for ill-conditioned systems.

We introduce a generic scheme for accelerating gradient-based optimization methods in the sense of Nesterov. The approach, called Catalyst, builds upon the inexact accelerated proximal point algorithm for minimizing a convex objective function, and consists of approximately solving a sequence of well-chosen auxiliary p…

Regularized LAEs learn principal components efficiently.

Yau's Affine Normal Descent optimizes smooth unconstrained problems with geometrically adapted directions.

A large body of animation research focuses on optimization of movement control, either as action sequences or policy parameters. However, as closed-form expressions of the objective functions are often not available, our understanding of the optimization problems is limited. Building on recent work on analyzing neural …

We orthogonalize the NSS model to condition and diagnose its ill-conditioned parameters.

Dual-space sampling tackles ill-conditioned inverse problems with Bayesian methods.

This paper accelerates distributed convex optimization by mitigating ill-conditioning issues.

A new optimization method reduces memory and compute requirements for deep learning.

AI-driven framework optimizes MCMC-based preconditioners for faster linear system solving.

New algorithms solve nonconvex-nonconcave minimax optimization problems.

ScaledGD improves gradient descent for ill-conditioned low-rank matrix estimation.

The total variation (TV) penalty, as many other analysis-sparsity problems, does not lead to separable factors or a proximal operatorwith a closed-form expression, such as soft thresholding for the $\ell\_1$ penalty. As a result, in a variational formulation of an inverse problem or statisticallearning estimation, it l…

New algorithm reduces sample complexity for sparse linear regression.

In this paper, we study large-scale convex optimization algorithms based on the Newton method applied to regularized generalized self-concordant losses, which include logistic regression and softmax regression. We first prove that our new simple scheme based on a sequence of problems with decreasing regularization para…

A new method for robust matrix completion overcomes limitations of existing approaches.

New method speeds up Gaussian process training and inference for large datasets.

We present a machine learning approach to the inversion of Fredholm integrals of the first kind. The approach provides a natural regularization in cases where the inverse of the Fredholm kernel is ill-conditioned. It also provides an efficient and stable treatment of constraints. The key observation is that the stabili…

Gaussian Processes (GPs) are a popular approach to predict the output of a parameterized experiment. They have many applications in the field of Computer Experiments, in particular to perform sensitivity analysis, adaptive design of experiments and global optimization. Nearly all of the applications of GPs require the …

Stochastic gradient descent approximates Gaussian process posteriors efficiently.

New insights into convergence of optimization methods for DAG structure learning.

Developed shrinkage methods for Poisson regression models with experts to handle multicollinearity.

New algorithm improves heteroskedastic PCA performance.

New insights into adversarial vulnerability linked to manifold separability issues.

New gradient methods solve multiscale optimization problems efficiently.

Consider Least Squares Monte Carlo (LSM) algorithm, which is proposed by Longstaff and Schwartz (2001) for pricing American style securities. This algorithm is based on the projection of the value of continuation onto a certain set of basis functions via the least squares problem. We analyze the stability of the algori…

Several methods have been recently proposed for estimating sparse Gaussian graphical models using $\ell_{1}$ regularization on the inverse covariance matrix. Despite recent advances, contemporary applications require methods that are even faster in order to handle ill-conditioned high dimensional modern day datasets. I…

In this paper, we extend the geometric descent method recently proposed by Bubeck, Lee and Singh to tackle nonsmooth and strongly convex composite problems. We prove that our proposed algorithm, dubbed geometric proximal gradient method (GeoPG), converges with a linear rate $(1-1/\sqrtκ)$ and thus achieves the optimal …