arXiv research

Paper proposes a working set algorithm for non-convex sparse regression with provable convergence.

Generalizes smoothness conditions for optimization methods.

Here we study non-convex composite optimization: first, a finite-sum of smooth but non-convex functions, and second, a general function that admits a simple proximal mapping. Most research on stochastic methods for composite optimization assumes convexity or strong convexity of each function. In this paper, we extend t…

Solves equality case in isoperimetric inequality for non-convex domains.

In this paper, we consider the problem of learning high-dimensional tensor regression problems with low-rank structure. One of the core challenges associated with learning high-dimensional models is computation since the underlying optimization problems are often non-convex. While convex relaxations could lead to polyn…

This work explores the non-convex optimization in compressive learning and the performance of heuristics.

Stochastic gradient descent is the method of choice for large scale optimization of machine learning objective functions. Yet, its performance is greatly variable and heavily depends on the choice of the stepsizes. This has motivated a large body of research on adaptive stepsizes. However, there is currently a gap in o…

Improved variance reduction for Riemannian non-convex optimization with adaptive batch size.

New algorithms for differentially private optimization in convex and non-convex settings with near-optimal rates.

Non-convex sparsity-inducing penalties have recently received considerable attentions in sparse learning. Recent theoretical investigations have demonstrated their superiority over the convex counterparts in several sparse learning settings. However, solving the non-convex optimization problems associated with non-conv…

New algorithms optimize non-smooth, non-convex objectives with improved complexity.

This paper improves convergence guarantees for SGD algorithms in non-convex smooth functions.

New method certifies generative models' robustness.

Linear speedup achieved in non-convex optimization for decentralized systems.

Paper checks SSC for matrix factorizations using Gurobi.

Algorithm optimizes non-convex functions using dueling comparisons.

SGD's uncertainty quantified in non-convex learning problems.

New framework for robust hypothesis testing using Sinkhorn uncertainty sets.

SGD converges to global minimum for structured non-convex functions.

Improved optimization guarantees for deep learning models with Nesterov acceleration.

Unified analysis of multi-attribute graph learning with non-convex penalties.

We consider online learning in an adversarial, non-convex setting under the assumption that the learner has an access to an offline optimization oracle. In the general setting of prediction with expert advice, Hazan et al. (2016) established that in the optimization-oracle model, online learning requires exponentially …

New bounds for SGLD show error decreases with more data.

Many classical algorithms are found until several years later to outlive the confines in which they were conceived, and continue to be relevant in unforeseen settings. In this paper, we show that SVRG is one such method: being originally designed for strongly convex objectives, it is also very robust in non-strongly co…

Paper shows non-convexity in solutions to Hessian equations.

Paper proposes a weak approximation of reflection coupling for non-convex optimization.

Paper estimates differences in multi-attribute Gaussian graphical models using non-convex penalties.

ProxSkip achieves linear speedup in distributed non-convex optimization.

The paper analyzes adaptive algorithms in non-convex optimization landscapes.

This paper tackles global Nash equilibrium in non-convex multi-player games.

AEGD optimizes non-convex functions with dynamic energy updates.

Develops a new interior-point technique for non-convex optimization problems.

New approach for distributed online optimization of non-convex losses with sublinear regret.

Study accelerates optimization methods in non-convex problems, but doesn't improve the algorithm's performance.

New bounds found for optimizing non-convex functions with noisy data.

Online SGD from random init solves non-smooth, non-convex phase retrieval.

A fast method for decentralized non-convex optimization over networks.

MAGIC method optimally estimates model predictions changes.

We study the boundary and lens rigidity problems on domains without assuming the convexity of the boundary. We show that such rigidities hold when the domain is a simply connected compact Riemannian surface without conjugate points. For the more general class of non-trapping compact Riemannian surfaces with no conjugat…

In machine learning research, the proximal gradient methods are popular for solving various optimization problems with non-smooth regularization. Inexact proximal gradient methods are extremely important when exactly solving the proximal operator is time-consuming, or the proximal operator does not have an analytic sol…

The paper tackles MAP inference over non-convex constraints in safety-critical settings.

The aim of this paper is to develop a general framework for training neural networks (NNs) in a distributed environment, where training data is partitioned over a set of agents that communicate with each other through a sparse, possibly time-varying, connectivity pattern. In such distributed scenario, the training prob…

The family of admissible positions in a transaction costs model is a random closed set, which is convex in case of proportional transaction costs. However, the convexity fails, e.g. in case of fixed transaction costs or when only a finite number of transfers are possible. The paper presents an approach to measure risks…

Develops a new fairness learning approach for multi-task regression models.

The online meta-learning framework is designed for the continual lifelong learning setting. It bridges two fields: meta-learning which tries to extract prior knowledge from past tasks for fast learning of future tasks, and online-learning which deals with the sequential setting where problems are revealed one by one. I…

Analyzes optimal learning rate schedules in high-dimensional non-convex optimization problems.

The paper derives new inequalities for non-convex domains and flows.

This paper tackles non-convex phase retrieval with structured assumptions.