arXiv research

Exact solver speeds up Weston-Watkins SVM subproblem significantly.

Improved Frank-Wolfe algorithms for large-scale optimization.

Pipeline decomposes portfolio optimization problems into smaller, solvable subproblems.

A new framework RTK accelerates diffusion inference by breaking down the process into fewer, more efficient subproblems.

In this paper we consider sparse approximation problems, that is, general $l_0$ minimization problems with the $l_0$-"norm" of a vector being a part of constraints or objective function. In particular, we first study the first-order optimality conditions for these problems. We then propose penalty decomposition (PD) me…

New approach uses dynamic programming to efficiently discover failures in autonomous vehicle simulations.

Paper presents a method to solve variational inequalities with general constraints without requiring analytic solutions.

Recently several methods were proposed for sparse optimization which make careful use of second-order information [10, 28, 16, 3] to improve local convergence rates. These methods construct a composite quadratic approximation using Hessian information, optimize this approximation using a first-order method, such as coo…

Two methods extend multivariate Kelly optimization to large problem sizes.

We show that Newton's method converges globally at a linear rate for objective functions whose Hessians are stable. This class of problems includes many functions which are not strongly convex, such as logistic regression. Our linear convergence result is (i) affine-invariant, and holds even if an (ii) approximate Hess…

A new method reduces the cost of solving large-scale linear models.

We describe a new technique for computing lower-bounds on the minimum energy configuration of a planar Markov Random Field (MRF). Our method successively adds large numbers of constraints and enforces consistency over binary projections of the original problem state space. These constraints are represented in terms of …

Paper studies fully implicit online learning algorithms for improved numerical stability.

New method for sparse kernel selection improves prediction accuracy.

Proposes STRON method for large-scale machine learning problems.

The problem of classification of Legendrian knots (links) up to isotopy in the class of Legendrian embeddings (Legendrian isotopy) naturally leads to the following two subproblems. The first of them is: which combinations of the three classical invariants can be realized by a Legendrian knot? (It is well-known that eac…

Iterated Amplification uses subproblem solutions to build training signals for complex tasks.

BlitzWS is a working set algorithm for convex problems with theoretical guarantees.

New method solves optimization problems faster than existing methods.

A new method learns interpretable decision rules using submodular optimization.

New method combines deep learning and splitting for high-dimensional PDEs.

New algorithm solves phase retrieval with adaptive stopping criteria.

MOBO-OSD optimizes multi-objective functions using orthogonal search directions.

We propose a communication- and computation-efficient distributed optimization algorithm using second-order information for solving ERM problems with a nonsmooth regularization term. Current second-order and quasi-Newton methods for this problem either do not work well in the distributed setting or work only for specif…

New methods help escape strict saddle points in nonsmooth optimization.

Faster algorithms for solving multichain MDPs under average-reward criterion.

In this paper, we consider solving a class of nonconvex and nonsmooth problems frequently appearing in signal processing and machine learning research. The traditional alternating direction method of multipliers encounters troubles in both mathematics and computations in solving the nonconvex and nonsmooth subproblem. …

Rotates MFVI for better Gaussian approximations.

In this paper we consider the problem of minimizing a convex function using a randomized block coordinate descent method. One of the key steps at each iteration of the algorithm is determining the update to a block of variables. Existing algorithms assume that in order to compute the update, a particular subproblem is …

This work tackles sparse coding in DLRA for interpretable multiway data.

Paper introduces a novel matrix-wise sparse MNNLS formulation and algorithm.

Efficient algorithm solves sparse nonconvex regression problems.

Efficiently representing real world data in a succinct and parsimonious manner is of central importance in many fields. We present a generalized greedy pursuit framework, allowing us to efficiently solve structured matrix factorization problems, where the factors are allowed to be from arbitrary sets of structured vect…

CD methods tackle nonconvex optimization with three terms, achieving critical points.

Two multifidelity trust-region methods use low-fidelity models for efficient optimization.

EnsemFDet detects fraud by solving subproblems on small graphs, scaling up e-commerce fraud detection.

We propose a novel general algorithm LHAC that efficiently uses second-order information to train a class of large-scale l1-regularized problems. Our method executes cheap iterations while achieving fast local convergence rate by exploiting the special structure of a low-rank matrix, constructed via quasi-Newton approx…

This work uses Lasry-Lions envelopes to solve nonconvex optimization problems.

A new method for aligning multiple distributions efficiently.

In regularized risk minimization, the associated optimization problem becomes particularly difficult when both the loss and regularizer are nonsmooth. Existing approaches either have slow or unclear convergence properties, are restricted to limited problem subclasses, or require careful setting of a smoothing parameter…

We propose a fast proximal Newton-type algorithm for minimizing regularized finite sums that returns an $ε$-suboptimal point in $\tilde{\mathcal{O}}(d(n + \sqrt{κd})\log(\frac{1}ε))$ FLOPS, where $n$ is number of samples, $d$ is feature dimension, and $κ$ is the condition number. As long as $n > d$, the proposed method…

Introduces PPMM algorithm for nonconvex robust regression problems.

In this paper we study general Schatten-$p$ quasi-norm (SPQN) regularized matrix minimization problems. In particular, we first introduce a class of first-order stationary points for them, and show that the first-order stationary points introduced in [11] for an SPQN regularized $vector$ minimization problem are equiva…

This paper considers the problem of estimating multiple related Gaussian graphical models from a $p$-dimensional dataset consisting of different classes. Our work is based upon the formulation of this problem as group graphical lasso. This paper proposes a novel hybrid covariance thresholding algorithm that can effecti…

New SMC algorithm reduces memory usage for online clustering.

Proposes a new algorithm for solving optimization problems with stochastic objectives and equality constraints.

Extends branch and bound for probabilistic neural network verification.

Stochastic approach improves neural network training for kinetic simulations.