arXiv research

Iterated Amplification uses subproblem solutions to build training signals for complex tasks.

A new method reduces the cost of solving large-scale linear models.

By reducing optimization to a sequence of smaller subproblems, working set algorithms achieve fast convergence times for many machine learning problems. Despite such performance, working set implementations often resort to heuristics to determine subproblem size, makeup, and stopping criteria. We propose BlitzWS, a wor…

Two methods extend multivariate Kelly optimization to large problem sizes.

New method for sparse kernel selection improves prediction accuracy.

Exact solver speeds up Weston-Watkins SVM subproblem significantly.

Regularized online learning is widely used in machine learning applications. In online learning, performing exact minimization ($i.e.,$ implicit update) is known to be beneficial to the numerical stability and structure of solution. In this paper we study a class of regularized online algorithms without linearizing the…

New method solves optimization problems faster than existing methods.

Pareto MTL finds optimal solutions for multiple tasks with different trade-offs.

In this paper we study general Schatten-$p$ quasi-norm (SPQN) regularized matrix minimization problems. In particular, we first introduce a class of first-order stationary points for them, and show that the first-order stationary points introduced in [11] for an SPQN regularized $vector$ minimization problem are equiva…

A new algorithm for solving constrained convex optimization problems efficiently.

Paper presents a method to solve variational inequalities with general constraints without requiring analytic solutions.

New Frank-Wolfe algorithm speeds up SVM-type multi-category learning.

We introduce a few variants on Frank-Wolfe style algorithms suitable for large scale optimization. We show how to modify the standard Frank-Wolfe algorithm using stochastic gradients, approximate subproblem solutions, and sketched decision variables in order to scale to enormous problems while preserving (up to constan…

Faster algorithms for solving multichain MDPs under average-reward criterion.

MOBO-OSD optimizes multi-objective functions using orthogonal search directions.

A new framework RTK accelerates diffusion inference by breaking down the process into fewer, more efficient subproblems.

In this paper we consider sparse approximation problems, that is, general $l_0$ minimization problems with the $l_0$-"norm" of a vector being a part of constraints or objective function. In particular, we first study the first-order optimality conditions for these problems. We then propose penalty decomposition (PD) me…

Pipeline decomposes portfolio optimization problems into smaller, solvable subproblems.

New approach uses dynamic programming to efficiently discover failures in autonomous vehicle simulations.

In the context of sparse recovery, it is known that most of existing regularizers such as $\ell_1$ suffer from some bias incurred by some leading entries (in magnitude) of the associated vector. To neutralize this bias, we propose a class of models with partial regularizers for recovering a sparse solution of a linear …

SUSTAIN algorithm tackles stochastic bilevel optimization with near-optimal complexity.

We describe a new technique for computing lower-bounds on the minimum energy configuration of a planar Markov Random Field (MRF). Our method successively adds large numbers of constraints and enforces consistency over binary projections of the original problem state space. These constraints are represented in terms of …

Generalized canonical correlation analysis (GCCA) aims at finding latent low-dimensional common structure from multiple views (feature vectors in different domains) of the same entities. Unlike principal component analysis (PCA) that handles a single view, (G)CCA is able to integrate information from different feature …

Paper introduces a novel matrix-wise sparse MNNLS formulation and algorithm.

A new method solves complex constrained minimax problems.

Proposes a new algorithm for solving optimization problems with stochastic objectives and equality constraints.

The problem of classification of Legendrian knots (links) up to isotopy in the class of Legendrian embeddings (Legendrian isotopy) naturally leads to the following two subproblems. The first of them is: which combinations of the three classical invariants can be realized by a Legendrian knot? (It is well-known that eac…

A new method solves a complex optimization problem efficiently.

In this paper, we study an optimal excess-of-loss reinsurance and investment problem for an insurer in defaultable market. The insurer can buy reinsurance and invest in the following securities: a bank account, a risky asset with stochastic volatility and a defaultable corporate bond. We discuss the optimal investment …

Gaussian graphical models are of great interest in statistical learning. Because the conditional independencies between different nodes correspond to zero entries in the inverse covariance matrix of the Gaussian distribution, one can learn the structure of the graph by estimating a sparse inverse covariance matrix from…

In this paper we consider general rank minimization problems with rank appearing in either objective function or constraint. We first establish that a class of special rank minimization problems has closed-form solutions. Using this result, we then propose penalty decomposition methods for general rank minimization pro…

A new algorithm solves the metric nearness problem efficiently.

We propose a new sparse regression method called the component lasso, based on a simple idea. The method uses the connected-components structure of the sample covariance matrix to split the problem into smaller ones. It then solves the subproblems separately, obtaining a coefficient vector for each one. Then, it uses n…

New algorithm solves phase retrieval with adaptive stopping criteria.

Paper proposes algorithms for BMF using integer programming.

Paper solves MV portfolio selection in jump-diffusion models with no-shorting constraint.

In this paper, we consider solving a class of nonconvex and nonsmooth problems frequently appearing in signal processing and machine learning research. The traditional alternating direction method of multipliers encounters troubles in both mathematics and computations in solving the nonconvex and nonsmooth subproblem. …

In this paper we consider the problem of minimizing a convex function using a randomized block coordinate descent method. One of the key steps at each iteration of the algorithm is determining the update to a block of variables. Existing algorithms assume that in order to compute the update, a particular subproblem is …

Efficient algorithm solves sparse nonconvex regression problems.

Two multifidelity trust-region methods use low-fidelity models for efficient optimization.

EnsemFDet detects fraud by solving subproblems on small graphs, scaling up e-commerce fraud detection.

Efficient algorithms solve joint graphical lasso problems.

In regularized risk minimization, the associated optimization problem becomes particularly difficult when both the loss and regularizer are nonsmooth. Existing approaches either have slow or unclear convergence properties, are restricted to limited problem subclasses, or require careful setting of a smoothing parameter…

Paper tackles bilevel optimization problems using penalty methods.

We propose a fast proximal Newton-type algorithm for minimizing regularized finite sums that returns an $ε$-suboptimal point in $\tilde{\mathcal{O}}(d(n + \sqrt{κd})\log(\frac{1}ε))$ FLOPS, where $n$ is number of samples, $d$ is feature dimension, and $κ$ is the condition number. As long as $n > d$, the proposed method…

Introduces PPMM algorithm for nonconvex robust regression problems.

This paper considers the problem of estimating multiple related Gaussian graphical models from a $p$-dimensional dataset consisting of different classes. Our work is based upon the formulation of this problem as group graphical lasso. This paper proposes a novel hybrid covariance thresholding algorithm that can effecti…