arXiv research

Paper proposes algorithms to minimize both dynamic and adaptive regret simultaneously.

Sharpness minimization algorithms don't solely improve generalization.

New algorithms improve submodular minimization via DC programming.

We develop a family of accelerated stochastic algorithms that minimize sums of convex functions. Our algorithms improve upon the fastest running time for empirical risk minimization (ERM), and in particular linear least-squares regression, across a wide range of problem settings. To achieve this, we establish a framewo…

Paper analyzes time series prediction using empirical risk minimization.

Survey of universal portfolio techniques for minimizing investment regret.

Introduces PPMM algorithm for nonconvex robust regression problems.

Minimal genus surfaces solve homology problems in finite complexes.

New learning algorithm for real analytic functions without gradient descent.

This paper begins with a study on the dual representations of risk and regret measures and their impact on modeling multistage decision making under uncertainty. A relationship between risk envelopes and regret envelopes is established by using the Lagrangian duality theory. Such a relationship opens a door to a decomp…

We solve minimal separator problems in AMP chain graphs and improve structure learning algorithms.

We decrease the $rms$ mean curvature and area of a variable surface with a fixed boundary by iterating a few times through a curvature-based variational algorithm. For a boundary with a known minimal surface, starting with a deliberately chosen non-minimal surface, we achieve up to 65 percent of the total possible decr…

New algorithm for online convex minimization over integer lattice.

Accelerates machine learning algorithms for sparse data.

Algorithm minimizes risk for multiclass classification of stochastic diffusion paths.

New random forest algorithms for PU learning minimize risk directly.

New algorithm for nonconvex optimization on constrained Riemannian manifolds converges quickly.

New algorithm learns FMDP structure while minimizing regret.

We extend the work of Narasimhan and Bilmes [30] for minimizing set functions representable as a dierence between submodular functions. Similar to [30], our new algorithms are guaranteed to monotonically reduce the objective function at every step. We empirically and theoretically show that the per-iteration cost of ou…

New algorithms minimize regret with global costs in online learning.

Many machine learning algorithms minimize a regularized risk, and stochastic optimization is widely used for this task. When working with massive data, it is desirable to perform stochastic optimization in parallel. Unfortunately, many existing stochastic optimization algorithms cannot be parallelized efficiently. In t…

Unbiased methods for alpha-divergence minimization struggle in high dimensions.

Unified algorithm for minimizing composite functions with flexible design.

Given an orientable surface with boundary and a free homotopy class, we present a purely combinatorial algorithm which produces a representative of that homotopy class with minimal self intersection.

In many estimation problems, e.g. linear and logistic regression, we wish to minimize an unknown objective given only unbiased samples of the objective function. Furthermore, we aim to achieve this using as few samples as possible. In the absence of computational constraints, the minimizer of a sample average of observ…

The paper tackles efficient online learning by achieving minimal regret with respect to the best expert.

Optimizes bilevel empirical risk minimization with improved oracle calls.

New algorithm minimizes worst-case regret in uncertain, time-varying dynamics.

Algorithm minimizes regret and converges to equilibria in Markov games.

Matrix completion has attracted much interest in the past decade in machine learning and computer vision. For low-rank promotion in matrix completion, the nuclear norm penalty is convenient due to its convexity but has a bias problem. Recently, various algorithms using nonconvex penalties have been proposed, among whic…

Paper presents efficient algorithms for convolutional neural networks using Winograd minimal filtering.

New algorithms achieve uniform stability for empirical risk minimization.

New algorithms delete user data from machine learning models efficiently.

We study the problem of online learning with a notion of regret defined with respect to a set of strategies. We develop tools for analyzing the minimax rates and for deriving regret-minimization algorithms in this scenario. While the standard methods for minimizing the usual notion of regret fail, through our analysis …

New algorithms help machines forget old data efficiently.

New algorithm solves online resource allocation problems efficiently.

We consider online algorithms under both the competitive ratio criteria and the regret minimization one. Our main goal is to build a unified methodology that would be able to guarantee both criteria simultaneously. For a general class of online algorithms, namely any Metrical Task System (MTS), we show that one can sim…

Support vector machines (SVMs) are an important tool in modern data analysis. Traditionally, support vector machines have been fitted via quadratic programming, either using purpose-built or off-the-shelf algorithms. We present an alternative approach to SVM fitting via the majorization--minimization (MM) paradigm. Alg…

We present theoretical guarantees for an alternating minimization algorithm for the dictionary learning/sparse coding problem. The dictionary learning problem is to factorize vector samples $y^{1},y^{2},\ldots, y^{n}$ into an appropriate basis (dictionary) $A^*$ and sparse vectors $x^{1*},\ldots,x^{n*}$. Our algorithm …

Proposes a new method for kernel density estimation using stagewise minimization and a simple dictionary.

New algorithms minimize simple and cumulative regret in contextual bandits.

Submodular function minimization is well studied, and existing algorithms solve it exactly or up to arbitrary accuracy. However, in many applications, such as structured sparse learning or batch Bayesian optimization, the objective function is not exactly submodular, but close. In this case, no theoretical guarantees e…

Bayesian optimization (BO) aims to minimize a given blackbox function using a model that is updated whenever new evidence about the function becomes available. Here, we address the problem of BO under partially right-censored response data, where in some evaluations we only obtain a lower bound on the function value. T…

This paper tackles robust submodular minimization for image segmentation and correspondence.

In this work we consider the stochastic minimization of nonsmooth convex loss functions, a central problem in machine learning. We propose a novel algorithm called Accelerated Nonsmooth Stochastic Gradient Descent (ANSGD), which exploits the structure of common nonsmooth loss functions to achieve optimal convergence ra…

New method uses robust estimators for Newton's method in empirical risk minimization.

Algorithm finds minimal volume hyperbolic links in 3-manifolds.

Differential privacy is concerned about the prediction quality while measuring the privacy impact on individuals whose information is contained in the data. We consider differentially private risk minimization problems with regularizers that induce structured sparsity. These regularizers are known to be convex but they…