arXiv research

Paper introduces techniques to learn higher-order programs, improving predictive accuracy and reducing learning times.

CEFOL uses deep learning for dynamic programming with recursive utility.

Hamiltonian Monte Carlo (HMC) is arguably the dominant statistical inference algorithm used in most popular "first-order differentiable" Probabilistic Programming Languages (PPLs). However, the fact that HMC uses derivative information causes complications when the target distribution is non-differentiable with respect…

This paper improves online learning algorithms for LP problems, achieving better regret bounds.

New algorithm reduces online decision-making regret with efficient LP re-solving and parallel first-order method.

Method solves complex optimization problems with high probability bounds.

Defines a calculus for integrating Moreau envelopes in differentiable programming.

In this paper we study a broad class of structured nonlinear programming (SNLP) problems. In particular, we first establish the first-order optimality conditions for them. Then we propose sequential convex programming (SCP) methods for solving them in which each iteration is obtained by solving a convex programming pro…

Deep learning solves dynamic programming with recursive utility.

Study non-Abelian gauge theories using Poisson bracket structures.

We develop a new Low-level, First-order Probabilistic Programming Language (LF-PPL) suited for models containing a mix of continuous, discrete, and/or piecewise-continuous variables. The key success of this language and its compilation scheme is in its ability to automatically distinguish parameters the density functio…

New algorithms optimize without knowing problem parameters.

Study General Relativity using field theories and Poisson brackets.

Paper develops a TR-SSQP method for noisy optimization with heavy-tailed noise.

Motivated by big data applications, first-order methods have been extremely popular in recent years. However, naive gradient methods generally converge slowly. Hence, much efforts have been made to accelerate various first-order methods. This paper proposes two accelerated methods towards solving structured linearly co…

This book is a graduate-level introduction to probabilistic programming. It not only provides a thorough background for anyone wishing to use a probabilistic programming system, but also introduces the techniques needed to design and build these systems. It is aimed at people who have an undergraduate-level understandi…

Deep learning methods capable of handling relational data have proliferated over the last years. In contrast to traditional relational learning methods that leverage first-order logic for representing such data, these deep learning methods aim at re-representing symbolic relational data in Euclidean spaces. They offer …

New framework tackles bi-level optimization without LLS condition.

New metric derived for robust optimization in stochastic control problems.

Nonnegative matrix factorization (NMF) has been shown to be identifiable under the separability assumption, under which all the columns(or rows) of the input data matrix belong to the convex cone generated by only a few of these columns(or rows) [1]. In real applications, however, such separability assumption is hard t…

New method solves optimization problems with stochastic objectives and constraints.

Optimal Control Theory optimizes neural networks, improving robustness and efficiency.

Paper models graph edge dependencies using latent variables for community detection.

Unified model learns concepts across domains like left and right.

We consider discriminative dictionary learning in a distributed online setting, where a network of agents aims to learn a common set of dictionary elements of a feature space and model parameters while sequentially receiving observations. We formulate this problem as a distributed stochastic program with a non-convex o…

New method solves complex constrained optimization problems.

A biclustering algorithm finds dense disjoint subgraphs in weighted bipartite graphs.

Paper proposes a novel RRL framework that learns from images and incorporates expert knowledge.

A new method solves diagonally constrained SDPs quickly and accurately.

In this paper, a direct continuation of math.DG/0411165, we generalize S. Lie's linearization criterion of an ordinary second order differential equation to the case of several independent variables (x^1, x^2 ..., x^n), n >1, and a single dependent variable y. Strikingly, as in math.DG/0411165, the (complicated) charac…

Paper proposes a method to efficiently cluster stretched mixtures.

In this paper we study a continuous time, optimal stochastic investment problem under limited resources in a market with N firms. The investment processes are subject to a time-dependent stochastic constraint. Rather than using a dynamic programming approach, we exploit the concavity of the profit functional to derive …

Paper develops zeroth and first order stochastic Frank-Wolfe algorithms for constrained optimization.

Optimal transportation, or computing the Wasserstein or ``earth mover's'' distance between two distributions, is a fundamental primitive which arises in many learning and statistical settings. We give an algorithm which solves this problem to additive $ε$ with $\tilde{O}(1/ε)$ parallel depth, and $\tilde{O}\left(n^2/ε\…

New algorithms reduce complexity for solving nonconvex optimization problems with stochastic objectives and constraints.

Maximizes determinant of vector sums under matroid constraints.

New MIP framework solves high-dimensional $\ell_0\ell_2$-regularized regression problems.

New method solves stochastic optimization problems with random models.

We revisit the optimal investment and consumption model of Davis and Norman (1990) and Shreve and Soner (1994), following a shadow-price approach similar to that of Kallsen and Muhle-Karbe (2010). Making use of the completeness of the model without transaction costs, we reformulate and reduce the Hamilton-Jacobi-Bellma…

The L1-regularized Gaussian maximum likelihood estimator (MLE) has been shown to have strong statistical guarantees in recovering a sparse inverse covariance matrix, or alternatively the underlying graph structure of a Gaussian Markov Random Field, from very limited samples. We propose a novel algorithm for solving the…

Given the superposition of a low-rank matrix plus the product of a known fat compression matrix times a sparse matrix, the goal of this paper is to establish deterministic conditions under which exact recovery of the low-rank and sparse components becomes possible. This fundamental identifiability issue arises with tra…

New scalable methods for robust model learning from large datasets.

A new method solves variational inequality problems with multiple constraints without needing optimal Lagrange multipliers.

Proposes a new algorithm to improve Frank-Wolfe method efficiency.

First, we review the Dirac operator folklore about basic analytic and geometrical properties of operators of Dirac type on compact manifolds with smooth boundary and on closed partitioned manifolds and show how these properties depend on the construction of a canonical invertible double and are related to the concept o…

We consider the problem of covariance matrix estimation in the presence of latent variables. Under suitable conditions, it is possible to learn the marginal covariance matrix of the observed variables via a tractable convex program, where the concentration matrix of the observed variables is decomposed into a sparse ma…

Examines how first-order differential operators can be equivalently transformed.

New vector fields integrate first-order ODEs.