arXiv research

New method improves neural network verification by considering multivariate input space of ReLU neurons.

Statistical image reconstruction (SIR) methods are studied extensively for X-ray computed tomography (CT) due to the potential of acquiring CT scans with reduced X-ray dose while maintaining image quality. However, the longer reconstruction time of SIR methods hinders their use in X-ray CT in practice. To accelerate st…

Label assignment problems with large state spaces are important tasks especially in computer vision. Often the pairwise interaction (or smoothness prior) between labels assigned at adjacent nodes (or pixels) can be described as a function of the label difference. Exact inference in such labeling tasks is still difficul…

Differentiable relaxation for inferring partial orders from noisy linear data.

We propose convex relaxations for convolutional neural nets with one hidden layer where the output weights are fixed. For convex activation functions such as rectified linear units, the relaxations are convex second order cone programs which can be solved very efficiently. We prove that the relaxation recovers the glob…

Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.

This work analyzes machine learning for Lagrangian Relaxation in MILP.

Paper relaxes optimal transport using convex functions for data science.

In this note we compare two recently proposed semidefinite relaxations for the sparse linear regression problem by Pilanci, Wainwright and El Ghaoui (Sparse learning via boolean relaxations, 2015) and Dong, Chen and Linderoth (Relaxation vs. Regularization A conic optimization perspective of statistical variable select…

New method controls linear systems with adversarial disturbances.

We study singular stochastic control of a two dimensional stochastic differential equation, where the first component is linear with random and unbounded coefficients. We derive existence of an optimal relaxed control and necessary conditions for optimality in the form of a mixed relaxed-singular maximum principle in a…

Many high dimensional sparse learning problems are formulated as nonconvex optimization. A popular approach to solve these nonconvex optimization problems is through convex relaxations such as linear and semidefinite programming. In this paper, we study the statistical limits of convex relaxations. Particularly, we con…

Paper solves NP-hard sparse mixed linear regression problem with provable guarantees.

MAP inference for general energy functions remains a challenging problem. While most efforts are channeled towards improving the linear programming (LP) based relaxation, this work is motivated by the quadratic programming (QP) relaxation. We propose a novel MAP relaxation that penalizes the Kullback-Leibler divergence…

Training neural networks with verifiable robustness guarantees is challenging. Several existing approaches utilize linear relaxation based neural network output bounds under perturbation, but they can slow down training by a factor of hundreds depending on the underlying network architectures. Meanwhile, interval bound…

Structured prediction is used in areas such as computer vision and natural language processing to predict structured outputs such as segmentations or parse trees. In these settings, prediction is performed by MAP inference or, equivalently, by solving an integer linear program. Because of the complex scoring functions …

Improved neural network robustness certification through tighter convex relaxations.

Paper revisits set membership estimation for linear systems with relaxed disturbance bounds.

The relaxed maximum entropy problem is concerned with finding a probability distribution on a finite set that minimizes the relative entropy to a given prior distribution, while satisfying relaxed max-norm constraints with respect to a third observed multinomial distribution. We study the entire relaxation path for thi…

We study the problem of controlling linear time-invariant systems with known noisy dynamics and adversarially chosen quadratic losses. We present the first efficient online learning algorithms in this setting that guarantee $O(\sqrt{T})$ regret under mild assumptions, where $T$ is the time horizon. Our algorithms rely …

This work interprets SFA through variational inference, relaxing linearity constraints.

Renet improves Elastic Net by dynamically selecting between convex blending and refitting, enhancing prediction accuracy.

PEREGRiNN verifies safety of ReLU NNs by penalizing relaxation in a greedy manner.

A new method for efficient causal structure learning at scale.

New method closes certification gap for adversarially trained models.

We consider the problem of computing upper and lower bounds on the price of a European basket call option, given prices on other similar baskets. Although this problem is very hard to solve exactly in the general case, we show that in some instances the upper and lower bounds can be computed via simple closed-form expr…

New conic quadratic formulations improve outlier detection in regression models.

This paper develops an ensemble learning-based linearization approach for power flow, which differs from the network-parameter based direct current (DC) power flow or other extended versions of linearization. As a novel data-driven linearization through data mining, it firstly applies the polynomial regression (PR) as …

We propose an SDP relaxation for the Gromov-Wasserstein distance, providing globally optimal solutions.

Consider a dataset of vector-valued observations that consists of noisy inliers, which are explained well by a low-dimensional subspace, along with some number of outliers. This work describes a convex optimization problem, called REAPER, that can reliably fit a low-dimensional model to this type of data. This approach…

We show that the spectral norm of a random $n_1\times n_2\times \cdots \times n_K$ tensor (or higher-order array) scales as $O\left(\sqrt{(\sum_{k=1}^{K}n_k)\log(K)}\right)$ under some sub-Gaussian assumption on the entries. The proof is based on a covering number argument. Since the spectral norm is dual to the tensor…

New conditions ensure Dantzig-Wolfe relaxation matches rank-constrained optimization problems.

New method solves subspace optimization problems efficiently.

This work optimizes RL algorithms using entropy regularisation for continuous-time LQ problems.

We consider the problem of computing optimal policies in average-reward Markov decision processes. This classical problem can be formulated as a linear program directly amenable to saddle-point optimization methods, albeit with a number of variables that is linear in the number of states. To address this issue, recent …

Bell's Theorem shows that quantum mechanical correlations can violate the constraints that the causal structure of certain experiments impose on any classical explanation. It is thus natural to ask to which degree the causal assumptions -- e.g. locality or measurement independence -- have to be relaxed in order to allo…

Plain vanilla K-means clustering has proven to be successful in practice, yet it suffers from outlier sensitivity and may produce highly unbalanced clusters. To mitigate both shortcomings, we formulate a joint outlier detection and clustering problem, which assigns a prescribed number of datapoints to an auxiliary outl…

Near isometric orthogonal embeddings to lower dimensions are a fundamental tool in data science and machine learning. In this paper, we present the construction of such embeddings that minimizes the maximum distortion for a given set of points. We formulate the problem as a non convex constrained optimization problem. …

Study bounds financial path expectations using martingale distributions.

New methods optimize sums of bivariate functions on finite domains.

We develop a convex relaxation method for analyzing neural network generalization.

We give a polynomial-time algorithm for learning latent-state linear dynamical systems without system identification, and without assumptions on the spectral radius of the system's transition matrix. The algorithm extends the recently introduced technique of spectral filtering, previously applied only to systems with a…

New MIP approach for efficient change-point detection.

The importance of accurate recommender systems has been widely recognized by academia and industry. However, the recommendation quality is still rather low. Recently, a linear sparse and low-rank representation of the user-item matrix has been applied to produce Top-N recommendations. This approach uses the nuclear nor…

Quantized Stochastic Primal-Dual Methods for Distributed Optimization

Gradient descent converges linearly for overparameterized linear networks.

Convolutional neural networks are among the most successful architectures in deep learning with this success at least partially attributable to the efficacy of spatial invariance as an inductive bias. Locally connected layers, which differ from convolutional layers only in their lack of spatial invariance, usually perf…

Efficient algorithm predicts unknown linear systems with long-term memory.