Paper relaxes optimal transport using convex functions for data science.
problem Optimal transport problem on finite spaces.
method Relaxation via strictly convex functions (Kullback-Leibler divergence, Bregman divergences). Gradient descent iterative process.
result Mathematical foundations and iterative process for the relaxed optimal transport problem.
New method relaxes optimization problems to find solutions more reliably.
problem Optimizing functions with stochastic or non-differentiable elements.
method Using measure theory and Fourier analysis to impose structure on optimization problems.
result Consistency of optimal values, Lipschitzness of gradients, and convexity are key traits for fast and reliable optimization.
New conditions prevent gaps in optimal control problems.
problem Preventing gaps in optimal control problems with state constraints.
method Developed new sufficient conditions not relying on convexity.
result Derived bounds for the size of the relaxation gap.
Many high dimensional sparse learning problems are formulated as nonconvex optimization. A popular approach to solve these nonconvex optimization problems is through convex relaxations such as linear and semidefinite programming. In this paper, we study the statistical limits of convex relaxations. Particularly, we con…
We propose an SDP relaxation for the Gromov-Wasserstein distance, providing globally optimal solutions.
problem Matching objects between incomparable spaces using the Gromov-Wasserstein distance.
method Semi-definite programming (SDP) relaxation of the GW distance.
result The SDP relaxation provides globally optimal solutions for the GW distance in some instances.
We relax indicator matrices to form a manifold for faster optimization.
problem Optimizing indicator matrices is NP-hard.
method Developed a Riemannian manifold (RIM) and Riemannian optimization methods.
result RIM manifold optimization is significantly faster and yields better results.
Finding efficient and provable methods to solve non-convex optimization problems is an outstanding challenge in machine learning and optimization theory. A popular approach used to tackle non-convex problems is to use convex relaxation techniques to find a convex surrogate for the problem. Unfortunately, convex relaxat…
Two-stage nonconvex algorithm and convex relaxation both achieve optimal accuracy in noisy blind deconvolution.
problem Solving bilinear systems of equations with random noise under different designs.
method Two-stage nonconvex algorithm and convex relaxation.
result Both methods achieve minimax-optimal accuracy in the presence of random noise.
Improved neural network robustness certification through tighter convex relaxations.
problem Certifying neural network robustness to perturbed and adversarial inputs.
method Exploiting ReLU network structure, novel partition-based certification procedure.
result Tightens existing linear programming relaxations to achieve zero relaxation error asymptotically.
NeuralSort optimizes sorting networks using continuous relaxations.
problem Non-differentiability of sorting operator hinders gradient-based optimization.
method Continuous relaxation of sorting operator to unimodal row-stochastic matrices, enabling gradient-based optimization.
result Gradient-based stochastic optimization over permutations is achieved.
Variable selection is a fundamental task in statistical data analysis. Sparsity-inducing regularization methods are a popular class of methods that simultaneously perform variable selection and model estimation. The central problem is a quadratic optimization problem with an l0-norm penalty. Exactly enforcing the l0-no…
New method improves neural network verification by considering multivariate input space of ReLU neurons.
problem Improving the effectiveness of neural network verification algorithms.
method A new tightened convex relaxation for ReLU neurons considering multivariate input space.
result Our convex relaxation is significantly stronger than the commonly used univariate-input relaxation.
Proposes a continuous relaxation for discrete Bayesian optimization.
problem Efficiently optimizing discrete data with limited target observations.
method Continuous relaxation of objective function, incorporating prior knowledge.
result Optimization can be computationally tractable with few observations.
A novel method relaxes binary constraints to non-negative spheres for multi-matching and clustering.
problem Optimization problems over binary matrices with injectivity constraints.
method Non-negative spherical relaxation followed by conditional power iteration.
result Automatic adjustment of the continuous parameter related to universe size.
Improves scalability of Bayesian optimization for combinatorial spaces.
problem Optimizing expensive functions over large combinatorial spaces.
method Parametrized Submodular Relaxation (PSR) to solve AFO problems for BOCS.
result Significant improvements in scalability and accuracy for BOCS model.
The (constrained) minimization of a ratio of set functions is a problem frequently occurring in clustering and community detection. As these optimization problems are typically NP-hard, one uses convex or spectral relaxations in practice. While these relaxations can be solved globally optimally, they are often too loos…
Paper uses relaxation techniques to find optimal brokerage fees with private signals.
problem Finding optimal brokerage fees for clients with private trading signals.
method Relaxation techniques to establish contract existence in asymmetric information settings.
result Existence of optimal brokerage fees established in a market model with private signals.
Sparse principal component analysis (PCA) involves nonconvex optimization for which the global solution is hard to obtain. To address this issue, one popular approach is convex relaxation. However, such an approach may produce suboptimal estimators due to the relaxation effect. To optimally estimate sparse principal su…
Paper improves understanding of noisy matrix completion using convex relaxation and nonconvex optimization.
problem Estimating a low-rank matrix from noisy partial entries.
method Combining convex relaxation and the nonconvex Burer-Monteiro approach.
result Convex relaxation achieves near-optimal estimation errors for noisy matrix completion.
New method compresses graphs using optimal transport and improves classification.
problem Efficiently compressing graphs while preserving classification accuracy.
method Optimal transport seeded with prior information, Boolean relaxations for exact solutions.
result Exact O(dlogd) algorithm for graph projection, outperforms state-of-the-art methods. In this note we compare two recently proposed semidefinite relaxations for the sparse linear regression problem by Pilanci, Wainwright and El Ghaoui (Sparse learning via boolean relaxations, 2015) and Dong, Chen and Linderoth (Relaxation vs. Regularization A conic optimization perspective of statistical variable select…
We study singular stochastic control of a two dimensional stochastic differential equation, where the first component is linear with random and unbounded coefficients. We derive existence of an optimal relaxed control and necessary conditions for optimality in the form of a mixed relaxed-singular maximum principle in a…
Study uses DRL with Lagrangian relaxation to solve temporal control tasks with STL constraints.
problem Optimal control problems with temporal logic constraints.
method Extended CMDP formulation, Lagrangian relaxation, two-phase constrained DRL algorithm.
result Demonstrated learning performance of the proposed algorithm through simulations.
Extends optimal transport to dynamic and martingale settings.
problem Dynamic and martingale relaxation of optimal transport problems.
method Extends Benamou-Brenier formula to weak optimal transport and introduces barycentric optimal transport.
result Relates barycentric optimal transport to martingale Benamou-Brenier formula.
The paper develops sum-of-squares relaxations for computing f-divergences.
problem Computing f-divergences from non-centered covariance matrices. method Sum-of-squares relaxations for convex optimization.
result Sum-of-squares relaxations make computations tractable.
We optimize saddle-point problems for large-scale Markov decision processes.
problem Optimizing policies in large-scale Markov decision processes.
method Characterized conditions for convergence and designed an optimization algorithm.
result Our algorithm converges faster and is state-space independent.
Paper proposes Adaptive Pareto Exploration for identifying Pareto optimal arms in multi-objective scenarios.
problem Identifying Pareto optimal arms in multi-objective scenarios with relaxed constraints.
method Adaptive Pareto Exploration strategy for different relaxations of Pareto Set Identification.
result Reduction in sample complexity when identifying at most k Pareto optimal arms.
New method closes certification gap for adversarially trained models.
problem Certifying robustness of adversarially trained neural networks.
method Nonconvex low-rank SDP relaxation with polynomial-time optimization.
result Strong certifications comparable to SDP methods, but with fewer variables.
New approach to convex hulls for low-rank problems.
problem Characterizing convex hulls for low-rank sets.
method Matrix perspective function and orthogonal projection matrices.
result Strong relaxations for various low-rank problems.
In this paper, we study a nonconvex continuous relaxation of MAP inference in discrete Markov random fields (MRFs). We show that for arbitrary MRFs, this relaxation is tight, and a discrete stationary point of it can be easily reached by a simple block coordinate descent algorithm. In addition, we study the resolution …
Proposes a faster second-order method for MDPs.
problem Slow convergence of first-order value iteration methods in MDPs.
method Applies Newton-Raphson method to successive relaxation value iteration scheme.
result Second-order convergence and faster convergence to optimal solution.
New convex method solves densest subgraph problem with high probability.
problem Finding the densest subgraph in a graph.
method Convex relaxation of low-rank plus sparse decomposition of adjacency matrices.
result Optimal solution of convex relaxation recovers densest subgraph with high probability.
A new method learns fair classifiers without sacrificing accuracy.
problem Designing fair classifiers that do not discriminate based on sensitive attributes.
method A model-agnostic multi-objective architecture using a differentiable relaxation of fairness notions.
result Our method achieves lower loss of accuracy compared to current debiasing algorithms.
Study optimal consumption with relaxed benchmarks and drawdown constraints.
problem Optimal consumption under relaxed benchmark tracking and consumption drawdown constraint.
method Transformed stochastic control problem into regular control problem with state-control constraints, then solved using dual transform and optimal consumption behavior.
result Closed-form solution for optimal investment and consumption in feedback form.
Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.
problem Exploration-exploitation dilemma in linear quadratic regulator (LQR) setting.
method Relax optimistic optimization into a constrained extended LQR problem, then solve using Riccati equations.
result Computes ε-optimistic controller efficiently with O(log(1/ε)) Riccati equations. Proposes SOR Q-learning for faster optimal value function computation in RL.
problem Finding optimal value function in Markov Decision Processes (MDPs).
method Successive Over-Relaxation (SOR) applied to Q-learning algorithm.
result SOR Q-learning converges faster to optimal value function compared to standard Q-learning.
A new GP interpolation method for better predictive distributions in ranges of interest.
problem Improving predictive distributions in specific ranges of interest.
method Relaxed Gaussian process interpolation, relaxing interpolation constraints outside ranges of interest.
result Better predictive distributions in ranges of interest, especially in non-stationary cases.
SAM improves deep learning by relaxing Bayes objective.
problem Improving generalization in deep learning models.
method SAM as a relaxation of Bayes objective using Fenchel biconjugate.
result SAM connects adversarial and Bayesian methods for robustness.
Optimal transport relaxations improve Wasserstein GAN training efficiency.
problem Training inefficiency in Wasserstein GANs.
method Optimal transport relaxations with a minimization step over a small region.
result Running time improvements with no performance degradation.
New algorithm solves large cardinality-constrained clustering problems.
problem Optimizing clustering with cardinality constraints.
method Branch-and-cut technique with SDP relaxation and polyhedral cuts.
result Solves real-world instances 10 times larger than previous methods.
CRA improves UL-based CO solvers by dynamically smoothing and enforcing discreteness.
problem Local optima and artificial rounding issues in UL-based CO solvers.
method Continuous Relaxation Annealing (CRA) strategy that dynamically shifts from continuous to discrete solutions.
result Significantly enhances UL-based CO solver performance and eliminates artificial rounding.
Statistical image reconstruction (SIR) methods are studied extensively for X-ray computed tomography (CT) due to the potential of acquiring CT scans with reduced X-ray dose while maintaining image quality. However, the longer reconstruction time of SIR methods hinders their use in X-ray CT in practice. To accelerate st…
A new method for categorical variational inference using discrete normalizing flows.
problem Challenges in optimizing variational approximations for discrete latent variables.
method Differentiable reparameterization using a mixture of discrete normalizing flows.
result Improves optimization of evidence lower bound and reduces sensitivity to hyperparameters.
Paper analyzes Birkhoff relaxation for graph alignment, providing theoretical guarantees.
problem Finding vertex correspondence between two graphs to maximize edge overlap.
method Birkhoff relaxation as a convex relaxation of the quadratic assignment problem (QAP).
result Theoretical guarantees on the performance of Birkhoff relaxation under specific conditions.
In this paper, we propose an unifying view of several recently proposed structured sparsity-inducing norms. We consider the situation of a model simultaneously (a) penalized by a set- function de ned on the support of the unknown parameter vector which represents prior knowledge on supports, and (b) regularized in Lp-n…
We suggest using the max-norm as a convex surrogate constraint for clustering. We show how this yields a better exact cluster recovery guarantee than previously suggested nuclear-norm relaxation, and study the effectiveness of our method, and other related convex relaxations, compared to other clustering approaches.
Spectral Clustering as a relaxation of the normalized/ratio cut has become one of the standard graph-based clustering methods. Existing methods for the computation of multiple clusters, corresponding to a balanced k-cut of the graph, are either based on greedy techniques or heuristics which have weak connection to th…
New approach for fair graph clustering using semidefinite relaxation.
problem Ensuring equitable representation in network analysis.
method Semidefinite relaxation approach for NP-hard optimization problem.
result Optimal accuracy-fairness trade-off achieved.