Paper studies PSGD for constrained optimization problems and its statistical properties.
problem Online inference for constrained optimization problems.
method Stochastic gradient descent with projection (PSGD) for constrained optimization.
result Limiting distribution of PSGD-based estimates under linear-equality constraints.
c-lasso is a Python tool for robust and sparse regression with linear constraints.
problem Sparse and robust linear regression with linear constraints.
method Estimates coefficients and scale under linear constraints using perspective M-estimators.
result Provides estimators for various loss functions with linear constraints.
We address the problem of solving convex optimization problems with many convex constraints in a distributed setting. Our approach is based on an extension of the alternating direction method of multipliers (ADMM) that recently gained a lot of attention in the Big Data context. Although it has been invented decades ago…
Eigen-decomposition simplifies quadratic programming with equality constraints.
problem Optimizing solutions under linear equality constraints in quadratic programming.
method Eigenvalue decomposition of the quadratic term matrix to project optimal solutions.
result Established a linear mapping between EQP formulations with and without diagonalized Q. Variable selection is one of the most important tasks in statistics and machine learning. To incorporate more prior information about the regression coefficients, the constrained Lasso model has been proposed in the literature. In this paper, we present an inexact augmented Lagrangian method to solve the Lasso problem …
We present a stochastic setting for optimization problems with nonsmooth convex separable objective functions over linear equality constraints. To solve such problems, we propose a stochastic Alternating Direction Method of Multipliers (ADMM) algorithm. Our algorithm applies to a more general class of nonsmooth convex …
Due to the lack of reliable market information, building financial term-structures may be associated with a significant degree of uncertainty. In this paper, we propose a new term-structure interpolation method that extends classical spline techniques by additionally allowing for quantification of uncertainty. The prop…
An important problem in fiber-optic communications is to invert the nonlinear Schrödinger equation in real time to reverse the deterministic effects of the channel. Interestingly, the popular split-step Fourier method (SSFM) leads to a computation graph that is reminiscent of a deep neural network. This observation all…
In this paper, we focus on solving an important class of nonconvex optimization problems which includes many problems for example signal processing over a networked multi-agent system and distributed learning over networks. Motivated by many applications in which the local objective function is the sum of smooth but po…
We propose a low-complexity sub-banded DSP architecture for digital backpropagation where the walk-off effect is compensated using simple delay elements. For a simulated 96-Gbaud signal and 2500 km optical link, our method achieves a 2.8 dB SNR improvement over linear equalization.
Support vector machines (SVMs) are successful modeling and prediction tools with a variety of applications. Previous work has demonstrated the superiority of the SVMs in dealing with the high dimensional, low sample size problems. However, the numerical difficulties of the SVMs will become severe with the increase of t…
Proposes a new algorithm for solving optimization problems with stochastic objectives and equality constraints.
problem Optimization problems with stochastic objectives and deterministic equality constraints.
method Trust-region stochastic sequential quadratic programming (TR-StoSQP) with adaptive relaxation techniques.
result Established a global almost sure convergence guarantee for TR-StoSQP.
Boosted Difference of Convex Functions Algorithm solves VaR constrained portfolio optimization.
problem Designing VaR optimal portfolios under financial regulations.
method Boosted Difference of Convex Functions Algorithm (BDCA) with a novel line search framework.
result BDCA linearly converges to a Karush-Kuhn-Tucker point for VaR constrained portfolio problems.
ODCGM solves non-convex optimization on manifolds with simpler projections.
problem Minimizing non-convex functions over smooth manifolds.
method Orthogonal Directions Constrained Gradient Method (ODCGM) that projects onto a vector space.
result ODCGM converges to the manifold with near-optimal oracle complexities.
We consider the problem of computing a Wasserstein barycenter for a set of discrete probability distributions with finite supports, which finds many applications in areas such as statistics, machine learning and image processing. When the support points of the barycenter are pre-specified, this problem can be modeled a…
Proposes PKM for soft K-means clustering.
problem Soft K-means (m=1) unsolved since 1981.
method Probabilistic K-Means (PKM) via nonlinear programming.
result Proposed methods solve PKM efficiently.
In this work, we consider one challenging training time attack by modifying training data with bounded perturbation, hoping to manipulate the behavior (both targeted or non-targeted) of any corresponding trained classifier during test time when facing clean samples. To achieve this, we proposed to use an auto-encoder-l…
A new algorithm tackles submodular bandit problems with multiple constraints.
problem Addressing diversified retrieval and online learning with budget constraints.
method Non-greedy algorithm focusing on upper-confidence bounds.
result High-probability upper bound of an approximation regret matching fast offline algorithm's ratio.
This work proposes an online learning approach to tighten constraints in stochastic control problems.
problem Solving chance-constrained stochastic optimal control problems is computationally challenging.
method Reformulate chance constraints as a binary regression problem and use a GP model to learn constraint-tightening parameters online.
result The approach tightens constraints more effectively, leading to lower costs in numerical experiments.
We study constrained clustering, where constraints guide the clustering process. In existing works, two categories of constraints have been widely explored, namely pairwise and cardinality constraints. Pairwise constraints enforce the cluster labels of two instances to be the same (must-link constraints) or different (…
Simplifies neural network constraints with computationally efficient method.
problem Implementing hard output constraints in neural networks.
method Additional neural network layer for output constraints.
result Computational simplicity with complexity O(n*m) for linear constraints.
Reduces Lie (bi-)algebroids and Dirac manifolds using constraint vector bundles.
problem Reduction of Lie (bi-)algebroids and Dirac manifolds.
method Introduces constraint manifolds and constraint vector bundles; proves constraint Serre-Swan theorem; introduces Cartan calculus for constraint forms and multivector fields; shows compatibility with reduction.
result Reduction procedure for Lie (bi-)algebroids and Dirac manifolds.
Optimistic algorithm reduces regret and constraint violations in online convex optimization with adversarial constraints.
problem Online convex optimization with adversarial constraints.
method Improved algorithm using accurate predictions of loss and constraint functions.
result Improved bounds on regret and cumulative constraint violations.
Holistic GLMs add constraints for better model quality.
problem Improving classical linear regression models.
method Sparsity-inducing, sign-coherence, and linear constraints.
result Holistic GLMs reliably solve GLMs for various responses.
A new ML method teaches constraints directly to models.
problem Addressing safety and fairness in AI systems.
method Directly teaching constraint satisfaction to ML models using a constraint solver.
result Empirically, our approach performs well on fairness and synthetic constraints.
Paper tackles constrained bandit problems with a new learning framework.
problem Optimizing a black-box reward function subject to a black-box constraint function over a continuous space.
method Rectified Pessimistic-Optimistic Learning (RPOL) framework, incorporating optimistic and pessimistic GP bandit learning.
result RPOL achieves sublinear regret and minimal cumulative constraint violation.
In the present paper, the minimal investment risk for a portfolio optimization problem with imposed budget and investment concentration constraints is considered using replica analysis. Since the minimal investment risk is influenced by the investment concentration constraint (as well as the budget constraint), it is i…
Survey of Gaussian process constraints for modeling expensive data.
problem Modeling expensive data with physical constraints.
method Overview of various Gaussian process constraints and their implementation.
result Discussion of computational challenges introduced by constraints.
This paper considers online convex optimization over a complicated constraint set, which typically consists of multiple functional constraints and a set constraint. The conventional online projection algorithm (Zinkevich, 2003) can be difficult to implement due to the potentially high computation complexity of the proj…
We provide a dynamic programming principle for stochastic optimal control problems with expectation constraints. A weak formulation, using test functions and a probabilistic relaxation of the constraint, avoids restrictions related to a measurable selection but still implies the Hamilton-Jacobi-Bellman equation in the …
Iterative method learns unknown constraints for MPC control.
problem Learning to satisfy unknown polyhedral state constraints in iterative MPC.
method Collects and improves estimates of unknown constraints using collected data, designs an MPC controller to satisfy the estimated constraints.
result Robust and probabilistic guarantees of constraint satisfaction as a function of task iterations.
We reformulate data-dependent constraints to ensure they are always met with high probability.
problem Ensuring fairness and stability in machine learning models with data-dependent constraints.
method Calibrated reformulation of constraints to guarantee satisfaction with a specified probability.
result Our method guarantees that fairness constraints are met at test time with high probability.
New algorithm reduces regret and constraint violation in online convex optimization with complex constraints.
problem Online convex optimization with multiple functional constraints and a simple constraint set.
method Instance-dependent bound using online primal-dual mirror-prox algorithm in general normed spaces.
result Achieves an O(√V*(T)) regret and O(1) constraint violation, improving over previous works.
The paper explores how to learn models that respect constraints in probabilistic learning.
problem Learning models that respect declared constraints in probabilistic learning.
method Mathematical inquiry on tractable probabilistic models like sum-product networks.
result Determines conditions under which constraints can be integrated with model learning.
Algorithm ensures privacy while strictly adhering to constraints.
problem Differential privacy with linear constraints that must be strictly followed.
method Developed an algorithm that releases a nearly-optimal solution satisfying constraints with probability 1.
result Achieved nearly optimal performance while preserving privacy and strictly adhering to constraints.
Proposes NUV priors for half-space and box constraints.
problem Adding constraints to linear Gaussian models without computational cost.
method Introduces NUV representations for half-space and box constraints.
result Adds constraints to linear Gaussian models without affecting computational tractability.
Meta-gradient D4PG optimizes performance and constraint adherence in RL.
problem Balancing performance and adherence to complex constraints in RL.
method Uses meta-gradients to find a balance between expected return and minimizing constraint violations.
result Meta-gradient D4PG consistently outperforms baselines across MuJoCo domains.
Geometrically characterizes virtual nonlinear nonholonomic constraints using symplectic methods.
problem Characterizing virtual nonlinear nonholonomic constraints geometrically.
method Geometric characterization using symplectic structures and Chetaev equations.
result A unique control law exists to satisfy virtual constraints, and closed-loop dynamics are projections of uncontrolled dynamics.
This work is a further study on the Generalized Constraint Neural Network (GCNN) model [1], [2]. Two challenges are encountered in the study, that is, to embed any type of prior information and to select its imposing schemes. The work focuses on the second challenge and studies a new constraint imposing scheme for equa…
The paper improves Gaussian processes by adding sum constraints, enhancing prediction accuracy.
problem Improving Gaussian process predictions with background knowledge constraints.
method Conditioning the prior distribution on sum constraints to ensure fulfillment of linear and nonlinear constraints.
result The approach fulfills constraints with high precision and improves prediction accuracy.
Develops a new method for optimizing with uncertain data.
problem Uncertainty in real-world optimization problems.
method Combines chance constraints and constraint learning for mixed-integer linear optimization.
result Data-driven solution for setting probabilistic bounds on learned constraints.
Efficient algorithms decide algebraic constraints of causal graphs.
problem Distinguish causal graphs with latent confounders.
method Study algebraic constraints and propose efficient algorithms.
result Decide equivalence or subset of algebraic constraints.
The paper introduces MU for NMF with β-divergences and disjoint constraints.
problem Nonnegative matrix factorization with constraints.
method Design multiplicative updates for NMF based on β-divergences with disjoint constraints. result Multiplicative updates satisfy constraints and decrease the objective function.
FISAR uses neural networks to optimize safe reinforcement learning with forward-invariant constraints.
problem Safe reinforcement learning with constraints in safety-critical environments.
method Imposing linear constraints on policy parameters' updating dynamics, using a DNN-based optimizer to satisfy these constraints.
result The policy decreases constraint violation and maximizes cumulative reward monotonically.
Solves Einstein constraint equations on compact manifolds with specified boundaries.
problem Solving Einstein constraint equations with specified boundaries.
method Studies conformal constraint equations with low regularity assumptions.
result Solves Einstein constraint equations on compact manifolds with specified boundaries.
We propose a general method for deformation quantization of any second-class constrained system on a symplectic manifold. The constraints determining an arbitrary constraint surface are in general defined only locally and can be components of a section of a non-trivial vector bundle over the phase-space manifold. The c…
Algorithm finds real line mapping from points under ordinal constraints.
problem Finding a mapping from points to real line under ordinal constraints.
method Approximation algorithm for dense case in O(n7)+(1/ε)O(1/ε1/8)n time. result Computes a solution satisfying (1−O(ε1/8))-fraction of all constraints. HardCoRe-NAS finds fitting neural networks adhering to hard resource constraints.
problem Finding fitting neural networks that adhere to hard resource constraints.
method Accurate formulation of resource requirement and scalable search method.
result HardCoRe-NAS generates state-of-the-art architectures strictly satisfying hard resource constraints.