CoRR learns convex relaxations for smooth functions via random point evaluations.
problem Solving non-convex optimization problems efficiently and provably.
method Estimating convex envelopes of smooth functions by fitting a convex function to random point evaluations.
result The solution of CoRR converges to the global optimizer of a function with a specified error rate.
New convex relaxations solve sparse regression problems efficiently.
problem Sparse regression with ℓ0 constraint is NP-hard. method Rank-one convexification for semidefinite optimization.
result Stronger and more general convex relaxations for sparse regression.
SR3 framework improves sparse regression solutions.
problem Sparse regression problems in various fields.
method SR3 framework solves relaxed regularized regression problems.
result SR3 provides superior solutions with faster algorithms.
New approach to convex hulls for low-rank problems.
problem Characterizing convex hulls for low-rank sets.
method Matrix perspective function and orthogonal projection matrices.
result Strong relaxations for various low-rank problems.
Safe screening rules reduce ℓ0-regression computation by fixing 76% of variables.
problem Efficiently solving ℓ0-regression problems with large datasets. method Convex relaxation and safe screening rules to eliminate variables.
result 76% of variables can be fixed to their optimal values, reducing computational burden.
Joint sparsity regularization in multi-task learning has attracted much attention in recent years. The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks. Although this approach leads to a simple convex formulation, it suffers from several issues due to the loosenes…
A nearly tight convex relaxation for sparse Naive Bayes features.
problem Feature selection in large-scale Naive Bayes classification.
method Proposes a convex relaxation for the combinatorial maximum-likelihood problem of feature selection in Naive Bayes.
result The convex relaxation bounds become tight as marginal feature contributions decrease, providing a nearly optimal solution.
New regularizers tighten convex relaxation bounds for neural networks.
problem Large gap between certifiable and empirical robustness in neural networks.
method Two regularizers to train neural networks yielding tighter convex relaxation bounds.
result Higher certified accuracy with proposed regularizers.
Convex relaxations improve CNNs with fixed weights.
problem Improving CNNs with fixed weights.
method Convex relaxations for CNNs with fixed weights using second order cone programs.
result The relaxation recovers the global minimum under a planted model assumption.
New method clusters variables using robust nodewise regression.
problem Variable clustering in multi-factor models.
method Distributionally robust nodewise regression with convex relaxation and ADMM.
result Superior performance in numerical studies.
New method tightens convex relaxations for permutation matrix problems without lifting.
problem Optimizing quadratic problems over permutation matrices.
method Lifting-free convex relaxation approach.
result Proves at least as tight as existing methods and performs better experimentally.
New model tackles region-sparse regression with hierarchical Gaussian process.
problem Sparse and dependent parameter vectors in regression settings.
method Hierarchical model with transformed Gaussian process and structured Fourier coefficients.
result Substantial improvements over comparable methods in simulated and real datasets.
A new framework for sparse regression models with slow variations.
problem Parameter estimation for sparse regression models with slow variations.
method Formulated as a mixed-integer optimization problem, then reformulated as a binary convex optimization problem with a novel relaxation technique.
result Efficiently solves the problem to provable optimality using a cutting plane-type algorithm.
Many high dimensional sparse learning problems are formulated as nonconvex optimization. A popular approach to solve these nonconvex optimization problems is through convex relaxations such as linear and semidefinite programming. In this paper, we study the statistical limits of convex relaxations. Particularly, we con…
Study identifies key differences in convex relaxations for combinatorial penalties.
problem Understanding which structures are preserved by convex relaxations for combinatorial penalties.
method Examined homogeneous and non-homogeneous convex relaxations, introduced lower combinatorial envelope, and proposed adaptive estimator.
result Identified new necessary and sufficient conditions for support recovery in convex monotone regularizers.
New method improves neural network verification by considering multivariate input space of ReLU neurons.
problem Improving the effectiveness of neural network verification algorithms.
method A new tightened convex relaxation for ReLU neurons considering multivariate input space.
result Our convex relaxation is significantly stronger than the commonly used univariate-input relaxation.
Paper tackles non-convex tensor regression with gradient descent.
problem Learning high-dimensional tensor regression with low-rank structure.
method Projected gradient descent on non-convex constraint set.
result Non-convex projected gradient descent provides superior statistical error and run-time.
NPMR uses nuclear norm penalty for multinomial regression, predicting baseball outcomes.
problem Predicting at bat outcomes in baseball with improved accuracy.
method Nuclear penalized multinomial regression (NPMR) applied to MLB data.
result NPMR provides better prediction probabilities for batter-pitcher matchups.
Paper introduces methods to create fair and accurate regression models.
problem Creating fair and accurate regression models.
method Mixed-integer optimization methods, exact formulations, branch-and-bound algorithm, coordinate descent algorithm.
result Developed methods produce fair and accurate models with reduced training times.
Active-set algorithm improves Cox regression for shape-restricted covariates.
problem Improving Cox regression for shape-restricted covariates.
method Shape-restricted inference using active-set optimization for spline basis expansion.
result Active-set algorithm produces accurate linear covariate effect estimates.
Improved neural network robustness certification through tighter convex relaxations.
problem Certifying neural network robustness to perturbed and adversarial inputs.
method Exploiting ReLU network structure, novel partition-based certification procedure.
result Tightens existing linear programming relaxations to achieve zero relaxation error asymptotically.
Paper relaxes optimal transport using convex functions for data science.
problem Optimal transport problem on finite spaces.
method Relaxation via strictly convex functions (Kullback-Leibler divergence, Bregman divergences). Gradient descent iterative process.
result Mathematical foundations and iterative process for the relaxed optimal transport problem.
Non-convex regularizers usually improve the performance of sparse estimation in practice. To prove this fact, we study the conditions of sparse estimations for the sharp concave regularizers which are a general family of non-convex regularizers including many existing regularizers. For the global solutions of the regul…
Algorithm for exact partitioning of high-order models using convex tensor relaxation.
problem Exact partitioning of high-order models.
method Defining a general class of m-degree Homogeneous Polynomial Models, relaxing the high-order combinatorial problem to a convex conic form problem, defining the Carathéodory symmetric tensor cone, and constructing a primal-dual certificate. result The solution of the convex relaxation is correct and provides a statistical upper bound for exact partitioning.
Convex geometry explains optimal neural network parameters.
problem Understanding optimal parameters in over-parameterized neural networks.
method Convex geometry, extreme points, linear spline interpolation, kernel matrix, cutting-plane algorithm.
result Optimal network parameters can be characterized as interpretable closed-form formulas.
Group-based sparsity models are proven instrumental in linear regression problems for recovering signals from much fewer measurements than standard compressive sensing. The main promise of these models is the recovery of "interpretable" signals through the identification of their constituent groups. In this paper, we e…
Paper improves understanding of noisy matrix completion using convex relaxation and nonconvex optimization.
problem Estimating a low-rank matrix from noisy partial entries.
method Combining convex relaxation and the nonconvex Burer-Monteiro approach.
result Convex relaxation achieves near-optimal estimation errors for noisy matrix completion.
The paper analyzes trace regression with low-rank matrices under various regularization methods.
problem Estimating low-rank matrices with near-optimal error bounds under unknown regularization parameters.
method General spikiness notion, restricted strong convexity of sampling operator, cross-validation for parameter selection.
result Cross-validated estimators select near-optimal penalty parameters and outperform theory-inspired approaches.
Unified convex relaxation framework for neural network robustness verification.
problem Inability to achieve tight verification of neural networks against adversarial attacks.
method Unified convex relaxation framework for neural networks of various architectures and nonlinearities.
result Exact solution to convex-relaxed problem does not significantly improve verification gap.
Robust sparse reduced rank regression for high-dimensional data with heavy-tailed noise.
problem Analyzing large, complex high-dimensional data with heavy-tailed random noise.
method Convex relaxation of a rank- and sparsity-constrained non-convex optimization problem solved using the alternating direction method of multipliers.
result Established non-asymptotic estimation error bounds under Frobenius and nuclear norms, quantifying the tradeoff between heavy-tailedness and statistical bias.
Two-stage nonconvex algorithm and convex relaxation both achieve optimal accuracy in noisy blind deconvolution.
problem Solving bilinear systems of equations with random noise under different designs.
method Two-stage nonconvex algorithm and convex relaxation.
result Both methods achieve minimax-optimal accuracy in the presence of random noise.
A number of recent work studied the effectiveness of feature selection using Lasso. It is known that under the restricted isometry properties (RIP), Lasso does not generally lead to the exact recovery of the set of nonzero coefficients, due to the looseness of convex relaxation. This paper considers the feature selecti…
Paper proposes a new convex relaxation for low-rank approximation problems.
problem Finding low-rank approximations with convex constraints in data analysis.
method Proposes a new convex relaxation using the convex envelope of the squared Frobenius norm and rank constraint.
result Solutions to the convex relaxation coincide with the original non-convex problem under certain conditions.
Based on a new atomic norm, we propose a new convex formulation for sparse matrix factorization problems in which the number of nonzero elements of the factors is assumed fixed and known. The formulation counts sparse PCA with multiple factors, subspace clustering and low-rank sparse bilinear regression as potential ap…
Bounds on chemical reaction network relaxation rates using convex analysis.
problem Understanding relaxation dynamics in chemical reaction networks.
method Convex analysis, generalized gradient flows, singular values of stoichiometric matrix.
result Bounds on Kullback-Leibler divergence to equilibrium for CRNs.
The paper develops sum-of-squares relaxations for computing f-divergences.
problem Computing f-divergences from non-centered covariance matrices. method Sum-of-squares relaxations for convex optimization.
result Sum-of-squares relaxations make computations tractable.
Paper relaxes assumptions for non-parametric estimation in pairwise learning.
problem Generalization performance of non-parametric estimation for pairwise learning.
method Significantly relaxes restrictive assumptions, constructs structured deep ReLU neural network, and designs targeted hypothesis space.
result Establishes a sharp oracle inequality for empirical minimizer with general hypothesis space for Lipschitz continuous pairwise losses.
Paper solves graph matching problem using convex relaxation to the simplex.
problem Finding the best alignment between two graphs.
method Introduces a new convex relaxation onto the unit simplex and uses mirror descent scheme.
result Shows exact recovery of ground truth permutation with high probability.
We study the total least squares (TLS) problem that generalizes least squares regression by allowing measurement errors in both dependent and independent variables. TLS is widely used in applied fields including computer vision, system identification and econometrics. The special case when all dependent and independent…
Although many convex relaxations of clustering have been proposed in the past decade, current formulations remain restricted to spherical Gaussian or discriminative models and are susceptible to imbalanced clusters. To address these shortcomings, we propose a new class of convex relaxations that can be flexibly applied…
The paper explores new insights into variable selection using convex relaxation and semidefinite programming.
problem Sparsity-inducing regularization methods for variable selection in statistical data analysis.
method The paper introduces a new perspective on sparsity-inducing penalties using perspective relaxation and semidefinite programming.
result The perspective relaxation can be solved by a semidefinite relaxation and provides a probabilistic interpretation.
Paper solves NP-hard sparse mixed linear regression problem with provable guarantees.
problem Sparse mixed linear regression on unlabeled data.
method Invex relaxation for intractable problem with theoretical guarantees.
result Exact recovery of data labels and close approximation of regression parameters.
New framework tackles unsupervised learning without generative models.
problem Tackles unsupervised learning without assuming a generative model.
method Formal theoretical framework based on worst-case performance and convex relaxations.
result Efficient learning of unsupervised models using convex optimization.
Study differentially private linear regression with heavy-tailed data.
problem Differentially private ℓ1-norm linear regression with heavy-tailed data. method Exponential mechanism for ℓ2-norm bounded second moment; relaxation to ℓ2-norm bounded θ-th moment; coordinate-wise bounded moments. result Achieved upper bounds for privacy-preserving linear regression under various moment conditions.
Paper relaxes SGD privacy and generalization guarantees for non-smooth convex losses.
problem Privacy and generalization in SGD for non-smooth convex losses.
method Relaxes Lipschitz and strong smoothness assumptions to Hölder smoothness, proving (ε,δ)-DP and optimal excess risk. result Noisy SGD with α-Hölder smooth losses achieves optimal excess risk with linear gradient complexity for α≥1/2. We suggest using the max-norm as a convex surrogate constraint for clustering. We show how this yields a better exact cluster recovery guarantee than previously suggested nuclear-norm relaxation, and study the effectiveness of our method, and other related convex relaxations, compared to other clustering approaches.
Proposes new convex relaxations for certifying spatial robustness of neural networks.
problem Lack of provable guarantees for robustness against vector field transformations.
method Novel convex relaxations for certifying robustness against vector field transformations.
result First time providing a certificate of robustness against vector field transformations.
PEREGRiNN verifies safety of ReLU NNs by penalizing relaxation in a greedy manner.
problem Formal verification of safety specifications for ReLU NNs.
method Uses a relaxed convex program to verify polytopic input/output constraints, penalizing relaxation and forcing largest relaxations to early layers.
result Significantly faster and more properties verified compared to other approaches.