The stochastic block model (SBM) is a popular tool for community detection in networks, but fitting it by maximum likelihood (MLE) involves a computationally infeasible optimization problem. We propose a new semidefinite programming (SDP) solution to the problem of fitting the SBM, derived as a relaxation of the MLE. W…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “SDP relaxation”
Hidden Integrality and Semi-random Robustness of SDP Relaxation for Sub-Gaussian Mixture Modelstat.ML
We consider the problem of estimating the discrete clustering structures under the Sub-Gaussian Mixture Model. Our main results establish a hidden integrality property of a semidefinite programming (SDP) relaxation for this problem: while the optimal solution to the SDP is not integer-valued in general, its estimation …
On the Tightness of Semidefinite Relaxations for Certifying Robustness to Adversarial Examplesmath.OC
Geometric technique determines exactness of SDP robustness certificate.
problem Certifying robustness of neural networks to adversarial examples.
method Geometric projection onto hyperbola, SDP relaxation of ReLU activation.
result SDP certificate is exact for a single hidden layer under mild assumptions.
Efficient, Certifiably Optimal Clustering with Applications to Latent Variable Graphical Modelsstat.ML
Motivated by the task of clustering either variables or points into groups, we investigate efficient algorithms to solve the Peng-Wei (P-W) -means semi-definite programming (SDP) relaxation. The P-W SDP has been shown in the literature to have good statistical properties in a variety of settings, but rem…
We have observed an interesting, yet unexplained, phenomenon: Semidefinite programming (SDP) based relaxations of maximum likelihood estimators (MLE) tend to be tight in recovery problems with noisy data, even when MLE cannot exactly recover the ground truth. Several results establish tightness of SDP based relaxations…
Recently, Awasthi et al. introduced an SDP relaxation of the -means problem in . In this work, we consider a random model for the data points in which balls of unit radius are deterministically distributed throughout , and then in each ball, points are drawn according to a common ro…
Resolving a conjecture of Abbe, Bandeira and Hall, the authors have recently shown that the semidefinite programming (SDP) relaxation of the maximum likelihood estimator achieves the sharp threshold for exactly recovering the community structure under the binary stochastic block model of two equal-sized clusters. The s…
Paper presents a randomized algorithm for SPCA with high probability approximation.
problem Sparse Principal Component Analysis (SPCA) is NP-hard.
method Based on basic SDP relaxation, the algorithm constructs deterministic and randomized solutions.
result The algorithm achieves an approximation ratio of at most the sparsity constant with high probability.
SDP achieves optimal error in noisy phase synchronization.
problem Phase synchronization with noisy measurements.
method SDP relaxation of Maximum Likelihood Estimation (MLE).
result Achieves error bound of under normalized squared loss, matching minimax lower bound.
New algorithm solves large cardinality-constrained clustering problems.
problem Optimizing clustering with cardinality constraints.
method Branch-and-cut technique with SDP relaxation and polyhedral cuts.
result Solves real-world instances 10 times larger than previous methods.
Optimal neural network approximation for Wasserstein gradient direction via convex optimization.
problem Approximating Wasserstein gradient direction with limited data.
method Two-layer networks with squared-ReLU activations, SDP relaxation.
result Optimal approximation of Wasserstein gradient direction in two-layer networks.
Maximum A posteriori Probability (MAP) inference in graphical models amounts to solving a graph-structured combinatorial optimization problem. Popular inference algorithms such as belief propagation (BP) and generalized belief propagation (GBP) are intimately related to linear programming (LP) relaxation within the She…
We study the problem of controlling linear time-invariant systems with known noisy dynamics and adversarially chosen quadratic losses. We present the first efficient online learning algorithms in this setting that guarantee regret under mild assumptions, where is the time horizon. Our algorithms rely …
The framework of Integral Quadratic Constraints of Lessard et al. (2014) reduces the computation of upper bounds on the convergence rate of several optimization algorithms to semi-definite programming (SDP). Followup work by Nishihara et al. (2015) applies this technique to the entire family of over-relaxed Alternating…
Tuning Over-Relaxed ADMMstat.ML
The framework of Integral Quadratic Constraints (IQC) reduces the computation of upper bounds on the convergence rate of several optimization algorithms to a semi-definite program (SDP). In the case of over-relaxed Alternating Direction Method of Multipliers (ADMM), an explicit and closed form solution to this SDP was …
We study the statistical performance of semidefinite programming (SDP) relaxations for clustering under random graph models. Under the Synchronization model, Censored Block Model and Stochastic Block Model, we show that SDP achieves an error rate of the form \[ \exp\Big[-\big(1-o(1)\big)\bar{n} I^* \Bi…
Several probabilistic models from high-dimensional statistics and machine learning reveal an intriguing --and yet poorly understood-- dichotomy. Either simple local algorithms succeed in estimating the object of interest, or even sophisticated semi-definite programming (SDP) relaxations fail. In order to explore this p…
New algorithm improves clustering accuracy without sacrificing scalability.
problem Improving clustering accuracy for large datasets.
method Nonnegative low-rank semidefinite programming with Burer-Monteiro factorization.
result Significantly smaller mis-clustering errors compared to existing methods.
New SDP method certifies neural network robustness across all classes efficiently.
problem Certifying robustness of neural networks across multiple classes.
method Quadratic model + SDP relaxation + pruning strategy.
result Significant computational speed-up and scalability to large datasets.
We propose an SDP relaxation for the Gromov-Wasserstein distance, providing globally optimal solutions.
problem Matching objects between incomparable spaces using the Gromov-Wasserstein distance.
method Semi-definite programming (SDP) relaxation of the GW distance.
result The SDP relaxation provides globally optimal solutions for the GW distance in some instances.
A fast algorithm for -means clustering using subsampled SDP.
problem Efficiently solving large-scale -means clustering problems.
method Sketch-and-Lift (SL) approach for approximating SDP relaxed -means.
result SL method achieves similar exact recovery threshold as full SDP on full dataset.
SyncRank recovers global ranking from noisy comparisons with theoretical guarantees.
problem Recovering a global ranking from noisy pairwise comparisons.
method Complex-valued data model and SDP relaxation for exact ranking recovery.
result SyncRank achieves exact ranking recovery with high probability above a critical noise threshold of O(sqrt(n / log n)).
A new Wasserstein -means method for clustering probability distributions.
problem Clustering probability distributions using the Wasserstein metric.
method Distance-based -means with SDP relaxation for Wasserstein barycenters.
result Distance-based -means outperforms centroid-based -means for clustering probability distributions.
Paper uses SDP for community detection with side information.
problem Community detection in graphs with additional non-graph data.
method Formulates SDP relaxation for maximum likelihood node labeling with side information.
result SDP achieves same exact recovery threshold as maximum likelihood with side information.
Algorithm recovers function samples from noisy modulo samples with high probability.
problem Recovering function samples from noisy modulo samples.
method Two-stage algorithm involving k-NN regression and SDP relaxation.
result Uniform error rate of for function samples.
A new method for clustering heterogeneous data using likelihood-adjusted SDP.
problem Clustering heterogeneous data with different cluster shapes and sizes.
method Iterative likelihood-adjusted semidefinite programming (iLA-SDP) method.
result iLA-SDP achieves lower mis-clustering errors compared to other methods.
Clustering is one of the most important unsupervised problems in machine learning and statistics. Among many existing algorithms, kernel k-means has drawn much research attention due to its ability to find non-linear cluster boundaries and its inherent simplicity. There are two main approaches for kernel k-means: SVD o…
New method closes certification gap for adversarially trained models.
problem Certifying robustness of adversarially trained neural networks.
method Nonconvex low-rank SDP relaxation with polynomial-time optimization.
result Strong certifications comparable to SDP methods, but with fewer variables.
In this paper, we analyze different preconditionings designed to enhance robustness of pure-pixel search algorithms, which are used for blind hyperspectral unmixing and which are equivalent to near-separable nonnegative matrix factorization algorithms. Our analysis focuses on the successive projection algorithm (SPA), …
AMP algorithms can be efficiently simulated by SDPs even with corrupted data.
problem Optimizing average-case optimization problems with corrupted data.
method Local statistics hierarchy semidefinite programs (SDPs) simulate AMP algorithms robustly.
result Robust guarantees for many AMP algorithms are offered, contrasting with strong lower bounds for SDPs.
Diffusion -means clustering on manifolds: provable exact recovery via semidefinite relaxationsmath.ST
We introduce the {\it diffusion -means} clustering method on Riemannian submanifolds, which maximizes the within-cluster connectedness based on the diffusion distance. The diffusion -means constructs a random walk on the similarity graph with vertices as data points randomly sampled on the manifolds and edges as …
We study a semidefinite programming (SDP) relaxation of the maximum likelihood estimation for exactly recovering a hidden community of cardinality from an symmetric data matrix , where for distinct indices , if are both in the community and otherwise, for …
A large number of problems in optimization, machine learning, signal processing can be effectively addressed by suitable semidefinite programming (SDP) relaxations. Unfortunately, generic SDP solvers hardly scale beyond instances with a few hundreds variables (in the underlying combinatorial problem). On the other hand…
We determine the information-theoretic cutoff value on separation of cluster centers for exact recovery of cluster labels in a -component Gaussian mixture model with equal cluster sizes. Moreover, we show that a semidefinite programming (SDP) relaxation of the -means clustering method achieves such sharp threshol…
New SDP algorithm recovers large clusters in SBM with small clusters of any size.
problem Graph clustering in SBM with large and small clusters.
method Semidefinite programming (SDP) with novel techniques to handle small clusters.
result Proves exact recovery of large clusters regardless of small cluster sizes.
We introduce a model-free relax-and-round algorithm for k-means clustering based on a semidefinite relaxation due to Peng and Wei. The algorithm interprets the SDP output as a denoised version of the original data and then rounds this output to a hard clustering. We provide a generic method for proving performance guar…
We present a novel algorithm for overcomplete independent components analysis (ICA), where the number of latent sources k exceeds the dimension p of observed variables. Previous algorithms either suffer from high computational complexity or make strong assumptions about the form of the mixing matrix. Our algorithm does…
This paper introduces a general multi-class approach to weakly supervised classification. Inferring the labels and learning the parameters of the model is usually done jointly through a block-coordinate descent algorithm such as expectation-maximization (EM), which may lead to local minima. To avoid this problem, we pr…
Algorithm recovers sparse PCA support from incomplete data.
problem Sparse PCA with incomplete and noisy data.
method Semidefinite program (SDP) relaxation of non-convex -regularized PCA.
result SDP enables exact recovery of true support of sparse leading eigenvector.
We consider clustering problems where the goal is to determine an optimal partition of a given point set in Euclidean space in terms of a collection of affine subspaces. While there is vast literature on heuristics for this kind of problem, such approaches are known to be susceptible to poor initializations and getting…
We consider semidefinite programs (SDPs) of size n with equality constraints. In order to overcome scalability issues, Burer and Monteiro proposed a factorized approach based on optimizing over a matrix Y of size by such that is the SDP variable. The advantages of such formulation are twofold: the di…
Estimating the leading principal components of data, assuming they are sparse, is a central task in modern high-dimensional statistics. Many algorithms were developed for this sparse PCA problem, from simple diagonal thresholding to sophisticated semidefinite programming (SDP) methods. A key theoretical question is und…
We propose a flexible convex relaxation for the phase retrieval problem that operates in the natural domain of the signal. Therefore, we avoid the prohibitive computational cost associated with "lifting" and semidefinite programming (SDP) in methods such as PhaseLift and compete with recently developed non-convex techn…
Simplifies solving noisy SDPs for low rank matrix recovery problems.
problem Solving SDPs with noisy data for low rank matrix recovery problems.
method Identifies conditions called simplicity to limit error in noisy SDP solutions.
result Simple SDPs can be efficiently solved and their approximate solutions trusted.
This paper establishes a statistical versus computational trade-off for solving a basic high-dimensional machine learning problem via a basic convex relaxation method. Specifically, we consider the {\em Sparse Principal Component Analysis} (Sparse PCA) problem, and the family of {\em Sum-of-Squares} (SoS, aka Lasserre/…
Maximum a posteriori (MAP) inference over discrete Markov random fields is a fundamental task spanning a wide spectrum of real-world applications, which is known to be NP-hard for general graphs. In this paper, we propose a novel semidefinite relaxation formulation (referred to as SDR) to estimate the MAP assignment. A…
The problem of low-rank approximation with convex constraints, which appears in data analysis, system identification, model order reduction, low-order controller design and low-complexity modelling is considered. Given a matrix, the objective is to find a low-rank approximation that meets rank and convex constraints, w…
SDP Feasibility Problems and sos Representation Ranks for OT-FKM Type Isoparametric Polynomialsmath.DG
The paper studies SDP feasibility and sos ranks for specific polynomials.
problem Characterizing sos representations of nonnegative polynomials.
method Explicit SDP formulation based on Clifford systems.
result Quantitative rank bounds for sos representations, with rigidity.