Three algorithms improve starting solutions for clustering problems.
problem Improving solutions for the Minimum Sum of Squares Clustering Problem.
method Proposed three algorithms to create starting solutions.
result Found new best known solutions for 5 out of 24 challenging instances.
New method finds global minima using function evaluations and kernel approximations.
problem Finding global minima of smooth functions with limited evaluations.
method Approximates the function using infinite sums of square smooth functions and solves the optimization problem with polynomial time complexity.
result Achieves optimal number of function evaluations with theoretical guarantees and nearly optimal convergence rate.
In this paper we consider the use of the space vs. time Kronecker product decomposition in the estimation of covariance matrices for spatio-temporal data. This decomposition imposes lower dimensional structure on the estimated covariance matrix, thus reducing the number of samples required for estimation. To allow a sm…
Minimum sum-of-squares clustering (MSSC) is a widely used clustering model, of which the popular K-means algorithm constitutes a local minimizer. It is well known that the solutions of K-means can be arbitrarily distant from the true MSSC global optimum, and dozens of alternative heuristics have been proposed for this …
New algorithm solves large cardinality-constrained clustering problems.
problem Optimizing clustering with cardinality constraints.
method Branch-and-cut technique with SDP relaxation and polyhedral cuts.
result Solves real-world instances 10 times larger than previous methods.
New proof shows how to identify DAGs with weakly increasing errors.
problem Identifying the true DAG in models with weakly increasing error variances.
method Minimum-trace DAG method and hill climbing algorithm with R2R neighborhood.
result Hill climbing algorithm without strict local optima under weakly increasing error variances.
New method clusters non-spherical Gaussian mixtures with fewer samples and time.
problem Clustering non-spherical Gaussian mixtures with arbitrary component covariances.
method Sum-of-Squares method for finding low-dimensional projections.
result Improved clustering algorithms with fewer samples and time complexity.
Estimation is the computational task of recovering a hidden parameter x associated with a distribution Dx, given a measurement y sampled from the distribution. High dimensional estimation problems arise naturally in statistics, machine learning, and complexity theory. Many high dimensional estimation problems ca…
We define the "sum of squares of the wavelengths" of a Riemannian surface (M,g) to be the regularized trace of the inverse of the Laplacian. We normalize by scaling and adding a constant, to obtain a "mass", which is scale invariant and vanishes at the round sphere. This is an anlaog for closed surfaces of the ADM mass…
Bayesian method recovers causal structure in SEMs with equal error variances.
problem Recovering causal structure in SEMs with equal error variances.
method Bayesian DAG selection method using g-priors and the key property of minimum expected squared errors.
result The method consistently recovers the true graph without additional distributional assumptions.
The paper analyzes the risk of a least squares estimator under a spike covariance model.
problem Risk analysis of the least squares estimator under a spike covariance model.
method Assumes spike covariance matrices, studies risk as d/nightarrow∞. result Risk of the minimum norm least squares estimator vanishes compared to the null estimator.
New algorithms learn graph structures privately, matching best results.
problem Private learning of graph structures with multiple blocks.
method Sum-of-squares relaxation and exponential mechanism for score function.
result Matches statistical utility of previous best non-private methods.
To a compact Riemann surface of genus g can be assigned a principally polarized abelian variety (PPAV) of dimension g, the Jacobian of the Riemann surface. The Schottky problem is to discern the Jacobians among the PPAVs. Buser and Sarnak showed, that the square of the first successive minimum, the squared norm of the …
Hilbert's 17th problem asks that whether every nonnegative polynomial can be a sum of squares of rational functions. It has been answered affirmatively by Artin. However, the question as to whether a given nonnegative polynomial is a sum of squares of polynomials is still a central question in real algebraic geometry. …
Regularization for matrix factorization (MF) and approximation problems has been carried out in many different ways. Due to its popularity in deep learning, dropout has been applied also for this class of problems. Despite its solid empirical performance, the theoretical properties of dropout as a regularizer remain qu…
The paper explores the information-theoretic nature of excess risk in machine learning.
problem Understanding the excess risk in machine learning models.
method Formulates the minimax excess risk as a zero-sum game and modifies it to allow swapping of the order of play.
result Proves that under certain conditions, the duality gap is zero, allowing for the application of Bayesian results to provide bounds on minimax excess risk.
Paper solves outlier robust mean estimation near breakdown point.
problem Estimating mean in presence of adversarial outliers.
method Sum-of-Squares approach to optimize error rate efficiently.
result Achieves optimal error rate for all ε ∈ [0, 1/2).
The paper models financial markets using information theory to minimize information.
problem Understanding the dynamics of financial markets.
method Modeling financial market dynamics with independent stationary scalar diffusions, interpreting the market as a communication system, and minimizing information-theoretical joint information.
result Financial market dynamics are represented by squared radial Ornstein-Uhlenbeck processes with additivity and self-similarity properties.
We propose a minimum distance estimation method for robust regression in sparse high-dimensional settings. The traditional likelihood-based estimators lack resilience against outliers, a critical issue when dealing with high-dimensional noisy data. Our method, Minimum Distance Lasso (MD-Lasso), combines minimum distanc…
New algorithm finds corrupted vertices in graphs with few queries.
problem Adversarial tampering of graph edges and vertices.
method Active learning algorithm with polynomial query complexity.
result Efficiently recovers corrupted vertices with small query complexity.
Algorithm distinguishes Gaussian mixtures from pure Gaussians in quasi-polynomial time.
problem Distinguishing mixtures of Gaussian components from pure Gaussians, especially when components are well-separated.
method Sum-of-Squares method, quasi-polynomial time algorithm, bipartitioning sample to separate components.
result Algorithm can reliably distinguish between mixtures and pure Gaussians in quasi-polynomial time.
This book introduces linear models and their theories rigorously.
problem Understanding linear models and their theories.
method Explains linear models from three perspectives, introduces maximum likelihood estimation, and proves least squares is the best unbiased linear model.
result Least squares is the best unbiased linear model in terms of mean squared error.
New framework reduces sum-of-squares proof degree, speeding up clustering and robust moment estimation.
problem Sum-of-squares proof optimization and faster algorithms for clustering and robust moment estimation.
method Introducing new variables to reduce the degree of sum-of-squares proofs.
result Significantly faster algorithms for clustering and robust moment estimation with the same statistical guarantees.
Paper uses SC to estimate hidden interference for WSRM.
problem Maximizing sum-rate with hidden interfering sources.
method Synthetic control (SC) for estimating counterfactual interference in WMMSE.
result SC-WMMSE outperforms original WMMSE in convergence and objective.
In recent years, optimization theory has been greatly impacted by the advent of sum of squares (SOS) optimization. The reliance of this technique on large-scale semidefinite programs however, has limited the scale of problems to which it can be applied. In this paper, we introduce DSOS and SDSOS optimization as linear …
We show that every thin position for a connected sum of small knots is obtained in an obvious way: place each summand in thin position so that no two summands intersect the same level surface, then connect the lowest minimum of each summand to the highest maximum of the adjacent summand below.
New method for NMF without tuning parameter.
problem Finding latent structures in noisy data matrices.
method Inspired by square-root lasso, proposes a tuning-free minimum-volume NMF.
result Optimal tuning parameter value is noise level-independent.
Polynomial-time algorithm estimates edge density of random graphs with privacy and robustness.
problem Estimating edge density of random graphs while maintaining privacy and robustness.
method Sum-of-squares algorithm for robust edge density estimation and reduction from privacy to robustness.
result Optimal error rate up to logarithmic factors, matching theoretical lower bounds.
New algorithm estimates transport maps with nearly optimal error.
problem Estimating smooth transport maps efficiently and accurately.
method Solving semi-dual formulation of optimal transport with kernel sums-of-squares.
result Statistical L2 error on maps nearly matches minimax lower-bounds. The paper shows how multi-task learning in neural networks is similar to kernel regression and Hilbert spaces.
problem Understanding the solutions to multi-task shallow ReLU neural network learning problems.
method Analyzing the properties of solutions to multi-task shallow ReLU neural network learning problems, proving uniqueness and equivalence to minimum-norm interpolation problems in Hilbert spaces.
result The solutions to multi-task neural network interpolation problems are almost always unique and coincide with the solution to a minimum-norm interpolation problem in a Sobolev (Reproducing Kernel) Hilbert Space.
Paper introduces a new kernel model for PSD-valued functions with theoretical guarantees and applications.
problem Enforcing positive semi-definiteness (PSD) in function models with good performance and theoretical guarantees.
method Kernel sum-of-squares model for PSD-valued functions, extending previous models for non-negative scalar functions.
result The model constitutes a universal approximator of PSD functions and can represent any smooth and strongly convex function.
New perspective on Heegaard splittings using square complexes and combinatorial measurements.
problem Measuring obstructions to Heegaard splittings in 3-manifolds.
method Square complexes and Guirardel's core, augmented Heegaard diagrams.
result Augmented Heegaard diagrams provide a new way to describe Heegaard splittings with desirable properties.
This paper proposes a new Nystrom-based clustering algorithm for large-scale data.
problem Spectral clustering's high computational complexity for large-scale data.
method Centroid Minimum Sum of Squared Similarities (CMS3) sampling procedure with eigen spectrum shape heuristic.
result Competitive low-rank approximations in test datasets compared to state-of-the-art methods.
New bounds on homological eigenvalues relate to Weil-Petersson length.
problem Bounding growth of homological eigenvalues for pseudo-Anosov automorphisms.
method Established inequality linking homological Jensen square sum to Weil-Petersson translation length.
result Homological Jensen square sum grows at most linearly with covering degree compared to Weil-Petersson translation length.
This study explains gradient flow dynamics in neural networks for small initialisation.
problem Understanding the training dynamics of neural networks for small initialisation.
method Analysis of gradient flow dynamics for one-hidden layer ReLU networks with orthogonal inputs.
result Gradient flow converges to zero loss and characterizes implicit bias towards minimum variation norm.
We consider the minimum error entropy (MEE) criterion and an empirical risk minimization learning algorithm in a regression setting. A learning theory approach is presented for this MEE algorithm and explicit error bounds are provided in terms of the approximation ability and capacity of the involved hypothesis space w…
The paper develops sum-of-squares relaxations for computing f-divergences.
problem Computing f-divergences from non-centered covariance matrices. method Sum-of-squares relaxations for convex optimization.
result Sum-of-squares relaxations make computations tractable.
We consider the question of existence of ramified covers over P_1 matching certain prescribed ramification conditions. This problem has already been faced in a number of papers, but we discuss alternative approaches for an existence proof, involving elliptic curves and universal ramified covers with signature. We also …
New flag-no-square 4-manifolds discovered with unique triangulations.
problem Identifying 4-manifolds with specific triangulation properties.
method Introduced star-connected-sum operation to preserve flag-no-square property; derived new constructions.
result Found non-aspherical flag-no-square 4-manifolds, answering a question by Przytycki and Swiatkowski.
We prove the Chern-Weil formula for SU(n+1)-singular connections over the complement of an embedded oriented surface in smooth four manifolds. The expression of the representation of a number as a sum of nonvanishing squares is given in terms of the representations of a number as a sum of squares. Using the number theo…
In order to model entanglements of polymers in a confined region, we consider the linking numbers and writhes of cycles in random linear embeddings of complete graphs in a cube. Our main results are that for a random linear embedding of Kn in a cube, the mean sum of squared linking numbers and the mean sum of square…
We consider a zero-sum continuous time stopping game in which the pay-off is revealed in the maximum of the two stopping times instead of the minimum, which is the case in Dynkin games.
Key to the imposition of appropriate minimum capital requirements on a daily basis requires accurate volatility estimation. Here, measures are presented based on discrete estimation of aggregated high frequency UK futures realisations underpinned by a continuous time framework. Squared and absolute returns are incorpor…
Formula for Bergman kernel of complex hyperbolic manifolds proved.
problem Calculating the Bergman kernel of complex hyperbolic manifolds.
method Expressed as a sum over geodesic loops.
result Maximum and minimum of the Bergman kernel function proved.
We consider two problems that arise in machine learning applications: the problem of recovering a planted sparse vector in a random linear subspace and the problem of decomposing a random low-rank overcomplete 3-tensor. For both problems, the best known guarantees are based on the sum-of-squares method. We develop new …
Efficiently estimates prediction error in regression with Gaussian covariates under privacy constraints.
problem Private regression with Gaussian covariates under differential privacy constraints.
method Sum-of-Squares framework combined with robust estimators.
result Sample-optimal private regression algorithm with optimal error rates.
Adding linear layers to ReLU networks favors functions with low mixed variation.
problem Understanding function space bias in overparameterized neural networks.
method Examined a family of networks with varying depths and same capacity but different representation costs, focusing on the effect of adding linear layers to the input side.
result Adding linear layers to shallow ReLU networks results in a bias towards functions with low mixed variation, which can be well approximated by single- or multi-index models.
We obtain the first polynomial-time algorithm for exact tensor completion that improves over the bound implied by reduction to matrix completion. The algorithm recovers an unknown 3-tensor with r incoherent, orthogonal components in Rn from r⋅O~(n1.5) randomly observed entries of the tensor…