Paper proposes a new clustering model that preserves cluster recovery with fewer dimensions.
problem Clustering high-dimensional data with limited embedding dimensions.
method Randomly projected convex clustering model with improved embedding dimension.
result Cluster recovery can be preserved with fewer dimensions, independent of data points.
We discuss a general notion of "sparsity structure" and associated recoveries of a sparse signal from its linear image of reduced dimension possibly corrupted with noise. Our approach allows for unified treatment of (a) the "usual sparsity" and "usual ℓ1 recovery," (b) block-sparsity with possibly overlapping blo…
Higher-order tensors can represent scores in a rating system, frames in a video, and images of the same subject. In practice, the measurements are often highly quantized due to the sampling strategies or the quality of devices. Existing works on tensor recovery have focused on data losses and random noises. Only a few …
Study shows overparameterization helps shallow neural networks recover signals in high dimensions.
problem Signal recovery in shallow neural networks with overparameterization.
method Gradient flow on population risk, Gaussian distribution assumption, high-dimensional limit analysis.
result Minimal overparameterization is sufficient for strong recovery of signals.
We study the effect of the quality and quantity of side information on the recovery of a hidden community of size K=o(n) in a graph of size n. Side information for each node in the graph is modeled by a random vector with the following features: either the dimension of the vector is allowed to vary with n, while …
In this paper, we study the recovery of a signal from a set of noisy linear projections (measurements), when such projections are unlabeled, that is, the correspondence between the measurements and the set of projection vectors (i.e., the rows of the measurement matrix) is not known a priori. We consider a special case…
Spectral flow connects manifold geometry to rigidity criteria.
problem Tackling rigidity of simply-connected closed manifolds.
method Spectral deformation flow and invariant-based approach.
result Spherical profile is the unique manifold-compatible asymptotic realization.
RBM learns in high dimensions via AMP and GD, reaching optimal weak recovery.
problem Learning from high-dimensional data with RBM.
method AMP and GD analysis for RBM training in high dimensions.
result RBM reaches optimal weak recovery threshold in spiked covariance model.
We study graph matching with correlated Gaussian features and find thresholds for exact recovery.
problem Graph matching with correlated Gaussian features.
method Information-theoretic thresholds and conditions for exact and almost exact recovery.
result Contextual information introduces a richer structure, with thresholds for exact and almost exact recovery no longer coinciding.
The paper evaluates samplers on multi-modal targets, focusing on mode separation and recovery.
problem Handling multi-modality in sampling.
method Synthetic experimental setting focusing on mode relative importance recovery.
result Illustrates the challenges and potential of samplers in multi-modality.
Improved subspace recovery algorithm with dimension-independent error and polynomial time.
problem Efficiently recover a covariance matrix from a mix of inliers and adversarial outliers.
method List-decodable subspace recovery algorithm with faster fixed-polynomial time and less restrictive distributional assumptions.
result Achieved dimension-independent error guarantee of O(1/α) with poly(1/α d^O(1)) time complexity.
The problem of population recovery refers to estimating a distribution based on incomplete or corrupted samples. Consider a random poll of sample size n conducted on a population of individuals, where each pollee is asked to answer d binary questions. We consider one of the two polling impediments: (a) in lossy pop…
This study optimizes multi-modal learning thresholds and algorithms in high dimensions.
problem Optimizing multi-modal learning performance in high-dimensional data.
method Analytical quantification and derivation of AMP algorithm with state evolution analysis.
result Bayes-optimal performance and recovery thresholds derived for multi-modal data.
Sign-RIP improves robust low-rank matrix recovery by preserving norms even with corrupted measurements.
problem Robust low-rank matrix recovery in the presence of corrupted measurements.
method Proposed Sign-RIP, a robust restricted isometry property.
result Sign-RIP guarantees uniform convergence of subdifferentials in robust low-rank matrix recovery.
New method tackles non-smooth tensor data for better recovery.
problem Non-smooth changes in tensor data degrade traditional t-SVD methods.
method Learnable tensor nuclear norm, Alternating Proximal Multiplier Method (APMM), multi-objective tensor recovery framework.
result The proposed method effectively recovers tensor data with non-smooth changes.
Hyperbolic embeddings offer excellent quality with few dimensions when embedding hierarchical data structures like synonym or type hierarchies. Given a tree, we give a combinatorial construction that embeds the tree in hyperbolic space with arbitrarily low distortion without using optimization. On WordNet, our combinat…
Study of Langevin dynamics for tensor PCA recovery in high dimensions.
problem Recovering hidden signal vectors (spikes) from noisy Gaussian tensor observations.
method Langevin dynamics approach for nonconvex optimization.
result Sample complexity matches the single-spike case but degrades for all spikes.
We present a mathematical analysis of a non-convex energy landscape for robust subspace recovery. We prove that an underlying subspace is the only stationary point and local minimizer in a specified neighborhood under a deterministic condition on a dataset. If the deterministic condition is satisfied, we further show t…
Develops PRPCA for smooth image recovery combining low-rank and smoothness.
problem Image matrix recovery under low-rank and smoothness assumptions.
method Projected Robust PCA (PRPCA) framework combining low-rank and smoothness.
result Explicit statistical guarantees for PRPCA, reducing matrix dimensionality.
Paper develops a new algorithm for sparse signal recovery.
problem Sparse signal recovery from noisy observations.
method Iterative Stochastic Optimization using Stochastic Mirror Descent.
result Linear convergence during preliminary phase of the routine.
Study recovers spike order in noisy tensor estimation without SNR assumptions.
problem Estimating multiple signal vectors from noisy tensor observations.
method Gradient flow optimization of a nonconvex function.
result Determines sample complexity for efficient permutation recovery.
This paper improves sample efficiency in noisy inductive matrix completion with side-information.
problem Improving sample efficiency in noisy inductive matrix completion with side-information.
method Nonconvex projected gradient descent algorithm with spectral initialization.
result Achieves linear convergence and stable recovery at a sample complexity governed by the effective side-information dimension.
ReLU networks learn simple models even with many parameters, overcoming traditional wisdom.
problem Generalization of overparameterized neural networks.
method Convex optimization and sparse recovery perspective applied to two-layer ReLU networks with standard weight decay.
result ReLU networks learn simple models that explain the data, analogous to sparse recovery in compressed sensing.
A new method for signal recovery in high dimensions using projections and diffusion models.
problem Recovering a latent signal from noisy observations with unknown support.
method Metric projection estimator based on score matching in a diffusion model.
result The posterior distribution concentrates near the metric projection of the observed signal.
This paper establishes conditions for sparse signal recovery with sparse measurements.
problem Recovering the support of a sparse signal using noisy projections with sparse measurement matrices.
method Establishes sufficient conditions for successful sparse recovery using sparse measurement matrices.
result A phase transition threshold for sparse recovery in the sparse setting is discovered, revealing a trade-off between sampling complexity and measurement sparsity.
This paper improves diffusion models for low-dimensional data.
problem Theoretical foundations of diffusion models are lacking for low-dimensional data.
method Score approximation, estimation, and distribution recovery of diffusion models on low-dimensional data.
result Sample complexity bounds for distribution estimation using diffusion models are provided.
Study generalizes matrix completion with side info in low noise settings.
problem Matrix completion with side information in low noise conditions.
method Inductive matrix completion with i.i.d. subgaussian noise, uniform sampling, and side information.
result Generalization bounds with noise scaling, convergence to zero, and logarithmic dependence on matrix size.
Study on Gaussian-width complexity on statistical manifolds and its applications in learning and recovery.
problem Understanding the geometry of statistical manifolds and its implications for learning and recovery.
method Analysis of Fisher width and inverse-Fisher width, proving their complementary roles and establishing a relation between them.
result Established a sharp relation between Fisher width and inverse-Fisher width, showing they cannot reduce relative to Euclidean scale.
Federated learning supports exact support recovery with minimal communication.
problem Learning the exact support of sparse linear regression in federated learning.
method One-shot communication algorithm for exact support recovery without optimization.
result Polynomial sample complexity and logarithmic number of clients required.
New method recovers clusters in non-convex finite metric spaces with oracle queries.
problem Exact recovery of clusters in non-convex finite metric spaces.
method Introducing (β,γ)-convexity and a deterministic algorithm using oracle queries. result Clusters can be recovered using O(k2logn+k2(6/βγ)dens(X)) same-cluster queries. Study spectral estimators for multi-index models to recover low-dimensional signal subspaces.
problem Recovering low-dimensional signal subspaces in multi-index models.
method Spectral estimators for multi-index models.
result Precise asymptotic characterization of spectral methods' performance, revealing a phase transition for weak recovery.
Paper proposes fast, robust methods for low-rank matrix recovery.
problem Estimating low-rank matrices from incomplete or corrupted data.
method Scaled subgradient methods for nonsmooth, nonconvex formulations.
result Methods converge almost dimension-free and condition-number independent.
We develop an efficient algorithm to find confidence ellipsoids with volume guarantees in high dimensions.
problem Finding robust confidence ellipsoids in high-dimensional data.
method Polynomial time algorithm using primal-dual structure and geometric Brascamp-Lieb inequality.
result Algorithm finds ellipsoids within a O(β)γd volume factor of best β-conditioned ellipsoid. Convex optimization method recovers low-rank matrices from rank-one projections efficiently.
problem Recovering low-rank matrices from limited rank-one projections.
method Unlifted convex optimization with subgradient method.
result The estimator succeeds with high probability if the number of measurements exceeds r2(d1+d2) up to logarithmic factors. In this paper, we consider parameter recovery for non-overlapping convolutional neural networks (CNNs) with multiple kernels. We show that when the inputs follow Gaussian distribution and the sample size is sufficiently large, the squared loss of such CNNs is locally strongly convex in a basin of attraction…
PDGMM-VAE uses adaptive priors for better ICA recovery.
problem Nonlinear ICA recovery of latent source signals.
method Adaptive per-dimension Gaussian mixture model priors in a variational autoencoder.
result PDGMM-VAE effectively recovers source-specific non-Gaussian marginals.
The paper sets information-theoretic lower bounds for neural networks' parameter recovery and excess risk.
problem Establishing sample complexity lower bounds for neural network parameters and excess risk.
method Using information-theoretic tools, the paper proves lower bounds by constructing a generative network.
result Proves information-theoretic lower bounds for exact parameter recovery and positive excess risk.
Manifold embedding algorithms map high-dimensional data down to coordinates in a much lower-dimensional space. One of the aims of dimension reduction is to find intrinsic coordinates that describe the data manifold. The coordinates returned by the embedding algorithm are abstract, and finding their physical or domain-r…
Distributed-OMP recovers sparse vectors with low communication costs.
problem High-dimensional sparse linear regression with limited computation and communication.
method Distributed orthogonal matching pursuit (OMP) scheme.
result Support of the regression vector can be recovered with linear communication per machine and logarithmic in dimension.
New method avoids spurious critical points for low-rank matrix recovery.
problem Low-rank matrix recovery problems on Riemannian manifold.
method Riemannian gradient descent with random initialization.
result Riemannian gradient descent avoids spurious critical points and converges nearly linearly.
New bounds for adaptive control in high dimensions without fixed state space.
problem Adaptive control of linear systems in high or infinite dimensions.
method Novel perturbation bound for certainty equivalence, scaling with prediction error.
result First regret bounds for LQR in infinite dimensional systems, independent of ambient dimension.
We derive an arbitrage free relationship between recovery swap rates, digital default swap spreads and conventional CDS spreads, and argue that the fair forward recovery rate used in recovery swaps must contain a convexity premium over the expected recovery value.
This paper derives sufficient conditions for local recovery of coordinate dictionaries comprising a Kronecker-structured dictionary that is used for representing Kth-order tensor data. Tensor observations are assumed to be generated from a Kronecker-structured dictionary multiplied by sparse coefficient tensors that …
Tyler's M-estimator's phase transition at DS-SNR = 1 is resolved.
problem Robust Subspace Recovery
method Tyler's M-estimator
result TME converges exactly to the true subspace for DS-SNR >= 1 under a new stability condition.
Many applications concern sparse signals, for example, detecting anomalies from the differences between consecutive images taken by surveillance cameras. This paper focuses on the problem of recovering a K-sparse signal x in N dimensions. In the mainstream framework of compressed sensing (CS), the vector x is recovered…
SGD recovers multiple signal vectors in noisy tensor PCA.
problem Estimating multiple signal vectors from noisy tensor observations.
method Online stochastic gradient descent (SGD) in high dimensions with detailed analysis of correlations.
result Sequential elimination of correlations allows recovery of all spikes from Np−2 samples. Paper analyzes IHT's performance in sparse recovery problems.
problem Generalization performance of Iterative Hard Thresholding (IHT).
method Sparse generalization theory under algorithmic stability.
result IHT achieves convergence rates in sparse excess risk.
Study sparse function recovery from indirect noisy observations using ℓ1-regularization.
problem Recovering sparse functions from indirect, noisy observations.
method Proposes an ℓ1-regularized empirical risk minimizer and analyzes its statistical properties. result Established almost-sure consistency and derived high-probability convergence rates in prediction and ℓ1 norms.