Paper addresses concentration of distances for fractional quasi p-norms, identifying conditions for concentration and anti-concentration.
problem Understanding concentration of distances for fractional quasi p-norms in high dimensions.
method Analyzes conditions for concentration and anti-concentration of distances for fractional quasi p-norms.
result Identifies conditions for concentration and anti-concentration of fractional quasi p-norms, ruling out some approaches and specifying conditions for control.
In this note, we derive concentration inequalities for random vectors with subGaussian norm (a generalization of both subGaussian random vectors and norm bounded random vectors), which are tight up to logarithmic factors.
Improved estimation of concentration using half-spaces for adversarial vulnerability.
problem Understanding the concentration of measure phenomenon and its impact on adversarial vulnerability.
method Extending Gaussian Isoperimetric Inequality to non-spherical Gaussian measures and arbitrary ℓ_p-norms, using half-spaces to estimate concentration.
result Proposed method finds tighter intrinsic robustness bounds, providing evidence against concentration as a cause of adversarial vulnerability.
Develops inequalities for high-dimensional linear processes with dependent innovations.
problem Estimating high-dimensional VAR(p) systems and HAC covariance estimation.
method Concentration inequalities for l∞ norm of vector linear processes with sub-Weibull, mixingale innovations. result Obtained concentration bounds for the maximum entrywise norm of lag-h autocovariance matrices. We survey recent results related to the concentration of eigenfunctions. We also prove some new results concerning ball-concentration, as well as showing that eigenfunctions saturating lower bounds for L1-norms must also, in a measure theoretical sense, have extreme concentration near a geodesic.
New method tightens sub-Gaussian concentration inequalities.
problem Estimating variance-type parameters of sub-Gaussian distributions.
method Using sub-Gaussian intrinsic moment norm to maximize normalized moments.
result Provides tighter sub-Gaussian concentration inequalities.
Sharp bounds on quasimode norms on compact space forms.
problem Characterize compact manifolds using quasimode decay rates.
method Analyzes upper and lower bounds of quasimode norms on compact space forms.
result Characterizes compact manifolds of constant curvature using quasimode decay rates.
The paper generalizes product inequalities for random vectors and their applications.
problem Understanding concentration of measure for products of random vectors.
method Develops expressions for the concentration of functionals of random vectors based on product norms.
result Provides generalized Hanson-Wright inequalities and applications to random matrices.
The paper studies how norms of random vectors are preserved by random projections.
problem Understanding how random matrix affects norms of random vectors.
method Proved the distribution of the norm of random vector is preserved by random projection.
result Random matrix preserves the distribution of the norm of random vectors with i.i.d. entries.
Sharp concentration inequalities for sub-Orlicz random variables with phase transition at α=2.
problem Developing concentration inequalities for sub-Orlicz random variables with phase transition.
method New theoretical analysis framework involving variance and min/max functions of Orlicz tails.
result Sharp concentration inequalities with phase transition at α=2 for sub-Orlicz random variables.
The paper introduces a new method for tail bounds of random vectors and matrices.
problem Estimating norms of random vectors and matrices under moment assumptions.
method Variational tail bounds for norms of random vectors and matrices.
result Dimension-free concentration inequalities for various norms of random vectors and matrices.
We consider the problem of approximately reconstructing a partially-observed, approximately low-rank matrix. This problem has received much attention lately, mostly using the trace-norm as a surrogate to the rank. Here we study low-rank matrix reconstruction using both the trace-norm, as well as the less-studied max-no…
Concerns about interpretability, computational resources, and principled inductive priors have motivated efforts to engineer sparse neural models for NLP tasks. If sparsity is important for NLP, might well-trained neural models naturally become roughly sparse? Using the Taxi-Euclidean norm to measure sparsity, we find …
New method approximates complex kernel norms with random features, making learning tractable.
problem Complexity of learning with kernel methods in high dimensions.
method Random features approximations to Fp norms, focusing on p>1. result For p>1, the number of random features required is polynomial in the sample size, making learning tractable. This paper gives new concentration inequalities for the spectral norm of a wide class of matrix martingales in continuous time. These results extend previously established Freedman and Bernstein inequalities for series of random matrices to the class of continuous time processes. Our analysis relies on a new supermarti…
This article concerns upper bounds for L∞-norms of random approximate eigenfunctions of the Laplace operator on a compact aperiodic Riemannian manifold (M,g). We study fλ chosen uniformly at random from the space of L2-normalized linear combinations of Laplace eigenfunctions with eigenvalues in the inte…
Improved location estimation for high-dimensional data with finite sample size.
problem Estimating the shift in high-dimensional data with limited samples.
method Smoothed estimators and bounds on subgamma vectors.
result Convergence to Cramér-Rao bound for finite sample sizes.
In this article, we prove energy quantization for approximate (intrinsic and extrinsic) biharmonic maps into spheres where the approximate map is in LlogL. Moreover, we demonstrate that if the LlogL norm of the approximate maps does not concentrate, the image of the bubbles are connected without necks.
Estimates for eigenfunctions and quasimodes on compact manifolds.
problem Characterizing eigenfunctions and quasimodes on compact manifolds.
method Sharp Lq-estimates for log-quasimodes, focusing on small Lebesgue exponents. result No characterization possible for q>qc. Study confirms fractional norms and quasinorms do not help overcome curse of dimensionality.
problem Overcoming the curse of dimensionality in machine learning.
method Systematic testing of fractional norms and quasinorms (p<1) on classification problems.
result Distance concentration behavior is qualitatively the same for all norms and quasinorms as dimensionality increases.
Study on bending energy of surfaces with curvature concentration, deriving new lower bounds.
problem Analyzing the Willmore energy of surfaces with curvature concentration.
method Using isoperimetric inequalities and framed loops, derive new lower bounds for the bending energy.
result Optimal blowup rates of the Willmore energy when curvature is concentrated.
Quantum ML predicts data with improved speed and accuracy.
problem Predicting data using maximum likelihood in a quantum setting.
method Quantum states embedding and minimization of quantum relative entropy.
result Unified framework for classical and quantum LLMs with performance guarantees.
The paper constructs local solutions concentrating near singular points of spinors.
problem Constructing solutions near singular points of spinors.
method Constructs local solutions parameterized by ε, concentrating near singular points.
result Local solutions concentrate in tubular neighborhoods of singular points, converging to original spinors after renormalization.
Study spectral properties of sparse random graphs to recover latent vectors.
problem Recovering latent vectors in sparse random geometric graphs.
method Analyzes spectral concentration and uses orthogonal polynomial expansions, decoupling, and matrix concentration.
result Sharpens spectral norm bounds and proves exact recovery for Gaussian mixture models.
The paper proves concentration inequalities for two-sample rank processes and applies them to ranking performance criteria.
problem Measuring the performance of ranking statistics between two populations.
method Proves concentration inequalities for two-sample rank processes indexed by VC classes of scoring functions.
result Generalization capacity of empirical maximizers of ranking performance criteria is investigated.
Many problems can be formulated as recovering a low-rank tensor. Although an increasingly common task, tensor recovery remains a challenging problem because of the delicacy associated with the decomposition of higher order tensors. To overcome these difficulties, existing approaches often proceed by unfolding tensors i…
From concentration inequalities for the suprema of Gaussian or Rademacher processes an inequality is derived. It is applied to sharpen existing and to derive novel bounds on the empirical Rademacher complexities of unit balls in various norms appearing in the context of structured sparsity and multitask dictionary lear…
We present a novel notion of outlier, called the Concentration Free Outlier Factor, or CFOF. As a main contribution, we formalize the notion of concentration of outlier scores and theoretically prove that CFOF does not concentrate in the Euclidean space for any arbitrary large dimensionality. To the best of our knowled…
The study analyzes how covariance estimation errors affect the global minimum-variance portfolio under heavy-tailed distributions.
problem The impact of covariance estimation errors on the global minimum-variance portfolio under heavy-tailed distributions.
method Characterization of covariance-estimation error's effect on GMVP suboptimality, derivation of regret identity and bound, application to heavy-tailed returns.
result The decision geometry of GMVP regret is invariant to a (p-1)-dimensional projection of the error matrix, with invariance to the covariance-scale direction as an exact special case.
New method learns shared structures in non-linear tasks.
problem Learning shared linear representations in non-linear tasks.
method Convex optimization with structural assumptions.
result Rank and clustered estimators recover shared structures under certain conditions.
We obtain a Bernstein-type inequality for sums of Banach-valued random variables satisfying a weak dependence assumption of general type and under certain smoothness assumptions of the underlying Banach norm. We use this inequality in order to investigate in the asymptotical regime the error upper bounds for the broad …
We provide a necessary and sufficient condition that Lp-norms, 2<p<6, of eigenfunctions of the square root of minus the Laplacian on 2-dimensional compact boundaryless Riemannian manifolds M are small compared to a natural power of the eigenvalue λ. The condition that ensures this is that their L2 norms ove…
Recently theoretical guarantees have been obtained for matrix completion in the non-uniform sampling regime. In particular, if the sampling distribution aligns with the underlying matrix's leverage scores, then with high probability nuclear norm minimization will exactly recover the low rank matrix. In this article, we…
New study shows how model complexity affects test risk, challenging classical theory.
problem Understanding how test risk scales with model complexity for large over-parametrized deep networks.
method Developed norm-based capacity measures for random features based estimators, providing precise characterization of estimator's norm concentration and test error.
result Predicted learning curve shows a phase transition from under- to over-parameterization, confirming classical U-shaped behavior with appropriate capacity measures.
Given i.i.d. observations of a random vector X∈Rp, we study the problem of estimating both its covariance matrix Σ∗, and its inverse covariance or concentration matrix {Θ∗=(Σ∗)−1.} We estimate Θ∗ by minimizing an ℓ1-penalized log-determinant Bregman divergence; in the multivariate G…
Study on extremizers for Sobolev inequality on curved manifolds.
problem Existence of extremizers for the sharp p-Sobolev inequality on Riemannian manifolds with nonnegative curvature. method Nonsmooth concentration compactness methods and Mosco-convergence results for Cheeger energy.
result Almost extremal functions are close to radial Euclidean bubbles and almost zero globally under nonnegative curvature.
Paper provides finite-sample guarantees for Wasserstein DRO without dimensionality curse.
problem Tackles empirical success of Wasserstein DRO in operations and ML with performance guarantees.
method Develops non-asymptotic framework for analyzing out-of-sample performance and generalization bound.
result First finite-sample guarantee for generic Wasserstein DRO problems without curse of dimensionality.
Study on estimating distances between covariance operators and Gaussian processes.
problem Estimating distances between covariance operators and Gaussian processes.
method Riemannian distances, concentration results for Hilbert space-valued random variables, RKHS covariance and cross-covariance operators.
result Both distances converge in the Hilbert-Schmidt norm and can be consistently and efficiently estimated.
We address the problem of computing reliable policies in reinforcement learning problems with limited data. In particular, we compute policies that achieve good returns with high confidence when deployed. This objective, known as the \emph{percentile criterion}, can be optimized using Robust MDPs~(RMDPs). RMDPs general…
New property ensures neural networks generalize well with limited data.
problem Limited training data limits model generalization in neural networks.
method Introduces NeuRIP, a uniform concentration event for ReLU networks.
result All shallow ReLU networks generalize uniformly if they achieve NeuRIP.
New method shows stochastic momentum can converge quickly on optimization problems.
problem Improving convergence of stochastic optimization methods.
method Stochastic heavy ball momentum with minibatching.
result Stochastic heavy ball momentum retains fast linear rate on quadratic problems.
Extends Mahalanobis distance to Banach spaces for anomaly detection.
problem Anomaly detection in infinite-dimensional spaces.
method Generalizes Mahalanobis distance to Banach spaces via Cameron-Martin norm and variance norm.
result Kernelized nearest-neighbour Mahalanobis distance outperforms traditional methods for time series novelty detection.
Paper improves distributed mean estimation and variance reduction without relying on input norm.
problem Distributed mean estimation and variance reduction with large input norms.
method Quantization and lattice theory connection for improved error bounds.
result Output error bounds depend only on input distance, not norm.
SGD noise helps select flat minima by concentrating in sharp directions and being proportional to loss value.
problem Understanding the implicit regularization of SGD and selecting flat minima in over-parameterized models.
method Relating SGD's linear stability to the Frobenius norm of the Hessian and analyzing the alignment property of SGD noise.
result Flat minima are linearly stable for SGD, and their sharpness is bounded independently of model size and sample size.
DoRA improves adaptation efficiency for large models by factoring norms and fusing kernels.
problem High-rank DoRA is computationally expensive and infeasible on common GPUs.
method Factored norms and fused Triton kernels to reduce memory and speed up computation.
result Fused implementation is up to 2.0x faster for inference and 1.9x faster for gradient computation.
Tensor rank and low-rank tensor decompositions have many applications in learning and complexity theory. Most known algorithms use unfoldings of tensors and can only handle rank up to n⌊p/2⌋ for a p-th order tensor in Rnp. Previously no efficient algorithm can decompose 3rd order ten…
Unified PAC-Bayesian framework for deep learning generalization.
problem Limitations of existing PAC-Bayesian norm-based bounds for deep neural networks.
method Unified framework using anisotropic Gaussian posteriors and sensitivity matrix.
result Comparable or tighter generalization bounds compared to state-of-the-art approaches.
Unified analysis of MPLE for Ising models with bounded operator norm or infinity norm.
problem Estimating Ising models in Total Variation distance with limited samples.
method Maximum Pseudo-Likelihood Estimator (MPLE) for two general classes of Ising models.
result Unified framework for polynomial-time estimation in TV distance for two general classes of Ising models.