New norms derived from box-norm improve multitask learning performance.
problem Improving multitask learning performance in matrix completion and prediction.
method Derived new norms (box-norm, spectral k-support, spectral box-norm) and improved algorithms to compute them.
result New norms provide state-of-the-art performance in matrix completion and multitask learning.
Paper generalizes spectral k k k -support norm for better matrix completion performance.
problem Matrix completion problems with spectral decay.
method Introduced spectral ( k , p ) (k,p) ( k , p ) -support norm and developed conditional gradient method. result Performance improvement on matrix completion benchmarks with varying p p p . The k k k -support norm is a regularizer which has been successfully applied to sparse vector prediction problems. We show that it belongs to a general class of norms which can be formulated as a parameterized infimum over quadratics. We further extend the k k k -support norm to matrices, and we observe that it is a special …
We derive a novel norm that corresponds to the tightest convex relaxation of sparsity combined with an ℓ 2 \ell_2 ℓ 2 penalty. We show that this new {\em k k k -support norm} provides a tighter relaxation than the elastic net and is thus a good replacement for the Lasso or the elastic net in sparse prediction problems. Through …
We propose a Generalized Dantzig Selector (GDS) for linear models, in which any norm encoding the parameter structure can be leveraged for estimation. We investigate both computational and statistical aspects of the GDS. Based on conjugate proximal operator, a flexible inexact ADMM framework is designed for solving GDS…
IRKSN algorithm achieves sparse recovery with wider applicability conditions.
problem Sparse recovery challenges due to NP-hard nature and restrictive conditions.
method IRKSN algorithm based on k k k -support norm regularizer. result Achieves sparse recovery with explicit constants and standard linear rate.
Unified analysis of matrix completion with structural constraints.
problem Matrix completion under general structural constraints.
method Unified analysis using generic chaining and characterizations of Gaussian widths.
result Unified upper bounds on sample complexity and estimation error.
Dropout improves neural network performance by promoting low-rank solutions.
problem Improving neural network generalization through regularization.
method Analyzing Dropout, DropBlock, and DropConnect as regularizers for linear networks and extending to deep networks.
result Dropout, DropBlock, and DropConnect induce low-rank solutions and can be computed in closed form.
Spectral norm regularization improves deep learning models' generalizability.
problem High sensitivity to input perturbation degrades deep learning model performance.
method Spectral norm regularization, penalizing high spectral norm of weight matrices.
result Models trained with spectral norm regularization show better generalizability.
Exact spectral norm regularization improves neural network generalization.
problem Improving neural network generalization while protecting against noise.
method Exact spectral norm regularization of the Jacobian.
result Improved generalization performance compared to previous methods.
Equivalence of norms on manifolds with curvature bounds established.
problem Establishing equivalence of norms on manifolds with bounded sectional curvature.
method Using spectral projector and thickness condition for subsets.
result Constant in equivalence depends only on manifold dimension, curvature bounds, and frequency threshold.
New bounds improve deep learning performance efficiently.
problem Improving generalization and robustness of deep learning models.
method Deriving four provable upper bounds on spectral norm of convolution layers, differentiable and efficient.
result Minimum of four bounds is a tight, differentiable and efficient upper bound on spectral norm.
We introduce here a natural functional associated to any b ∈ Q H ∗ ( M , ω ) b \in QH_* (M, ω) b ∈ Q H ∗ ( M , ω ) : \emph{spectral length functional}, on the space of "generalized paths" in Ham ( M , ω ) \text {Ham}(M, ω) Ham ( M , ω ) , closely related to both the Hofer length functional and spectral invariants and establish some of its properties. This functional is smooth on its…
The study connects norms and filtrations on section rings of projective manifolds.
problem Understanding norms and filtrations on section rings of polarized projective manifolds.
method Analyzes submultiplicative norms and their equivalence to sup-norms, discusses applications to spectral theory and holomorphic extension.
result Injective and projective tensor norms on symmetric algebras are asymptotically equivalent.
Muon dynamics study uses spectral Wasserstein flow for optimization stability.
problem Optimizing deep learning models with gradient normalization.
method Introduces Spectral Wasserstein distances for matrix flows, proving equivalence with Benamou--Brenier formulation.
result Gradient-flow interpretation of mean-field normalized training dynamics.
Spectral regularization simplifies sequence models by focusing on grammatical simplicity.
problem Sequence modeling challenges in learning tasks.
method Introduces spectral regularization based on Hankel matrices and trace norm, addressing bi-infinite matrices with an unbiased estimator.
result Demonstrates spectral regularization's potential benefits on Tomita grammars.
We show that the spectral norm of a random n 1 × n 2 × ⋯ × n K n_1\times n_2\times \cdots \times n_K n 1 × n 2 × ⋯ × n K tensor (or higher-order array) scales as O ( ( ∑ k = 1 K n k ) log ( K ) ) O\left(\sqrt{(\sum_{k=1}^{K}n_k)\log(K)}\right) O ( ( ∑ k = 1 K n k ) log ( K ) ) under some sub-Gaussian assumption on the entries. The proof is based on a covering number argument. Since the spectral norm is dual to the tensor…
Muon optimizer improves deep learning with spectral norm constraints.
problem Improving optimization algorithms in deep learning.
method Theoretical analysis of Muon optimizer within the Lion- K \mathcal{K} K family. result Muon implicitly solves an optimization problem enforcing spectral norm constraints.
Study on neural networks' sample complexity with one hidden layer.
problem Understanding how sample complexity is affected by network architecture and norm constraints.
method Norm-based uniform convergence bounds for scalar-valued one-hidden-layer networks, focusing on spectral and Frobenius norms.
result Spectral norm control is insufficient for uniform convergence guarantees, but Frobenius norm control is sufficient, with conditions.
Improved singular value approximation for convolutional layers.
problem Improving accuracy of singular value approximation for linear convolutional layers.
method Developed a new spectral density matrix method for singular value approximation with improved accuracy and reduced computational complexity.
result Obtained moderate improvement in singular value distribution compared to circular approximation.
New bounds adaptively control spectral complexity of trained Transformers.
problem Understanding why Transformers generalize well in machine learning.
method Spectrum-adaptive post hoc generalization bounds for multi-layer Transformers.
result Bounds adaptively trade off spectral complexity against dimension and depth factors.
Extended Gauss-Markov theorem for linear estimation with bounded bias.
problem Linear estimation with bounded bias operator.
method Derive optimal estimator formulas for Nuclear and Spectral norms, analyze generalization error.
result Cross-validated Nuclear and Spectral regressors outperform Ridge regression in simulations.
Study spectral distribution of twisted Laplacian on high genus hyperbolic surfaces.
problem Estimating spectral distribution of twisted Laplacian on hyperbolic surfaces.
method Estimate spectral distribution by supremum norm of harmonic form; show small supremum norm for high genus surfaces; prove uniform Weyl law.
result Prove uniform Weyl law for real parts of spectrum on high genus hyperbolic surfaces.
The study proves properties of spectral selectors for contact manifolds and applies them to contact big fibers and geodesics.
problem Properties of spectral selectors for contact manifolds.
method Algebraic properties of spectral selectors for strongly orderable contact manifolds.
result Established contact big fiber theorem and constructed norms on contactomorphism group universal cover.
New analysis shows how data distribution affects distributed SGD performance.
problem Understanding how data distribution impacts convergence rates in distributed SGD.
method Proposes a new analysis method that relates convergence rates to the spectral norm of the sample covariance matrix.
result Data distribution significantly influences convergence rates in distributed SGD.
The paper improves tensor completion bounds using spectral gap.
problem Theoretical limitations in tensor completion, especially for deterministic sampling.
method Bounding the generalization error of tensor completion methods using spectral gap.
result Improved bounds on tensor completion error, reducing rank dependence.
Proposes a new spectral embedding norm for better cluster separation in unbalanced datasets.
problem Challenges traditional spectral clustering in unbalanced datasets, especially in anomaly detection.
method Introduces the spectral embedding norm, summing the squared values of the first I I I normalized eigenvectors. result Demonstrates improved performance in separating clusters from background in various datasets.
Optimal estimates for spectral projection norms on compact manifolds.
problem Estimating norms of spectral projection operators on compact manifolds.
method Analyzing spectral windows with logarithmic growth and applying curvature constraints.
result Optimal estimates for L 2 ( M ) o L q ( M ) L^2(M) o L^q(M) L 2 ( M ) o L q ( M ) norms are derived, saturating on flat or negatively curved manifolds. This work simplifies proximal mapping for low-rank norms.
problem Efficient computation of proximal mappings for low-rank inducing norms.
method Reduces proximal mapping to nested binary search, solving simpler problems analytically.
result Simplified computation of proximal mappings for various norms.
The study improves norms of spectral projectors on specific surfaces.
problem Improving the L 2 o L ∞ L^2 o L^{\infty} L 2 o L ∞ norm of spectral projectors on certain surfaces. method Quantum Integrability, joint basis of eigenfunctions, Lagrangian oscillatory functions, caustics, BKW decay.
result Polynomial improvement on the L 2 o L ∞ L^2 o L^{\infty} L 2 o L ∞ norm for generic simple spheres of revolution and the Euclidean disk. Proves spectral gap bounds for Teichmüller geodesics on flat surfaces.
problem Quantify spectral gaps for Teichmüller geodesics.
method Bounding spectral gaps in terms of geometric quantities on flat surfaces.
result Quantitative non-uniform hyperbolicity of Teichmüller geodesic flow.
New bounds for CNNs show better generalization than previous models.
problem Improving understanding of CNNs' generalization ability.
method Proposed tighter generalization bounds for CNNs by exploiting the sparse and permutation structure of weight matrices and spectral norms of convolution operations.
result Theoretical and experimental results show tighter bounds for CNNs than existing bounds.
Algorithm estimates covariance from noisy data efficiently.
problem Estimating covariance from a noisy set of points.
method Spectral techniques for list-decodable covariance estimation.
result Efficient algorithm with poly(1/α) sample and time complexity.
Paper solves TRPCA problem for tensor data with new tensor nuclear norm.
problem Exact recovery of tensor low-rank and sparse components.
method Introduces tensor-tensor product and new tensor nuclear norm to solve TRPCA.
result The new tensor nuclear norm guarantees exact recovery of tensor data.
Recent research in off-the-grid compressed sensing (CS) has demonstrated that, under certain conditions, one can successfully recover a spectrally sparse signal from a few time-domain samples even though the dictionary is continuous. In particular, atomic norm minimization was proposed in \cite{tang2012csotg} to recove…
Paper provides a performance guarantee for spectral clustering.
problem Finding the global solution to the minimum ratio cut problem.
method Two-step spectral clustering method with a rounding step, analyzed using two-to-infinity norm perturbation bounds.
result Spectral clustering is guaranteed to output the global solution under certain conditions.
This paper proves a generalization bound for complex-valued neural networks scaling with spectral complexity.
problem Ensuring the performance of complex-valued neural networks on unseen data.
method Theoretical derivation using Maurey Sparsification Lemma and Dudley Entropy Integral, empirical validation on various datasets.
result The spectral complexity of weight matrices is a significant factor in the generalization ability of complex-valued neural networks.
This paper improves covariance estimation with minimal data.
problem Estimating covariance from few compressive measurements.
method Back-projections of compressive samples for consistent estimation.
result Single linear measurement suffices for consistent covariance estimation.
Study precise sample covariance error for Gaussian centered data.
problem Precise characterization of sample covariance error for Gaussian data.
method Developed a Random Duality Theory (RDT) framework to determine upper and lower bounds.
result Upper and lower bounds match in large-dimensional contexts, matching the spectral norm's limiting value.
Study extends neural network approximation to time-varying PDEs using Fourier-Lebesgue spaces.
problem Limitation to static PDEs and different time-domain regularity.
method Extend spectral Barron spaces to anisotropic weighted Fourier-Lebesgue spaces, measure approximation error in Bochner-Sobolev norm.
result Established bound on approximation rate for functions in anisotropic weighted Fourier-Lebesgue spaces.
New spectral results on lens spaces and related spaces.
problem Spectral theory of lens spaces and related locally symmetric spaces.
method Use of Molien's formula and manipulation of one-norm generating function associated to a congruence lattice.
result First examples of Riemannian manifolds isospectral on p-forms for all p but not strongly isospectral were constructed.
Estimates spectral projections restricted to uniformly embedded submanifolds.
problem Estimating spectral projections on submanifolds of manifolds with nonpositive curvature.
method Estimates the L 2 ( M ) o L q ( Σ ) L^2(M) o L^q(Σ) L 2 ( M ) o L q ( Σ ) norm of spectral projection operators. result Sharp spectral projection estimates for small spectral windows.
Study of unitary and groupoid orbits of normal operators, focusing on manifold structures and spectral conditions.
problem Understanding the manifold structures of orbits of normal operators under different norm topologies.
method Unified treatment of unitary and groupoid orbits, using moment maps and conditional expectations.
result Differentiable structures for orbits and necessary spectral conditions for norm closure and submanifold properties.
Defines spectral selectors on lens spaces for contactomorphisms.
problem Understanding the geometry of contactomorphism groups on lens spaces.
method Using Givental's non-linear Maslov index, defines spectral selectors.
result Standard Reeb flow is a geodesic for specific lens spaces.
In this paper, we consider low rank matrix estimation using either matrix-version Dantzig Selector A ^ λ d \hat{A}_λ^d A ^ λ d or matrix-version LASSO estimator A ^ λ L \hat{A}_λ^L A ^ λ L . We consider sub-Gaussian measurements, i . e . i.e. i . e . , the measurements X 1 , … , X n ∈ R m × m X_1,\ldots,X_n\in\mathbb{R}^{m\times m} X 1 , … , X n ∈ R m × m have i . i . d . i.i.d. i . i . d . sub-Gaussian entries. Suppose $\textrm…
New SAM method improves model robustness with spectral inner perturbation and Muon optimizer.
problem Improving model robustness to small parameter perturbations.
method Introducing a spectral inner perturbation step in SAM combined with Muon optimizer.
result Spectral inner perturbation combined with Muon optimizer achieves best validation accuracy on ImageNet-1K.
Study spectral norms of random kernel matrices for privacy applications.
problem Analyzing privacy in non-parametric regression methods.
method Investigate spectral norms of random kernel matrices and their applications to privacy.
result Obtain tight upper bounds on spectral norms of random kernel matrices.
A new model reduces noise and speeds up subspace segmentation.
problem Subspace segmentation from noisy data.
method Group norm regularized factorization model (GNRFM) with AALM algorithm.
result The method is faster and more robust to noise.