One of the challenges in the study of generative adversarial networks is the instability of its training. In this paper, we propose a novel weight normalization technique called spectral normalization to stabilize the training of the discriminator. Our new normalization technique is computationally light and easy to in…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “Spectral Normalization”
A new method speeds up spectral normalization for neural nets.
problem Efficiently controlling the spectral norm of convolutional layers.
method Depthwise separable convolutions with spectral normalization.
result Significant reduction in computational and memory costs.
Spectral normalization stabilizes GANs by controlling gradient explosion and vanishing.
problem Stability and sample quality issues in GAN training.
method Spectral normalization controls gradient explosion and vanishing, improving GAN training stability and sample quality.
result Bidirectional Scaled Spectral Normalization (BSSN) outperforms standard spectral normalization in sample quality and training stability.
Spectral clustering is a technique that clusters elements using the top few eigenvectors of their (possibly normalized) similarity matrix. The quality of spectral clustering is closely tied to the convergence properties of these principal eigenvectors. This rate of convergence has been shown to be identical for both th…
New method improves GAN training stability and quality.
problem Improving training stability and sample quality in GANs.
method Proposes a new method for Lipschitz continuity in GANs that is efficient and unbiased.
result Demonstrates the effectiveness of the new method in various GAN training scenarios.
The multiplier spectral curve of a conformal torus in the 4-sphere is essentially, see arXiv:0712.2311, given by all Darboux transforms of the conformal torus. In the particular case when the conformal immersion is a Hamiltonian stationary torus in Euclidean 4-space, the left normal of the immersion is harmonic, hence …
Muon dynamics study uses spectral Wasserstein flow for optimization stability.
problem Optimizing deep learning models with gradient normalization.
method Introduces Spectral Wasserstein distances for matrix flows, proving equivalence with Benamou--Brenier formulation.
result Gradient-flow interpretation of mean-field normalized training dynamics.
Deep neural networks (DNNs) have set benchmarks on a wide array of supervised learning tasks. Trained DNNs, however, often lack robustness to minor adversarial perturbations to the input, which undermines their true practicality. Recent works have increased the robustness of DNNs by fitting networks using adversarially…
Proposes MSN to improve DNN performance and speed.
problem Improving Deep Learning model regularization and performance.
method Empirical approach to study Spectral Normalization (SN) and Mean Spectral Normalization (MSN).
result MSN significantly improves DNN performance and speed.
Solves the Wiegold problem by showing free products of left-orderable groups have normal rank > 1.
problem Wiegold problem about groups of normal rank > 1
method Topological argument and intricate construction of left-orders
result Free products of nontrivial left-orderable groups have normal rank > 1
Bayesian model improves traffic prediction with uncertainty estimates.
problem Lack of uncertainty estimates in deep-learning traffic models.
method Proposes a Bayesian recurrent neural network with spectral normalization.
result Spectral normalization improves uncertainty estimates and generalizability.
Develops an oblique projection technique to approximate a foliation for non-normal dynamics.
problem Modeling dynamics far from a primary Spectral Submanifold (SSM) in non-normal systems.
method Oblique projection technique based on experimental data.
result Approximates a stable invariant foliation for non-normal dynamics efficiently.
Generative models create paintings that match training data.
problem Creating realistic paintings using machine learning.
method Used Spectral Normalization GAN (SN-GAN) and SN-GAN with Gradient Penalty to generate paintings.
result SN-GAN produced paintings most comparable to the training dataset.
The paper proves geometric and spectral alignment for deep neural networks.
problem Understanding the singular spectra of deep neural network layers.
method Proves deterministic quotient-geometric estimates for singular spectra of Frobenius-normalized layer factors.
result Exact power-law spectra form a trace-normalized Cartan orbit under Frobenius normalization.
The study of spectral-tightness in Riemannian manifolds and its topological implications.
problem Understanding the spectral properties of Riemannian manifolds and their coverings.
method Analyzing the fundamental group and the Euclidean local de Rham factor to characterize spectral-tightness.
result Spectral-tightness is a topological property of the fundamental group, and it can be characterized by the dimension of the Euclidean local de Rham factor.
An important form of prior information in clustering comes in form of cannot-link and must-link constraints. We present a generalization of the popular spectral clustering technique which integrates such constraints. Motivated by the recently proposed -spectral clustering for the unconstrained problem, our method is…
This paper presents a margin-based multiclass generalization bound for neural networks that scales with their margin-normalized "spectral complexity": their Lipschitz constant, meaning the product of the spectral norms of the weight matrices, times a certain correction factor. This bound is empirically investigated for…
We study spectral asymptotics for the Laplace operator on differential forms on a Riemannian foliated manifold equipped with a bundle-like metric in the case when the metric is blown up in directions normal to the leaves of the foliation. The asymptotical formula for the eigenvalue distribution function is obtained. Th…
We prove the existence of metrics maximizing the first eigenvalue normalized by area on closed, non-orientable surfaces assuming two spectral gap conditions. These spectral gap conditions are proved by the authors in \cite{MS3}.
Study beta function for convex billiard maps, linking spectral invariants.
problem Understanding spectral invariants of convex billiard maps.
method Birkhoff normal form via constructive generating functions, explicit beta function formula.
result Linked spectral invariants to beta function for convex billiard maps.
This work uses neural density estimation to analyze laser-induced breakdown spectroscopy data, enabling accurate predictions and uncertainty quantification.
problem Inference of probability densities in high-dimensional spectral data is often intractable.
method Normalizing flows on structured spectral latent spaces for density estimation and uncertainty quantification.
result The approach enables generation of realistic spectral samples and accurate prediction of state vectors with well-calibrated uncertainties.
For general Riemannian foliations, spectral asymptotics of the Laplacian is studied when the metric on the ambient manifold is blown up in directions normal to the leaves (adiabatic limit). The number of ``small'' eigenvalues is given in terms of the differentiable spectral sequence of the foliation. The asymptotics of…
ULES embeds dynamic networks with stability guarantees.
problem Stability of time-varying node embeddings in evolving networks.
method Unfolded Laplacian Spectral Embedding (ULSE) using normalized Laplacian operators.
result ULES satisfies cross-sectional and longitudinal stability under dynamic stochastic block model.
Develops spectral estimators for network structure with nodal covariates.
problem Identifying observed and unobserved factors affecting network structure.
method Spectral estimators for unobserved blocks and covariates in stochastic blockmodels.
result Asymptotic normality of estimators and superior performance compared to existing methods.
New complexity measure shows similar generalization bounds for CNNs and non-CNNs.
problem Understanding why CNNs generalize well despite fitting random labels.
method Theoretical and empirical investigation of spectral complexity measure insensitivity to CNN invariances.
result Spectral complexity measure results in the same upper bound complexity estimates for CNNs and non-CNNs, contradicting common intuition.
Sparse and Smooth: improved guarantees for Spectral Clustering in the Dynamic Stochastic Block Modelstat.ML
Improved spectral clustering guarantees for dynamic stochastic block models.
problem Analyzing Spectral Clustering in dynamic stochastic block models.
method Extending guarantees to sparse and smooth DSBM, linking sparsity and smoothness.
result Improved error bounds for consistent recovery in dynamic DSBM.
Study of unitary and groupoid orbits of normal operators, focusing on manifold structures and spectral conditions.
problem Understanding the manifold structures of orbits of normal operators under different norm topologies.
method Unified treatment of unitary and groupoid orbits, using moment maps and conditional expectations.
result Differentiable structures for orbits and necessary spectral conditions for norm closure and submanifold properties.
High-dimensional inference for sparse spectral precision matrices
problem Inference on the spectral precision matrix at a fixed frequency
method Full likelihood-based inference using neighboring discrete Fourier transforms
result Simultaneous control of regularization, finite-sample truncation, and smoothing biases
Spectral algorithms improve under covariate shift with novel weighted techniques.
problem Improving spectral algorithms' performance under covariate shift.
method Analysis of spectral algorithms in non-parametric regression over RKHS, proposing a weighted spectral algorithm with clipped weights.
result Normalized weighted spectral algorithm achieves optimal capacity-independent convergence rates, and clipped weights can approach optimal capacity-dependent rates.
Spectral risk measures are attractive risk measures as they allow the user to obtain risk measures that reflect their subjective risk-aversion. This paper examines spectral risk measures based on an exponential utility function, and finds that these risk measures have nice intuitive properties. It also discusses how th…
A new method for spectral barycentre of graph datasets.
problem Creating a summary graph from a set of graphs with community structure.
method Using multiscale spectral distance based on normalized graph Laplacian eigenvalues.
result The barycentre inherits the topological structure of the graphs in the sample dataset.
Spectral clustering with edge counting detects communities in sparse models.
problem Detecting communities in sparse latent space models.
method Spectral clustering followed by edge counting.
result Algorithm achieves consistency and optimality for a broad class of models.
Clustering of data sets is a standard problem in many areas of science and engineering. The method of spectral clustering is based on embedding the data set using a kernel function, and using the top eigenvectors of the normalized Laplacian to recover the connected components. We study the performance of spectral clust…
A new training method for GANs improves image generation quality.
problem Training GANs is challenging and unstable.
method Consistency regularization to stabilize GAN training.
result CR-GAN achieves best FID scores for unconditional image generation.
Batch normalization prevents rank collapse in deep networks, improving training stability.
problem Rank collapse in randomly initialized deep networks with increasing depth.
method Investigates spectral instabilities in random matrices and uses batch normalization to avoid rank collapse.
result Batch normalization prevents rank collapse in both linear and ReLU networks, improving training stability.
Networks or graphs can easily represent a diverse set of data sources that are characterized by interacting units or actors. Social networks, representing people who communicate with each other, are one example. Communities or clusters of highly connected actors form an essential feature in the structure of several emp…
A new measure quantifies how risk-averse different risk measures are.
problem Measuring the degree of risk aversion among different risk measures.
method Two axioms: normalization and linearity. Two formulas for the functional.
result Quantifies the degree of risk aversion among spectral risk measures.
We present a novel spectral embedding of graphs that incorporates weights assigned to the nodes, quantifying their relative importance. This spectral embedding is based on the first eigenvectors of some properly normalized version of the Laplacian. We prove that these eigenvectors correspond to the configurations of lo…
In this paper, we propose guaranteed spectral methods for learning a broad range of topic models, which generalize the popular Latent Dirichlet Allocation (LDA). We overcome the limitation of LDA to incorporate arbitrary topic correlations, by assuming that the hidden topic proportions are drawn from a flexible class o…
Sharp spectral estimates for negatively curved foliations.
problem Estimating the bottom of the spectrum of Riemannian foliations.
method Analyzing the normal exponential map and using it to derive spectral estimates.
result Sharp estimates for the bottom of the spectrum of Riemannian foliations.
This paper establishes the consistency of spectral approaches to data clustering. We consider clustering of point clouds obtained as samples of a ground-truth measure. A graph representing the point cloud is obtained by assigning weights to edges based on the distance between the points they connect. We investigate the…
New Hermite approximations accelerate convergence with adaptive coordinate transformations.
problem Accelerating convergence of spectral approximations for Hermite expansions.
method Using normalizing flows for adaptive coordinate transformations and deriving error estimates.
result Error estimates for Hermite expansions under adaptive coordinate transformations.
This article concerns new off-diagonal estimates on the remainder and its derivatives in the pointwise Weyl law on a compact n-dimensional Riemannian manifold. As an application, we prove that near any non self-focal point, the scaling limit of the spectral projector of the Laplacian onto frequency windows of constant …
Probabilistic Skip Connections for Deterministic Uncertainty Quantification in Deep Neural Networkscs.LG
Proposes PSCs for UQ in deep nets without retraining.
problem Estimating uncertainty in deep nets with a single pass.
method Identifies sensitive, smooth intermediate layer, fits probabilistic model.
result PSCs achieve UQ and OOD detection performance matching existing methods.
A new method boosts graph neural networks by preventing over-smoothing and over-squashing.
problem Graph Neural Networks struggle with long-range signals and over-smoothing/over-squashing.
method Proposes PowerEmbed, a layer-wise normalization technique inspired by spectral graph embedding.
result PowerEmbed prevents over-smoothing and avoids over-squashing, improving performance on heterophilous graphs.
Paper establishes statistical inference for pairwise comparison models.
problem Statistical inference for pairwise comparison models when the number of subjects diverges.
method Identifies Fisher information matrix as a weighted graph Laplacian for asymptotic normality.
result Near-optimal asymptotic normality result for maximum likelihood estimator.
Spectral clustering achieves strong consistency in the stochastic block model under certain conditions.
problem Achieving strong consistency in spectral clustering for the stochastic block model.
method Entrywise analysis of the Fielder eigenvector of graph Laplacians.
result Spectral clustering achieves exact recovery of hidden communities under matching information-theoretic limits.
Frölicher inequalitiesmath.DG
Proves Frölicher inequality on complex manifolds.
problem Calculating Betti and Hodge numbers.
method Uses spectral projectors of to build an injection.
result New proof of classical Frölicher inequality.