One of the challenges in the study of generative adversarial networks is the instability of its training. In this paper, we propose a novel weight normalization technique called spectral normalization to stabilize the training of the discriminator. Our new normalization technique is computationally light and easy to in…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Spectral normalization”
A new method speeds up spectral normalization for neural nets.
problem Efficiently controlling the spectral norm of convolutional layers.
method Depthwise separable convolutions with spectral normalization.
result Significant reduction in computational and memory costs.
Spectral clustering is a technique that clusters elements using the top few eigenvectors of their (possibly normalized) similarity matrix. The quality of spectral clustering is closely tied to the convergence properties of these principal eigenvectors. This rate of convergence has been shown to be identical for both th…
Spectral normalization stabilizes GANs by controlling gradient explosion and vanishing.
problem Stability and sample quality issues in GAN training.
method Spectral normalization controls gradient explosion and vanishing, improving GAN training stability and sample quality.
result Bidirectional Scaled Spectral Normalization (BSSN) outperforms standard spectral normalization in sample quality and training stability.
The multiplier spectral curve of a conformal torus in the 4-sphere is essentially, see arXiv:0712.2311, given by all Darboux transforms of the conformal torus. In the particular case when the conformal immersion is a Hamiltonian stationary torus in Euclidean 4-space, the left normal of the immersion is harmonic, hence …
Muon dynamics study uses spectral Wasserstein flow for optimization stability.
problem Optimizing deep learning models with gradient normalization.
method Introduces Spectral Wasserstein distances for matrix flows, proving equivalence with Benamou--Brenier formulation.
result Gradient-flow interpretation of mean-field normalized training dynamics.
Deep neural networks (DNNs) have set benchmarks on a wide array of supervised learning tasks. Trained DNNs, however, often lack robustness to minor adversarial perturbations to the input, which undermines their true practicality. Recent works have increased the robustness of DNNs by fitting networks using adversarially…
Lipschitz continuity recently becomes popular in generative adversarial networks (GANs). It was observed that the Lipschitz regularized discriminator leads to improved training stability and sample quality. The mainstream implementations of Lipschitz continuity include gradient penalty and spectral normalization. In th…
Solves the Wiegold problem by showing free products of left-orderable groups have normal rank > 1.
problem Wiegold problem about groups of normal rank > 1
method Topological argument and intricate construction of left-orders
result Free products of nontrivial left-orderable groups have normal rank > 1
Bayesian model improves traffic prediction with uncertainty estimates.
problem Lack of uncertainty estimates in deep-learning traffic models.
method Proposes a Bayesian recurrent neural network with spectral normalization.
result Spectral normalization improves uncertainty estimates and generalizability.
Develops an oblique projection technique to approximate a foliation for non-normal dynamics.
problem Modeling dynamics far from a primary Spectral Submanifold (SSM) in non-normal systems.
method Oblique projection technique based on experimental data.
result Approximates a stable invariant foliation for non-normal dynamics efficiently.
The paper proves geometric and spectral alignment for deep neural networks.
problem Understanding the singular spectra of deep neural network layers.
method Proves deterministic quotient-geometric estimates for singular spectra of Frobenius-normalized layer factors.
result Exact power-law spectra form a trace-normalized Cartan orbit under Frobenius normalization.
The study of spectral-tightness in Riemannian manifolds and its topological implications.
problem Understanding the spectral properties of Riemannian manifolds and their coverings.
method Analyzing the fundamental group and the Euclidean local de Rham factor to characterize spectral-tightness.
result Spectral-tightness is a topological property of the fundamental group, and it can be characterized by the dimension of the Euclidean local de Rham factor.
An important form of prior information in clustering comes in form of cannot-link and must-link constraints. We present a generalization of the popular spectral clustering technique which integrates such constraints. Motivated by the recently proposed -spectral clustering for the unconstrained problem, our method is…
This paper presents a margin-based multiclass generalization bound for neural networks that scales with their margin-normalized "spectral complexity": their Lipschitz constant, meaning the product of the spectral norms of the weight matrices, times a certain correction factor. This bound is empirically investigated for…
We study spectral asymptotics for the Laplace operator on differential forms on a Riemannian foliated manifold equipped with a bundle-like metric in the case when the metric is blown up in directions normal to the leaves of the foliation. The asymptotical formula for the eigenvalue distribution function is obtained. Th…
We prove the existence of metrics maximizing the first eigenvalue normalized by area on closed, non-orientable surfaces assuming two spectral gap conditions. These spectral gap conditions are proved by the authors in \cite{MS3}.
Study beta function for convex billiard maps, linking spectral invariants.
problem Understanding spectral invariants of convex billiard maps.
method Birkhoff normal form via constructive generating functions, explicit beta function formula.
result Linked spectral invariants to beta function for convex billiard maps.
This work uses neural density estimation to analyze laser-induced breakdown spectroscopy data, enabling accurate predictions and uncertainty quantification.
problem Inference of probability densities in high-dimensional spectral data is often intractable.
method Normalizing flows on structured spectral latent spaces for density estimation and uncertainty quantification.
result The approach enables generation of realistic spectral samples and accurate prediction of state vectors with well-calibrated uncertainties.
For general Riemannian foliations, spectral asymptotics of the Laplacian is studied when the metric on the ambient manifold is blown up in directions normal to the leaves (adiabatic limit). The number of ``small'' eigenvalues is given in terms of the differentiable spectral sequence of the foliation. The asymptotics of…
ULES embeds dynamic networks with stability guarantees.
problem Stability of time-varying node embeddings in evolving networks.
method Unfolded Laplacian Spectral Embedding (ULSE) using normalized Laplacian operators.
result ULES satisfies cross-sectional and longitudinal stability under dynamic stochastic block model.
Deep convolutional neural networks (CNNs) have been shown to be able to fit a random labeling over data while still being able to generalize well for normal labels. Describing CNN capacity through a posteriori measures of complexity has been recently proposed to tackle this apparent paradox. These complexity measures a…
Study of unitary and groupoid orbits of normal operators, focusing on manifold structures and spectral conditions.
problem Understanding the manifold structures of orbits of normal operators under different norm topologies.
method Unified treatment of unitary and groupoid orbits, using moment maps and conditional expectations.
result Differentiable structures for orbits and necessary spectral conditions for norm closure and submanifold properties.
Deep Neural Networks (DNNs) have begun to thrive in the field of automation systems, owing to the recent advancements in standardising various aspects such as architecture, optimization techniques, and regularization. In this paper, we take a step towards a better understanding of Spectral Normalization (SN) and its po…
High-dimensional inference for sparse spectral precision matrices
problem Inference on the spectral precision matrix at a fixed frequency
method Full likelihood-based inference using neighboring discrete Fourier transforms
result Simultaneous control of regularization, finite-sample truncation, and smoothing biases
Spectral algorithms improve under covariate shift with novel weighted techniques.
problem Improving spectral algorithms' performance under covariate shift.
method Analysis of spectral algorithms in non-parametric regression over RKHS, proposing a weighted spectral algorithm with clipped weights.
result Normalized weighted spectral algorithm achieves optimal capacity-independent convergence rates, and clipped weights can approach optimal capacity-dependent rates.
Spectral risk measures are attractive risk measures as they allow the user to obtain risk measures that reflect their subjective risk-aversion. This paper examines spectral risk measures based on an exponential utility function, and finds that these risk measures have nice intuitive properties. It also discusses how th…
A new method for spectral barycentre of graph datasets.
problem Creating a summary graph from a set of graphs with community structure.
method Using multiscale spectral distance based on normalized graph Laplacian eigenvalues.
result The barycentre inherits the topological structure of the graphs in the sample dataset.
Spectral clustering with edge counting detects communities in sparse models.
problem Detecting communities in sparse latent space models.
method Spectral clustering followed by edge counting.
result Algorithm achieves consistency and optimality for a broad class of models.
Clustering of data sets is a standard problem in many areas of science and engineering. The method of spectral clustering is based on embedding the data set using a kernel function, and using the top eigenvectors of the normalized Laplacian to recover the connected components. We study the performance of spectral clust…
We examined the use of modern Generative Adversarial Nets to generate novel images of oil paintings using the Painter By Numbers dataset. We implemented Spectral Normalization GAN (SN-GAN) and Spectral Normalization GAN with Gradient Penalty, and compared their outputs to a Deep Convolutional GAN. Visually, and quantit…
Batch normalization prevents rank collapse in deep networks, improving training stability.
problem Rank collapse in randomly initialized deep networks with increasing depth.
method Investigates spectral instabilities in random matrices and uses batch normalization to avoid rank collapse.
result Batch normalization prevents rank collapse in both linear and ReLU networks, improving training stability.
Networks or graphs can easily represent a diverse set of data sources that are characterized by interacting units or actors. Social networks, representing people who communicate with each other, are one example. Communities or clusters of highly connected actors form an essential feature in the structure of several emp…
A new measure quantifies how risk-averse different risk measures are.
problem Measuring the degree of risk aversion among different risk measures.
method Two axioms: normalization and linearity. Two formulas for the functional.
result Quantifies the degree of risk aversion among spectral risk measures.
We present a novel spectral embedding of graphs that incorporates weights assigned to the nodes, quantifying their relative importance. This spectral embedding is based on the first eigenvectors of some properly normalized version of the Laplacian. We prove that these eigenvectors correspond to the configurations of lo…
In this paper, we propose guaranteed spectral methods for learning a broad range of topic models, which generalize the popular Latent Dirichlet Allocation (LDA). We overcome the limitation of LDA to incorporate arbitrary topic correlations, by assuming that the hidden topic proportions are drawn from a flexible class o…
Sharp spectral estimates for negatively curved foliations.
problem Estimating the bottom of the spectrum of Riemannian foliations.
method Analyzing the normal exponential map and using it to derive spectral estimates.
result Sharp estimates for the bottom of the spectrum of Riemannian foliations.
This paper establishes the consistency of spectral approaches to data clustering. We consider clustering of point clouds obtained as samples of a ground-truth measure. A graph representing the point cloud is obtained by assigning weights to edges based on the distance between the points they connect. We investigate the…
This article concerns new off-diagonal estimates on the remainder and its derivatives in the pointwise Weyl law on a compact n-dimensional Riemannian manifold. As an application, we prove that near any non self-focal point, the scaling limit of the spectral projector of the Laplacian onto frequency windows of constant …
New Hermite approximations accelerate convergence with adaptive coordinate transformations.
problem Accelerating convergence of spectral approximations for Hermite expansions.
method Using normalizing flows for adaptive coordinate transformations and deriving error estimates.
result Error estimates for Hermite expansions under adaptive coordinate transformations.
Probabilistic Skip Connections for Deterministic Uncertainty Quantification in Deep Neural Networkscs.LG
Proposes PSCs for UQ in deep nets without retraining.
problem Estimating uncertainty in deep nets with a single pass.
method Identifies sensitive, smooth intermediate layer, fits probabilistic model.
result PSCs achieve UQ and OOD detection performance matching existing methods.
A new method boosts graph neural networks by preventing over-smoothing and over-squashing.
problem Graph Neural Networks struggle with long-range signals and over-smoothing/over-squashing.
method Proposes PowerEmbed, a layer-wise normalization technique inspired by spectral graph embedding.
result PowerEmbed prevents over-smoothing and avoids over-squashing, improving performance on heterophilous graphs.
Spectral clustering achieves strong consistency in the stochastic block model under certain conditions.
problem Achieving strong consistency in spectral clustering for the stochastic block model.
method Entrywise analysis of the Fielder eigenvector of graph Laplacians.
result Spectral clustering achieves exact recovery of hidden communities under matching information-theoretic limits.
Paper establishes statistical inference for pairwise comparison models.
problem Statistical inference for pairwise comparison models when the number of subjects diverges.
method Identifies Fisher information matrix as a weighted graph Laplacian for asymptotic normality.
result Near-optimal asymptotic normality result for maximum likelihood estimator.
Frölicher inequalitiesmath.DG
Proves Frölicher inequality on complex manifolds.
problem Calculating Betti and Hodge numbers.
method Uses spectral projectors of to build an injection.
result New proof of classical Frölicher inequality.
In this paper, we prove that the two well-known natural normalizations of Hamiltonian functions on the symplectic manifold canonically relates the action spectra of different normalized Hamiltonians on {\it arbitrary} symplectic manifolds . The natural class of normalized Hamiltonians consists of those w…
Graph Laplacians adapt to different manifold dimensions, while Dirichlet energies converge to a tensorized Dirichlet energy.
problem Understanding machine learning methods for data with varying intrinsic dimensions.
method Γ-convergence of graph Dirichlet energies and spectral convergence of graph Laplacians on intersecting manifolds of varying dimensions.
result Normalized Dirichlet energy converges to a tensorized Dirichlet energy that adapts to all dimensions simultaneously.
Sparse and Smooth: improved guarantees for Spectral Clustering in the Dynamic Stochastic Block Modelstat.ML
In this paper, we analyse classical variants of the Spectral Clustering (SC) algorithm in the Dynamic Stochastic Block Model (DSBM). Existing results show that, in the relatively sparse case where the expected degree grows logarithmically with the number of nodes, guarantees in the static case can be extended to the dy…