Weight normalization speeds up matrix sensing problems.
problem Matrix sensing with overparameterization.
method Generalized weight normalization with Riemannian optimization.
result WN achieves linear convergence, improving speed and complexity.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “matrix normalization”
On the impact of activation and normalization in obtaining isometric embeddings at initializationcs.LG
Layer normalization with activations prevents Gram matrix rank collapse at initialization.
problem Rank collapse in Gram matrices at initialization slows training in deep networks.
method Proved that layer normalization, with activation layers, biases Gram matrix towards identity matrix at exponential rate.
result Layer normalization with activations biases Gram matrix towards identity matrix at exponential rate with depth at initialization.
New method normalizes matrix features for robust low-rank approximation.
problem Robust feature normalization for low-rank matrix approximation.
method Learn quantile normalization operators jointly with matrix factorization.
result Improves quality of low-rank representation of data.
This paper develops the exact linear relationship between the leading eigenvector of the unnormalized modularity matrix and the eigenvectors of the adjacency matrix. We propose a method for approximating the leading eigenvector of the modularity matrix, and we derive the error of the approximation. There is also a comp…
In this paper we form relations for the determination of the elements of the Eötvös matrix of the Earth's normal gravity field. In addition a relation between the Gauss curvature of the normal equipotential surface and the Gauss curvature of the actual equipotential surface both passing through the point P is presented…
Matrix profile has been recently proposed as a promising technique to the problem of all-pairs-similarity search on time series. Efficient algorithms have been proposed for computing it, e.g., STAMP, STOMP and SCRIMP++. All these algorithms use the z-normalized Euclidean distance to measure the distance between subsequ…
We prove a central limit theorem for the components of the eigenvectors corresponding to the largest eigenvalues of the normalized Laplacian matrix of a finite dimensional random dot product graph. As a corollary, we show that for stochastic blockmodel graphs, the rows of the spectral embedding of the normalized La…
This work interprets diffusion score matching using normalizing flows for better model training and evaluations.
problem Limitations of diffusion score matching when dealing with certain types of distributions.
method The approach involves interpreting the diffusion matrix using normalizing flows to provide better interpretation and usage of diffusion score matching.
result Diffusion score matching is equivalent to the original score matching evaluated in the transformed space defined by the normalizing flow.
Doubly-stochastic normalization improves robustness to heteroskedastic noise.
problem Robustness to heteroskedastic noise in affinity matrix construction.
method Doubly-stochastic normalization of the Gaussian kernel.
result Doubly-stochastic normalization converges to clean matrix with rate under heteroskedastic noise.
NMF with specific constraints is equivalent to LDA.
problem Dimensionality reduction of non-negative data.
method NMF with normalization constraints and Dirichlet prior.
result NMF with these constraints is equivalent to LDA.
New method clusters matrix-variate data with outliers.
problem Clustering matrix-variate data with outliers.
method Iterative approach using subset log-likelihoods.
result Extends OCLUST algorithm to matrix-variate normal data.
New method for hyperparameter tuning in sparse matrix factorization.
problem Hyperparameter tuning in sparse matrix factorization.
method Numerical method based on evaluating the zero point of normalization factor in sparse matrix prior.
result Our method outperforms existing algorithms in ground-truth sparse matrix reconstruction.
New method proves asymptotic normality for matrix sensing problems.
problem Proving asymptotic normality for matrix sensing under general convex losses.
method Riemannian geometry to handle degeneracy of the Hessian due to rotational symmetry.
result Proves as .
Algorithm counts intersections of normal curves efficiently.
problem Efficiently solving word problems in mapping class groups of punctured surfaces.
method Fast algorithm for counting intersections of normal curves on triangulated surfaces.
result Efficient solution of the word problem for mapping class groups of punctured surfaces.
The paper studies matrix normalization and graph balancing using a new functional and gradient descent.
problem Matrix normalization and graph balancing.
method A new functional called the non-normal energy, and gradient descent.
result Gradient descent of the non-normal energy converges to balanced graphs and preserves spectra and realness of weights.
In recent years, data have become increasingly higher dimensional and, therefore, an increased need has arisen for dimension reduction techniques for clustering. Although such techniques are firmly established in the literature for multivariate data, there is a relative paucity in the area of matrix variate, or three-w…
Spectral clustering is a technique that clusters elements using the top few eigenvectors of their (possibly normalized) similarity matrix. The quality of spectral clustering is closely tied to the convergence properties of these principal eigenvectors. This rate of convergence has been shown to be identical for both th…
PSMM method optimizes matrix sufficient dimension reduction.
problem Feature matrices with row- and column-wise interpretations require efficient dimension reduction.
method PSMM method converts matrix problem into classification problems using rank-1 normal matrix.
result PSMM outperforms existing methods and provides strong interpretability.
MuonEq improves training of matrix-valued parameters by rebalancing momentum before orthogonalization.
problem Training matrix-valued parameters with orthogonalized-update optimizers like Muon.
method MuonEq introduces three lightweight pre-orthogonalization equilibration schemes: two-sided row/column normalization (RC), row normalization (R), and column normalization (C).
result Row/column normalization acts as a zeroth-order surrogate for whitening and improves the geometry seen by orthogonalization.
The ubiquitous proliferation of online social networks has led to the widescale emergence of relational graphs expressing unique patterns in link formation and descriptive user node features. Matrix Factorization and Completion have become popular methods for Link Prediction due to the low rank nature of mutual node fr…
In (exploratory) factor analysis, the loading matrix is identified only up to orthogonal rotation. For identifiability, one thus often takes the loading matrix to be lower triangular with positive diagonal entries. In Bayesian inference, a standard practice is then to specify a prior under which the loadings are indepe…
Paper solves a key problem in learning from high-dimensional covariance matrices.
problem Computing normalizing factors for Riemannian Gaussian distributions on high-dimensional covariance matrices.
method Equivalence with random matrix theory and log-normal matrix ensembles to approximate normalizing factors.
result Efficient approximation of normalizing factors with decreasing error as dimension increases.
Undirected graphs can be used to describe matrix variate distributions. In this paper, we develop new methods for estimating the graphical structures and underlying parameters, namely, the row and column covariance and inverse covariance matrices from the matrix variate data. Under sparsity conditions, we show that one…
Paper improves matrix-valued data classification using nonparametric LDA.
problem Classification of matrix-valued data in neuroimaging and signal processing.
method Nonparametric LDA based on NPMLE for vectorized and scaled matrices.
result Improves classification performance across various data structures.
CoreFlow models matrix-valued distributions efficiently, preserving shared low-rank structure.
problem Learning matrix-valued distributions from high-dimensional and incomplete data.
method Low-rank flow model that learns shared row/column subspaces and trains a normalizing flow on the core.
result CoreFlow improves generation quality in few-sample regimes and remains competitive in data-rich settings.
In this note we answer a question of G. Lecué, by showing that column normalization of a random matrix with iid entries need not lead to good sparse recovery properties, even if the generating random variable has a reasonable moment growth. Specifically, for every we construct a random vector …
Muon dynamics study uses spectral Wasserstein flow for optimization stability.
problem Optimizing deep learning models with gradient normalization.
method Introduces Spectral Wasserstein distances for matrix flows, proving equivalence with Benamou--Brenier formulation.
result Gradient-flow interpretation of mean-field normalized training dynamics.
Statistical Inference for Matching Decisions via Matrix Completion under Dependent Missingnessstat.ME
The paper addresses statistical inference in matching markets with dependent missingness.
problem Statistical inference for two-sided matching markets with matching-induced dependence.
method Non-convex algorithm based on Grassmannian gradient descent, debiasing and projection framework.
result Near-optimal entrywise convergence rates for various matching mechanisms.
The paper improves matrix completion with auxiliary covariates using LS estimation.
problem Matrix completion with noisy data and auxiliary covariates.
method Iterative least squares estimation with statistical properties derived.
result Asymptotic normal distributions of estimators for low-rank matrix and coefficient matrix.
Causal Matrix Completionecon.EM
New method corrects bias in missing data for matrix completion.
problem Missing data bias in matrix completion.
method Causal model and synthetic nearest neighbors (SNN) method.
result Synthetic nearest neighbors (SNN) method provides consistent and normal estimates.
Paper proposes an online estimator for covariance matrix of SGD iterates.
problem Quantifying variability and randomness of SGD-based estimates in online learning.
method Proposes a fully online estimator for covariance matrix of ASGD using SGD iterates.
result Establishes consistency of the online estimator and shows comparable convergence rate to offline methods.
Nonnegative matrix factorization (NMF) has been shown recently to be tractable under the separability assumption, under which all the columns of the input data matrix belong to the convex cone generated by only a few of these columns. Bittorf, Recht, Ré and Tropp (`Factoring nonnegative matrices with linear programs', …
Missing data estimation is an important challenge with high-dimensional data arranged in the form of a matrix. Typically this data matrix is transposable, meaning that either the rows, columns or both can be treated as features. To model transposable data, we present a modification of the matrix-variate normal, the mea…
This paper is on the normal approximation of singular subspaces when the noise matrix has i.i.d. entries. Our contributions are three-fold. First, we derive an explicit representation formula of the empirical spectral projectors. The formula is neat and holds for deterministic matrix perturbations. Second, we calculate…
In this paper, we use a new approach to prove that the largest eigenvalue of the sample covariance matrix of a normally distributed vector is bigger than the true largest eigenvalue with probability 1 when the dimension is infinite. We prove a similar result for the smallest eigenvalue.
Several variants of recurrent neural networks (RNNs) with orthogonal or unitary recurrent matrices have recently been developed to mitigate the vanishing/exploding gradient problem and to model long-term dependencies of sequences. However, with the eigenvalues of the recurrent matrix on the unit circle, the recurrent s…
The paper develops inference methods for high-dimensional multi-task regression with row-sparse coefficients.
problem Inference for high-dimensional multi-task regression with unknown coefficient matrix under row-sparsity.
method Proposes chi-square and normal inference methodologies using MT Lasso with de-biasing scheme and interaction matrix.
result Derives asymptotic normal and chi-square distribution results for valid confidence intervals and ellipsoids.
New optimizers control network width scaling, improving stability and transfer across different model sizes.
problem Designing stable optimizers for networks of varying widths.
method Interpreting optimizers as steepest descent under mean-normalized operator norms, enabling layerwise composability and width-independent bounds.
result New optimizers like row normalization and column normalization provide stable learning-rate transfer across different model widths.
We are concerned with an approximation problem for a symmetric positive semidefinite matrix due to motivation from a class of nonlinear machine learning methods. We discuss an approximation approach that we call {matrix ridge approximation}. In particular, we define the matrix ridge approximation as an incomplete matri…
This paper is a further extension of the method proposed in Itkin, 2014 as applied to another set of jump-diffusion models: Inverse Normal Gaussian, Hyperbolic and Meixner. To solve the corresponding PIDEs we accomplish few steps. First, a second-order operator splitting on financial processes (diffusion and jumps) is …
Improved bipartite link prediction using 2-hop paths.
problem Link prediction in bipartite networks without node attributes.
method Multiply reconstructed adjacency matrix with symmetrically normalized training adjacency matrix to form 2-hop paths.
result 2-hop paths improve link prediction performance.
Low-rank matrix regression refers to the instances of recovering a low-rank matrix based on specially designed measurements and the corresponding noisy outcomes. In the last decade, numerous statistical methodologies have been developed for efficiently recovering the unknown low-rank matrices. However, in some applicat…
We investigate the analogy between the large N expansion in normal matrix models and the asymptotic expansion of the determinant of the Hilb map, appearing in the study of critical metrics on complex manifolds via projective embeddings. This analogy helps to understand the geometric meaning of the expansion of matrix m…
New connections on symmetric spaces with invariant properties.
problem Understanding invariant connections on hermitian symmetric spaces.
method Introduced a class of -invariant connections on homogeneous bundles over hermitian symmetric spaces.
result Parameter space of connections is a normal variety with a canonical anti-holomorphic involution.
Study on neural networks with non-normal interactions reveals unique spectral properties.
problem Understanding episodic memory encoding in the brain.
method Developed a neural network model with non-Hermitian couplings and applied random matrix theory.
result Spectral density of the model is non-uniform and can transition to chaos, providing computational benefits.
MANO normalizes logits to estimate test accuracy without labels.
problem Estimating test accuracy of OOD samples without labels.
method Applies norm to normalized logits.
result Achieves state-of-the-art performance across various architectures.
The Sinkhorn-Knopp algorithm converges quickly but the number of iterations is poorly understood.
problem Understanding the number of iterations required for the Sinkhorn-Knopp algorithm to converge.
method Analyzing the Sinkhorn-Knopp algorithm for matrices with a specific density threshold.
result The Sinkhorn-Knopp algorithm requires iterations for matrices with density .
A new method for learning Bayesian neural networks using layerwise inference.
problem Learning Bayesian neural networks efficiently and accurately.
method Bayesian layerwise inference, treating neural networks as stacked Bayesian linear models, with pseudo-targets defined by backpropagated gradients.
result The method converges quickly and performs well on various benchmarks.