Improved singular value approximation for convolutional layers.
problem Improving accuracy of singular value approximation for linear convolutional layers.
method Developed a new spectral density matrix method for singular value approximation with improved accuracy and reduced computational complexity.
result Obtained moderate improvement in singular value distribution compared to circular approximation.
Ranky solves SVD for large sparse matrices in distributed systems.
problem Rank problem in large sparse matrices for SVD.
method Distributed approach to solve rank problem.
result Recovers SVD with negligible error for large sparse matrices.
A low rank matrix X has been contaminated by uniformly distributed noise, missing values, outliers and corrupt entries. Reconstruction of X from the singular values and singular vectors of the contaminated matrix Y is a key problem in machine learning, computer vision and data science. In this paper we show that common…
Paper analyzes singular subspace estimation in noisy matrix models.
problem Estimating low-rank signals in noisy matrix data.
method Asymptotic distributional theory, extreme value theory, saddle point approximation, random matrix theory.
result Plug-in test statistic based on two-to-infinity norm has higher power for detecting structured alternatives.
Study shows XRP price correlates with transaction network metrics.
problem Understanding the relationship between cryptoasset price and network metrics.
method Analysis of correlation tensor spectra, random matrix theory comparison, singular values investigation.
result Distinct correlation between XRP price and singular values during bubble and non-bubble periods.
It is well known that the initialization of weights in deep neural networks can have a dramatic impact on learning speed. For example, ensuring the mean squared singular value of a network's input-output Jacobian is O(1) is essential for avoiding the exponential vanishing or explosion of gradients. The stronger condi…
This work analyzes self-attention matrices using random matrix theory.
problem Understanding the theoretical behavior of self-attention layers in neural networks.
method Asymptotic spectral analysis of the attention matrix, Gaussian equivalence, and linearization.
result The singular value distribution of the attention matrix is asymptotically characterized by a linear model.
Random matrix analysis reveals that neural network weights are mostly random, with some indicating learned information.
problem Understanding how neural networks store information needed for tasks.
method Random matrix theory (RMT) applied to weight matrices of trained deep neural networks.
result Most singular values and eigenvectors of trained neural networks follow universal RMT predictions, suggesting they are random and do not contain system-specific information.
The paper proves a distribution claim for neural network Jacobians.
problem Distribution of singular values in deep neural networks.
method Free probability and random matrix theory techniques.
result Singular value distribution matches for specific cases.
We show how to analyze and interpret the correlation structures, the conditional expectation values and correlation coefficients of exchangeable Bernoulli random variables. We study implied default distributions for the iTraxx-CJ tranches and some popular probabilistic models, including the Gaussian copula model, Beta …
New method approximates MMD using pseudo-differential operators and singular values.
problem Approximating MMD with pseudo-differential operators and singular values.
method Corresponding pseudo-differential operators to Mercer kernels, approximating p(x,y) with its first r singular values. result The new MMD distance measures the difference of two distributions with respect to r∗ local moments, where r∗ depends on singular values decay rate. We regard pre-trained residual networks (ResNets) as nonlinear systems and use linearization, a common method used in the qualitative analysis of nonlinear systems, to understand the behavior of the networks under small perturbations of the input images. We work with ResNet-56 and ResNet-110 trained on the CIFAR-10 dat…
Distributed model training suffers from communication overheads due to frequent gradient updates transmitted between compute nodes. To mitigate these overheads, several studies propose the use of sparsified stochastic gradients. We argue that these are facets of a general sparsification method that can operate on any p…
We present a general method to detect and extract from a finite time sample statistically meaningful correlations between input and output variables of large dimensionality. Our central result is derived from the theory of free random matrices, and gives an explicit expression for the interval where singular values are…
Kaczmarz++ accelerates convergence for ill-conditioned systems.
problem Solving ill-conditioned linear systems efficiently.
method Adaptive momentum acceleration, Tikhonov-regularized projections, and memoization.
result Kaczmarz++ converges faster than Krylov methods on ill-conditioned systems.
Unified approach to totally ramified values in various surface theories.
problem Totally ramified values in value distribution theory, normal family theory, and Gauss maps of surfaces.
method Bloch--Ros principle applied to various surface theories.
result Unified approach to phenomena concerning totally ramified values.
We use matricial free energy to regularize autoencoders, producing Gaussian-like codes.
problem Generating Gaussian-like codes for autoencoders.
method Define a differentiable loss function based on singular values of the code matrix, minimizing matricial free energy.
result Minimizing matricial free energy results in Gaussian-like codes that generalize.
Note on minimal maps' uniqueness via singular values.
problem Uniqueness of minimal maps into \(\mathbb{R}^n\).
method Using singular values and convexity of area functional, proving local linearity of singular value vectors.
result Improved uniqueness theorem for minimal graphs.
New technique stabilizes singular values in concatenated matrices.
problem How singular values of concatenated matrices relate to individual components.
method Developed perturbation technique extending classical results to concatenated matrices.
result Dominant singular values remain stable under small perturbations in submatrices.
Study inequalities for singular values of rectangular matrices.
problem Inequalities for singular values of rectangular matrices.
method Study convex cones associated to isotropic representations of symmetric spaces.
result Describe inequalities by cohomological conditions.
Reproducing kernel Hilbert spaces (RKHSs) play an important role in many statistics and machine learning applications ranging from support vector machines to Gaussian processes and kernel embeddings of distributions. Operators acting on such spaces are, for instance, required to embed conditional probability distributi…
Study differentiable maps on hypersurface links, finding fold maps with circle singular value sets.
problem Understanding differentiable maps on hypersurface links.
method Restricting holomorphic functions to hypersurface links and analyzing the resulting maps.
result Found fold maps with concentric circle singular value sets.
Study how firm liquidation regimes affect shareholder value and stability.
problem Balancing shareholder value and financial stability during firm liquidation.
method Modelled forced liquidation in reduced form, solved singular stochastic control problem.
result Combining distress regions below and above ruin threshold improves both shareholder value and firm survival.
Study evaluates thresholds for removing noise from DNN weights using random matrix theory.
problem Removing noise from deep neural network weights for better approximation.
method Model weights as signal + noise, use random matrix theory to estimate thresholds, evaluate using cosine similarity.
result Proposed threshold estimation method improves approximation quality.
In this paper, we develop the notion of entropy for uniform hypergraphs via tensor theory. We employ the probability distribution of the generalized singular values, calculated from the higher-order singular value decomposition of the Laplacian tensors, to fit into the Shannon entropy formula. We show that this tensor …
New method estimates high-dimensional GoM models efficiently.
problem Estimating GoM models for high-dimensional polytomous data.
method Flattening three-way quasi-tensor into a matrix, performing singular value decomposition.
result Established finite-sample error bounds for estimated parameters.
New method for handling multi-dimensional singular controls with jump costs in mean-field problems.
problem Handling jump costs in multi-dimensional singular controls.
method Introducing two-layer parametrisations to interpolate jumps on both distributional and pathwise levels.
result Derivation of a DPP and characterisation of the value function as a minimal super-solution to a quasi-variational inequality.
Deterministic bounds for tensor singular values and vectors, differing from matrix cases.
problem Spectral learning of higher-order orthogonally decomposable tensors.
method Deterministic perturbation bounds for singular values and vectors of orthogonally decomposable tensors.
result Perturbation affects each essential singular value/vector in isolation, independent of multiplicity and distance from other singular values.
We introduce the concept of singular values for the Riemann curvature tensor, a central mathematical tool in Einstein's theory of general relativity. We study the properties related to the singular values, and investigate five typical cases to show its relationship to the Ricci scalar and other invariants.
Study shows singular set of certain graphs has codimension 1.
problem Understanding the singular set of specific graph structures.
method Proved using the area stationarity condition.
result Singular set has codimension 1.
Solves initial value problem for harmonic maps on specific manifolds.
problem Initial value problem for harmonic maps on cohomogeneity one manifolds.
method Setup and solve the initial value problem using equivariant harmonic maps and regular-singular systems.
result Local existence of harmonic maps in a neighborhood of singular orbits.
A new method selects regions of interest in GC-MS data without prior target selection.
problem Challenges in GC-MS data analysis due to fragmentation and shared fragment ions.
method Uses a pseudo F-ratio moving window (ψFRMV) to automatically select regions of interest. result Algorithm can accurately identify signal regions in GC-MS data.
The paper solves conditions for extending circle-valued Morse functions.
problem Conditions for extending circle-valued Morse functions on closed orientable surfaces.
method Provided necessary and sufficient conditions for the existence of a non-singular extension.
result Necessary and sufficient conditions for the existence of a non-singular extension of a circle-valued Morse function.
A statistical model or a learning machine is called regular if the map taking a parameter to a probability distribution is one-to-one and if its Fisher information matrix is always positive definite. If otherwise, it is called singular. In regular statistical models, the Bayes free energy, which is defined by the minus…
Recent work (Pennington et al, 2017) suggests that controlling the entire distribution of Jacobian singular values is an important design consideration in deep learning. Motivated by this, we study the distribution of singular values of the Jacobian of the generator in Generative Adversarial Networks (GANs). We find th…
Unified framework for singular statistical models using observable charts.
problem Non-identifiability and breakdown of classical asymptotic theory in singular models.
method Invariant framework based on observable charts to define local coordinate systems in model space.
result Observable order provides a lower bound on KL divergence vanishing rate in singular models.
New framework for higher-order singular-value derivatives of rectangular matrices.
problem Challenging to derive higher-order Fréchet derivatives of singular values in real rectangular matrices.
method Using Kato's analytic perturbation theory for self-adjoint operators and embedding rectangular matrices into block self-adjoint operators.
result Closed-form expressions for the n-th order spectral variations of singular values. Study optimal liquidation with multiple regimes using BSDEs with singular terminal values.
problem Optimal liquidation with regime switching in dark pools.
method Introduced a system of BSDEs with jumps and singular terminal values.
result Existence and uniqueness results for the BSDE system are obtained.
Proves continuity and singular set dimension for 2D maps with Q values.
problem Interior regularity of 2D Q-valued maps. method Strong concentration-compactness theorem for equicontinuous maps.
result 2D Q-valued maps are Hölder continuous with singular set dimension ≤1. The complex Lie superalgebras g of type D(2,1;a) - also denoted by osp(4,2;a) - are usually considered for "non-singular" values of the parameter a, for which they are simple. In this paper we introduce five suitable integral forms of g, that are well-defined at singular valu…
Study of singular curves in a specific type of hyperbolic distribution.
problem Characterizing singular curves in hyperbolic (4,7)-distributions. method Introduced hyperbolic (4,7)-distributions of type C3, described singular curves via prolongations. result Completely described singular curves for hyperbolic (4,7)-distributions of type C3. Optimal rank-adaptive matrix estimation from linear measurements.
problem Estimating high-dimensional matrices from linear measurements with adaptive rank selection.
method Combines Least-Squares estimator with universal singular value thresholding.
result Algorithm performance nearly matches fundamental limits.
New method for high-dimensional manifold-based inference tackles latent responses.
problem Inference on latent right factor vectors in multi-task learning with large numbers of responses and features.
method SOFARI-R method with two variants: one for strongly orthogonal factors and another for weakly orthogonal factors.
result Bias-corrected estimators for latent right factor vectors with asymptotically normal distributions and justified asymptotic variance estimates.
Fast and accurate methods for low-rank learning problems.
problem Partial singular value decomposition and numerical rank estimation of huge matrices.
method Krylov subspaces and Ritz vectors for fast and accurate solutions.
result Advantages over traditional methods in accuracy and speed.
In the early 1980's Almgren developed a theory of Dirichlet energy minimizing multi-valued functions, proving that the Hausdorff dimension of the singular set (including branch points) of such a function is at most (n−2), where n is the dimension of its domain. Almgren used this result in an essential way to show t…
New method approximates high-dimensional probability densities efficiently.
problem Approximating high-dimensional probability densities accurately and efficiently.
method Hierarchical tensor-network approach using randomized SVD and linear equations.
result The method effectively approximates high-dimensional densities with linear complexity.
Bayesian neural networks can be simplified by parameterizing weights as rank-r matrices, reducing parameter count and improving performance.
problem High parameter count in standard Bayesian neural networks.
method Parameterize weights as W=ABop with A∈Rmimesr, B∈Rnimesr, inducing a singular posterior. result PAC-Bayes generalization bounds and loss bounds show improved performance with fewer parameters.
The paper studies phase transitions in random matrices and tensor unfolding for detecting signals.
problem Phase transitions in singular values and vectors of large random matrices.
method Analysis of singular values and vectors of long rectangular random matrices, and tensor unfolding algorithm for asymmetric rank-one spiked tensor models.
result An exact threshold for tensor unfolding to detect signals, independent of unfolding procedure.