Gradient descent converges linearly for deep linear networks under specific conditions.
problem Speed of convergence in gradient descent for deep linear neural networks.
method Analysis of gradient descent training for deep linear neural networks minimizing ℓ2 loss. result Gradient descent converges linearly under specific conditions on layer dimensions, initialization, and initial loss.
Analyzes the structure and rank of neural network Hessians.
problem Understanding redundancy in overparameterized neural networks.
method Theoretical tools to analyze Hessian map range and rank deficiency.
result Exact formulas and tight upper bounds for Hessian rank of deep linear networks.
MaxVol NMF maximizes the volume of H in NMF for better sparse and interpretable solutions.
problem Finding interpretable and unique NMF solutions.
method Dual approach to MinVol NMF, maximizing the volume of H. result MaxVol NMF solutions correspond to clustering columns in disjoint clusters.
A new algorithm solves constrained optimization problems with stochastic gradients.
problem Nonlinear equality constrained optimization with rank-deficient Jacobians.
method Step decomposition strategy combining normal and tangential steps.
result Convergence guarantees in rank-deficient Jacobian cases.
New method identifies causal direction with latent confounders.
problem Identifying causal direction in presence of multiple latent variables.
method Use of joint higher-order cumulant matrix properties.
result Causal asymmetry can be seen from rank deficiency properties of cumulant matrices.
Gradient descent with preconditioning finds global optima in overparameterized nonconvex factorization.
problem Finding global optima in nonconvex Burer-Monteiro factorization.
method Preconditioned gradient descent for overparameterized nonconvex function minimization.
result Gradient descent with preconditioning achieves linear convergence in the overparameterized case.
Soft-Radial Projection solves gradient saturation in constrained deep learning.
problem Gradient saturation in deep learning models when integrating hard constraints.
method Introduces Soft-Radial Projection, a differentiable layer that maps predictions onto constraint boundaries without rank-deficient Jacobians.
result Improves convergence and solution quality over state-of-the-art methods.
New method differentiates square-root Kalman filters robustly.
problem Gradient calculation issues in square-root Kalman filters.
method Closed-form chain rule derived from Gramian identity, resolves non-orthogonal and rank-deficient issues.
result Robust automatic differentiation for Kalman filters, resolving numerical stability and gradient issues.
Regular integer lattices are characterized by k unit vectors that build up their generator matrices. These have rank k for D-lattices, and are rank-deficient for A-lattices, for E_6 and E_7. We count lattice points inside hypercubes centered at the origin for all three types, as if classified by maximum infinity norm i…
The paper analyzes a five-factor capital market model and facilitates exact simulation.
problem Analyzing and simulating a five-factor capital market model.
method Using a Vasicek interest rate model, mean-reverting excess return, and realized inflation with expectation, the paper derives the necessary distributional results and describes practical methods to overcome rank deficiency.
result Exact simulation from the model can be achieved by sampling from a seven-dimensional normal distribution.
DBKs enable scalable GPs with tractable inference for large datasets.
problem Scaling Gaussian processes to large and complex datasets while maintaining tractable inference.
method DBKs constructed from neural-network-parameterized basis functions with explicit low-rank structure, enabling linear-complexity inference.
result DBKs provide a unified perspective and improve predictive accuracy, uncertainty quantification, and computational efficiency.
Study nondifferentiable metrics in general relativity, resolving causality issues and limits evolution scenarios.
problem Causality issues and evolution scenarios in black hole interiors with closed timelike geodesics.
method Method of equivalence on Courant algebroids to derive new differential invariants.
result Resolved causality issues and limited evolution scenarios for gravitational collapse.
Efficiently clusters nodes in Gaussian graphical models from data.
problem Clustering nodes in Gaussian graphical models directly from data.
method Clusters nodes based on the similarity of their network neighborhoods defined by partial correlations. Uses matrix factors for limited data.
result Demonstrates improved clustering of nodes in Gaussian graphical models.
Kernel approximation using randomized feature maps has recently gained a lot of interest. In this work, we identify that previous approaches for polynomial kernel approximation create maps that are rank deficient, and therefore do not utilize the capacity of the projected feature space effectively. To address this chal…
Gradient descent recovers planted weights in shallow neural networks with quadratic activations.
problem Learning shallow neural networks with quadratic activations and planted weights.
method Analysis of optimization landscape, gradient descent, semicircle law for Wishart ensemble.
result Gradient descent can recover planted weights if initialized below an energy barrier.
Slow feature analysis (SFA) is a method for extracting slowly varying features from a quickly varying multidimensional signal. An open source Matlab-implementation sfa-tk makes SFA easily useable. We show here that under certain circumstances, namely when the covariance matrix of the nonlinearly expanded data does not …
New algorithms estimate matrix leverage scores using rank revealing and randomization.
problem Estimating leverage scores for matrices of arbitrary rank.
method Combining rank revealing methods with randomized dimensionality reduction.
result Effective estimators for leverage scores, even in rank deficient cases.
New method improves cross-validation for sparse reduced rank regression models.
problem Inconsistent parameter selection in cross-validation for high-dimensional data.
method Proposes cross-validation of projection-selection patterns to avoid inconsistency issues.
result Develops new scale-free information criteria for minimax optimal error rate.
SGD's training dynamics align with Hessian and gradient spectra in high-dimensional classification tasks.
problem Understanding the spectra of Hessian and gradient matrices in high-dimensional classification tasks.
method Rigorous analysis of SGD dynamics and spectra of Hessian and gradient matrices.
result SGD trajectory and emergent outlier eigenspaces align with a common low-dimensional subspace in multi-class high-dimensional mixtures and neural networks.
Geometric families of low-rank covariances improve flexibility and tractability in high dimensions.
problem Interpolating and identifying covariance matrices in high dimensions with limited data.
method Differential geometric construction of low-rank covariance families, interpolation on manifolds, and distance minimization for identification.
result Differential geometric covariance families offer significant flexibility and computational tractability.
Develops methods to find most probable paths on complex manifolds.
problem Identifying optimal paths for manifold-valued processes, especially those with non-trivial structures.
method Constructs a general approach to defining and identifying most probable paths by measuring the Onsager-Machlup function on the anti-development of such processes.
result Derives explicit equations for development most probable paths that encompass various manifold-valued processes.
New method for NMF without tuning parameter.
problem Finding latent structures in noisy data matrices.
method Inspired by square-root lasso, proposes a tuning-free minimum-volume NMF.
result Optimal tuning parameter value is noise level-independent.
Riemannian gradient descent escapes some spurious critical points on low-rank matrix manifold.
problem Spurious critical points on the boundary of low-rank matrix manifold.
method Riemannian gradient descent with dynamical low-rank approximation and rescaled gradient flow.
result Riemannian gradient descent escapes some spurious critical points on the boundary of the manifold.
Factor analysis, a classical multivariate statistical technique is popularly used as a fundamental tool for dimensionality reduction in statistics, econometrics and data science. Estimation is often carried out via the Maximum Likelihood (ML) principle, which seeks to maximize the likelihood under the assumption that t…
We present evolution equations for a family of paths that results from anisotropically weighting curve energies in non-linear statistics of manifold valued data. This situation arises when performing inference on data that have non-trivial covariance and are anisotropic distributed. The family can be interpreted as mos…
Geometric approach to quantum thermodynamics models state spaces and processes.
problem Quantum thermodynamics in the regime of non-equilibrium states.
method Contact geometry and principal fiber bundles to model quantum state spaces and processes.
result Geometric formulation reveals the fundamental thermodynamic relations and unattainability of the third law.
New algorithms for SSMF with weaker identifiability conditions than SSC.
problem Identifying unique decompositions in simplex-structured matrix factorization.
method Extracting facets containing the largest number of points to ensure identifiability.
result Our algorithms recover unique decompositions under weaker conditions than SSC.
New method corrects Laplace/BIC errors in singular models, revealing effective dimension.
problem Laplace/BIC errors in singular models due to incorrect effective dimension assumption.
method RLCT (real log canonical threshold) to correct effective dimension in linear models.
result Correct evidence slope and effective dimension estimation in linear settings.
New method generates synthetic time series paths with more flexibility.
problem Restrictions in generating synthetic paths using Brownian reference.
method Introduces Triangular-Reference Schrödinger Bridges (TR-SBTS) for time series generation.
result Generates synthetic paths with more flexibility in stochastic volatility and correlated noise.
New score-based methods identify causal structures with latent variables.
problem Identifying causal structures involving latent variables.
method Score-based methods with identifiability guarantees.
result Score equivalence and consistency for latent variable causal models.
Transformer learns context and regularization for ICL in inverse problems.
problem Learning context and effective regularization for transformer-based in-context learning (ICL) in inverse problems.
method Introduced a linear transformer to learn inverse mapping from contextual examples to weight vectors, addressing rank-deficient problems.
result Transformer implicitly learns a prior distribution and effective regularization strategy, outperforming traditional methods.
SON-NMF estimates nonnegative rank on-the-fly for NMF.
problem Estimating the nonnegative rank of data in NMF.
method Sum-of-norms (SON) regularization to reduce rank, combined with a first-order BCD algorithm.
result SON-NMF can automatically estimate the rank from data without prior knowledge.
This paper studies non-asymptotic model selection for the general case of arbitrary design matrices and arbitrary nonzero entries of the signal. In this regard, it generalizes the notion of incoherence in the existing literature on model selection and introduces two fundamental measures of coherence---termed as the wor…
Optimal hashing embeddings reduce linear least squares solving time.
problem Efficiently solving large-scale linear least squares problems.
method Optimal hashing sketching matrices for linear least squares.
result Ski-LLS outperforms state-of-the-art solvers on various problem types.
New methods predict brain age from MEG/EEG without source modeling.
problem Predicting brain age from MEG/EEG data without source localization.
method Two Riemannian approaches to vectorize rank-reduced covariance matrices for regression.
result Data-driven Riemannian methods outperform sensor-space estimators and biophysics models.
This paper solves the intractability barrier in non-parametric information geometry by introducing a novel framework.
problem The intractability barrier in non-parametric information geometry due to the Fisher-Rao metric being a functional.
method Introducing an Orthogonal Decomposition of the Tangent Space and deriving the Covariate Fisher Information Matrix (cFIM).
result Established a rigorous foundation for the G-entropy and provided fundamental limits of variance for semi-parametric estimators.
The paper examines partial regularity of Lipschitz solutions to minimal surface system.
problem Understanding the regularity of solutions to the minimal surface system.
method Investigation of stationary, integral weak, and viscosity solutions; interior gradient estimate using maximum principle.
result Partial regularity results for Lipschitz solutions, including interior gradient estimate.
Unique ancient solutions found for anisotropic curve shortening flow.
problem Finding unique solutions for anisotropic curve shortening flow.
method Constructing translating and ancient solutions under given conditions.
result Unique ancient and translating solutions found for anisotropic curve shortening flow.
The paper constructs solutions to a critical Dirac equation on spheres.
problem Solving the critical Dirac equation on spheres with singularities.
method Constructing Delaunay-type solutions and another kind of singular solutions.
result The constructed solutions are building blocks for singular solutions on Spin manifolds.
Paper classifies ancient solutions to 3D Ricci flow.
problem Classifying ancient solutions to 3D Ricci flow.
method Proves uniqueness of solutions based on classification criteria.
result Ancient solutions are either shrinking spheres or Perelman's Type II solutions.
New findings on κ-solutions with round cylinder as asymptotic shrinker.
problem Characterizing κ-solutions with specific asymptotic behavior. method Analysis of Ricci flow in dimensions n≥4. result Uniformly Positive Isoperimetric Constant (PIC) for κ-solutions. Study higher-dimensional Ricci flow solutions, proving uniqueness.
problem Classifying ancient solutions to the Ricci flow on Sn. method Extending [13] to higher dimensions, proving uniqueness.
result Ancient solutions are either shrinking spheres or Type II solutions.
Ancient solutions of Ricci flow with Type I growth are classified.
problem Understanding ancient solutions of Ricci flow with specific curvature growth.
method Analyzing ancient solutions with Type I curvature growth in arbitrary dimensions.
result Ancient solutions with Type I growth are classified into specific types.
We construct low regularity solutions of the vacuum Einstein constraint equations. In particular, on 3-manifolds we obtain solutions with metrics in $H^s\loc$ with s>23. The theory of maximal asymptotically Euclidean solutions of the constraint equations descends completely the low regularity setting. Moreove…
Researchers find entire solutions to magnetic Ginzburg-Landau equations in 4D.
problem Finding entire solutions to magnetic Ginzburg-Landau equations in 4D.
method Using Lyapunov-Schmidt reduction.
result Existence of entire solutions and saddle type solutions with specific zero sets.
New ancient solutions found for curvature flow in 2D.
problem Ancient solutions for curvature flow in 2D.
method Constructing and classifying convex ancient solutions.
result All convex ancient solutions classified for α∈(32,1). The study approximates nearly optimal Lasso solutions using convex hulls.
problem Finding diverse yet nearly optimal Lasso solutions.
method Formulate problem as approximating nearly optimal solutions with a convex hull of sampled extreme points. Use a greedy algorithm to select a small number of points.
result The proposed algorithm can approximate the solution set well and obtain diverse Lasso solutions.
Let n≥3 and m=n+2n−2. We construct 5-parameters, 4-parameters, 3-parameters ancient solutions of the equation vt=(vm)xx+v−vm, v>0, in R×(−∞,T) for some T∈R. This equation arises in the study of Yamabe flow. We obtain various properties of the ancient so…