Orthogonal Random Features reduce kernel approximation error and speed up computation.
problem Gaussian kernel approximation error reduction and speed up computation.
method Replacing random Gaussian matrix with a scaled random orthogonal matrix, and using structured discrete orthogonal matrices.
result Significantly decreases kernel approximation error and reduces computation time from O(d2) to O(dlogd). A new optimizer preserves orthogonality constraints on matrices efficiently.
problem Optimization on Stiefel manifold with orthogonality constraints.
method Interplay between continuous and discrete dynamics leading to a gradient-based optimizer with momentum.
result The method optimizes matrices on Stiefel manifold efficiently and accurately.
Study isotropy groups for complex orthogonal and skew-symmetric matrices.
problem Understanding isotropy subgroups of orthogonal similarity transformations.
method Analysis of group structure of nonsingular block matrices.
result Group structure of isotropy subgroups related to block Toeplitz matrices.
Computes isotropy subgroups of orthogonal matrices acting on Hermitian matrices.
problem Computing isotropy subgroups of orthogonal matrices acting on Hermitian matrices.
method Algorithm for solving a matrix equation to compute isotropy subgroups.
result Computed isotropy subgroups of orthogonal matrices acting on Hermitian matrices.
Study the geometric properties of skew symmetric matrices and orthogonal groups.
problem Understanding the geometric properties of skew symmetric matrices and orthogonal groups.
method Investigate the differential-geometric properties of the exponential map and Riemannian structure.
result Connections between skew symmetric matrices and orthogonal groups are revealed.
Algorithm finds isotropy subgroups of orthogonal similarity on symmetric matrices.
problem Computing isotropy subgroups of orthogonal similarity on symmetric matrices.
method Algorithmic procedure solving a Toeplitz matrix equation.
result Structure of isotropy subgroups described.
Fast approximates orthogonal matrices for PCA.
problem Approximating orthogonal matrices for fast computation.
method Optimization over structured matrices and greedy algorithm.
result Efficient approximation balances accuracy and speed.
New method for sampling orthogonal matrices using Hamiltonian Monte-Carlo.
problem Sampling from posterior distributions of orthogonal matrices in Bayesian models.
method Proposes a new sampling scheme based on Hamiltonian Monte-Carlo and Riemannian optimization.
result New method is comparable or faster in time per iteration and more sample-efficient than conventional methods.
Optimizing over the set of orthogonal matrices is a central component in problems like sparse-PCA or tensor decomposition. Unfortunately, such optimization is hard since simple operations on orthogonal matrices easily break orthogonality, and correcting orthogonality usually costs a large amount of computation. Here we…
New probabilistic CCA extensions with moment matching for multi-view models.
problem Estimating multi-view models with identifiability guarantees.
method Moment matching techniques, generalized covariance matrices, non-orthogonal joint diagonalization.
result Improved sample complexity and simplified algorithms.
Constructs orthogonal coordinates in curved spaces.
problem Separating variables in curved spaces.
method Explicit construction of orthogonal coordinates and transformations.
result Explicit formulas for Killing tensors and Stäckel matrices.
A new algorithm POGO optimizes thousands of orthogonal matrices efficiently.
problem Optimizing thousands of orthogonal constraints at scale is computationally expensive.
method Revisits Landing algorithm, uses modern adaptive optimizers, reduces hyperparameters.
result POGO optimizes thousands of orthogonal matrices in minutes, outperforming alternatives.
Improved machine learning performance through structured random orthogonal embeddings.
problem Improving accuracy and speed in machine learning applications.
method Structured random orthogonal matrices for dimensionality reduction and kernel approximation.
result Significant improvement in accuracy and speed compared to existing methods.
A Clifford algebra model for M"obius geometry is presented. The notion of Ribaucour pairs of orthogonal systems in arbitrary dimensions is introduced, and the structure equations for adapted frames are derived. These equations are discretized and the geometry of the occuring discrete nets and sphere congruences is disc…
scoRNN improves RNN performance with simpler orthogonal weight matrices.
problem Vanishing and exploding gradients in RNNs.
method Parametrizing orthogonal recurrent weight matrices with a scaled Cayley transform.
result scoRNN achieves superior results with fewer parameters than other unitary RNNs.
This work proves the asymptotic freeness of layerwise Jacobians in MLPs with Haar orthogonal matrices.
problem Proving the asymptotic freeness of layerwise Jacobians in multilayer perceptrons (MLPs).
method Replacing each layer's parameter matrix with itself multiplied by a Haar orthogonal matrix, and using the invariance of the MLP.
result Proves the asymptotic freeness of layerwise Jacobians in MLPs with Haar orthogonal matrices.
The paper solves PDEs from matrices with orthogonal columns, linking them to Hessian metrics and symmetric spaces.
problem Solving third order PDEs for strictly convex smooth functions.
method Geometric methods using Hessian metrics and symmetric spaces.
result Explicit solutions and a family of non-generic solutions with applications in Poisson geometry and Kahler structures.
One reflection suffices for orthogonal weights, reducing GPU usage.
problem Efficiently computing orthogonal weight matrices without high GPU utilization.
method Use an auxiliary neural network to compute one reflection instead of many.
result One reflection is sufficient for orthogonal weights, improving GPU utilization.
Proposes a new RNN structure to improve expressivity without sacrificing stability.
problem Exploding and vanishing gradient problems in RNNs and reduced expressivity.
method Introduces a non-normal RNN structure using Schur decomposition and splitting.
result Enhances expressivity while maintaining stability and training speed.
Orthogonal random features approximate a Bessel kernel, offering sharper bounds than random Fourier features.
problem Approximating Gaussian kernel efficiently for large datasets.
method Use of Haar orthogonal matrices to construct orthogonal random features and analyze their bias and variance.
result Orthogonal random features approximate a Bessel kernel, not the Gaussian kernel, with sharper bounds.
Discrete conjugate systems are quadrilateral nets with all planar faces. Discrete orthogonal systems are defined by the additional property of all faces being concircular. Their geometric properties allow one to consider them as proper discretization of conjugate, resp. orthogonal coordinate systems of classical differ…
Researchers derive exclusive Racah matrix for complex representation with non-trivial multiplicities.
problem Constructing and understanding Racah matrices for complex representations with non-trivial multiplicities.
method Using effective field theory for arborescent knots, they deduced the exclusive Racah matrix for representation R=[3,1] with non-trivial multiplicities. result The exclusive Racah matrix $ar S$ for representation R=[3,1] is operator valued and depends on basis choices in intertwiner spaces. A new algorithm avoids retractions to optimize orthogonal matrices efficiently.
problem Optimizing functions over the manifold of orthogonal matrices efficiently.
method Landing algorithm that avoids retractions using potential energy.
result The landing algorithm is faster and less prone to numerical errors than retraction-based methods.
AuON is a linear-time optimizer that improves upon Muon's performance without approximate orthogonal matrices.
problem High memory and computational costs of orthogonal momentum updates.
method AuON uses normalized nonlinear scaling and a 'emergency brake' to handle exploding attention logits.
result AuON achieves strong performance without approximate orthogonal matrices, preserving structural alignment and reconditioning.
Efficiently optimizes orthogonal and Stiefel matrices on parallel units.
problem Optimization over orthogonal groups on parallel units.
method CWY and T-CWY transforms for parametrization and optimization.
result CWY and T-CWY methods lead to convergence on parallel units.
Chevalley theorems extended to isotropic functions on matrix spaces.
problem Extending Chevalley theorems to isotropic functions on matrix spaces.
method Proving ultradifferentiable Chevalley restriction theorems for various ultradifferentiable classes.
result Isotropic functions on symmetric matrices have ultradifferentiable regularity if and only if their diagonal restrictions do.
Improved Kalman filter for Stiefel manifold measurements.
problem Improving accuracy in measurements on Stiefel manifolds.
method Generalization of extended Kalman filter for Stiefel manifold-valued measurements.
result Significant improvement over raw measurements.
Curious structure of special orthogonal, unitary, and symplectic groups as products of Grassmannians discovered.
problem Understanding the structure of special orthogonal, unitary, and symplectic groups.
method Expressing these groups as products of Grassmannians realized as involution matrices.
result Special orthogonal, special unitary, and symplectic groups can be expressed as products of their corresponding Grassmannians.
Paper derives local Plücker formulas for special orthogonal groups.
problem Deriving Plücker formulas for special orthogonal groups.
method Reduction to classical A_n case.
result Local Plücker formulas for special orthogonal groups derived.
A new method for efficiently computing derivatives of skew-symmetric matrix exponentials.
problem Efficient computation of derivatives for skew-symmetric matrices.
method Characterization of invertibility, construction of nearby logarithm, and efficient implementation.
result Explicit formulae for differentiation and its inverse of skew-symmetric matrix exponentials.
A geometric approach discretizes confocal quadrics, leading to novel discrete nets.
problem Discretization of confocal quadrics.
method Geometric characterization and factorizable orthogonal coordinate systems.
result Explicit computation of coordinate functions for discrete confocal quadrics.
This paper explores how Transformers predict next tokens in autoregressive tasks.
problem Understanding the success of Transformers in autoregressive learning.
method Trained a Transformer on a next-token prediction task, focusing on commuting orthogonal matrices.
result Trained Transformers can be seen as implementing gradient descent for a specific objective function.
New discretizations of principal curvature lines discovered.
problem Discretizing principal curvature line parametrizations.
method Generalization of polar pairs of line congruences in the Lie quadric.
result New discretizations of orthogonal and Gauss-orthogonal parametrizations.
Bayesian inference on orthogonal matrices using Givens representation.
problem Posterior inference in models with orthogonal matrix parameters.
method Givens representation for posterior inference over the Stiefel manifold.
result Effective methods for transforming densities over the Stiefel manifold into Euclidean space.
Characterizes the local diffeomorphism structure of the exponential in the set of skew-symmetric matrices.
problem Characterizing the local diffeomorphism structure of the exponential in the set of skew-symmetric matrices.
method Introduce the diffeomorphic logarithm of special orthogonal matrices and an efficient algorithm.
result The region containing the principal logarithm has a special multiplicity structure.
New metrics improve landing algorithms for orthogonality constraints.
problem Optimizing landing algorithms with orthogonality constraints.
method Proposed a family of metrics over full-rank matrices to enhance landing algorithms.
result Natural extension of β-metric improves landing performance.
Orthogonal coding matrices improve multi-class classification accuracy across various datasets.
problem Improving multi-class classification accuracy using orthogonal coding matrices.
method Optimized orthogonal coding matrices for multi-class classification, compared with other methods.
result Orthogonal coding matrices generally outperform random ECCs and are faster than 1 vs. 1.
Study real logarithms of semi-simple matrices, focusing on differential structure.
problem Understanding the differential structure of real logarithms of semi-simple matrices.
method Examines the differential structure of real logarithms of semi-simple matrices under specific matrix types.
result Characterizes the differential structure of real logarithms of semi-simple matrices.
FedSPDnet improves federated learning for SPD matrices, outperforming existing methods.
problem Federated learning for SPD matrices with orthogonality constraints.
method Two efficient aggregation strategies: ProjAvg and RLAvg, preserving geometric structure.
result FedSPDnet outperforms federated EEGnet in F1 score and robustness to federation and partial participation.
Optimizes embedding accuracy for data variance and error.
problem Efficiently embedding data while minimizing distortion.
method Uses Johnson-Lindenstrauss embeddings with orthogonal matrices and singular-value latent variables.
result Achieves best accuracy in variance, mean-squared error, and length distortion.
Study of discrete period matrices on embedded graphs, relating to Riemann surfaces.
problem Understanding discrete conformal structures on surfaces via period matrices.
method Combinatorial interpretation of period matrices, using homological quasi-trees and Laplacian determinants.
result Derived a combinatorial analogue of the Weil-Petersson potential and related it to homological quasi-trees.
Study differential properties of matrix square roots in specific cases.
problem Understanding matrix square roots in semi-simple, symmetric, and orthogonal cases.
method Analysis of differential and metric structures of real square roots of matrices under specific conditions.
result Differential properties of matrix square roots in semi-simple, symmetric, and orthogonal cases.
Cyclidic nets are introduced as discrete analogs of curvature line parametrized surfaces and orthogonal coordinate systems. A 2-dimensional cyclidic net is a piecewise smooth C1-surface built from surface patches of Dupin cyclides, each patch being bounded by curvature lines of the supporting cyclide. An explicit de…
New discrete cmc surfaces defined from sphere packings and combinatorics.
problem Creating constant mean curvature surfaces from discrete data.
method Discrete cmc surfaces defined via sphere packings and combinatorial patterns.
result Construction of discrete cmc surfaces from orthogonal ring patterns.
Study finds Calabi-Yau models' operator spectra match random matrix theory.
problem Understanding spectra of Calabi-Yau sigma models.
method Numerical methods for Ricci-flat metrics, averaging over complex structure moduli space.
result Spectrum matches Gaussian orthogonal ensemble of random matrix theory.
Confocal quadrics lie at the heart of the system of confocal coordinates (also called elliptic coordinates, after Jacobi). We suggest a discretization which respects two crucial properties of confocal coordinates: separability and all two-dimensional coordinate subnets being isothermic surfaces (that is, allowing a con…
SpecNet2 improves spectral embedding without orthogonalization, achieving better performance and efficiency.
problem Improving spectral embedding methods for better performance and efficiency.
method Optimizes an equivalent objective of the eigen-problem without orthogonalization, allowing separate row and column sampling.
result Local and global convergence of the new objective using batch-based gradient descent is proven, and improved performance and efficiency are demonstrated on simulated and image datasets.
Deep neural networks' Jacobian spectrum becomes well-conditioned with orthogonal weights.
problem Understanding and handling the Jacobian spectrum of deep neural networks.
method Applying free probability theory to show almost sure asymptotic freeness of Jacobians in the wide limit.
result Layer-wise Jacobians of deep neural networks with orthogonal weights are almost surely asymptotically free.