Orthogonal coding matrices improve multi-class classification accuracy across various datasets.
problem Improving multi-class classification accuracy using orthogonal coding matrices.
method Optimized orthogonal coding matrices for multi-class classification, compared with other methods.
result Orthogonal coding matrices generally outperform random ECCs and are faster than 1 vs. 1.
A new feature coding method for invariant features using tensor products.
problem Learning invariant features for transformations represented by orthogonal matrices.
method Group-invariant feature vector using tensor-product representations of basic representations.
result Group-invariant feature vector contains sufficient discriminative information for linear classifiers.
Subgradient descent learns orthogonal dictionaries efficiently.
problem Sparse coding and dictionary learning for data representation.
method Subgradient descent algorithm with random initialization.
result Subgradient descent can recover orthogonal dictionaries under mild assumptions.
AuON is a linear-time optimizer that improves upon Muon's performance without approximate orthogonal matrices.
problem High memory and computational costs of orthogonal momentum updates.
method AuON uses normalized nonlinear scaling and a 'emergency brake' to handle exploding attention logits.
result AuON achieves strong performance without approximate orthogonal matrices, preserving structural alignment and reconditioning.
Study isotropy groups for complex orthogonal and skew-symmetric matrices.
problem Understanding isotropy subgroups of orthogonal similarity transformations.
method Analysis of group structure of nonsingular block matrices.
result Group structure of isotropy subgroups related to block Toeplitz matrices.
Computes isotropy subgroups of orthogonal matrices acting on Hermitian matrices.
problem Computing isotropy subgroups of orthogonal matrices acting on Hermitian matrices.
method Algorithm for solving a matrix equation to compute isotropy subgroups.
result Computed isotropy subgroups of orthogonal matrices acting on Hermitian matrices.
Study the geometric properties of skew symmetric matrices and orthogonal groups.
problem Understanding the geometric properties of skew symmetric matrices and orthogonal groups.
method Investigate the differential-geometric properties of the exponential map and Riemannian structure.
result Connections between skew symmetric matrices and orthogonal groups are revealed.
Algorithm finds isotropy subgroups of orthogonal similarity on symmetric matrices.
problem Computing isotropy subgroups of orthogonal similarity on symmetric matrices.
method Algorithmic procedure solving a Toeplitz matrix equation.
result Structure of isotropy subgroups described.
Fast approximates orthogonal matrices for PCA.
problem Approximating orthogonal matrices for fast computation.
method Optimization over structured matrices and greedy algorithm.
result Efficient approximation balances accuracy and speed.
New method for sampling orthogonal matrices using Hamiltonian Monte-Carlo.
problem Sampling from posterior distributions of orthogonal matrices in Bayesian models.
method Proposes a new sampling scheme based on Hamiltonian Monte-Carlo and Riemannian optimization.
result New method is comparable or faster in time per iteration and more sample-efficient than conventional methods.
Optimizing over the set of orthogonal matrices is a central component in problems like sparse-PCA or tensor decomposition. Unfortunately, such optimization is hard since simple operations on orthogonal matrices easily break orthogonality, and correcting orthogonality usually costs a large amount of computation. Here we…
Constructs orthogonal coordinates in curved spaces.
problem Separating variables in curved spaces.
method Explicit construction of orthogonal coordinates and transformations.
result Explicit formulas for Killing tensors and Stäckel matrices.
A new algorithm POGO optimizes thousands of orthogonal matrices efficiently.
problem Optimizing thousands of orthogonal constraints at scale is computationally expensive.
method Revisits Landing algorithm, uses modern adaptive optimizers, reduces hyperparameters.
result POGO optimizes thousands of orthogonal matrices in minutes, outperforming alternatives.
Improved machine learning performance through structured random orthogonal embeddings.
problem Improving accuracy and speed in machine learning applications.
method Structured random orthogonal matrices for dimensionality reduction and kernel approximation.
result Significant improvement in accuracy and speed compared to existing methods.
Orthogonal Random Features reduce kernel approximation error and speed up computation.
problem Gaussian kernel approximation error reduction and speed up computation.
method Replacing random Gaussian matrix with a scaled random orthogonal matrix, and using structured discrete orthogonal matrices.
result Significantly decreases kernel approximation error and reduces computation time from O(d2) to O(dlogd). scoRNN improves RNN performance with simpler orthogonal weight matrices.
problem Vanishing and exploding gradients in RNNs.
method Parametrizing orthogonal recurrent weight matrices with a scaled Cayley transform.
result scoRNN achieves superior results with fewer parameters than other unitary RNNs.
This work proves the asymptotic freeness of layerwise Jacobians in MLPs with Haar orthogonal matrices.
problem Proving the asymptotic freeness of layerwise Jacobians in multilayer perceptrons (MLPs).
method Replacing each layer's parameter matrix with itself multiplied by a Haar orthogonal matrix, and using the invariance of the MLP.
result Proves the asymptotic freeness of layerwise Jacobians in MLPs with Haar orthogonal matrices.
A new algorithm speeds up matrix operations in Neural Networks.
problem Time-consuming matrix operations in Neural Networks.
method An algorithm that increases the degree of parallelism of matrix multiplication.
result The algorithm speeds up several matrix operations in Neural Networks.
The paper solves PDEs from matrices with orthogonal columns, linking them to Hessian metrics and symmetric spaces.
problem Solving third order PDEs for strictly convex smooth functions.
method Geometric methods using Hessian metrics and symmetric spaces.
result Explicit solutions and a family of non-generic solutions with applications in Poisson geometry and Kahler structures.
One reflection suffices for orthogonal weights, reducing GPU usage.
problem Efficiently computing orthogonal weight matrices without high GPU utilization.
method Use an auxiliary neural network to compute one reflection instead of many.
result One reflection is sufficient for orthogonal weights, improving GPU utilization.
Proposes a new RNN structure to improve expressivity without sacrificing stability.
problem Exploding and vanishing gradient problems in RNNs and reduced expressivity.
method Introduces a non-normal RNN structure using Schur decomposition and splitting.
result Enhances expressivity while maintaining stability and training speed.
Orthogonal random features approximate a Bessel kernel, offering sharper bounds than random Fourier features.
problem Approximating Gaussian kernel efficiently for large datasets.
method Use of Haar orthogonal matrices to construct orthogonal random features and analyze their bias and variance.
result Orthogonal random features approximate a Bessel kernel, not the Gaussian kernel, with sharper bounds.
Researchers derive exclusive Racah matrix for complex representation with non-trivial multiplicities.
problem Constructing and understanding Racah matrices for complex representations with non-trivial multiplicities.
method Using effective field theory for arborescent knots, they deduced the exclusive Racah matrix for representation R=[3,1] with non-trivial multiplicities. result The exclusive Racah matrix $ar S$ for representation R=[3,1] is operator valued and depends on basis choices in intertwiner spaces. A new algorithm avoids retractions to optimize orthogonal matrices efficiently.
problem Optimizing functions over the manifold of orthogonal matrices efficiently.
method Landing algorithm that avoids retractions using potential energy.
result The landing algorithm is faster and less prone to numerical errors than retraction-based methods.
Efficiently optimizes orthogonal and Stiefel matrices on parallel units.
problem Optimization over orthogonal groups on parallel units.
method CWY and T-CWY transforms for parametrization and optimization.
result CWY and T-CWY methods lead to convergence on parallel units.
Chevalley theorems extended to isotropic functions on matrix spaces.
problem Extending Chevalley theorems to isotropic functions on matrix spaces.
method Proving ultradifferentiable Chevalley restriction theorems for various ultradifferentiable classes.
result Isotropic functions on symmetric matrices have ultradifferentiable regularity if and only if their diagonal restrictions do.
Improved Kalman filter for Stiefel manifold measurements.
problem Improving accuracy in measurements on Stiefel manifolds.
method Generalization of extended Kalman filter for Stiefel manifold-valued measurements.
result Significant improvement over raw measurements.
Curious structure of special orthogonal, unitary, and symplectic groups as products of Grassmannians discovered.
problem Understanding the structure of special orthogonal, unitary, and symplectic groups.
method Expressing these groups as products of Grassmannians realized as involution matrices.
result Special orthogonal, special unitary, and symplectic groups can be expressed as products of their corresponding Grassmannians.
Paper derives local Plücker formulas for special orthogonal groups.
problem Deriving Plücker formulas for special orthogonal groups.
method Reduction to classical A_n case.
result Local Plücker formulas for special orthogonal groups derived.
A new method for efficiently computing derivatives of skew-symmetric matrix exponentials.
problem Efficient computation of derivatives for skew-symmetric matrices.
method Characterization of invertibility, construction of nearby logarithm, and efficient implementation.
result Explicit formulae for differentiation and its inverse of skew-symmetric matrix exponentials.
Proposes a new graph representation method using tensor products.
problem Dynamic graph representation and theoretical properties.
method Bind-and-sum approach in hyperdimensional computing (HDC), tensor product as binding operation.
result Memory vs. size analysis of graph representation size scaling.
New framework solves low-rank optimization problems to certifiable optimality.
problem Low-rank optimization problems with certifiable solutions.
method Mixed-Projection Conic Optimization framework using symmetric projection matrices and outer-approximation algorithms.
result Solves low-rank problems to certifiable optimality, outperforming existing methods.
This paper explores how Transformers predict next tokens in autoregressive tasks.
problem Understanding the success of Transformers in autoregressive learning.
method Trained a Transformer on a next-token prediction task, focusing on commuting orthogonal matrices.
result Trained Transformers can be seen as implementing gradient descent for a specific objective function.
Bayesian inference on orthogonal matrices using Givens representation.
problem Posterior inference in models with orthogonal matrix parameters.
method Givens representation for posterior inference over the Stiefel manifold.
result Effective methods for transforming densities over the Stiefel manifold into Euclidean space.
Characterizes the local diffeomorphism structure of the exponential in the set of skew-symmetric matrices.
problem Characterizing the local diffeomorphism structure of the exponential in the set of skew-symmetric matrices.
method Introduce the diffeomorphic logarithm of special orthogonal matrices and an efficient algorithm.
result The region containing the principal logarithm has a special multiplicity structure.
New metrics improve landing algorithms for orthogonality constraints.
problem Optimizing landing algorithms with orthogonality constraints.
method Proposed a family of metrics over full-rank matrices to enhance landing algorithms.
result Natural extension of β-metric improves landing performance.
Study real logarithms of semi-simple matrices, focusing on differential structure.
problem Understanding the differential structure of real logarithms of semi-simple matrices.
method Examines the differential structure of real logarithms of semi-simple matrices under specific matrix types.
result Characterizes the differential structure of real logarithms of semi-simple matrices.
FedSPDnet improves federated learning for SPD matrices, outperforming existing methods.
problem Federated learning for SPD matrices with orthogonality constraints.
method Two efficient aggregation strategies: ProjAvg and RLAvg, preserving geometric structure.
result FedSPDnet outperforms federated EEGnet in F1 score and robustness to federation and partial participation.
Optimizes embedding accuracy for data variance and error.
problem Efficiently embedding data while minimizing distortion.
method Uses Johnson-Lindenstrauss embeddings with orthogonal matrices and singular-value latent variables.
result Achieves best accuracy in variance, mean-squared error, and length distortion.
Study differential properties of matrix square roots in specific cases.
problem Understanding matrix square roots in semi-simple, symmetric, and orthogonal cases.
method Analysis of differential and metric structures of real square roots of matrices under specific conditions.
result Differential properties of matrix square roots in semi-simple, symmetric, and orthogonal cases.
Study finds Calabi-Yau models' operator spectra match random matrix theory.
problem Understanding spectra of Calabi-Yau sigma models.
method Numerical methods for Ricci-flat metrics, averaging over complex structure moduli space.
result Spectrum matches Gaussian orthogonal ensemble of random matrix theory.
A new optimizer preserves orthogonality constraints on matrices efficiently.
problem Optimization on Stiefel manifold with orthogonality constraints.
method Interplay between continuous and discrete dynamics leading to a gradient-based optimizer with momentum.
result The method optimizes matrices on Stiefel manifold efficiently and accurately.
We found a way to code meanders and show they are idempotent.
problem Understanding and coding meandric permutations.
method We established a bijection between meanders and Gauss diagrams, and used this to construct matrices that are idempotent.
result Meandric permutations are idempotent over the field GF(2).
SpecNet2 improves spectral embedding without orthogonalization, achieving better performance and efficiency.
problem Improving spectral embedding methods for better performance and efficiency.
method Optimizes an equivalent objective of the eigen-problem without orthogonalization, allowing separate row and column sampling.
result Local and global convergence of the new objective using batch-based gradient descent is proven, and improved performance and efficiency are demonstrated on simulated and image datasets.
Deep neural networks' Jacobian spectrum becomes well-conditioned with orthogonal weights.
problem Understanding and handling the Jacobian spectrum of deep neural networks.
method Applying free probability theory to show almost sure asymptotic freeness of Jacobians in the wide limit.
result Layer-wise Jacobians of deep neural networks with orthogonal weights are almost surely asymptotically free.
Deep networks without non-linearities are equivalent to shallow ones.
problem Training deep orthogonal linear networks with no non-linearity.
method Riemannian gradient descent and gradient descent on factorization.
result Training deep overparametrized networks is equivalent to shallow ones.
Deterministic bounds for tensor singular values and vectors, differing from matrix cases.
problem Spectral learning of higher-order orthogonally decomposable tensors.
method Deterministic perturbation bounds for singular values and vectors of orthogonally decomposable tensors.
result Perturbation affects each essential singular value/vector in isolation, independent of multiplicity and distance from other singular values.
Formula for Laplace-Beltrami on orthogonal group in Euclidean coords.
problem Computing Laplace-Beltrami on constrained submanifolds.
method Embedded gradient vector field method, explicit formula derivation.
result Explicit formula for Laplace-Beltrami on orthogonal group.