Study shows how varying levels of supervision and orthonormality constraints affect generalization errors in subspace fitting.
problem Effects of varying levels of supervision and orthonormality constraints on generalization errors in subspace fitting.
method Flexible family of problems connecting unsupervised and supervised subspace fitting tasks, explored over a supervision-orthonormality plane.
result Generalization errors of subspace fitting problems follow double descent trends as they become more supervised and less orthonormally constrained.
New algorithm improves deep learning models' robustness without sacrificing accuracy.
problem Low-rank methods compromise model robustness against adversarial perturbations.
method Robust low-rank training via approximate orthonormal constraints.
result Ensures well-conditioning and better adversarial robustness without sacrificing model accuracy.
We shed new insights on the two commonly used updates for the online k-PCA problem, namely, Krasulina's and Oja's updates. We show that Krasulina's update corresponds to a projected gradient descent step on the Stiefel manifold of the orthonormal k-frames, while Oja's update amounts to a gradient descent step using…
Finsler space is differentiable manifold for which Minkowski space is the fiber of the tangent bundle. To understand structure of the reference frame in Finsler space, we need to understand the structure of orthonormal basis in Minkowski space. In this paper, we considered the definition of orthonormal basis in Minkows…
Finsler space is differentiable manifold for which Minkowski space is the fiber of the tangent bundle. To understand structure of the reference frame in Finsler space, we need to understand the structure of orthonormal basis in Minkowski space. In this paper, I considered the definition of orthonormal basis in Minkowsk…
Proves well-posedness for Einstein equations with totally geodesic timelike boundary condition.
problem Initial boundary value problem for Einstein equations with specific geometric boundary condition.
method ADM system, parallelly propagated orthonormal frame, modified evolution equations, hyperbolic systems, constraints propagation.
result First well-posedness result for Einstein equations with totally geodesic timelike boundary condition.
ManifoldFlow relaxes fixed-spectrum Stiefel layers to learn a positive spectrum.
problem Fixed-spectrum Stiefel layers impose rigid spectral constraints.
method Introduces ManifoldFlow, a relaxation that learns a positive spectrum while keeping the basis on the Stiefel manifold.
result Learnable SPD spectrum improves performance in various settings.
We develop Fourier methods to expand translation-invariant kernels.
problem Constructing orthonormal expansions for translation-invariant kernels.
method Fourier analytic technique to derive explicit expansions.
result Explicit expansions for various kernels (Matérn, Cauchy, Gaussian).
We show that every unimodular Lie algebra, of dimension at most 4, equipped with an inner product, possesses an orthonormal basis comprised of geodesic elements. On the other hand, we give an example of a solvable unimodular Lie algebra of dimension 5 that has no orthonormal geodesic basis, for any inner product.
Strictly enforcing orthonormality constraints on parameter matrices has been shown advantageous in deep learning. This amounts to Riemannian optimization on the Stiefel manifold, which, however, is computationally expensive. To address this challenge, we present two main contributions: (1) A new efficient retraction ma…
The ability to decompose a signal in an orthonormal basis (a set of orthogonal components, each normalized to have unit length) using a fast numerical procedure rests at the heart of many signal processing methods and applications. The classic examples are the Fourier and wavelet transforms that enjoy numerically effic…
Isometry pursuit identifies orthonormal submatrices from wide matrices.
problem Identifying isometric embeddings from wide matrices.
method A convex algorithm combining normalization and multitask basis pursuit.
result The method identifies isometric embeddings from interpretable dictionaries.
We generalize the orthonormal basis for the Gaussian RKHS described in \cite{MinhGaussian2010} to an infinite, continuously parametrized, family of orthonormal bases, along with some implications. The proofs are direct generalizations of those in \cite{MinhGaussian2010}.
Generalisation of a deep neural network (DNN) is one major concern when employing the deep learning approach for solving practical problems. In this paper we propose a new technique, named approximated orthonormal normalisation (AON), to improve the generalisation capacity of a DNN model. Considering a weight matrix W …
Categorical regressor variables are usually handled by introducing a set of indicator variables, and imposing a linear constraint to ensure identifiability in the presence of an intercept, or equivalently, using one of various coding schemes. As proposed in Yuan and Lin [J. R. Statist. Soc. B, 68 (2006), 49-67], the gr…
We show the existence of uniformly bounded sequences of increasing numbers of orthonormal sections of powers Lk of a positive holomorphic line bundle L on a compact Kähler manifold M. In particular, we construct for each positive integer k, orthonormal sections s1k,…,snkk in H0(M,Lk), $n_k\geβ…
Dictionary learning is the task of determining a data-dependent transform that yields a sparse representation of some observed data. The dictionary learning problem is non-convex, and usually solved via computationally complex iterative algorithms. Furthermore, the resulting transforms obtained generally lack structure…
New method uses almost orthonormal bases to prove low-degree lower bounds in complex statistical models.
problem Proving statistical-computational gaps in high-dimensional models with planted structures.
method Constructing an almost orthonormal polynomial basis under the planted distribution.
result Established new low-degree lower bounds for various complex models.
PolyILR: A Tree-Structured Orthonormal Decomposition of Compositional Data
problem Representing compositional data with hierarchical structure
method PolyILR: A canonical orthonormal decomposition of the Aitchison tangent space aligned with any tree topology
result PolyILR yields stable, interpretable features and enables inference at multiscale tree resolution
Two regularization techniques improve GCNN explainability and preference from chemists.
problem Difficulty in rationalizing molecular graph neural network predictions.
method Batch Representation Orthonormalization (BRO) and Gini regularization applied during GCNN training.
result Regularization improves GCNN attribution methods and preference from chemists.
Study shows properties of Gromov-Hausdorff limit of frame bundles for non-collapsed manifolds.
problem Characterizing the Gromov-Hausdorff limit of orthonormal frame bundles of non-collapsed manifolds with bounded Ricci curvature.
method Analysis of the Gromov-Hausdorff limit space of orthonormal frame bundles equipped with an almost canonical metric.
result The singular set of the limit space has codimension ≥4 and the complement contains an open and dense C1,α-Riemannian manifold. Paper introduces RKHM for more explicit variable structures analysis.
problem Explicitly analyzing structures among variables.
method Orthonormal systems in Hilbert C∗-modules, RKHM. result Theoretical and practical procedures for RKHM orthonormalization.
We study the holomorphic unitary representations of the Jacobi group based on Siegel-Jacobi domains. Explicit polynomial orthonormal bases of the Fock spaces based on the Siegel-Jacobi disk are obtained. The scalar holomorphic discrete series of the Jacobi group for the Siegel-Jacobi disk is constructed and polynomial …
Batch normalization has become ubiquitous in many state-of-the-art nets. It accelerates training and yields good performance results. However, there are various other alternatives to normalization, e.g. orthonormalization. The objective of this paper is to explore the possible alternatives to channel normalization with…
In the present work we construct a lift of a metric g on a 2-dimensional oriented Riemannian manifold M to a metric g^ on the total space P of the orthonormal frame bundle of M. We call this lift the \textit {Wagner lift}. Viktor Vladimirovich Wagner (1908 -1981) proposed a technique to extend a metric d…
Multi-view clustering is an important and fundamental problem. Many multi-view subspace clustering methods have been proposed, and most of them assume that all views share a same coefficient matrix. However, the underlying information of multi-view data are not fully exploited under this assumption, since the coefficie…
BOOOM optimizes orthonormal matrices without needing gradients.
problem Optimizing over the Stiefel manifold in non-convex, non-smooth settings.
method Global Givens rotation-based parametrization and Recursive Modified Pattern Search.
result BOOOM achieves strong performance across various optimization problems.
Bayesian method maps high-dimensional inputs to lower dimensions for efficient multi-fidelity Gaussian Process modeling.
problem Efficiently modeling high-dimensional inputs with low-dimensional latent variables for multi-fidelity Gaussian Processes.
method Bayesian approach with orthonormal projection matrix inference using Markov Chain Monte Carlo (MCMC) and Geodesic Monte Carlo sampling.
result Optimal transformations identified that improve computational efficiency in multi-fidelity Gaussian Process modeling.
The Frenet frame is generally known an orthonormal vector frame for curves. But, it does not always meet the needs of curve characterizations. In this study, with the help of associated curves of any spatial curve we obtained a new orthonormal frame which has the property that the second vector makes a constant angle w…
Paper provides linear convergence guarantees for KZIHT and KZPT methods.
problem Solving linear equation systems with sparse constraints.
method Combines Kaczmarz and iterative thresholding methods, using reshuffling data sampling.
result KZIHT and KZPT converge linearly to sparse solutions.
Higher-dimensional Milnor frames are characterized and contrasted with 3D Heisenberg and 4D nilpotent Lie algebras.
problem Characterizing higher-dimensional Milnor frames and their properties.
method Definition and classification of higher-dimensional Milnor frames and their relationship to known Lie algebras.
result Higher-dimensional Milnor frames are isomorphic to direct sums of 3D Heisenberg and 4D nilpotent Lie algebras and an abelian Lie algebra.
Quasi-orthonormal encoding reduces high dimensionality for categorical data.
problem High dimensionality and low sample size issues in categorical data encoding.
method Quasi-orthonormal encoding (QOE) for categorical data.
result QOE reduces dimensionality and improves machine learning performance.
We discuss conditions for the integrability of an almost complex structure defined on the total space of an induced Hopf S^3-bundle over a Sasakian manifold . As an application, we obtain an uncountable family of inequivalent complex structures on the Stiefel manifolds of orthonormal 2-frames in C^{n+1}, non compatible…
New proof of past stability for Kasner solutions in (3+1)-dimensional Einstein vacuum spacetime.
problem Stability of Kasner singularities in (3+1)-dimensional Einstein vacuum spacetime. method Developed (2+1) orthonormal-frame decomposition and symmetrization argument, applying Fuchsian techniques. result Perturbed solutions are asymptotically pointwise Kasner, geodesically incomplete, and crushing at the Big Bang singularity.
The study explores rigid constraints on almost Ricci-Bourguignon solitons on contact metric three-manifolds.
problem Investigating constraints on almost Ricci-Bourguignon solitons on contact metric three-manifolds.
method Using a local orthonormal \(\varphi\)-basis, derived the full component form of the almost Ricci-Bourguignon soliton equation.
result For contact metric three-manifolds satisfying \(Qξ=σξ\), a collinear potential field must vanish on the non-Sasakian region whenever \(ξ(σ)=0\).
For any compact Riemannian manifold (M,g) and its heat kernel embedding map psit from M into l2 constructed in [BBG], we study the higher derivatives of psit with respect to an orthonormal basis at x on M. As the heat flow time t goes to 0, it turns out the limiting angles between these derivative vect…
An almost-Riemannian structure on a surface is a generalized Riemannian structure whose local orthonormal frames are given by Lie bracket generating pairs of vector fields that can become collinear. The distribution generated locally by orthonormal frames has maximal rank at almost every point of the surface, but in ge…
The paper studies horizontal semimartingales on Riemannian manifolds and their connections to Euclidean spaces.
problem Stochastic lifts and anti-developments of semimartingales on Riemannian manifolds.
method Using stochastic differential geometry with jumps, the paper establishes correspondences between discontinuous semimartingales and their lifts.
result The paper extends previous results to include geodesics and small jumps, enabling the construction of martingales from local martingales.
The aim of this paper is to write an explicit orthonormal parallelization for all parallelizable products of spheres, using an explicit isomorphism with a trivial vector bundle.
A machine learning method selects optimal orthonormal bases for functional data analysis.
problem Lack of formal criteria for choosing initial orthonormal bases in functional data methods.
method Proposes a machine learning algorithm to learn and place knots for efficient orthogonal spline bases (splinets).
result Demonstrates efficiency, especially for sparse functional data and complex physical systems.
Graph clustering method uses templates to match vertices and outperforms classical methods.
problem Graph clustering with additional structural information.
method Formulates graph clustering as template matching, using orthonormal matrices for embedding.
result Method outperforms classical methods, especially for challenging cases.
Unified multi-view learning framework using OPLS with regularization and deep extensions.
problem Improving multi-view learning for classification and feature extraction.
method Orthonormalized Partial Least Squares (OPLS) with regularization and deep extensions.
result Unified multi-view learning framework with improved performance.
We propose an intriguingly simple method for the construction of adversarial images in the black-box setting. In constrast to the white-box scenario, constructing black-box adversarial images has the additional constraint on query budget, and efficient attacks remain an open problem to date. With only the mild assumpti…
We consider the control problem where, given an orthonormal tangent frame in the hyperbolic plane or three dimensional hyperbolic space, one is allowed to transport the frame a fixed distance r>0 along the geodesic in direction of the first vector, or rotate it in place a right angle. We characterize the values of …
The paper proves fundamental theorems for timelike surfaces in Minkowski 4-space.
problem Analyzing timelike surfaces without minimal points in Minkowski space.
method Introducing a pseudo-orthonormal frame field and deriving derivative formulas, proving a Bonnet-type theorem.
result Timelike surfaces are uniquely determined by six functions satisfying natural conditions.
New IRL algorithm for continuous state spaces with formal guarantees.
problem Finding a reward function for expert behavior in continuous state spaces.
method Modeling the system using orthonormal functions and providing correctness proofs.
result Proof of correctness and formal guarantees on sample and time complexity.
Muon optimizer improves federated learning performance.
problem Improving federated learning performance with Muon optimizer.
method Proposed FedMuon algorithm with orthonormalized update direction.
result FedMuon's learning rate is independent of problem-specific parameters.
A special p-form is a p-form which, in some orthonormal basis {e_μ}, has components φ_{μ_1...μ_p} = φ(e_{μ_1},..., e_{μ_p}) taking values in {-1,0,1}. We discuss graphs which characterise such forms.