Study removes bias from chest X-ray embeddings using orthogonalization.
problem Reduces bias in chest X-ray embeddings due to protected features.
method Orthogonalization technique to remove protected feature effects.
result Orthogonalization removes bias and makes predictions of protected attributes infeasible.
Transformers learn to recall with non-orthogonal embeddings in realistic settings.
problem Understanding how transformers store and retrieve knowledge in practical scenarios.
method Analyzing a single-layer transformer with random embeddings trained on a token-retrieval task.
result Explicit formulas for the model's storage capacity reveal a multiplicative dependence on sample size, embedding dimension, and sequence length.
The paper explores how semantic independence can be captured in text embeddings using partial orthogonality.
problem Capturing semantic independence in text embeddings.
method Developed a theory and methods based on partial orthogonality to demonstrate semantic independence.
result Partial orthogonality captures semantic independence in text embeddings.
SpecNet2 improves spectral embedding without orthogonalization, achieving better performance and efficiency.
problem Improving spectral embedding methods for better performance and efficiency.
method Optimizes an equivalent objective of the eigen-problem without orthogonalization, allowing separate row and column sampling.
result Local and global convergence of the new objective using batch-based gradient descent is proven, and improved performance and efficiency are demonstrated on simulated and image datasets.
Optimizes embedding accuracy for data variance and error.
problem Efficiently embedding data while minimizing distortion.
method Uses Johnson-Lindenstrauss embeddings with orthogonal matrices and singular-value latent variables.
result Achieves best accuracy in variance, mean-squared error, and length distortion.
SOLAR improves search efficiency and accuracy with sparse, orthogonal embeddings.
problem Bottleneck of indexing large dense vectors and NNS for query efficiency and accuracy.
method Proposes SOLAR embeddings: sparse, orthogonal, learned, and random vectors across multiple GPUs.
result Successfully trains 500K dimensional SOLAR embeddings for 1.6M books and multi-label classification.
In this paper, we propose a scalable algorithm for spectral embedding. The latter is a standard tool for graph clustering. However, its computational bottleneck is the eigendecomposition of the graph Laplacian matrix, which prevents its application to large-scale graphs. Our contribution consists of reformulating spect…
Are Graph Neural Networks (GNNs) fair? In many real world graphs, the formation of edges is related to certain node attributes (e.g. gender, community, reputation). In this case, standard GNNs using these edges will be biased by this information, as it is encoded in the structure of the adjacency matrix itself. In this…
In every dimension n≥3 we introduce a class of orthogonal graph-manifolds and prove that the fundamental group of any orthogonal graph-manifold quasi-isometrically embeds into a product of n trees. As a consequence, we obtain that asymptotic and linearly-controlled asymptotic dimensions of such group are equal t…
Minimal dimensions found for flag manifolds embeddings.
problem Finding the smallest dimensions for flag manifolds embeddings.
method Equivariant embeddings of orthogonal and unitary groups acting on real and complex flag manifolds.
result Minimal dimensions achieved at isospectral models.
Paper proposes a KGE framework that reduces training time and carbon footprint.
problem Efficient KGE learning with reduced computational cost and environmental impact.
method Full batch learning, Orthogonal Procrustes Analysis, non-negative-sampling training.
result Significant reduction in training time and carbon footprint compared to state-of-the-art approaches.
Formula for Laplace-Beltrami on orthogonal group in Euclidean coords.
problem Computing Laplace-Beltrami on constrained submanifolds.
method Embedded gradient vector field method, explicit formula derivation.
result Explicit formula for Laplace-Beltrami on orthogonal group.
Explores tensor products in hyperdimensional computing.
problem Understanding tensor products in hyperdimensional computing.
method Generalized results from graph embeddings to vector symbolic architectures and hyperdimensional computing.
result Tensor product is the most general and expressive representation with errorless unbinding and detection.
Paper derives local Plücker formulas for special orthogonal groups.
problem Deriving Plücker formulas for special orthogonal groups.
method Reduction to classical A_n case.
result Local Plücker formulas for special orthogonal groups derived.
Embed spherical quandles into Lie groups smoothly.
problem Embedding spherical quandles into Lie groups.
method Construct smooth embeddings into conjugation quandles of Lie groups.
result Embeddings into orthogonal, Spin, or Pin groups in dimensions 1 and 3 compared with Bergman and Akita's.
We examine a class of embeddings based on structured random matrices with orthogonal rows which can be applied in many machine learning applications including dimensionality reduction and kernel approximation. For both the Johnson-Lindenstrauss transform and the angular kernel, we show that we can select matrices yield…
Conformal Autoencoders infer intrinsic dimensionality and impose invariance.
problem Detecting intrinsic dimensionality and imposing invariance in nonlinear manifold data.
method Imposing orthogonality conditions on latent variables to infer intrinsic dimensionality and build coordinate invariance.
result The method can infer intrinsic dimensionality and build coordinate invariance on submanifolds.
Classifies linear embeddings of grassmannians and ind-grassmannians.
problem Understanding linear embeddings of grassmannians and ind-grassmannians.
method Classification through isomorphism of Picard groups and direct limits.
result Most linear embeddings of grassmannians are equivariant.
Geometrically transforms word embeddings into a common space for better comparison.
problem Comparing embeddings from different sources is challenging.
method Applies orthogonal rotations and Mahalanobis scaling to transform embeddings into a shared latent space.
result The method improves word similarity and analogy tasks.
Develops a new framework for temporal anchoring in deep embedding spaces.
problem Temporal anchoring in deep embedding spaces, especially drift and convergence issues.
method Operator-theoretic framework with drift maps and event-indexed blocks, proving convergence theorems and equivalence theorems.
result Proves convergence theorems and equivalence theorems for the proposed framework.
Minimal sphere dimension for equivariant embedding of circles.
problem Embedding a bouquet of circles into a sphere.
method Finding the minimal dimension of the sphere for equivariant embedding.
result The minimal dimension is 2g−1. CAMEL enhances manifold embedding and learning with curvature metrics.
problem High-dimensional data classification, dimension reduction, and visualization.
method CAMEL uses a Riemannian manifold with curvature metrics for enhanced expressibility and interpretability.
result CAMEL outperforms state-of-the-art methods on high-dimensional datasets.
The Variational Autoencoder (VAE) is a powerful architecture capable of representation learning and generative modeling. When it comes to learning interpretable (disentangled) representations, VAE and its variants show unparalleled performance. However, the reasons for this are unclear, since a very particular alignmen…
Characterizes quasi-isometric embeddings in coarsely Lipschitz category.
problem Understanding quasi-isometric embeddings in geometric terms.
method Formalizes quasi-isometric embeddings as regular monomorphisms in coarsely Lipschitz category.
result Quasi-isometric embeddings are equivalently characterised as effective, strong, or extremal monomorphisms.
Proposes a novel method for detecting novelty in multi-modal data.
problem Challenges in detecting novelty in high-dimensional, multi-modal data.
method Orthogonalized latent space for disentangling features and defining novelty score.
result Proposed method outperforms state-of-the-art algorithms in novelty detection.
An orthogonal complex structure on a domain in R^4 is a complex structure which is integrable and is compatible with the Euclidean metric. This gives rise to a first order system of partial differential equations which is conformally invariant. We prove two Liouville-type uniqueness theorems for solutions of this syste…
In the machine learning field, dimensionality reduction is an important task. It mitigates the undesired properties of high-dimensional spaces to facilitate classification, compression, and visualization of high-dimensional data. During the last decade, researchers proposed many new (non-linear) techniques for dimensio…
New tensors capture intrinsic embedding data of conformal hypersurfaces.
problem Classifying hypersurface invariants in conformal manifolds.
method Constructing curvatures and conformal fundamental forms.
result Finite family of tensors captures extrinsic embedding data.
Proposes EOT eigenmaps for aligning and embedding multiple datasets.
problem Aligning and embedding multiple datasets with shared structures but individual distortions.
method Entropic Optimal Transport (EOT) eigenmaps, leveraging leading singular vectors of EOT plan matrix.
result Proves theoretical guarantees and favorable properties for aligning and embedding datasets.
A new knot invariant measures crossings in three orthogonal directions.
problem Defining a new knot invariant for certain knot diagrams.
method Defining the simultaneous crossing number for knots with doubly transvergent diagrams.
result The limit of the ratio of the new invariant to the usual crossing number is at most 8.
The paper finds an upper limit for the length of geodesic chords on Riemannian manifolds.
problem Finding the maximum length of geodesic chords on Riemannian manifolds.
method Establishing an upper bound for geodesic chord length using geometric bounds on the manifold.
result An upper bound for the length of geodesic chords is derived, with a specific example for 2-dimensional spheres.
We characterize those spacetimes which admit a isometric (or conformal) embedding in some Lorentz-Minkowski space L^N. In particular, any globally hyperbolic spacetime can be isometrically embedded in L^N. This is proven by a result of its own interest: the construction of a smooth time function whose gradient is bound…
In this article, we study a free boundary isometric embedding problem for abstract Riemannian two-manifolds with the topology of the disc. Under the assumption of positive Gauss curvature and geodesic curvature of the boundary being equal to one, we show that any such disc may be isometrically embedded into the Euclide…
Bayesian method maps high-dimensional inputs to lower dimensions for efficient multi-fidelity Gaussian Process modeling.
problem Efficiently modeling high-dimensional inputs with low-dimensional latent variables for multi-fidelity Gaussian Processes.
method Bayesian approach with orthonormal projection matrix inference using Markov Chain Monte Carlo (MCMC) and Geodesic Monte Carlo sampling.
result Optimal transformations identified that improve computational efficiency in multi-fidelity Gaussian Process modeling.
We prove the existence of a family of embedded doubly periodic minimal surfaces of (quotient) genus g with orthogonal ends that generalizes the classical doubly periodic surface of Scherk and the genus-one Scherk surface of Karcher. The proof of the family of immersed surfaces is by induction on genus, while the proo…
Globally hyperbolic spacetimes with timelike boundary (M=M∪∂M,g) are the natural class of spacetimes where regular boundary conditions (eventually asymptotic, if M is obtained by means of a conformal embedding) can be posed. ∂M represents the naked singularities and c…
In this paper we explore the "vector semantics" problem from the perspective of "almost orthogonal" property of high-dimensional random vectors. We show that this intriguing property can be used to "memorize" random vectors by simply adding them, and we provide an efficient probabilistic solution to the set membership …
This paper proves area-minimizing cones over Grassmannian manifolds.
problem Determine if cones over Grassmannian manifolds are area-minimizing.
method Detailed descriptions of embedding maps using Hermitian orthogonal projectors, re-proving area-minimization using Lawlor's Curvature Criterion.
result All cones over Grassmannian manifolds are area-minimizing except for oriented real Grassmannians.
A fast feature selection method using OLS and SOCC for classification.
problem Feature selection for linear classification.
method Orthogonal Least Squares (OLS) with Squared Orthogonal Correlation Coefficient (SOCC).
result The proposed method outperforms other feature selection methods in speed and accuracy.
Sharp fractional Sobolev inequalities on closed manifolds identified.
problem Critical fractional Sobolev embedding on closed Riemannian manifolds.
method Intrinsic heat-kernel based framework, determining optimal coefficients, proving sharp inequalities.
result Sharp p-power inequality and almost sharp inequality established. Near isometric orthogonal embeddings to lower dimensions are a fundamental tool in data science and machine learning. In this paper, we present the construction of such embeddings that minimizes the maximum distortion for a given set of points. We formulate the problem as a non convex constrained optimization problem. …
IMA addresses non-identifiability in nonlinear ICA by assuming orthogonal Jacobian columns.
problem Non-identifiability in nonlinear ICA.
method IMA assumes orthogonal Jacobian columns and extends to manifold settings.
result IMA circumvents non-identifiability issues and can be beneficial for higher-dimensional observations.
Deep neural networks trained using a softmax layer at the top and the cross-entropy loss are ubiquitous tools for image classification. Yet, this does not naturally enforce intra-class similarity nor inter-class margin of the learned deep representations. To simultaneously achieve these two goals, different solutions h…
Synthetic construction of Hopf fibration in 4D space.
problem Visualizing 4D objects in 3D space.
method Double orthogonal projection method to visualize 4D space.
result Direct synthetic construction of 3-sphere fibers from 2-sphere points.
The kernel embedding algorithm is an important component for adapting kernel methods to large datasets. Since the algorithm consumes a major computation cost in the testing phase, we propose a novel teacher-learner framework of learning computation-efficient kernel embeddings from specific data. In the framework, the h…
New Zoll families of minimal spheres found in spheres and projective spaces.
problem Finding new Zoll families of minimal spheres in various spaces.
method Equivariant constructions using Nash-Moser-Hamilton implicit function theorem.
result First examples of metrics on real projective spaces with Zoll families of minimal projective hyperplanes.
We analyze the spectral clustering procedure for identifying coarse structure in a data set x1,…,xn, and in particular study the geometry of graph Laplacian embeddings which form the basis for spectral clustering algorithms. More precisely, we assume that the data is sampled from a mixture model supported on …
Parsimonious Dynamic Mode Decomposition selects sparse modes robustly.
problem Manual tuning of sparsity parameters in traditional DMD.
method Time-delay embedding and Orthogonal Matching Pursuit.
result Autonomously determines optimally sparse subset of modes.