Paper proposes a KGE framework that reduces training time and carbon footprint.
problem Efficient KGE learning with reduced computational cost and environmental impact.
method Full batch learning, Orthogonal Procrustes Analysis, non-negative-sampling training.
result Significant reduction in training time and carbon footprint compared to state-of-the-art approaches.
Method preserves correlations in synthetic data.
problem Preserving dependence structure of original data.
method Orthogonal Procrustes problem for restoring Pearson correlation.
result Restores Pearson correlation structure while preserving feature distributions and downstream tasks performance.
Multivariate Analysis (MVA) comprises a family of well-known methods for feature extraction that exploit correlations among input variables of the data representation. One important property that is enjoyed by most such methods is uncorrelation among the extracted features. Recently, regularized versions of MVA methods…
Study matches two noisy point clouds with geometric transformations and relabeling.
problem Matching two noisy point clouds with orthogonal transformations and relabeling.
method Information-theoretic results and Ping-Pong algorithm for computational alignment.
result The Ping-Pong algorithm retrieves the planted signal after one step.
Solves a fundamental problem in statistics and imaging with new methods.
problem Generalized Orthogonal Procrustes Problem (GOPP)
method Semidefinite relaxation (SDR) and generalized power method (GPM)
result GPM converges linearly to the global minimizer under large signal-to-noise ratio.
New measures link neural representation geometry to decoding ability.
problem Understanding how neural representations relate to decoding ability.
method Showed that popular similarity measures can be interpreted from a decoding perspective.
result Proved that measures like CKA and CCA quantify alignment between optimal linear readouts.
Proposes BONMI for integrating noisy matrices from multi-source data.
problem Integrating noisy matrices from multi-source data with block-wise missingness.
method Exploits orthogonal Procrustes problem to align eigenspaces and completes missing blocks.
result Statistical rate for eigenspace of underlying matrix comparable to independently missing assumption.
The problem of estimating sparse eigenvectors of a symmetric matrix attracts a lot of attention in many applications, especially those with high dimensional data set. While classical eigenvectors can be obtained as the solution of a maximization problem, existing approaches formulate this problem by adding a penalty te…
A new algorithm reduces communication in distributed SVD by p factors.
problem Efficiently compute SVD in distributed systems.
method LocalPower algorithm with weighted aggregation and periodic decay of iterations.
result Reduces communication cost by a factor of p. The Procrustes distance is used to quantify the similarity or dissimilarity of (3-dimensional) shapes, and extensively used in biological morphometrics. Typically each (normalized) shape is represented by N landmark points, chosen to be homologous (i.e. corresponding to each other), as far as possible, and the Procrust…
A new method for computing shape barycenters from point clouds using Procrustes-Wasserstein distance.
problem Computing representative shapes from point clouds with precise alignment and shape preservation.
method Developed a new distance metric (Procrustes-Wasserstein) and algorithms for computing barycenters.
result Superior performance in precise alignment and shape preservation compared to existing OT approaches.
Extends metrics for SPD matrices to infinite dimensions.
problem Lack of generalized forms for Riemannian metrics.
method Unitized Hilbert-Schmidt operators and extended Mahalanobis norm.
result Improved performance in high-dimensional comparisons.
We solve a key problem in cross-lingual learning using a novel approach.
problem Aligning word embeddings across different languages.
method We devise a direct solution to the Wasserstein-Procrustes problem.
result Our method improves existing UCL approaches significantly.
Modified Wasserstein metric for Gaussian distributions, invariant to isometries.
problem Distance measurement for latent Gaussian distributions invariant to isometries.
method Modified Benamou-Brenier approach leading to a Procrustes Wasserstein metric.
result For Gaussian distributions, the metric reduces to Euclidean distance between eigenvalues.
One of the common tasks in unsupervised learning is dimensionality reduction, where the goal is to find meaningful low-dimensional structures hidden in high-dimensional data. Sometimes referred to as manifold learning, this problem is closely related to the problem of localization, which aims at embedding a weighted gr…
We present the Procrustes measure, a novel measure based on Procrustes rotation that enables quantitative comparison of the output of manifold-based embedding algorithms (such as LLE (Roweis and Saul, 2000) and Isomap (Tenenbaum et al, 2000)). The measure also serves as a natural tool when choosing dimension-reduction …
LDLE embeds manifolds in lower dimensions with low distortion.
problem Embedding manifolds in lower dimensions with low distortion.
method Constructs local views using global eigenvectors of the graph Laplacian, registers them using Procrustes analysis, and tears manifolds apart for intrinsic dimension embedding.
result LDLE preserves distances up to a constant scale with low distortion.
A new method for aligning datasets without known correspondences.
problem Aligning datasets from different domains without labeled correspondences.
method Integrates MDS and Wasserstein Procrustes for joint optimization of embeddings and correspondences.
result Maps datasets to a common low-dimensional space without labeled correspondences.
Geometric stability measures neural network robustness, distinguishing from similarity metrics.
problem Lack of robustness in neural network representations.
method Introduces geometric stability, quantified by Shesha metric measuring self-consistency.
result Stability and similarity are uncorrelated, revealing distinct properties of neural network robustness.
Robustly computes intrinsic coordinates on point clouds using resampling and averaging.
problem Computing intrinsic coordinates on noisy or outlier-prone point clouds.
method Subsample data, vary hyperparameters, cluster candidate embeddings, identify representative embeddings, and average them using Procrustes analysis.
result Robust to noise and outliers, validated on synthetic and real data.
New method preserves privacy while detecting communities in distributed networks.
problem Privacy-preserving community detection in locally distributed multi-layer networks.
method Privacy-preserving Distributed Spectral Clustering (ppDSC) using randomized response mechanism.
result Developed a novel algorithm that maintains community structure while protecting privacy.
We consider the task of aligning two sets of points in high dimension, which has many applications in natural language processing and computer vision. As an example, it was recently shown that it is possible to infer a bilingual lexicon, without supervised data, by aligning word embeddings trained on monolingual data. …
We propose a data aggregation-based algorithm with monotonic convergence to a global optimum for a generalized version of the L1-norm error fitting model with an assumption of the fitting function. The proposed algorithm generalizes the recent algorithm in the literature, aggregate and iterative disaggregate (AID), whi…
DFRot improves LLMs by reducing outlier and massive activation effects.
problem Reducing outlier and massive activation effects in rotated LLMs.
method Weighted loss function and orthogonal Procrustes transforms for rotation matrix refinement.
result DFRot achieves dual free (Outlier-Free and Massive Activation-Free) with significant improvements in perplexity.
Algorithm recovers permutations of high-dimensional Gaussian vectors with constant correlation.
problem Recovering permutations of high-dimensional Gaussian vectors with constant correlation.
method Computing and comparing weighted counts of specially chosen wide trees.
result Polynomial-time algorithm for exact recovery at constant correlation.
FedPower improves eigenspace estimation privacy in federated learning.
problem Privacy breaches and communication challenges in federated eigenspace estimation.
method FedPower uses a power method with local power iterations and global aggregation, weighted by OPT, and adds Gaussian noise for privacy.
result FedPower provides convergence bounds and demonstrates effectiveness in experiments.
Novel prior for orthogonal functions improves functional component estimation.
problem Improving orthogonality in functional principal component analysis.
method Sequential adaptive priors for orthogonal functions using hierarchical conditionally normal distributions.
result Proposed prior leads to nearly orthogonal posterior estimates.
Proposes FOAGP for efficient orthogonal effect decomposition of black-box computer experiments.
problem Challenges in sensitivity analysis of black-box computer experiments with complex, nonlinear functional outputs.
method Functional-output orthogonal additive Gaussian process (FOAGP) with conditional orthogonality constraint.
result Demonstrates effectiveness in orthogonal effect decomposition and variance decomposition through simulations and real-world application.
Study finds whitepaper narratives do not predict market factor structure.
problem Predicting market behavior from cryptocurrency whitepaper claims.
method Zero-shot NLP classification combined with CP tensor decomposition of market data.
result Weak alignment between whitepaper claims and market statistics and latent factors.
Change detection in dynamic networks is an important problem in many areas, such as fraud detection, cyber intrusion detection and health care monitoring. It is a challenging problem because it involves a time sequence of graphs, each of which is usually very large and sparse with heterogeneous vertex degrees, resultin…
Representational similarity analysis (RSA) has been shown to be an effective framework to characterize brain-activity profiles and deep neural network activations as representational geometry by computing the pairwise distances of the response patterns as a representational dissimilarity matrix (RDM). However, how to p…
Mapping and translating professional but arcane clinical jargons to consumer language is essential to improve the patient-clinician communication. Researchers have used the existing biomedical ontologies and consumer health vocabulary dictionary to translate between the languages. However, such approaches are limited b…
Improves model predictability by mixing forecasts and orthogonalizing models.
problem Redundant models contaminate model space and degrade predictive performance.
method Principal Component Analysis for model orthogonalization in Bayesian forecast mixing.
result Better prediction accuracy and excellent uncertainty quantification.
Unified framework for multi-view learning with orthogonal projections.
problem Learning individual orthogonal projections for multiple views.
method Successive approximations via eigenvectors, iterative Krylov subspace method.
result Consistently competitive and often better than existing methods.
Proposes a method to align language and image data.
problem Aligning language and image data for better understanding.
method Uses triplet loss to learn consistent embeddings of language and images.
result Outperforms four baselines across multiple metrics.
Paper optimizes tensor deflation for non-orthogonal signals.
problem Recovering low-rank signals from noisy tensors with correlated components.
method Developed an asymptotic analysis and optimized deflation procedure using random tensor theory.
result Proposed an efficient tensor deflation algorithm that optimizes a parameter introduced in the deflation mechanism.
TGCCA analyzes higher-order tensors using orthogonal rank-R CP decomposition.
problem Handling higher-order structures in multi-block data analysis.
method Tensor Generalized Canonical Correlation Analysis (TGCCA) with orthogonal rank-R CP decomposition.
result TGCCA outperforms state-of-the-art methods on simulated and real data.
DFSOS improves sparse discriminant analysis for high-dimensional data.
problem Sparse discriminant analysis in high-dimensional settings with feature selection.
method Deflation-Free Sparse Optimal Scoring (DFSOS) using Bregman iteration and orthogonality-constrained optimization.
result DFSOS achieves comparable or better classification accuracy than deflation-based methods.
Deep model predicts shapes of curves with multiple covariates.
problem Predicting shapes of planar curves with various covariates.
method Deep learning model using complex-valued functions, conditional covariance smoother with modality-specific encoders.
result Model accurately predicts shapes of curves with multimodal covariates.
PROD method improves high-dimensional regression by handling strong correlations.
problem Violation of Irrepresentable Condition in LASSO for high-dimensional data.
method PROD procedure based on orthogonal decomposition of design matrix.
result PROD enhances performance of high-dimensional penalized regression.
Python package for SPD matrix distances, reproducible and extensible.
problem Computing distances between SPD matrices for various applications.
method Unified, extensible framework supporting multiple SPD metrics.
result Reproducible and accessible SPD matrix comparison tool.
We have observed an interesting, yet unexplained, phenomenon: Semidefinite programming (SDP) based relaxations of maximum likelihood estimators (MLE) tend to be tight in recovery problems with noisy data, even when MLE cannot exactly recover the ground truth. Several results establish tightness of SDP based relaxations…
This paper concerns dictionary learning, i.e., sparse coding, a fundamental representation learning problem. We show that a subgradient descent algorithm, with random initialization, can provably recover orthogonal dictionaries on a natural nonsmooth, nonconvex ℓ1 minimization formulation of the problem, under mi…
Proposes a new method for analyzing multimodal neuroimaging data.
problem Combining interpretability and flexibility in multimodal data analysis.
method Orthogonalized kernel debiased machine learning approach.
result Established consistency and asymptotic normality of the estimated primary parameter.
We propose an efficient algorithm for solving orthogonal canonical correlation analysis (OCCA) in the form of trace-fractional structure and orthogonal linear projections. Even though orthogonality has been widely used and proved to be a useful criterion for pattern recognition and feature extraction, existing methods …
LOFT separates subspace rotation and transformation for orthogonal fine-tuning.
problem Conflating subspace rotation and transformation in orthogonal fine-tuning.
method LOFT explicitly separates subspace rotation and transformation, using task-aware support selection.
result LOFT recovers principal-subspace orthogonal adaptation and improves efficiency-performance trade-off.
Orthogonal Matching Pursuit (OMP) has long been considered a powerful heuristic for attacking compressive sensing problems; however, its theoretical development is, unfortunately, somewhat lacking. This paper presents an improved Restricted Isometry Property (RIP) based performance guarantee for T-sparse signal reconst…
Novel tRSA combines geometry and topology for brain and model analysis.
problem Traditional RSA overlooks topological information in neural representations.
method Topological RSA (tRSA) using nonlinear monotonic transforms.
result Robust model comparisons and novel insights into neural computation.