Novel prior for orthogonal functions improves functional component estimation.
problem Improving orthogonality in functional principal component analysis.
method Sequential adaptive priors for orthogonal functions using hierarchical conditionally normal distributions.
result Proposed prior leads to nearly orthogonal posterior estimates.
New method relaxes PCA orthogonality constraints using explained variance of correlated components.
problem Difficulty in using PCA for sparse design due to orthogonality constraints and non-differentiable penalty.
method Introduce expvar(Y) to measure variance explained by correlated components, relax orthogonality constraints.
result Two expvar(Y) definitions suitable for block PCA formulations without orthogonality constraints.
Paper optimizes tensor deflation for non-orthogonal signals.
problem Recovering low-rank signals from noisy tensors with correlated components.
method Developed an asymptotic analysis and optimized deflation procedure using random tensor theory.
result Proposed an efficient tensor deflation algorithm that optimizes a parameter introduced in the deflation mechanism.
We propose a penalized orthogonal-components regression (POCRE) for large p small n data. Orthogonal components are sequentially constructed to maximize, upon standardization, their correlation to the response residuals. A new penalization framework, implemented via empirical Bayes thresholding, is presented to effecti…
In the multiple linear regression setting, we propose a general framework, termed weighted orthogonal components regression (WOCR), which encompasses many known methods as special cases, including ridge regression and principal components regression. WOCR makes use of the monotonicity inherent in orthogonal components …
New method solves sparse PCA for multiple components efficiently.
problem Sparse PCA for multiple orthogonal components.
method Reformulates orthogonality as rank constraints, uses semidefinite relaxations and bounds.
result Exact solutions with near-optimal variance explained and orthogonality.
Improves model predictability by mixing forecasts and orthogonalizing models.
problem Redundant models contaminate model space and degrade predictive performance.
method Principal Component Analysis for model orthogonalization in Bayesian forecast mixing.
result Better prediction accuracy and excellent uncertainty quantification.
We propose orthogonality as a necessary condition for disentangling aleatoric and epistemic uncertainty.
problem Jointly estimating aleatoric and epistemic uncertainty is problematic and non-trivial.
method We propose orthogonality as a necessary condition for disentanglement and construct UDE to measure orthogonality and consistency.
result Orthogonality and consistency are necessary and sufficient criteria for disentanglement.
msPCA solves sparse PCA for multiple components efficiently.
problem Sparse principal component analysis with multiple components.
method Alternating maximization algorithm for sparse loading vectors, with orthogonality or zero correlation constraints.
result Achieves high variance explained with sparse components and controlled feasibility violations.
New algorithms improve tensor CP decomposition under mild conditions.
problem Improving tensor CP decomposition with theoretical guarantees under mild incoherence conditions.
method Composite PCA and Concurrent Orthogonalization algorithms.
result Theoretical guarantees and practical superiority over existing methods.
Improves statistical learning bounds with self-concordant losses.
problem Statistical prediction with nuisance components.
method Orthogonal statistical learning with self-concordant loss.
result Non-asymptotic bounds on excess risk improved by a dimension factor.
Semi-parametric framework for nonlinear system identification
problem Nonlinear system identification
method Orthogonal Gaussian process regression
result Interpretable models from incomplete physics
Method constructs orthogonal curvilinear coordinates in constant curvature spaces.
problem Creating orthogonal coordinates in spaces of constant curvature.
method Modification of Krichever's method for Euclidean space, applied to constant curvature spaces.
result Examples of orthogonal coordinate systems on the sphere and hyperbolic plane constructed.
The paper shows how gradient flow on over-parametrized tensor decomposition behaves like deflation.
problem Understanding the training dynamics of gradient flow on tensor decomposition.
method Empirical observation and mathematical proof of gradient flow dynamics for orthogonally decomposable tensors.
result Gradient flow dynamics for orthogonally decomposable tensors follows a tensor deflation process, recovering all tensor components.
A new method for sparse PCA using orthogonal rotations and soft-thresholding.
problem Sparse PCA with a new basis using orthogonal rotations.
method Initialize with leading principal components, apply kimesk orthogonal rotation, and soft-threshold the rotated components. result The proposed method is more stable and explains more variance compared to alternatives.
Bayesian SPCA method tackles orthogonality constraint with spike and slab prior.
problem Bayesian SPCA method for high-dimensional data with orthogonality constraint.
method Parameter-expanded coordinate ascent variational inference (PX-CAVI) with spike and slab prior.
result PX-CAVI algorithm outperforms existing SPCA approaches in performance.
We consider the problem of estimating multiple principal components using the recently-proposed Sparse and Functional Principal Components Analysis (SFPCA) estimator. We first propose an extension of SFPCA which estimates several principal components simultaneously using manifold optimization techniques to enforce orth…
Optimizing over the set of orthogonal matrices is a central component in problems like sparse-PCA or tensor decomposition. Unfortunately, such optimization is hard since simple operations on orthogonal matrices easily break orthogonality, and correcting orthogonality usually costs a large amount of computation. Here we…
Method estimates heterogeneous causal effects on networks using orthogonal learning.
problem Challenges in estimating causal effects on networks due to treatment effects on both treated and neighbors, and network homophily.
method Two-stage orthogonal learning framework: first stage uses graph neural networks for nuisance components, second stage residualizes and interpretable attention-based model for causal effects.
result Improves heterogeneous effect estimation and supports interpretable analyses.
We construct an explicit topological model (similar to the topological Springer fibers appearing in work of Khovanov and Russell) for every two-row Springer fiber associated with the even orthogonal group and prove that the respective topological model is homeomorphic to its corresponding Springer fiber. This confirms …
IMA addresses non-identifiability in nonlinear ICA by assuming orthogonal Jacobian columns.
problem Non-identifiability in nonlinear ICA.
method IMA assumes orthogonal Jacobian columns and extends to manifold settings.
result IMA circumvents non-identifiability issues and can be beneficial for higher-dimensional observations.
We develop a mean-field theory for multi-component ICA in high dimensions.
problem Understanding multi-component ICA in high-dimensional settings.
method Asymptotically exact mean-field theory for multi-component online ICA.
result Explicit learnability boundaries and competition conditions linking step size, data moments, and initialization.
A new classifier uses weighted orthogonal regression for robust classification with limited data.
problem Challenges in classification with insufficient training data.
method Exploits intrinsic structure of data through Eigen components with specific weights determined by eigenvalues.
result Robust learning in classification problems with limited data.
Lifts isometries in orbit spaces for compact groups.
problem Isometries in orbit spaces of compact groups.
method Equivariant isometry of original Euclidean space.
result Simple formula for connected component of isometry group.
Inverted file and asymmetric distance computation (IVFADC) have been successfully applied to approximate nearest neighbor search and subsequently maximum inner product search. In such a framework, vector quantization is used for coarse partitioning while product quantization is used for quantizing residuals. In the ori…
New findings on Kähler manifolds restrict orthogonal coordinates existence.
problem Existence of orthogonal coordinates on Kähler manifolds.
method Algebraic and geometric techniques applied to Kähler manifolds.
result No nontrivial self-dual Kähler 4-manifolds or Ricci-flat Kähler 4-manifolds support orthogonal coordinates.
LOFT separates subspace rotation and transformation for orthogonal fine-tuning.
problem Conflating subspace rotation and transformation in orthogonal fine-tuning.
method LOFT explicitly separates subspace rotation and transformation, using task-aware support selection.
result LOFT recovers principal-subspace orthogonal adaptation and improves efficiency-performance trade-off.
The study examines how gamma positivity and PL homeomorphism types affect simplicial spheres.
problem Understanding gamma positivity and its relation to PL homeomorphism types in simplicial spheres.
method Using edge contractions and the link condition as proxies for flagness, the study analyzes the effect of gamma positivity on simplicial spheres.
result The link condition has a trivial effect on gamma vectors of high-dimensional simplicial spheres with nonnegative gamma vectors.
New algorithms reduce orthogonality constraint enforcement time in machine learning.
problem Efficiently solving orthogonality constraints in machine learning.
method Extending the landing algorithm to Stiefel manifold, incorporating stochastic and variance reduction techniques.
result All proposed methods achieve the same convergence rate as Riemannian counterparts enforcing constraints.
Through Cayley and Langlands type correspondences, we give a geometric description of the moduli spaces of real orthogonal and symplectic Higgs bundles of any signature in the regular fibres of the Hitchin fibration. As applications of our methods, we complete the concrete abelianization of real slices corresponding to…
The paper argues for using Neyman orthogonal score for balancing in debiased machine learning.
problem Debiased machine learning requires a proper approach to balance covariates.
method The paper advocates for using Riesz regression with basis functions of X for balancing.
result Covariate balancing is only valid when the score-relevant regression error is a function of covariates alone.
Tensor CANDECOMP/PARAFAC (CP) decomposition is an important tool that solves a wide class of machine learning problems. Existing popular approaches recover components one by one, not necessarily in the order of larger components first. Recently developed simultaneous power method obtains only a high probability recover…
Principal component analysis (PCA) is an unsupervised method for learning low-dimensional features with orthogonal projections. Multilinear PCA methods extend PCA to deal with multidimensional data (tensors) directly via tensor-to-tensor projection or tensor-to-vector projection (TVP). However, under the TVP setting, i…
The Shapley value theory is used for risk allocation in non-orthogonal risk factors.
problem Risk allocation among non-orthogonal risk factors in financial portfolios.
method Using Shapley value from cooperative game theory to allocate risk contributions.
result Explicit formulas and numerical algorithms for calculating risk allocations are derived.
Sparse linear regression, which entails finding a sparse solution to an underdetermined system of linear equations, can formally be expressed as an l0-constrained least-squares problem. The Orthogonal Least-Squares (OLS) algorithm sequentially selects the features (i.e., columns of the coefficient matrix) to greedil…
Proposes σ-PCA to learn identifiable linear transformations without whitening.
problem Cannot identify axes with equal variances in PCA.
method Unified model for linear and nonlinear PCA, introducing a missing piece to eliminate rotational indeterminacy.
result Eliminates subspace rotational indeterminacy in PCA.
New PHO formula improves SSNs performance.
problem Suboptimal network estimation and degenerated predictions in SSNs.
method Non-invasive post-hoc orthogonalization (PHO) to identify model components.
result Better estimation and prediction quality in SSNs.
A new method for disentangled representations without supervision.
problem Learning disentangled representations in unsupervised learning.
method Constr-DRKM, a deep kernel method with orthogonality constraints.
result Constr-DRKM performs similarly to β-VAE on disentanglement metrics.
New framework extends ICA for non-independent variables, identifying pairwise mean independence.
problem Non-independent variables complicating ICA recovery.
method Algebraic recovery algorithm based on least-squares optimization over the orthogonal group.
result Pairwise mean independence is identifiable, robust to independence constraints.
We study the limiting case of the Krichever construction of orthogonal curvilinear coordinate systems when the spectral curve becomes singular. We show that the case when the curve is reducible and all its irreducible components are rational curves the construction procedure reduces to solving systems of linear equatio…
ICA accurately estimates treatment effects even with confounders.
problem Estimating treatment effects in the presence of confounding variables.
method Uses Independent Component Analysis (ICA) to identify latent sources and estimate mixing coefficients.
result Linear ICA can consistently estimate multiple treatment effects, even with Gaussian confounders, and is more sample-efficient than Orthogonal Machine Learning (OML).
We introduce three novel semi-parametric extensions of probabilistic canonical correlation analysis with identifiability guarantees. We consider moment matching techniques for estimation in these models. For that, by drawing explicit links between the new models and a discrete version of independent component analysis …
Principal component analysis (PCA) is largely adopted for chemical process monitoring and numerous PCA-based systems have been developed to solve various fault detection and diagnosis problems. Since PCA-based methods assume that the monitored process is linear, nonlinear PCA models, such as autoencoder models and kern…
In this article, we study rectifying curves in arbitrary dimensional Euclidean space. A curve is said to be a rectifying curve if, in all points of the curve, the orthogonal complement of its normal vector contains a fixed point. We characterize rectifying curves in the n-dimensional Euclidean space in different ways…
We address the problem of defining a group sparse formulation for Principal Components Analysis (PCA) - or its equivalent formulations as Low Rank approximation or Dictionary Learning problems - which achieves a compromise between maximizing the variance explained by the components and promoting sparsity of the loading…
New algorithms solve tensor problems with random components using SDP.
problem Exact tensor nuclear norm, decomposition, and completion for random tensors.
method Degree-4 Sum of Squares (SOS) semidefinite programs.
result Exact solutions for tensor nuclear norm, decomposition, and completion with random asymmetric components.
Let SO+(p,q) denote the identity connected component of the real orthogonal group with signature (p,q). We give a complete description of the spaces of continuous and generalized translation- and SO+(p,q)-invariant valuations, generalizing Hadwiger's classification of Euclidean isometry-invari…
We consider Fair Principal Component Analysis (FPCA) and search for a low dimensional subspace that spans multiple target vectors in a fair manner. FPCA is defined as a non-concave maximization of the worst projected target norm within a given set. The problem arises in filter design in signal processing, and when inco…