Geodesics in symmetrized bidisc have intrinsic orthogonality and distinguished directions.
problem Understanding orthogonality and distinguished geodesics in the symmetrized bidisc under the Carathéodory metric.
method Analyzing the complex tangent bundle of the symmetrized bidisc, splitting it into sharp and flat bundles, and defining orthogonality and geodesics.
result Geodesics in the symmetrized bidisc are orthogonal to flat geodesics, and the space is foliated by geodesics orthogonal to a fixed flat geodesic.
In optimization, the negative gradient of a function denotes the direction of steepest descent. Furthermore, traveling in any direction orthogonal to the gradient maintains the value of the function. In this work, we show that these orthogonal directions that are ignored by gradient descent can be critical in equilibri…
We classify six-dimensional Lie groups which admit a left-invariant half-flat SU(3)-structure and which split in a direct product of three-dimensional factors. Moreover, a complete list of those direct products is obtained which admit a left-invariant half-flat SU(3)-structure such that the three-dimensional factors ar…
A new knot invariant measures crossings in three orthogonal directions.
problem Defining a new knot invariant for certain knot diagrams.
method Defining the simultaneous crossing number for knots with doubly transvergent diagrams.
result The limit of the ratio of the new invariant to the usual crossing number is at most 8.
A new pruning method reduces neural network computation without retraining.
problem Efficiently reduce neural network computation while maintaining accuracy.
method Structured directional pruning via perturbation orthogonal projection.
result Achieves state-of-the-art pruned accuracy without retraining.
This paper investigates the use of multiple directions of stratification as a variance reduction technique for Monte Carlo simulations of path-dependent options driven by Gaussian vectors. The precision of the method depends on the choice of the directions of stratification and the allocation rule within each strata. S…
Equivalent tests for SGD batch size selection found.
problem Finding equivalent tests for adaptive batch size selection in SGD.
method Norm and inner product/orthogonality tests equivalence demonstration.
result Norm and inner product/orthogonality tests are equivalent under specific conditions.
Introduces new curvature concept for Kähler manifolds.
problem Optimizing curvature constraints for projective Kähler manifolds.
method Introduces weighted orthogonal Ricci curvature and proves vanishing theorems.
result Proves optimal curvature constraints for projective Kähler manifolds.
MOBO-OSD optimizes multi-objective functions using orthogonal search directions.
problem Challenging multi-objective optimization problem.
method Solves multiple constrained optimization problems along orthogonal search directions.
result Consistently outperforms state-of-the-art algorithms.
A new algorithm avoids retractions to optimize orthogonal matrices efficiently.
problem Optimizing functions over the manifold of orthogonal matrices efficiently.
method Landing algorithm that avoids retractions using potential energy.
result The landing algorithm is faster and less prone to numerical errors than retraction-based methods.
By this short preface we show the main idea and we will bring some definitions and concepts in each section.
Introduces MSW distances to improve SW metrics.
problem Redundant projections in SW distance.
method Imposes Markov structure on projecting directions.
result MSW distances improve SW metrics.
New framework detects directional influence in multivariate time series.
problem Detecting directional influence in multivariate time series.
method Order-constrained spectral non-invariance.
result Unique diagnostic functional for directional influence.
New findings on Kähler manifolds restrict orthogonal coordinates existence.
problem Existence of orthogonal coordinates on Kähler manifolds.
method Algebraic and geometric techniques applied to Kähler manifolds.
result No nontrivial self-dual Kähler 4-manifolds or Ricci-flat Kähler 4-manifolds support orthogonal coordinates.
LOFT separates subspace rotation and transformation for orthogonal fine-tuning.
problem Conflating subspace rotation and transformation in orthogonal fine-tuning.
method LOFT explicitly separates subspace rotation and transformation, using task-aware support selection.
result LOFT recovers principal-subspace orthogonal adaptation and improves efficiency-performance trade-off.
New differential geometry perspective on orthogonal RNNs.
problem Mitigating exploding and vanishing gradients in RNNs.
method Using tools from differential geometry, parameterizing vector fields via directional derivatives of scalar functions.
result Our approach achieves comparable or better results on benchmark tasks.
Paper finds conditions for benign overfitting in neural networks.
problem Benign overfitting in leaky ReLU two-layer neural networks.
method Established directional convergence and classification error bounds.
result Benign overfitting occurs with high probability on mixture data.
Modified Laplacian connects to Yang-Mills instantons on manifolds.
problem Understanding instantons on 4D manifolds.
method Infinite dimensional Lévy Laplacian defined on manifolds, parameterized by curves in orthogonal rotations.
result Instantons on 4D manifolds are related to the modified Lévy Laplacian under specific curve conditions.
We analyse the fine convergence properties of one parameter families of hyperbolic metrics, on a fixed underlying surface, that move always in a horizontal direction, i.e. orthogonal to the action of diffeomorphisms.
The radar experiment connects the geometry of spacetime with an observers measurement of spatial length. We investigate the radar experiment on Finsler spacetimes which leads to a general definition of radar orthogonality and radar length. The directions radar orthogonal to an observer form the spatial equal time surfa…
Characterizes non-degenerate cyclic metric Lie algebras.
problem Understanding the structure of non-solvable cyclic metric Lie algebras.
method Using sufficient conditions, cyclic quadruples, and double extension method.
result Complete characterization of non-degenerate cyclic metric Lie algebras.
We introduce Block Sparse Canonical Correlation Analysis which estimates multiple pairs of canonical directions (together a "block") at once, resulting in significantly improved orthogonality of the sparse directions which, we demonstrate, translates to more interpretable solutions. Our approach builds on the sparse CC…
Obtaining complete information about the shape of an object by looking at it from a single direction is impossible in general. In this paper, we theoretically study obtaining differential geometric information of an object from orthogonal projections in a number of directions. We discuss relations between (1) a space c…
Two-layer networks trained on low-dimensional subspaces are vulnerable to adversarial examples.
problem Vulnerability of two-layer neural networks to adversarial examples on low-dimensional subspaces.
method Analysis of gradient behavior and effect of initialization scale and regularization.
result Decreasing initialization scale or adding L2 regularization can improve robustness to adversarial perturbations orthogonal to the data.
This paper analyzes quantiles of heavy-tailed distributions, separating projection direction and quantile threshold effects.
problem Analyzing quantiles of heavy-tailed distributions with estimated parameters.
method Introduces a Q-Q orthogonality formulation to separate projection-direction and quantile-threshold effects.
result Decomposes the difference between empirical and population quantiles into three terms.
Method estimates heterogeneous causal effects on networks using orthogonal learning.
problem Challenges in estimating causal effects on networks due to treatment effects on both treated and neighbors, and network homophily.
method Two-stage orthogonal learning framework: first stage uses graph neural networks for nuisance components, second stage residualizes and interpretable attention-based model for causal effects.
result Improves heterogeneous effect estimation and supports interpretable analyses.
The paper develops methods to reduce deployment risk under dynamic covariate shifts.
problem Reduction of deployment risk under dynamic covariate shifts.
method Time-domain Poincare inequality and Jacobian-velocity theorem to identify and control directional tangent energy.
result Drift-aligned tangent regularization (DTR) reduces risk volatility and directional gain in low-rank drift regimes.
In 1931 Elie Cartan constructed a geometry which was rarely considered. Cartan proposed a way to define an infinitesimal metric ds starting from a variational problem on hypersurfaces in an n-dimensional manifold M. This distance depends not only of the point $\textsc{m}\in\mathcal{M}$ but on the orient…
A new method reduces complexity for optimizing large-scale problems with orthogonality constraints.
problem Optimizing large-scale problems with orthogonality constraints.
method Randomized Riemannian submanifold method that restricts updates to random submanifolds.
result Significantly reduces per-iteration complexity for large-scale problems.
In this note, we focus on smooth nonconvex optimization problems that obey: (1) all local minimizers are also global; and (2) around any saddle point or local maximizer, the objective has a negative directional curvature. Concrete applications such as dictionary learning, generalized phase retrieval, and orthogonal ten…
This work improves disentanglement in latent space models without sacrificing generation quality.
problem Trade-off between disentanglement and generation quality in latent space models.
method Manifold optimization with a sum of autoencoder and PCA reconstruction errors, on the Stiefel manifold.
result Improves disentanglement without sacrificing generation quality.
This work improves online SGD's sample complexity for multi-index models by considering higher-order terms.
problem Suboptimal sample complexity for learning multi-index models using online SGD.
method Focus on both second- and higher-order terms to improve sample complexity.
result Online SGD achieves ildeO(dPL−1) samples for multi-index models. In this paper, we propose the use of a black-box optimization method called deterministic Mesh Adaptive Direct Search (MADS) algorithm with orthogonal directions (Ortho-MADS) for the selection of hyperparameters of Support Vector Machines with a Gaussian kernel. Different from most of the methods in the literature that…
Scientific discovery is limited by hypothesis redundancy, and hybrid methods can exploit non-local exploration.
problem Limitation of scientific discovery due to hypothesis redundancy.
method Hybrid discovery systems combining structured local search with LLM-generated non-local proposals.
result Hybrid methods can exploit non-local exploration when three geometric conditions co-occur.
We introduce a novel harmonic analysis for functions defined on the vertices of a strongly connected directed graph of which the random walk operator is the cornerstone. As a first step, we consider the set of eigenvectors of the random walk operator as a non-orthogonal Fourier-type basis for functions over directed gr…
OrthoGrad improves neural calibration by constraining gradient updates orthogonally.
problem Overconfidence in neural networks, leading to poor uncertainty estimates.
method Orthogonal gradient updates to optimize for decision boundaries and reduce overconfidence.
result Significant improvements in test loss, predictive entropy, and confidence measures.
Given a convex set and an interior point close to the boundary, we prove the existence of a supporting hyperplane whose distance to the point is controlled, in a dimensionally quantified way, by the thickness of the convex set in the orthogonal direction. This result has important applications in the regularity theory …
Muon with Newton-Schulz converges to the same stationary point as SVD-polar, up to a constant factor.
problem Improving the convergence rate of Muon optimizer.
method Using Newton-Schulz steps for momentum orthogonalization, proving convergence rate and constant factor.
result Muon with Newton-Schulz converges to the same stationary point as SVD-polar, up to a constant factor.
The Variational Autoencoder (VAE) is a powerful architecture capable of representation learning and generative modeling. When it comes to learning interpretable (disentangled) representations, VAE and its variants show unparalleled performance. However, the reasons for this are unclear, since a very particular alignmen…
Classifies linear embeddings of grassmannians and ind-grassmannians.
problem Understanding linear embeddings of grassmannians and ind-grassmannians.
method Classification through isomorphism of Picard groups and direct limits.
result Most linear embeddings of grassmannians are equivariant.
We propose two nonlinear regression methods, named Adversarial Orthogonal Regression (AdOR) for additive noise models and Adversarial Orthogonal Structural Equation Model (AdOSE) for the general case of structural equation models. Both methods try to make the residual of regression independent from regressors while put…
Neural networks are achieving state of the art and sometimes super-human performance on learning tasks across a variety of domains. Whenever these problems require learning in a continual or sequential manner, however, neural networks suffer from the problem of catastrophic forgetting; they forget how to solve previous…
Develops a direct debiased machine learning framework using Bregman divergence.
problem Reduces bias in machine learning estimates of causal effects or structural models.
method Neyman targeted estimation and generalized Riesz regression using Bregman divergence.
result Improves estimation of parameters of interest in causal models.
New formulas for surface curvature when tangent vector points in asymptotic directions.
problem Formula limitations when tangent vector points in asymptotic directions.
method Analogous formulas for surfaces with cusp singular points.
result Invariants of cusp singular points of contour lines.
IMA addresses non-identifiability in nonlinear ICA by assuming orthogonal Jacobian columns.
problem Non-identifiability in nonlinear ICA.
method IMA assumes orthogonal Jacobian columns and extends to manifold settings.
result IMA circumvents non-identifiability issues and can be beneficial for higher-dimensional observations.
Synthetic construction of Hopf fibration in 4D space.
problem Visualizing 4D objects in 3D space.
method Double orthogonal projection method to visualize 4D space.
result Direct synthetic construction of 3-sphere fibers from 2-sphere points.
The paper finds non-Gaussian directions in high-dimensional data using Wasserstein distance.
problem Locating interesting non-Gaussian features in high-dimensional data.
method Projection pursuit using 2-Wasserstein distance to maximize the difference from Gaussian.
result Statistical guarantees for accurately approximating an unknown low-dimensional non-Gaussian subspace.
LEGO estimates tangent spaces more robustly than LPCA in noisy data.
problem Estimating tangent spaces in high-noise settings.
method Spectral method using graph Laplacian eigenvectors and gradient orthogonization.
result LEGO yields more robust tangent space estimates than LPCA.