arXiv research

FLORAS uses orthogonal sequences for SISO FL, offering both DP and convergence guarantees.

A diagonal metric sum_{i=1}^n g_{ii} dx_i^2 is termed Guichard_k if sum_{i=1}^{n-k}g_{ii}-sum_{i=n-k+1}^n g_{ii}=0. A hypersurface in R^{n+1} is isothermic_k if it admits line of curvature co-ordinates such that its induced metric is Guichard_k. Isothermic_1 surfaces in R^3 are the classical isothermic surfaces in R^3.…

Transformers learn to recall with non-orthogonal embeddings in realistic settings.

Novel prior for orthogonal functions improves functional component estimation.

Decomposing tensors into orthogonal factors is a well-known task in statistics, machine learning, and signal processing. We study orthogonal outer product decompositions where the factors in the summands in the decomposition are required to be orthogonal across summands, by relating this orthogonal decomposition to the…

A new method tests Expected Shortfall by analyzing both duration and severity of VaR violations.

For each integer $k \geq 2$, we apply gluing methods to construct sequences of minimal surfaces embedded in the round $3$-sphere. We produce two types of sequences, all desingularizing collections of intersecting Clifford tori. Sequences of the first type converge to a collection of $k$ Clifford tori intersecting with …

We propose a discretization of classical confocal coordinates. It is based on a novel characterization thereof as factorizable orthogonal coordinate systems. Our geometric discretization leads to factorizable discrete nets with a novel discrete analog of the orthogonality property. A discrete confocal coordinate system…

Muon optimizer simplifies matrix optimization with spectral orthogonalization.

A recent strategy to circumvent the exploding and vanishing gradient problem in RNNs, and to allow the stable propagation of signals over long time scales, is to constrain recurrent connectivity matrices to be orthogonal or unitary. This ensures eigenvalues with unit norm and thus stable dynamics and training. However …

Multi-head attention mechanism is capable of learning various representations from sequential data while paying attention to different subsequences, e.g., word-pieces or syllables in a spoken word. From the subsequences, it retrieves richer information than a single-head attention which only summarizes the whole sequen…

Universal preconditioning reduces sequential prediction regret.

In this paper, we construct local and global solutions to the Kähler-Ricci flow from a non-collapsed Kähler manifold with curvature bounded from below. Combines with the mollification technique of McLeod-Simon-Topping, we show that the Gromov-Hausdorff limit of sequence of complete noncompact non-collapsed Kähler manif…

OSA overcomes instability in skipless Transformers.

Parseval frames can be thought of as redundant or linearly dependent coordinate systems for Hilbert spaces, and have important applications in such areas as signal processing, data compression, and sampling theory. We extend the notion of a Parseval frame for a fixed Hilbert space to that of a moving Parseval frame for…

This paper explores how Transformers predict next tokens in autoregressive tasks.

Overparameterized models improve performance in sequential learning tasks.

New Performer model tackles long-sequence protein modeling.

A new meta-learner improves prediction of individualized outcomes in sequential decisions.

Several variants of recurrent neural networks (RNNs) with orthogonal or unitary recurrent matrices have recently been developed to mitigate the vanishing/exploding gradient problem and to model long-term dependencies of sequences. However, with the eigenvalues of the recurrent matrix on the unit circle, the recurrent s…

The paper uses double machine learning to estimate dynamic treatment effects robustly.

In a rotationally symmetric space $\oM$ around an axis A (whose precise definition includes all real space forms), we consider a domain $G$ limited by two equidistant hypersurfaces orthogonal to A. Let $M \subset \oM$ be a revolution hypersurface generated by a graph over A, with boundary in $\partial G$ and orthogonal…

A new method for sparse PCA using orthogonal rotations and soft-thresholding.

Reservoir Computing (RC) refers to a Recurrent Neural Networks (RNNs) framework, frequently used for sequence learning and time series prediction. The RC system consists of a random fixed-weight RNN (the input-hidden reservoir layer) and a classifier (the hidden-output readout layer). Here we focus on the sequence lear…

This paper shows that when the Riemannian metric on a contact manifold is blown up along the direction orthogonal to the contact distribution, the corresponding harmonic forms rescaled and normalized in the $L^2$-norms will converge to Rumin's harmonic forms. This proves a conjecture in Gromov `` Carnot-Caratheodory sp…

Fibonacci Ensembles use Fibonacci weights to improve ensemble learning, inspired by natural growth patterns.

Studying the isotropy orbits of compact symmetric spaces Reiswich introduced a family of explicit polynomials in one variable in order to describe the unique minimal isotropy orbit of compact symmetric spaces with Dynkin diagram of type $D_m$. Based on this geometric interpretation he conjectured that these polynomials…

Sequences of correlated binary patterns can represent many time-series data including text, movies, and biological signals. These patterns may be described by weighted combinations of a few dominant structures that underpin specific interactions among the binary elements. To extract the dominant correlation structures …

The colored Jones function of a knot is a sequence of Laurent polynomials. It was shown by TTQ. Le and the author that such sequences are $q$-holonomic, that is, they satisfy linear $q$-difference equations with coefficients Laurent polynomials in $q$ and $q^n$. We show from first principles that $q$-holonomic sequence…

The study extends Jacobi-orthogonality to indefinite scalar product spaces.

New characterization of Osserman tensors using Jacobi-orthogonality.

Study isotropy groups for complex orthogonal and skew-symmetric matrices.

New method detects RNA modifications without prior training, revealing novel sites.

DeepBlip estimates treatment effects over time using neural networks.

The paper studies how neural networks evolve representations, finding a unique fixed point for nonlinear activations.

A method for finding most influential sets reduces a complex problem to a sequence of simpler top-$k$ problems.

New findings on Kähler manifolds restrict orthogonal coordinates existence.

Constructs orthogonal coordinates in curved spaces.

OPT framework improves neural network generalization by learning an orthogonal transformation.

The paper studies surfaces in a bounded domain with orthogonal boundaries and proves curvature estimates.

Orthogonal random features approximate a Bessel kernel, offering sharper bounds than random Fourier features.

Proposes FOAGP for efficient orthogonal effect decomposition of black-box computer experiments.

Method constructs orthogonal curvilinear coordinates in constant curvature spaces.

We study the Chern-Simons partition function of orthogonal quantum group invariants, and propose a new orthogonal Labastida-Mariño-Ooguri-Vafa conjecture as well as degree conjecture for free energy associated to the orthogonal Chern-Simons partition function. We prove the degree conjecture and some interesting cases o…

A Clifford algebra model for M"obius geometry is presented. The notion of Ribaucour pairs of orthogonal systems in arbitrary dimensions is introduced, and the structure equations for adapted frames are derived. These equations are discretized and the geometry of the occuring discrete nets and sphere congruences is disc…

DONUT improves treatment effect estimation by enforcing orthogonality constraints.

New convergence guarantees for learning with unknown nuisance parameters.

MuonEq improves training of matrix-valued parameters by rebalancing momentum before orthogonalization.