arXiv research

Study differential properties of matrix square roots in specific cases.

New method efficiently learns positive-definite curvature for neural nets.

Efficiently computes matrix square roots and their inverses for large matrices.

Advanced optimization algorithms such as Newton method and AdaGrad benefit from second order derivative or second order statistics to achieve better descent directions and faster convergence rates. At their heart, such algorithms need to compute the inverse or inverse square root of a matrix whose size is quadratic of …

Study centers of quantum tori and skein algebras for even roots of unity.

This expository article introduces the topic of roots in a compact Lie group. Compared to the many other treatments of this standard topic, I intended for mine to be relatively elementary, example-driven, and free of unnecessary abstractions. Some familiarity with matrix groups and with maximal tori is assumed. This ar…

Paper develops efficient AltMin algorithm for SRPCP robust matrix recovery.

Covariance pooling is a feature pooling method with good classification accuracy. Because covariance features consist of second-order statistics, the scale of the feature elements are varied. Therefore, normalizing covariance features using a matrix square root affects the performance improvement. When pooling methods …

This paper proposes the recursive and square-root BLS algorithms to improve the original BLS for new added inputs, which utilize the inverse and inverse Cholesky factor of the Hermitian matrix in the ridge inverse, respectively, to update the ridge solution. The recursive BLS updates the inverse by the matrix inversion…

The paper extends ternary algebra concepts using cube roots of unity.

Optimal data splitting improves covariance matrix estimation in large datasets.

New method differentiates square-root Kalman filters robustly.

We study the stability vis a vis adversarial noise of matrix factorization algorithm for matrix completion. In particular, our results include: (I) we bound the gap between the solution matrix of the factorization method and the ground truth in terms of root mean square error; (II) we treat the matrix factorization as …

ROOT-SGD solves convex optimization problems with optimal nonasymptotic and near-optimal asymptotic performance.

New method for NMF without tuning parameter.

Efficient neural networks compute various differential operators cheaply.

Adaptive regularization methods pre-multiply a descent direction by a preconditioning matrix. Due to the large number of parameters of machine learning problems, full-matrix preconditioning methods are prohibitively expensive. We show how to modify full-matrix adaptive regularization in order to make it practical and e…

New algorithms achieve logarithmic regret in learning linear quadratic control systems.

Paper defines multiplicities for quaternion eigenvalues without complex matrix concepts.

We consider a fundamental algorithmic question in spectral graph theory: Compute a spectral sparsifier of random-walk matrix-polynomial $L_α(G)=D-\sum_{r=1}^dα_rD(D^{-1}A)^r$ where $A$ is the adjacency matrix of a weighted, undirected graph, $D$ is the diagonal matrix of weighted degrees, and $α=(α_1...α_d)$ are nonn…

EPMF factorizes matrices by adjusting their entries to match a specified power.

Alpha-based performance evaluation may fail to capture correlated residuals due to model errors. This paper proposes using the Generalized Information Ratio (GIR) to measure performance under misspecified benchmarks. Motivated by the theoretical link between abnormal returns and residual covariance matrix, GIR is deriv…

New insights into learning rates and batch sizes for neural networks using random matrix theory.

Paper improves efficiency in matrix computations for Gaussian processes.

We study $k$-GenEV, the problem of finding the top $k$ generalized eigenvectors, and $k$-CCA, the problem of finding the top $k$ vectors in canonical-correlation analysis. We propose algorithms $\mathtt{LazyEV}$ and $\mathtt{LazyCCA}$ to solve the two problems with running times linearly dependent on the input size and…

We construct a braiding operator in terms of the quantum dilogarithm function based on the quantum cluster algebra. We show that it is a q-deformation of the R-operator for which hyperbolic octrahedron is assigned. Also shown is that, by taking q to be a root of unity, our braiding operator reduces to the Kashaev R-mat…

We develop Square Root Graphical Models (SQR), a novel class of parametric graphical models that provides multivariate generalizations of univariate exponential family distributions. Previous multivariate graphical models [Yang et al. 2015] did not allow positive dependencies for the exponential and Poisson generalizat…

Proves a conjecture about matrix orders for pseudo-Anosov maps.

We consider the problem of completing a matrix with categorical-valued entries from partial observations. This is achieved by extending the formulation and theory of one-bit matrix completion. We recover a low-rank matrix $X$ by maximizing the likelihood ratio with a constraint on the nuclear norm of $X$, and the obser…

New bounds for private matrix approximation using Gaussian noise and Dyson Brownian Motion.

Layer normalization (LayerNorm) has been successfully applied to various deep neural networks to help stabilize training and boost model convergence because of its capability in handling re-centering and re-scaling of both inputs and weight matrix. However, the computational overhead introduced by LayerNorm makes these…

The completion of low rank matrices from few entries is a task with many practical applications. We consider here two aspects of this problem: detectability, i.e. the ability to estimate the rank $r$ reliably from the fewest possible random entries, and performance in achieving small reconstruction error. We propose a …

New methods for sketching non-PSD matrices improve regression and optimization tasks.

Method detects lithium-ion battery knee onset for early warning.

We study a class of scalar, linear, non-local Riemann-Hilbert problems (RHP) involving finite subgroups of PSL(2,C). We associate to such problems a (maybe infinite) root system and describe the relevance of the orbits of the Weyl group in the construction of its solutions. As an application, we study in detail the lar…

Efficient optimization method reduces Full AdaGrad complexity.

We define a family of the braid group representations via the action of the $R$-matrix (of the quasitriangular extension) of the restricted quantum $\mathfrak{sl}(2)$ on a tensor power of a simple projective module. This family is an extension of the Lawrence representation specialized at roots of unity. Although the c…

A distributed framework for reducing high-dimensional matrix-variate time series data.

We define a numerical method that provides a non-parametric estimation of the kernel shape in symmetric multivariate Hawkes processes. This method relies on second order statistical properties of Hawkes processes that relate the covariance matrix of the process to the kernel matrix. The square root of the correlation f…

A new optimization method reduces memory and compute requirements for deep learning.

The abundance of high-dimensional data in the modern sciences has generated tremendous interest in penalized estimators such as the lasso, scaled lasso, square-root lasso, elastic net, and many others. In this paper, we establish a general oracle inequality for prediction in high-dimensional linear regression with such…

New insights into Hessian structure of neural networks reveal two forces.

Explicit matrix presentations of Blanchfield pairings and twisted pairings for torus knots.

Paper uses Random Matrix Theory for optimal training-testing data split.

Paper examines LASSO for high-dimensional predictive regression, improving its performance in forecasting unemployment.

It is postulated that quantum gravity is a sum over causal structures coupled to matter via scale evolution. Quantized causal structures can be described by studying simple matrix models where matrices are replaced by an algebra of quantum mechanical observables. In particular, previous studies constructed quantum grav…

New algorithm reduces complexity for SPD manifold optimization.

New methods reveal colored Jones polynomials from quantum R-matrices and knot invariants.