arXiv research

Spectral regularization simplifies sequence models by focusing on grammatical simplicity.

We present a solution to scale spectral algorithms for learning sequence functions. We are interested in the case where these functions are sparse (that is, for most sequences they return 0). Spectral algorithms reduce the learning problem to the task of computing an SVD decomposition over a special type of matrix call…

New model mimics neural next item recommendation using Hankel matrices.

Paper speeds up GP inference by reducing precision matrix computation.

HOPE improves SSMs for long-memory tasks with robust initialization and training.

Constructs new topological theories in 2D not fitting standard axioms.

HSNLD solves robust Hankel recovery efficiently and robustly.

The paper tackles system identification via Hankel nuclear norm regularization, improving estimation rates and singular value gaps.

We propose a scheme for recycling Gaussian random vectors into structured matrices to approximate various kernel functions in sublinear time via random embeddings. Our framework includes the Fastfood construction as a special case, but also extends to Circulant, Toeplitz and Hankel matrices, and the broader family of s…

Algorithm learns linear systems from partial observations with near-optimal rate.

The paper reviews Hankel low-rank methods for time series analysis and forecasting.

Signals are generally modeled as a superposition of exponential functions in spectroscopy of chemistry, biology and medical imaging. For fast data acquisition or other inevitable reasons, however, only a small amount of samples may be acquired and thus how to recover the full signal becomes an active research topic. Bu…

In this paper we consider the low-rank matrix completion problem with specific application to forecasting in time series analysis. Briefly, the low-rank matrix completion problem is the problem of imputing missing values of a matrix under a rank constraint. We consider a matrix completion problem for Hankel matrices an…

We consider the problem of learning a low-rank matrix, constrained to lie in a linear subspace, and introduce a novel factorization for modeling such matrices. A salient feature of the proposed factorization scheme is it decouples the low-rank and the structural constraints onto separate factors. We formulate the optim…

Proposes BHT-ARIMA for forecasting multiple short time series.

The annihilating filter-based low-rank Hankel matrix approach (ALOHA) is one of the state-of-the-art compressed sensing approaches that directly interpolates the missing k-space data using low-rank Hankel matrix completion. The success of ALOHA is due to the concise signal representation in the k-space domain thanks to…

Algorithm learns graph operator from sparse space-time samples.

In the first part of the paper, comprising section 1 through 6, we introduce a sequence of functions in the tangent bundle TM of any smooth two-dimensional manifold M with smooth Riemannian metric g that correspond to the higher order Schwarzians of the linearized geodesic flow. With these functions and a classical the…

Let $γ: I \rightarrow \mathbb R^n$ be a parametric curve of class $C^{n+1}$, regular of order $n$. The Frenet-Serret apparatus of $γ$ at $γ(t)$ consists of a frame $e_1(t), \dots , e_n(t)$ and generalized curvature values $κ_1(t), \dots, κ_{n-1}(t)$. Associated with each point of $γ$ there are also local singular vecto…

New method controls linear systems with adversarial disturbances.

The paper tackles estimation of hidden state LTI systems of unknown order.

Noise-robust Koopman operator framework for control with improved stability and performance.

New nonconvex methods improve SysID efficiency and accuracy.

Reservoir computing's success depends on mapping different input time series to separable states.

This paper addresses network anomography, that is, the problem of inferring network-level anomalies from indirect link measurements. This problem is cast as a low-rank subspace tracking problem for normal flows under incomplete observations, and an outlier detection problem for abnormal flows. Since traffic data is lar…

The paper studies the problem of recovering a spectrally sparse object from a small number of time domain samples. Specifically, the object of interest with ambient dimension $n$ is assumed to be a mixture of $r$ complex multi-dimensional sinusoids, while the underlying frequencies can assume any value in the unit disk…

This paper explores robust recovery of a superposition of $R$ distinct complex exponential functions from a few random Gaussian projections. We assume that the signal of interest is of $2N-1$ dimensional and $R<<2N-1$. This framework covers a large class of signals arising from real applications in biology, automation,…

Improved modeling of chaotic systems using time-delay embeddings and Frenet-Serret frame.

Recent contributions have framed linear system identification as a nonparametric regularized inverse problem. Relying on $\ell_2$-type regularization which accounts for the stability and smoothness of the impulse response to be estimated, these approaches have been shown to be competitive w.r.t classical parametric met…

The paper explores the problem of \emph{spectral compressed sensing}, which aims to recover a spectrally sparse signal from a small random subset of its $n$ time domain samples. The signal of interest is assumed to be a superposition of $r$ multi-dimensional complex sinusoids, while the underlying frequencies can assum…

Predictive State Representations (PSRs) are powerful techniques for modelling dynamical systems, which represent a state as a vector of predictions about future observable events (tests). In PSRs, one of the fundamental problems is the learning of the PSR model of the underlying system. Recently, spectral methods have …

We present an efficient and practical algorithm for the online prediction of discrete-time linear dynamical systems with a symmetric transition matrix. We circumvent the non-convex optimization problem using improper learning: carefully overparameterize the class of LDSs by a polylogarithmic factor, in exchange for con…

Nyquist ghost artifacts in EPI are originated from phase mismatch between the even and odd echoes. However, conventional correction methods using reference scans often produce erroneous results especially in high-field MRI due to the non-linear and time-varying local magnetic field changes. Recently, it was shown that …

In this paper, we unravel a fundamental connection between weighted finite automata~(WFAs) and second-order recurrent neural networks~(2-RNNs): in the case of sequences of discrete symbols, WFAs and 2-RNNs with linear activation functions are expressively equivalent. Motivated by this result, we build upon a recent ext…

We study power expansions of the characteristic function of a linear operator $A$ in a $p|q$-dimensional superspace $V$. We show that traces of exterior powers of $A$ satisfy universal recurrence relations of period $q$. `Underlying' recurrence relations hold in the Grothendieck ring of representations of $\GL(V)$. The…

This paper concerns model reduction of dynamical systems using the nuclear norm of the Hankel matrix to make a trade-off between model fit and model complexity. This results in a convex optimization problem where this trade-off is determined by one crucial design parameter. The main contribution is a methodology to app…

The problem of low-rank approximation with convex constraints, which appears in data analysis, system identification, model order reduction, low-order controller design and low-complexity modelling is considered. Given a matrix, the objective is to find a low-rank approximation that meets rank and convex constraints, w…

We simplify matrix computations for block matrices, especially useful for covariance and correlation matrices.

Minimal submanifolds in matrix spaces proven for specific ranks.

Computes isotropy subgroups of orthogonal matrices acting on Hermitian matrices.

Study on random matrices in deep neural networks with IID entries.

Financial markets analyzed by reducing correlation matrix complexity.

Study isotropy groups for complex orthogonal and skew-symmetric matrices.

The paper deals with distribution of singular values of product of random matrices arising in the analysis of deep neural networks. The matrices resemble the product analogs of the sample covariance matrices, however, an important difference is that the population covariance matrices, which are assumed to be non-random…

We introduce Clique Matrices as an alternative representation of undirected graphs, being a generalisation of the incidence matrix representation. Here we use clique matrices to decompose a graph into a set of possibly overlapping clusters, de ned as well-connected subsets of vertices. The decomposition is based on a s…

Study of strictly accretive matrices using Finsler geometry.

Minimal spectral radii found for specific matrix types.

Researchers develop geodesics for a new metric on correlation matrices.