arXiv research

Proposes a model for identifying edges in low-rank dynamical networks.

Review of algorithms for linear system approximations.

We propose a method to infer stochastic low-rank RNNs from neural data.

New method finds efficient low-rank neural networks during training.

A new method reduces high-dimensional filtering to quadratic complexity.

AIR-Net adapts low-rank regularization dynamically for better image completion.

We consider the problem of modeling multivariate time series with parsimonious dynamical models which can be represented as sparse dynamic Bayesian networks with few latent nodes. This structure translates into a sparse plus low rank model. In this paper, we propose a Gaussian regression approach to identify such a mod…

SALT models combine ARHMM and SLDS for efficient, interpretable time-series analysis.

This work studies the linear approximation of high-dimensional dynamical systems using low-rank dynamic mode decomposition (DMD). Searching this approximation in a data-driven approach is formalised as attempting to solve a low-rank constrained optimisation problem. This problem is non-convex and state-of-the-art algor…

New approach uses compressible dynamics to train deep models efficiently.

Riemannian gradient descent escapes some spurious critical points on low-rank matrix manifold.

Dynamic Mode Decomposition (DMD) has emerged as a powerful tool for analyzing the dynamics of non-linear systems from experimental datasets. Recently, several attempts have extended DMD to the context of low-rank approximations. This extension is of particular interest for reduced-order modeling in various applicative …

Active learning method for neural population dynamics using optogenetics.

LoRA fine-tuning explained with gradient dynamics for low-rank perturbations.

Low-rank matrix factorizations arise in a wide variety of applications -- including recommendation systems, topic models, and source separation, to name just a few. In these and many other applications, it has been widely noted that by incorporating temporal information and allowing for the possibility of time-varying …

LASER compresses recursive model activations by exploiting their low-dimensional structure.

MOSAIC detects change points in dynamic networks with low-rank and sparse changes.

Sparsity-based approaches have been popular in many applications in image processing and imaging. Compressed sensing exploits the sparsity of images in a transform domain or dictionary to improve image recovery from undersampled measurements. In the context of inverse problems in dynamic imaging, recent research has de…

Optimizes wide low-rank neural networks for reduced parameters and cost.

Deep ReLU networks escape from the origin via saddle points with a low-rank bias.

Research reveals deep networks often learn low-rank structures, leading to more efficient training and fine-tuning.

FLAMBE tackles RL in low rank MDPs by learning features.

The paper examines how gradient descent stabilizes low-rank matrix factorization in noisy conditions.

The paper shows how gradient flow on over-parametrized tensor decomposition behaves like deflation.

New algorithm tackles dynamic query routing to multiple embedding models.

New algorithm catches moving subspaces in bandit problems.

A low-rank tensor model simplifies multi-dimensional Markov chains.

Paper tackles dynamic assortment with dual contexts, improving revenue in e-commerce.

New model reduces matrix factorization bias, yielding truly low-rank solutions.

Principal Components Analysis (PCA) is one of the most widely used dimension reduction techniques. Robust PCA (RPCA) refers to the problem of PCA when the data may be corrupted by outliers. Recent work by Cand{è}s, Wright, Li, and Ma defined RPCA as a problem of decomposing a given data matrix into the sum of a low-ran…

Gradient descent promotes low-rank solutions in tensor completion.

New model-free algorithms learn representations for low-rank MDPs efficiently.

This paper considers a new framework to detect communities in a graph from the observation of signals at its nodes. We model the observed signals as noisy outputs of an unknown network process, represented as a graph filter that is excited by a set of unknown low-rank inputs/excitations. Application scenarios of this m…

We consider dynamic pricing with many products under an evolving but low-dimensional demand model. Assuming the temporal variation in cross-elasticities exhibits low-rank structure based on fixed (latent) features of the products, we show that the revenue maximization problem reduces to an online bandit convex optimiza…

PSI-LinUCB improves scalability for large recommender systems.

Analyzes learning dynamics of RNNs under locality constraints.

UA-LQE improves value function learning by selectively erasing uncertain entries in Q-matrix.

Reduced modeling of a computationally demanding dynamical system aims at approximating its trajectories, while optimizing the trade-off between accuracy and computational complexity. In this work, we propose to achieve such an approximation by first embedding the trajectories in a reproducing kernel Hilbert space (RKHS…

A new method for efficient neural network fine-tuning using queryable low-rank update atoms.

LoRA fine-tuning causes forgetting, studied via particle system dynamics.

Low-rank structure emerges in neural networks during learning.

Algorithm for low-rank matrix bandits with heavy-tailed rewards, achieving nearly optimal regret bound.

TensorGuide improves LoRA efficiency and expressivity through joint tensor-train optimization.

Flora uses random projections to achieve high-rank updates with low memory usage.

AdaRL improves robust RL by adaptively adjusting policy complexity.

Reduced modeling in high-dimensional reproducing kernel Hilbert spaces offers the opportunity to approximate efficiently non-linear dynamics. In this work, we devise an algorithm based on low rank constraint optimization and kernel-based computation that generalizes a recent approach called "kernel-based dynamic mode d…

Often, large, high dimensional datasets collected across multiple modalities can be organized as a higher order tensor. Low-rank tensor decomposition then arises as a powerful and widely used tool to discover simple low dimensional structures underlying such data. However, we currently lack a theoretical understanding …

New method learns low-dimensional representations of nonlinear time series without supervision.