arXiv research

PILNO uses neural operators to solve PDEs efficiently on point clouds.

New method normalizes matrix features for robust low-rank approximation.

Randomized algorithm solves vector-valued regression problems with low-rank operators.

Paper proposes a new optimization framework for learning eigenfunctions of operators.

This work provides closed-form solutions and minimum achievable errors for a large class of low-rank approximation problems in Hilbert spaces. The proposed theorem generalizes to the case of bounded linear operators the previous results obtained in the finite dimensional case for the Frobenius norm. The theorem provide…

The problem of low rank matrix completion is considered in this paper. To exploit the underlying low-rank structure of the data matrix, we propose a hierarchical Gaussian prior model, where columns of the low-rank matrix are assumed to follow a Gaussian distribution with zero mean and a common precision matrix, and a W…

A new EnKF method for elliptic PDEs reduces dimensionality for accurate state estimation.

The low displacement rank (LDR) framework for structured matrices represents a matrix through two displacement operators and a low-rank residual. Existing use of LDR matrices in deep learning has applied fixed displacement operators encoding forms of shift invariance akin to convolutions. We introduce a class of LDR ma…

Low-rank modeling has a lot of important applications in machine learning, computer vision and social network analysis. While the matrix rank is often approximated by the convex nuclear norm, the use of nonconvex low-rank regularizers has demonstrated better recovery performance. However, the resultant optimization pro…

Low-rank modeling has many important applications in computer vision and machine learning. While the matrix rank is often approximated by the convex nuclear norm, the use of nonconvex low-rank regularizers has demonstrated better empirical performance. However, the resulting optimization problem is much more challengin…

In this letter, we propose an algorithm for recovery of sparse and low rank components of matrices using an iterative method with adaptive thresholding. In each iteration, the low rank and sparse components are obtained using a thresholding operator. This algorithm is fast and can be implemented easily. We compare it w…

Given the superposition of a low-rank matrix plus the product of a known fat compression matrix times a sparse matrix, the goal of this paper is to establish deterministic conditions under which exact recovery of the low-rank and sparse components becomes possible. This fundamental identifiability issue arises with tra…

We address some theoretical guarantees for Schatten-$p$ quasi-norm minimization ($p \in (0,1]$) in recovering low-rank matrices from compressed linear measurements. Firstly, using null space properties of the measurement operator, we provide a sufficient condition for exact recovery of low-rank matrices. This condition…

Proposes a faster Isomap algorithm by reducing eigenvalue decomposition complexity.

A new method for efficient neural network fine-tuning using queryable low-rank update atoms.

A federated model learns shared archetypes from heterogeneous clients in continual learning.

Unified approach for learning quantum operations from measurements.

New bound for neural networks with full-rank weights, independent of network width.

New algorithm improves tensor completion performance.

Paper proposes fast, robust methods for low-rank matrix recovery.

We propose a unified framework to solve general low-rank plus sparse matrix recovery problems based on matrix factorization, which covers a broad family of objective functions satisfying the restricted strong convexity and smoothness conditions. Based on projected gradient descent and the double thresholding operator, …

Estimates low-rank distributional matrices from incomplete samples.

Kernel methods are widespread in machine learning; however, they are limited by the quadratic complexity of the construction, application, and storage of kernel matrices. Low-rank matrix approximation algorithms are widely used to address this problem and reduce the arithmetic and storage cost. However, we observed tha…

Efficient solver for nonconvex tensor regularization reduces computational cost.

A new Riemannian framework optimizes LoRA for faster convergence and better performance.

Convex optimization method recovers low-rank matrices from rank-one projections efficiently.

Recent years have seen rapid advances in the data-driven analysis of dynamical systems based on Koopman operator theory and related approaches. On the other hand, low-rank tensor product approximations -- in particular the tensor train (TT) format -- have become a valuable tool for the solution of large-scale problems …

We consider training over-parameterized two-layer neural networks with Rectified Linear Unit (ReLU) using gradient descent (GD) method. Inspired by a recent line of work, we study the evolutions of network prediction errors across GD iterations, which can be neatly described in a matrix form. When the network is suffic…

This work improves robustness guarantees for neural networks using low rank representations.

We describe novel subgradient methods for a broad class of matrix optimization problems involving nuclear norm regularization. Unlike existing approaches, our method executes very cheap iterations by combining low-rank stochastic subgradients with efficient incremental SVD updates, made possible by highly optimized and…

Algorithm learns graph operator from sparse space-time samples.

We study the adaptive estimation of copula correlation matrix $Σ$ for the semi-parametric elliptical copula model. In this context, the correlations are connected to Kendall's tau through a sine function transformation. Hence, a natural estimate for $Σ$ is the plug-in estimator $\hatΣ$ with Kendall's tau statistic. We …

New method decomposes corrupted data matrices into sparse and low-rank components.

Bayesian framework for sequential learning tasks with low-rank approximations.

Low-rank matrix is desired in many machine learning and computer vision problems. Most of the recent studies use the nuclear norm as a convex surrogate of the rank operator. However, all singular values are simply added together by the nuclear norm, and thus the rank may not be well approximated in practical problems. …

In this paper, we present a framework for fitting multivariate Hawkes processes for large-scale problems both in the number of events in the observed history $n$ and the number of event types $d$ (i.e. dimensions). The proposed Low-Rank Hawkes Process (LRHP) framework introduces a low-rank approximation of the kernel m…

LORENZA improves LLM fine-tuning efficiency and generalization.

In this paper, we study the popularly dubbed matrix completion problem, where the task is to "fill in" the unobserved entries of a matrix from a small subset of observed entries, under the assumption that the underlying matrix is of low-rank. Our contributions herein, enhance our prior work on nuclear norm regularized …

This work tackles representation learning for RL in low-rank MDPs, improving sample efficiency.

Paper bounds the minimal rank for kernel ridge regression approximations.

LOT improves optimal transport for large datasets.

Compressing DNNs is important for the real-world applications operating on resource-constrained devices. However, we typically observe drastic performance deterioration when changing model size after training is completed. Therefore, retraining is required to resume the performance of the compressed models suitable for…

A new algorithm reduces sample complexity for learning Q-functions in reinforcement learning.

New method reduces inventory inaccuracies by 10x, saving retailers 4% annually.

We analyze a class of estimators based on convex relaxation for solving high-dimensional matrix decomposition problems. The observations are noisy realizations of a linear transformation $\mathfrak{X}$ of the sum of an approximately) low rank matrix $Θ^\star$ with a second matrix $Γ^\star$ endowed with a complementary …

ALF reduces network parameters and operations by 70% and 61%, respectively, on embedded hardware.

GLSKF improves tensor completion by capturing both global and local variations.

New guarantees for recovering matrices as low-rank plus sparse from fewer measurements.