arXiv research

Sparse LR-LSSVM improves kernel machine performance.

Proposes a method to learn a low-rank kernel matrix for graph-based clustering.

The study assesses low-rank approximations in Gaussian Process regression.

Kernel methods are widespread in machine learning; however, they are limited by the quadratic complexity of the construction, application, and storage of kernel matrices. Low-rank matrix approximation algorithms are widely used to address this problem and reduce the arithmetic and storage cost. However, we observed tha…

New method reduces summary points for datasets while maintaining quality.

Solves low-rank approximation problems in Hilbert spaces.

The study assesses low-rank approximations in Gaussian Process regression.

Estimates low-rank distributional matrices from incomplete samples.

Although the convolutional neural networks (CNNs) have become popular for various image processing and computer vision task recently, it remains a challenging problem to reduce the storage cost of the parameters for resource-limited platforms. In the previous studies, tensor decomposition (TD) has achieved promising co…

Determinantal point processes (DPPs) are an elegant model for encoding probabilities over subsets, such as shopping baskets, of a ground set, such as an item catalog. They are useful for a number of machine learning tasks, including product recommendation. DPPs are parametrized by a positive semi-definite kernel matrix…

Paper bounds the minimal rank for kernel ridge regression approximations.

Determinantal point processes (DPPs) have garnered attention as an elegant probabilistic model of set diversity. They are useful for a number of subset selection tasks, including product recommendation. DPPs are parametrized by a positive semi-definite kernel matrix. In this work we present a new method for learning th…

DM2L tackles missing labels in multi-label learning by modeling local and global rank structures.

This work studies low-rank approximation of a positive semidefinite matrix from partial entries via nonconvex optimization. We characterized how well local-minimum based low-rank factorization approximates a fixed positive semidefinite matrix without any assumptions on the rank-matching, the condition number or eigensp…

New algorithm tackles low-rank constraints in optimal transport problems.

Efficient and accurate low-rank approximations of multiple data sources are essential in the era of big data. The scaling of kernel-based learning algorithms to large datasets is limited by the O(n^2) computation and storage complexity of the full kernel matrix, which is required by most of the recent kernel learning a…

The paper reveals low-rank structure in neural network gradients, influenced by data and model parameters.

New GSM kernel for efficient GP regression of time series.

We investigate how to train kernel approximation methods that generalize well under a memory budget. Building on recent theoretical work, we define a measure of kernel approximation error which we find to be more predictive of the empirical generalization performance of kernel approximation methods than conventional me…

The paper examines how kernel approximations affect Gaussian process regression in large data applications.

We propose a novel class of kernels to alleviate the high computational cost of large-scale nonparametric learning with kernel methods. The proposed kernel is defined based on a hierarchical partitioning of the underlying data domain, where the Nyström method (a globally low-rank approximation) is married with a locall…

Estimates spatio-temporal Hawkes processes using tensor recovery.

New spectral methods improve matrix estimation in RL with low-rank structure.

Kernel clustering algorithm improved for large datasets using incomplete Cholesky factorization.

Improved kernel ridge regression using conjugate gradients.

In this paper, we present a framework for fitting multivariate Hawkes processes for large-scale problems both in the number of events in the observed history $n$ and the number of event types $d$ (i.e. dimensions). The proposed Low-Rank Hawkes Process (LRHP) framework introduces a low-rank approximation of the kernel m…

Algorithm POLO learns low-rank MDPs with adversarial changes in full-info feedback.

New model for high rank matrix completion with online and batch methods.

Low-rank tensor regression, a new model class that learns high-order correlation from data, has recently received considerable attention. At the same time, Gaussian processes (GP) are well-studied machine learning models for structure learning. In this paper, we demonstrate interesting connections between the two, espe…

A well-recognized limitation of kernel learning is the requirement to handle a kernel matrix, whose size is quadratic in the number of training examples. Many methods have been proposed to reduce this computational cost, mostly by using a subset of the kernel matrix entries, or some form of low-rank matrix approximatio…

The paper tackles pure exploration in multi-armed bandits with low rank structure using oblivious sampling.

GLSKF improves tensor completion by capturing both global and local variations.

PILNO uses neural operators to solve PDEs efficiently on point clouds.

Paper tackles matrix estimation under arbitrary noise, achieving minimax optimality.

Proposes a framework to extract ordered eigenfunctions from contextual kernels.

Develops new oracle inequalities for Gaussian ranking estimators.

Reduced modeling of a computationally demanding dynamical system aims at approximating its trajectories, while optimizing the trade-off between accuracy and computational complexity. In this work, we propose to achieve such an approximation by first embedding the trajectories in a reproducing kernel Hilbert space (RKHS…

This work tackles scalable sampling for nonsymmetric DPPs.

We consider fast kernel summations in high dimensions: given a large set of points in $d$ dimensions (with $d \gg 3$) and a pair-potential function (the {\em kernel} function), we compute a weighted sum of all pairwise kernel interactions for each point in the set. Direct summation is equivalent to a (dense) matrix-vec…

A new algorithm reduces sample complexity for learning Q-functions in reinforcement learning.

Deep networks can be biased to learn top eigenfunctions of the kernel outside the training set.

The smart grid vision entails advanced information technology and data analytics to enhance the efficiency, sustainability, and economics of the power grid infrastructure. Aligned to this end, modern statistical learning tools are leveraged here for electricity market inference. Day-ahead price forecasting is cast as a…

TensorGuide improves LoRA efficiency and expressivity through joint tensor-train optimization.

Boosting Nyström improves accuracy of matrix approximations.

BKTR models spatiotemporal data with scalable tensor regression.

New framework estimates eigenvalues of kernel matrices without full matrix construction.

The paper recovers missing data entries of high-rank matrices using polynomial polynomials.

The Nystrom method has been popular for generating the low-rank approximation of kernel matrices that arise in many machine learning problems. The approximation quality of the Nystrom method depends crucially on the number of selected landmark points and the selection procedure. In this paper, we present a novel algori…