arXiv research

Paper finds a lower bound for estimating low-rank matrices in logistic regression.

Novel Fréchet regression method handles errors-in-variables with low-rank covariates.

Low-rank tensor regression, a new model class that learns high-order correlation from data, has recently received considerable attention. At the same time, Gaussian processes (GP) are well-studied machine learning models for structure learning. In this paper, we demonstrate interesting connections between the two, espe…

The study assesses low-rank approximations in Gaussian Process regression.

The study assesses low-rank approximations in Gaussian Process regression.

Solves weakly supervised regression using low-rank approximations and manifold regularization.

Tensor regression networks achieve high compression rate of neural networks while having slight impact on performances. They do so by imposing low tensor rank structure on the weight matrices of fully connected layers. In recent years, tensor regression networks have been investigated from the perspective of their comp…

Paper studies quantized LRMR with random dithering for correlated tasks.

Proposes a new method for multivariate functional regression.

CP degeneracy affects tensor regression solutions, especially in high dimensions.

We solve robust regression and matrix completion problems with sparse and low-rank models.

Low-rank matrix regression refers to the instances of recovering a low-rank matrix based on specially designed measurements and the corresponding noisy outcomes. In the last decade, numerous statistical methodologies have been developed for efficiently recovering the unknown low-rank matrices. However, in some applicat…

Randomized algorithm solves vector-valued regression problems with low-rank operators.

Novel method for efficient low-rank matrix estimation and bandit algorithms.

This paper extends neural collapse to regression problems, revealing key features and structures.

Introduces a new model for mapping matrices to matrices, subsuming linear regression.

Paper bounds the minimal rank for kernel ridge regression approximations.

The paper examines how kernel approximations affect Gaussian process regression in large data applications.

Algorithm recovers multiple low-rank matrices from unlabeled data.

Paper develops inference methods for low-rank tensors without debiasing.

Paper develops DP methods for low-rank matrix estimation with near-optimal performance.

This paper studies how to sketch element-wise functions of low-rank matrices. Formally, given low-rank matrix A = [Aij] and scalar non-linear function f, we aim for finding an approximated low-rank representation of the (possibly high-rank) matrix [f(Aij)]. To this end, we propose an efficient sketching-based algorithm…

We consider the problem of modeling multivariate time series with parsimonious dynamical models which can be represented as sparse dynamic Bayesian networks with few latent nodes. This structure translates into a sparse plus low rank model. In this paper, we propose a Gaussian regression approach to identify such a mod…

BKTR models spatiotemporal data with scalable tensor regression.

Low-rank approximations of data matrices are an important dimensionality reduction tool in machine learning and regression analysis. We consider the case of categorical variables, where it can be formulated as the problem of finding low-rank approximations to Boolean matrices. In this paper we give what is to the best …

In this paper, we develop a novel procedure for low-rank tensor regression, namely \emph{\underline{I}mportance \underline{S}ketching \underline{L}ow-rank \underline{E}stimation for \underline{T}ensors} (ISLET). The central idea behind ISLET is \emph{importance sketching}, i.e., carefully designed sketches based on bot…

We propose a sparse and low-rank tensor regression model to relate a univariate outcome to a feature tensor, in which each unit-rank tensor from the CP decomposition of the coefficient tensor is assumed to be sparse. This structure is both parsimonious and highly interpretable, as it implies that the outcome is related…

Paper develops RGN method for estimating low-rank tensors from noisy measurements.

This work analyzes how transformers learn common linear regression tasks.

We simplify SSL by approximating redundant structural components with low-rank factorization.

Multitask learning, i.e. taking advantage of the relatedness of individual tasks in order to improve performance on all of them, is a core challenge in the field of machine learning. We focus on matrix regression tasks where the rank of the weight matrix is constrained to reduce sample complexity. We introduce the comm…

In this paper, we consider the problem of learning high-dimensional tensor regression problems with low-rank structure. One of the core challenges associated with learning high-dimensional models is computation since the underlying optimization problems are often non-convex. While convex relaxations could lead to polyn…

Develops TOFU for tensor bandits with low-rank structure.

Simplifies transfer learning with deep neural networks using ridge regression.

TSRGA scales multivariate linear regression for feature-distributed data.

Low-rank forecasting improves consistency in time series predictions.

Multi-view data have been routinely collected in various fields of science and engineering. A general problem is to study the predictive association between multivariate responses and multi-view predictor sets, all of which can be of high dimensionality. It is likely that only a few views are relevant to prediction, an…

Weight Decay induces low-rank weight matrices in neural networks, improving generalization.

New approach to convex hulls for low-rank problems.

We propose a unified framework for estimating low-rank matrices through nonconvex optimization based on gradient descent algorithm. Our framework is quite general and can be applied to both noisy and noiseless observations. In the general case with noisy observations, we show that our algorithm is guaranteed to linearl…

New model reduces matrix factorization bias, yielding truly low-rank solutions.

Develops a regression model for partially observed dynamic tensor data.

New algorithm recovers tensor factors from incomplete measurements efficiently.

In this paper, we solve a semi-supervised regression problem. Due to the lack of knowledge about the data structure and the presence of random noise, the considered data model is uncertain. We propose a method which combines graph Laplacian regularization and cluster ensemble methodologies. The co-association matrix of…

Gaussian processes (GP) are Bayesian non-parametric models that are widely used for probabilistic regression. Unfortunately, it cannot scale well with large data nor perform real-time predictions due to its cubic time cost in the data size. This paper presents two parallel GP regression methods that exploit low-rank co…

Gaussian processes (GP) are Bayesian non-parametric models that are widely used for probabilistic regression. Unfortunately, it cannot scale well with large data nor perform real-time predictions due to its cubic time cost in the data size. This paper presents two parallel GP regression methods that exploit low-rank co…

We study the residual bootstrap (RB) method in the context of high-dimensional linear regression. Specifically, we analyze the distributional approximation of linear contrasts $c^{\top} (\hatβ_ρ-β)$, where $\hatβ_ρ$ is a ridge-regression estimator. When regression coefficients are estimated via least squares, classical…

The effectiveness of supervised learning techniques has made them ubiquitous in research and practice. In high-dimensional settings, supervised learning commonly relies on dimensionality reduction to improve performance and identify the most important factors in predicting outcomes. However, the economic importance of …