A new tree method for tensor data improves regression accuracy.
problem Efficiently modeling tensor data for regression problems.
method Scalar-output regression tree models for scalar-on-tensor problems, and tensor-on-tensor problems using additive tree ensemble approaches.
result The tensor-input tree (TT) method outperforms tensor-input GP models in efficiency and accuracy.
Enhances tensor regression for interpretability and performance.
problem Interpreting and modeling multidimensional tensor data with structural heterogeneity.
method Generalized Nonnegative Structured Kruskal Tensor Regression (NS-KTR) with hybrid regularization and nonnegativity constraints.
result NS-KTR outperforms conventional methods in synthetic and real hyperspectral datasets.
Tensor Regression tackles high-dimensional data analysis.
problem Challenges in traditional data representation methods for high-dimensional data.
method Systematic study and analysis of tensor-based regression models.
result Provides solutions for specific regression tasks with multiway data.
New methods solve tensor-on-tensor regression with unknown rank, revealing benefits of over-parameterization.
problem Connecting tensor responses to tensor covariates with unknown intrinsic rank.
method Riemannian gradient descent and Riemannian Gauss-Newton methods for tensor-on-tensor regression.
result Riemannian optimization methods converge linearly and quadratically to a statistically optimal estimate in rank over-parameterized settings.
Sparse symmetric tensor regression reduces brain connectivity complexity.
problem Complex brain connectivity analysis in neuroimaging.
method Sparse symmetric tensor regression model for functional connectivity.
result Superior performance in Alzheimer's disease detection.
TRNN combines tensor geometry with neural network nonlinearity for HD data.
problem Modeling high-dimensional data with preserved tensor geometry and nonlinear interactions.
method Introduces TRNN that integrates tensor geometry and neural network nonlinearity.
result TRNN preserves tensor geometry while offering nonlinearity.
Tensor regression networks achieve high compression rate of neural networks while having slight impact on performances. They do so by imposing low tensor rank structure on the weight matrices of fully connected layers. In recent years, tensor regression networks have been investigated from the perspective of their comp…
Extends multivariate regression for tensor-variate data, identifying brain regions and facial characteristics.
problem Challenges in fitting regression models with multivariate responses and covariates.
method Low-rank tensor formats on regression coefficients and tensor-variate normal distribution for errors.
result Maximum likelihood estimators for tensor-on-tensor regression via block-relaxation algorithms.
CP degeneracy affects tensor regression solutions, especially in high dimensions.
problem CP degeneracy in tensor regression.
method Analysis of CP degeneracy and development of a penalized strategy.
result A general penalized strategy to overcome CP degeneracy in tensor regression.
Robust TOT regression method handles outliers in tensor data.
problem Outliers in tensor data affect standard TOT regression.
method ROTOT method using a single loss function for outliers and robust MPCA for predictor.
result ROTOT method reduces influence of both casewise and cellwise outliers.
Paper proposes robust tensor regression method for tensor data analysis.
problem Outliers in tensor data analysis can make existing methods sensitive.
method Nonconvex relaxation of tensor tubal rank in optimization framework.
result Global convergence of proposed estimation algorithm under mild assumptions.
NA0CT2 improves tensor regression predictions with ℓ0 regularization.
problem Improving tensor regression predictions with structural information.
method Noise-Augmented ℓ0 regularization on Tucker decomposition. result Achieves exact ℓ0 regularization on core tensor in linear and generalized linear tensor regression. Optimizes tensor rank selection for neural network compression.
problem Finding optimal tensor rank for regression models.
method Analyzes population expressions for training-testing discrepancy under Gaussian design.
result Optimal rank minimizes prediction error and aligns with cross-validation.
We propose a sparse and low-rank tensor regression model to relate a univariate outcome to a feature tensor, in which each unit-rank tensor from the CP decomposition of the coefficient tensor is assumed to be sparse. This structure is both parsimonious and highly interpretable, as it implies that the outcome is related…
KReTTaH uses tensor trains and Hadamard overparameterization for fast, interpretable multi-way data imputation.
problem Multi-way data imputation for high-dimensional functional MRI and dynamic graph recovery.
method Reformulates imputation as RKHS regression with TT-constrained coefficients and Hadamard overparameterization. Optimizes TT coefficients and kernel matrices on Riemannian manifolds.
result Consistently outperforms state-of-the-art methods in modeling accuracy.
KReTTaH uses tensor trains and Hadamard overparameterization for fast, interpretable multi-way data imputation.
problem Multi-way data imputation in high-dimensional spaces.
method Reformulates imputation as RKHS regression with TT-constrained coefficients, optimized on manifold frameworks.
result Consistently outperforms state-of-the-art methods in accuracy.
In modern data science, dynamic tensor data is prevailing in numerous applications. An important task is to characterize the relationship between such dynamic tensor and external covariates. However, the tensor data is often only partially observed, rendering many existing methods inapplicable. In this article, we deve…
Tensor Neural Networks improve regression accuracy and efficiency.
problem Nonparametric regression problems with complex, high-dimensional functions.
method Integrates statistical regression and numerical integration within a tensor neural network framework.
result Superior performance in approximation accuracy and generalization capacity compared to FFNs and RBNs.
Low-rank tensor regression, a new model class that learns high-order correlation from data, has recently received considerable attention. At the same time, Gaussian processes (GP) are well-studied machine learning models for structure learning. In this paper, we demonstrate interesting connections between the two, espe…
BKTR models spatiotemporal data with scalable tensor regression.
problem High computational cost in applying STVC to large-scale spatiotemporal data.
method Summarize STVC coefficients in a tensor, reformulate as low-rank tensor regression, incorporate GP priors for local dependencies.
result BKTR efficiently models large spatiotemporal datasets with reduced parameters and local dependencies.
Two methods preserve tensor structure for reduced dimensionality in tensor regression.
problem Reducing dimensionality of tensor predictors for improved interpretation and accuracy.
method Developed two tensor dimension reduction methods using Tucker and CP decompositions.
result Substantial improvement in accuracy over existing methods in simulations and applications.
Tensors are becoming prevalent in modern applications such as medical imaging and digital marketing. In this paper, we propose a sparse tensor additive regression (STAR) that models a scalar response as a flexible nonparametric function of tensor covariates. The proposed model effectively exploits the sparse and low-ra…
Proposes FATTNN for tensor-on-tensor regression with improved prediction and reduced computation.
problem Tensor-on-tensor regression with complex tensor structures and nonlinear relationships.
method Integrates tensor factor models into deep neural networks to handle nonlinearity and reduce data dimensionality.
result Significant improvements in prediction accuracy and computational efficiency over traditional methods.
Nonparametric extension of tensor regression is proposed. Nonlinearity in a high-dimensional tensor space is broken into simple local functions by incorporating low-rank tensor decomposition. Compared to naive nonparametric approaches, our formulation considerably improves the convergence rate of estimation while maint…
We theoretically and experimentally investigate tensor-based regression and classification. Our focus is regularization with various tensor norms, including the overlapped trace norm, the latent trace norm, and the scaled latent trace norm. We first give dual optimization methods using the alternating direction method …
Motivated by applications in neuroimaging analysis, we propose a new regression model, Sparse TensOr REsponse regression (STORE), with a tensor response and a vector predictor. STORE embeds two key sparse structures: element-wise sparsity and low-rankness. It can handle both a non-symmetric and a symmetric tensor respo…
Bayesian tensor train kernel machine uses Laplace approximation for scalable GP regression.
problem Scalability limitations of Gaussian process regression.
method Bayesian tensor train kernel machine with Laplace approximation and variational inference.
result VI replaces cross-validation and offers up to 65x faster training.
Modeling inverse dynamics is crucial for accurate feedforward robot control. The model computes the necessary joint torques, to perform a desired movement. The highly non-linear inverse function of the dynamical system can be approximated using regression techniques. We propose as regression method a tensor decompositi…
New method tackles tensor regression with robust Kaczmarz approach.
problem Reconstructing tensor signals from corrupted measurements.
method Quantile-based randomized Kaczmarz method for tensor linear systems.
result Improved convergence and robustness to adversarial corruptions.
Paper develops RGN method for estimating low-rank tensors from noisy measurements.
problem Estimating low-rank tensors from noisy linear measurements.
method Riemannian Gauss-Newton (RGN) method for efficient low-rank tensor estimation.
result First local quadratic convergence guarantee of RGN for low-rank tensor estimation in noisy settings.
New tensor model reduces GLM estimation error and sample complexity.
problem Estimating GLM coefficients with reduced sample complexity.
method Developed LSR tensor model and block coordinate descent algorithm.
result Minimax lower bound on estimation error, suggesting lower sample complexity.
Efficient tensor kernel method reduces memory usage and computational cost for sparse regression.
problem Memory and computational limitations in tensor kernel methods for sparse regression.
method Proposes a new tensor data layout and Nystrom subsampling approach to reduce memory and computational requirements.
result Improvements lead to more efficient tensor kernel methods for sparse regression.
This paper studies a tensor-structured linear regression model with a scalar response variable and tensor-structured predictors, such that the regression parameters form a tensor of order d (i.e., a d-fold multiway array) in Rn1×n2×⋯×nd. It focuses on the task of estimatin…
Tensor networks improve integration accuracy for high-dimensional problems.
problem Integration of high-dimensional functions with exponential convergence.
method Regression-free tensor network representations for integration.
result Exponential convergence achieved for non-analytic integrands.
We present an algorithm for supervised learning using tensor networks, employing a step of preprocessing the data by coarse-graining through a sequence of wavelet transformations. We represent these transformations as a set of tensor network layers identical to those in a multi-scale entanglement renormalization ansatz…
New method reduces uncertainty in high-dimensional circuits by automatically determining tensor rank and adaptive sampling.
problem Uncertainty quantification in high-dimensional circuits due to fabrication process variations.
method Tensor regression with ℓq/ℓ2 group-sparsity regularization for rank determination and adaptive sampling. result Captures uncertainty with only 100-600 simulation samples for 19-100 random variables.
A method for learning complex functions from data with reduced memory usage.
problem Learning highly nonlinear, multivariate functions from examples.
method Transforming function learning into tensor reconstruction, incrementally building tensors from rank-one terms.
result Efficient gradient-based algorithm with linear time complexity in sample size and tensor dimensions.
Recently, tensor data (or multidimensional array) have been generated in many modern applications, such as functional magnetic resonance imaging (fMRI) in neuroscience and videos in video analysis. Many efforts are made in recent years to predict the relationship between tensor features and univariate responses. Howeve…
Bayesian model predicts phenotype effects from multi-environmental factors.
problem Predict phenotype effects from multi-environmental trials.
method Bayesian tensor regression with spike-and-slab structure.
result Model outperforms previous methods in simulation and real-world data.
Develops TOFU for tensor bandits with low-rank structure.
problem Linear bandit models fail to capture high-dimensional, low-rank tensor structures.
method Develops TOFU, a tensor bandit algorithm that estimates low-dimensional subspaces and uses norm constraints.
result Improves regret bound by a multiplicative factor that grows exponentially in system order.
Paper finds a lower bound for estimating low-rank matrices in logistic regression.
problem Estimating low-rank coefficient matrices in logistic regression.
method Derives a minimax lower bound on the risk.
result The bound depends on matrix dimensions, rank, and sample size.
In this paper we study the variational problem associated to support vector regression in Banach function spaces. Using the Fenchel-Rockafellar duality theory, we give explicit formulation of the dual problem as well as of the related optimality conditions. Moreover, we provide a new computational framework for solving…
GRTR framework uses graph regularization to improve financial forecasting.
problem High computational costs and economic domain knowledge loss in tensor models.
method Graph-Regularized Tensor Regression (GRTR) framework incorporating economic domain knowledge.
result Improved performance in multi-way financial forecasting with reduced computational costs.
New algorithm solves ℓ0-norm constrained multilinear logistic regression for tensor data.
problem Non-convex and nonsmooth ℓ0-norm constraints in multilinear logistic regression. method APALM+ method for globally convergent optimization. result APALM+ ensures convergence to a first-order critical point. HAR regression improves performance on small datasets.
problem Small datasets with complex functions.
method Data-adaptive kernel ridge regression using tensor-product spline basis.
result Achieves n−1/3 convergence rate for right-continuous functions. Discriminative latent-variable models are typically learned using EM or gradient-based optimization, which suffer from local optima. In this paper, we develop a new computationally efficient and provably consistent estimator for a mixture of linear regressions, a simple instance of a discriminative latent-variable mode…
Proposes a method to handle sparse multiway count data with false zeros using zero-truncated Poisson regression.
problem Handling sparse multiway count data corrupted by false zeros.
method Zero-truncated Poisson regression with tensor completion.
result Accurate estimation of multiway count data from approximately IR2log22(I) non-zero counts. Improved image learning using elliptically contoured tensor-variate distributions.
problem Inadequate statistical analysis for tensor-valued data, especially with heavier or lighter tails.
method Developed a family of elliptically contoured tensor-variate distributions and derived their properties and procedures for estimation.
result Tensor-variate classification rules and tensor-on-tensor regression better predict and characterize data than TVN-based methods.