Enhances tensor regression for interpretability and performance.
problem Interpreting and modeling multidimensional tensor data with structural heterogeneity.
method Generalized Nonnegative Structured Kruskal Tensor Regression (NS-KTR) with hybrid regularization and nonnegativity constraints.
result NS-KTR outperforms conventional methods in synthetic and real hyperspectral datasets.
A new graph-based approach for estimating complex data with manifold structure.
problem Regression of large-scale, complex data with underlying geometric structure and noises.
method Constructing a skeleton graph to capture geometric structure, defining metrics, and applying nonparametric regression.
result Statistical guarantees and effectiveness demonstrated through simulations and real data examples.
New framework for network regression models accounting for community structure.
problem Inaccurate modeling of residual dependencies in network regression models.
method Modeling errors as community-based and exploiting exchangeability properties.
result Parsimonious standard errors for regression parameters.
FaStR improves scalability for time-aware RS with varying coefficients.
problem Limited applicability of structured regression models to large-scale data with categorical effects and many interactions.
method Combines structured additive regression and factorization approaches in a neural network-based model implementation.
result FaStR scales better and performs competitively with other time-aware RS in prediction performance.
Deep P-Spline automates DNN structure selection for complex regression problems.
problem Challenges in selecting optimal network structures for DNNs.
method Linking neuron selection to knot placement in basis expansion techniques, introducing a difference penalty for automated knot selection.
result Deep P-Spline extends model class and forms a latent variable modeling framework with theoretical guarantees.
Paper develops methods for semi-supervised Fréchet regression.
problem High costs of obtaining non-Euclidean labels.
method Proposes semi-supervised NW Fréchet regression and semi-supervised kNN Fréchet regression.
result Demonstrates superior performance over supervised methods.
Proposes a new method for multivariate functional regression.
problem Multivariate functional regression with complex relationships.
method Nested reduced-rank regularization (NRRR) approach.
result Consistent and effective in fitting multivariate functional regression models.
Paper develops neural network for distribution regression.
problem Regression with probability measures.
method Develops a novel fully connected neural network (FNN) for distribution inputs.
result Almost optimal learning rates for distribution regression derived.
Develops experimental design for discovering missing physics in bioreactors.
problem Discovering missing physics in incomplete model structures of process systems.
method Combines universal differential equations and symbolic regression with sequential experimental design.
result Successfully recovered true model structure of a bioreactor using machine learning techniques.
A successful approach to structured learning is to write the learning objective as a joint function of linear parameters and inference messages, and iterate between updates to each. This paper observes that if the inference problem is "smoothed" through the addition of entropy terms, for fixed messages, the learning ob…
We propose two nonlinear regression methods, named Adversarial Orthogonal Regression (AdOR) for additive noise models and Adversarial Orthogonal Structural Equation Model (AdOSE) for the general case of structural equation models. Both methods try to make the residual of regression independent from regressors while put…
The paper improves GP regression for sparse sensor data in structural mode shape reconstruction.
problem Reconstructing full-field structural mode shapes from sparse sensor data.
method Physics-Constrained Single-Output Gaussian Process (CONS-SOGP) framework.
result The proposed method provides more accurate and reliable mode shapes.
This paper extends neural collapse to regression problems, revealing key features and structures.
problem Understanding the structure learned by deep neural networks in regression tasks.
method Established Neural Regression Collapse (NRC) across different models, analyzing feature and weight alignments.
result Deep neural regression models exhibit a collapsed feature space, aligning with target dimensions and covariances.
Bayesian framework for sphere regression using Gaussian fields.
problem Nonparametric regression on the sphere with Gaussian priors.
method Isotropic Gaussian field priors, harmonic structure, exact posterior distributions, optimal spectral truncation, posterior contraction rates.
result Sharp posterior contraction rates for Gaussian priors with polynomially decaying angular power spectra.
Nash integrates covariate-specific side info into sparse regression via neural networks.
problem Sparse linear regression struggles with covariates exhibiting structure or coming from heterogeneous sources.
method Neural Adaptive Shrinkage (Nash) framework that integrates side information into sparse regression via neural networks. Uses split variational empirical Bayes algorithm.
result Nash improves accuracy and adaptability over existing methods in real data experiments.
We consider the problem of estimating a sparse multi-response regression function, with an application to expression quantitative trait locus (eQTL) mapping, where the goal is to discover genetic variations that influence gene-expression levels. In particular, we investigate a shrinkage technique capable of capturing a…
Proposes a neural network for contextual regression.
problem Improving model efficiency and interpretability in regression with contextual features.
method Simple contextual neural network (SCtxtNN) that separates context identification from context-specific regression.
result SCtxtNN achieves lower excess mean squared error and more stable performance than feed-forward neural networks.
R2T hybrid model improves robust regression for asymmetric noise.
problem Least-squares regression fails with asymmetric structured noise.
method Transformer encoder, compression NN, fixed symbolic equation.
result Median regression MSE of 6e-6 to 3.5e-5 on synthetic data.
Sparse symmetric tensor regression reduces brain connectivity complexity.
problem Complex brain connectivity analysis in neuroimaging.
method Sparse symmetric tensor regression model for functional connectivity.
result Superior performance in Alzheimer's disease detection.
Study reveals how manifold geometry impacts linear regression solutions.
problem Impact of manifold geometry on linear regression solutions.
method Linear regression applied to manifold-structured data, focusing on extrinsic geometry.
result Linear regression does not have a unique solution on flat manifolds.
A new method for linear regression using feature graphs and hierarchical shrinkage.
problem Estimating robust parameters for linear regression models.
method Hierarchical Feature Regression (HFR) estimator that constructs a supervised feature graph to shrink parameters towards group targets.
result Demonstrates good predictive accuracy and versatility compared to other regularization techniques.
A new Gaussian process regression method infers implicit manifold structure from data.
problem Scaling Gaussian process regression to high-dimensional data.
method Proposes a fully differentiable Gaussian process regression technique that infers implicit manifold structure from data.
result Improves predictive performance and calibration of standard Gaussian process regression in high-dimensional settings.
Tensor Neural Networks improve regression accuracy and efficiency.
problem Nonparametric regression problems with complex, high-dimensional functions.
method Integrates statistical regression and numerical integration within a tensor neural network framework.
result Superior performance in approximation accuracy and generalization capacity compared to FFNs and RBNs.
Reduced-rank method improves least-squares regression under output regularity.
problem Least-squares regression with infinite dimensional outputs.
method Reduced-rank method for solving least-squares problems with output regularity assumptions.
result Learning bounds and improved statistical performance compared to full-rank method.
SBT model uses randomized sharding and sub-models to improve Bayesian Additive Regression Trees.
problem Improving efficiency and accuracy of Bayesian Additive Regression Trees.
method Randomized sharding, sub-models, intersection tree structure, optimal design.
result Theoretical optimal weights and worst-case complexity of SBT model.
Proposes KAR for nonlinear causal discovery using kernel methods.
problem Learning causal relationships in nonlinear settings.
method Kernel anchor regression (KAR) with improved three-stage nonparametric regression.
result KAR outperforms existing methods in nonlinear causal discovery.
Novel Fréchet regression method handles errors-in-variables with low-rank covariates.
problem Regression with noisy and limited covariate data.
method Combines global Fréchet regression and principal component regression for low-rank structure.
result Improved efficiency and accuracy in high-dimensional and noisy data settings.
Develops a Bayesian framework for symbolic regression of scientific expressions.
problem Lack of principled uncertainty quantification and interpretability in existing symbolic regression methods.
method Hierarchical Bayesian framework with tree-structured symbolic expressions and Markov chain Monte Carlo inference.
result Robust performance on various datasets, including single-atom catalysis.
This paper studies statistical estimation in optional regression models.
problem Estimating parameters in regression models with optional semimartingale processes.
method Structural least squares (LS) estimates and their sequential versions.
result Strong consistency of LS-estimates and fixed accuracy of sequential LS-estimates.
Ridge regression analysis under varying sample size and dimensionality.
problem Prediction error analysis in asymptotic ridge regression.
method Characterization of prediction error based on covariance and parameter structure.
result Interpolation can be optimal even with bounded SNR if true parameter coefficients are larger on high-variance directions.
Metrics assess uncertainty structure and distribution for regression models.
problem Quantifying uncertainty in high-dimensional and nonlinear regression tasks.
method Two bounded comparison metrics for uncertainty structure and distribution.
result DNNs and DNOs provide encouraging uncertainty metric values in high dimensions.
Despite its importance, choosing the structural form of the kernel in nonparametric regression remains a black art. We define a space of kernel structures which are built compositionally by adding and multiplying a small number of base kernels. We present a method for searching over this space of structures which mirro…
This work analyzes how transformers learn common linear regression tasks.
problem Understanding how in-context learning operates in real-world applications with common task structures.
method Analyzing a linear attention model trained on low-rank regression tasks.
result Statistical fluctuations in finite pre-training data induce an implicit regularization, leading to a sharp phase transition in generalization error.
Deep neural nets can estimate regression with dependent data without the curse of dimensionality.
problem Regression with dependent data and structural assumptions on the regression function.
method Deep recurrent neural network estimate under suitable structural assumptions.
result Deep neural nets can circumvent the curse of dimensionality for regression with dependent data.
A new method combines multiple cancer datasets to improve analysis.
problem Combining multiple cancer datasets for comprehensive analysis.
method Multiple Augmented Reduced Rank Regression (maRRR) method.
result Improved power and insights from combining multiple cancer datasets.
Localized transfer learning improves nonparametric regression performance.
problem Improving nonparametric regression performance on target tasks.
method Localized transfer learning framework that models heterogeneity and partition covariate space into cells.
result Sharp minimax rates show local transfer mitigates the curse of dimensionality.
We study the problem of learning a sparse linear regression vector under additional conditions on the structure of its sparsity pattern. This problem is relevant in machine learning, statistics and signal processing. It is well known that a linear regression can benefit from knowledge that the underlying regression vec…
Paper proposes an algorithm to reconstruct optimal model structure from graph adjacency matrix.
problem Optimal model structure reconstruction from weighted colored graph adjacency matrix.
method Uses prize-collecting Steiner tree algorithm to reconstruct minimum spanning tree.
result Demonstrates the effectiveness of the prize-collecting Steiner tree algorithm for model structure reconstruction.
New trees-based models handle correlated data better.
problem Standard trees-based models ignore correlation structure.
method Explicitly accounts for correlation structure in splitting criterion, stopping rules, and fitted values.
result New approach superior to standard models in simulations and real data.
Paper presents a faster classical algorithm for principal component regression.
problem Efficiently solving principal component regression problems.
method Uses quantum-inspired linear algebra techniques.
result Achieves polylogarithmic runtime, significantly faster than state-of-the-art.
Modal regression estimates the local modes of the distribution of Y given X=x, instead of the mean, as in the usual regression sense, and can hence reveal important structure missed by usual regression methods. We study a simple nonparametric method for modal regression, based on a kernel density estimate (KDE) of …
Combining additive models and neural networks allows to broaden the scope of statistical regression and extend deep learning-based approaches by interpretable structured additive predictors at the same time. Existing attempts uniting the two modeling approaches are, however, limited to very specific combinations and, m…
Transformer attention layers solve single-location regression tasks.
problem Understanding token-wise sparsity and internal linear representations in attention-based models.
method Introduce single-location regression task and a simplified predictor based on self-attention layers.
result Transformer attention layers are asymptotically Bayes optimal and can learn underlying structures effectively.
Lecture notes on advanced linear regression methods.
problem Understanding the properties of linear regression estimators in high dimensions.
method Proposition-proof exploration of least squares, ridgeless, ridge, and lasso estimators.
result Detailed analysis of the existence, uniqueness, relations, computation, and non-asymptotic properties of these estimators.
New method for scalable inference in large-scale regression models with complex error structures.
problem Challenges in statistical inference for large-scale regression models with dependent errors.
method Generalized Method of Wavelet Moments with Exogenous variables (GMWMX).
result Statistical validity and scalability of GMWMX for linear models with complex error structures.
We study the following three fundamental problems about ridge regression: (1) what is the structure of the estimator? (2) how to correctly use cross-validation to choose the regularization parameter? and (3) how to accelerate computation without losing too much accuracy? We consider the three problems in a unified larg…
Improves kernel ridge regression by optimizing scale and feature parameters.
problem Kernel ridge regression with fixed kernel.
method Introduces a matrix parameter U to optimize scale and feature parameters.
result Solves a nonlinear variational problem to optimize U.
Physics-informed model predicts beam stiffness and monitors structural health.
problem Predicting and monitoring the stiffness of Euler-Bernoulli beams.
method Physics-informed Gaussian process model using the Euler-Bernoulli beam equation.
result Model accurately predicts bending stiffness and detects structural damage.