PROD method improves high-dimensional regression by handling strong correlations.
problem Violation of Irrepresentable Condition in LASSO for high-dimensional data.
method PROD procedure based on orthogonal decomposition of design matrix.
result PROD enhances performance of high-dimensional penalized regression.
Prevalidated ridge regression simplifies logistic regression for high-dimensional data.
problem Efficient probabilistic classification in high-dimensional data with logistic regression.
method Developed a prevalidated ridge regression model that matches logistic regression's performance but is more computationally efficient.
result Prevalidated ridge regression achieves similar classification error and log-loss to logistic regression for high-dimensional data.
Proposes GPLFR for predicting high-dimensional outputs with few data.
problem Predicting high-dimensional outputs from limited data.
method GPLFR combines Gaussian process and linear-Gaussian decoding for high-dimensional prediction.
result GPLFR outperforms existing methods in predicting high-dimensional outputs.
Overview of high-dimensional time series regression methods.
problem Estimation and inference with high-dimensional time series data.
method Limit theory for high-dimensional dependent data, asymptotic theory for time series regression, statistical learning methods.
result Main limit theory results and asymptotic theory for high-dimensional time series regression.
Tensor Regression tackles high-dimensional data analysis.
problem Challenges in traditional data representation methods for high-dimensional data.
method Systematic study and analysis of tensor-based regression models.
result Provides solutions for specific regression tasks with multiway data.
We simplify complex regression coefficients using linearization and feature comparison.
problem Interpreting high-dimensional regression coefficients from nonlinear responses.
method Developed a linearization method to derive feature coefficients and compare them with regression coefficients.
result Shows how regression coefficients relate to linearized feature coefficients and how they change under regularization.
Boosting ridge regression for high-dimensional data classification reduces computational cost and improves learning time.
problem High computational demand of inverting regularised covariance matrix in ridge regression for high-dimensional problems.
method Train an ensemble of ridge regressors in randomly projected subspaces, then combine them using adaptive boosting.
result Effective in terms of learning time and improved predictive performance in some cases.
Paper introduces semi-supervised linear extremile regression for high-dimensional data.
problem Challenges in high-dimensional extremile regression due to data sparsity and overfitting.
method Proposes semi-supervised learning for linear extremile regression, achieving n \sqrt{n} n -consistency. result Demonstrates improved estimation efficiency and performance in high-dimensional settings.
Survey of SDR methods for high-dimensional regression and embedding.
problem Reducing dimensionality in high-dimensional data.
method Involves both statistical and machine learning approaches, covering inverse and forward regression methods.
result Supervised Kernel Dimension Reduction is equivalent to supervised PCA.
Unified derivation of high-dimensional linear models using stochastic gradient descent.
problem Performance analysis of high-dimensional linear models trained with stochastic gradient descent.
method Derivation of a deterministic equivalence for the two-point function of a random matrix resolvent.
result Unified understanding of model performance including previously known and novel results.
In this article the package High-dimensional Metrics (\texttt{hdm}) is introduced. It is a collection of statistical methods for estimation and quantification of uncertainty in high-dimensional approximately sparse models. It focuses on providing confidence intervals and significance testing for (possibly many) low-dim…
The package High-dimensional Metrics (\Rpackage{hdm}) is an evolving collection of statistical methods for estimation and quantification of uncertainty in high-dimensional approximately sparse models. It focuses on providing confidence intervals and significance testing for (possibly many) low-dimensional subcomponents…
Study shows double descent curve in high-dimensional linear regression with random projections.
problem Understanding the generalization performance in high-dimensional settings with random projections.
method Fixed prediction problem, ridge regression estimator, minimum norm least-squares fit, random matrix theory, asymptotic equivalents.
result Exhibit a double descent curve for high-dimensional linear regression with random projections.
Nested model averaging improves high-dimensional linear regression performance.
problem High-dimensional linear regression with predictor ordering impact.
method Combining model averaging with regularized estimators on the solution path.
result Nested model averaging with lasso and SLOPE outperforms competing methods.
We study the problem of treatment effect estimation in randomized experiments with high-dimensional covariate information, and show that essentially any risk-consistent regression adjustment can be used to obtain efficient estimates of the average treatment effect. Our results considerably extend the range of settings …
Gaussian process regression loses locality in high dimensions, affecting molecular energy surface fitting.
problem Loss of locality in high-dimensional Gaussian process regression.
method Analysis of Matern family kernels and multi-zeta basis functions.
result The property of locality disappears in high dimensions, impacting regression quality.
Enhances kernel regression with network data for better predictions.
problem Improving predictive power in high-dimensional data.
method Combines kernel regression with network cohesion data to model nonlinearities.
result Significantly better predictive performances in high-dimensional data.
Penalized (or regularized) regression, as represented by Lasso and its variants, has become a standard technique for analyzing high-dimensional data when the number of variables substantially exceeds the sample size. The performance of penalized regression relies crucially on the choice of the tuning parameter, which d…
Paper proposes inference method for high-dimensional censored quantile regression.
problem Identifying heterogeneous effects of high-dimensional genetic biomarkers on survival outcomes.
method Combines low-dimensional model estimates based on multi-sample splittings and variable selection.
result Proposed estimator is consistent and asymptotically follows a Gaussian process.
Tensor Neural Networks improve regression accuracy and efficiency.
problem Nonparametric regression problems with complex, high-dimensional functions.
method Integrates statistical regression and numerical integration within a tensor neural network framework.
result Superior performance in approximation accuracy and generalization capacity compared to FFNs and RBNs.
New method optimizes model selection in high-dimensional regression models.
problem Model selection in high-dimensional misspecified regression models with covariate shift.
method Importance-weighted orthogonal greedy algorithm (IWOGA) and high-dimensional importance-weighted information criterion (HDIWIC).
result IWOGA + HDIWIC achieves optimal convergence rates in terms of prediction error.
The nullspace and regularization impact high-dimensional linear regression interpretability.
problem Interpreting high-dimensional linear regression coefficients in complex data.
method Optimization formulation to compare coefficients and physical knowledge.
result Regularization and z-scoring choices affect interpretability and true coefficient closeness.
Paper analyzes GLM-tron for high-dimensional ReLU regression, providing upper and lower bounds.
problem Learning a single ReLU neuron in high-dimensional settings with overparameterization.
method Perceptron-type algorithm GLM-tron, with finite-sample analysis.
result Sharp characterization of high-dimensional ReLU regression problems via GLM-tron, contrasting with SGD.
Paper introduces a new IV regression method for mixed-frequency data.
problem Estimating high-dimensional slope parameters in mixed-frequency data.
method Tikhonov-regularized estimator for high-dimensional linear IV regression.
result High-dimensional slope parameter can be accurately estimated using a low-frequency instrumental variable.
A scalable method for accurate inference of low-dimensional parameters in high-dimensional linear regression.
problem Statistical inference for low-dimensional parameters in high-dimensional linear regression models.
method Mean-field variational Bayes approach, focusing on nuisance parameters and conditional distributions.
result Competitive numerical performance and theoretical guarantees for estimation and uncertainty quantification.
Simplifies IV regression for high-dimensional instruments.
problem Nonlinear instrumental variable regression with high-dimensional instruments.
method Combines kernelized IV methods with an adaptive regression algorithm.
result Faster convergence and adaptability to feature dimensionality.
The paper analyzes how generated data improves adversarial training in high-dimensional regression.
problem Improving adversarial training in high-dimensional regression.
method Theoretical analysis of a two-stage training approach with generated data and pseudo-labels.
result Two-stage adversarial training achieves better performance than ridgeless training in high-dimensional linear regression.
Paper tackles distributed quantile regression with improved efficiency and support recovery.
problem Challenges in distributed estimation and support recovery for high-dimensional linear quantile regression.
method Transformed quantile regression into least-squares optimization, applied double-smoothing approach, developed efficient algorithm.
result Achieved near-oracle convergence rate and high support recovery accuracy.
Efficiently selects important variables in high-dimensional logistic regression.
problem Variable selection in high-dimensional logistic regression with binary responses.
method Developed a variational empirical Bayes approach for efficient model space marginal distribution.
result The variational approximation inherits strong selection consistency from the posterior distribution.
EP method speeds up Bayesian probit regression in high dimensions.
problem Computational challenges in high-dimensional Bayesian probit regression.
method Adapting EP approximation to multivariate Gaussian prior and skew-normal distribution.
result EP routine is computationally feasible in high-dimensional settings.
Paper tackles high-dimensional quantile regression with distribution shift using transfer learning.
problem Efficiency of knowledge transfer is severely impacted by distribution shift in high-dimensional regression.
method Proposes a novel transferable set and framework for three types of distribution shift: parameter, covariate, and residual.
result Establishes estimation error bounds and source detection consistency for the proposed method.
Improved VB algorithm for high-dimensional logistic regression with theoretical guarantees.
problem Sparse high-dimensional logistic regression model selection.
method Spike and slab variational Bayes approximation.
result Optimal convergence rates in ℓ 2 \ell_2 ℓ 2 and prediction loss for sparse truths. The paper tackles high-dimensional mixed linear regression with unknown parameters and proposes methods for estimation, confidence intervals, and hypothesis testing.
problem High-dimensional mixed linear regression with unknown parameters and covariance structure.
method Iterative high-dimensional EM algorithm for estimating regression vectors, debiased estimators for individual coordinates, and large-scale multiple testing procedure.
result Asymptotic normality of debiased estimators and FDR control for hypothesis testing.
New method bounds high-dimensional regression without estimating design covariance.
problem High-dimensional linear regression with random design.
method Error-in-operator approach that incorporates design covariance into empirical risk minimization.
result Dimension-free bounds on excess prediction risk derived.
PROBE algorithm efficiently solves sparse high-dimensional linear regression.
problem Sparse high-dimensional linear regression models with complex parameter spaces.
method Partitioned empirical Bayes ECM algorithm for computationally efficient MAP estimation.
result PROBE algorithm provides robust and efficient coordinate-wise optimization.
New method for high-dimensional linear regression using empirical Bayes.
problem Estimating prior in high-dimensional linear regression.
method Variational empirical Bayes approach with NPMLE and mean field approximation.
result Established asymptotic consistency and computational efficiency of the method.
New CLT for SGD in high-dimensional regression provides online inference.
problem Quantifying uncertainty in SGD for high-dimensional regression.
method Established a high-dimensional CLT for online SGD iterates.
result Developed an online approach for estimating variance in CLT.
In this paper, we propose a non-parametric conditional factor regression (NCFR)model for domains with high-dimensional input and response. NCFR enhances linear regression in two ways: a) introducing low-dimensional latent factors leading to dimensionality reduction and b) integrating an Indian Buffet Process as a prior…
Proposes a new robust expectile regression method for high-dimensional data.
problem Heterogeneity in high-dimensional data with heteroscedastic variance or inhomogeneous covariate effects.
method Iteratively reweighted ℓ1-penalization for robust expectile regression (retire).
result Oracle convergence rate after log(log d) iterations in high-dimensional settings.
Develops a fast algorithm for high-dimensional LASSO penalized quantile regression.
problem Computational challenges in high-dimensional ℓ 1 \ell_1 ℓ 1 penalized quantile regression. method Pathwise coordinate descent algorithm to solve exact coordinatewise minimum of the nonsmooth loss function.
result Algorithm runs faster than existing alternatives and maintains estimation accuracy.
Aggregates predictions from multiple regression models using random projections and kernel methods.
problem Combining predictions from multiple regression models to improve accuracy.
method Random projection of high-dimensional feature space, followed by kernel-based consensual aggregation.
result The aggregation scheme performs similarly to using the original high-dimensional features, with high probability.
High-dimensional regression models struggle with resampling methods.
problem Estimating uncertainty in high-dimensional supervised regression tasks.
method Investigation of bootstrap, subsampling, and jackknife methods in high-dimensional generalized linear models.
result Resampling methods exhibit double-descent behavior and are inconsistent in high dimensions.
Study high-dimensional Bayesian linear regression using variational inference.
problem High-dimensional Bayesian linear regression with product priors.
method Non-linear large deviations theory and variational inference.
result Unique optimizer in variational problem governs posterior distribution under separation condition.
Active learning improves GP regression on complex, high-dimensional data.
problem Improving Gaussian Process regression in high-dimensional spaces with discontinuous functions.
method Combines manifold learning with active learning to optimize data selection and reduce dimensionality.
result Superior performance over random learning in synthetic data experiments.
Proposes a transfer learning method for high-dimensional quantile regression.
problem Inadequate handling of heterogeneity and heavy tails in transfer learning.
method High-dimensional quantile regression framework with double transfer learning estimator.
result Established error bounds and valid confidence intervals for high-dimensional quantile regression coefficients.
New methods combine low and high-fidelity data for accurate surrogate modeling.
problem Challenges in surrogate modeling for high-dimensional outputs with limited training data.
method Projection-based multifidelity linear regression methods integrating low-fidelity and high-fidelity data.
result Multifidelity methods achieve up to 12% improvement in median accuracy compared to single-fidelity methods.
New framework for regression trees with multivariate response and dynamic mean vectors.
problem Characterizing and implementing regression trees for multivariate responses.
method High dimensional model with dynamic mean vectors over multi-dimensional change axes.
result Optimal rate of convergence and asymptotic valid confidence intervals for change points.
Study on reducing dimensionality in high-dimensional regression with kernel methods and stability analysis.
problem Analyzing errors in high-dimensional regression with dimensionality reduction and kernel regression.
method Derive a stability result for kernel regression with Wasserstein distance and apply it to PCA to deduce convergence rates.
result Two-step procedure yields useful convergence rates in semi-supervised settings.