Survey of SDR methods for high-dimensional regression and embedding.
problem Reducing dimensionality in high-dimensional data.
method Involves both statistical and machine learning approaches, covering inverse and forward regression methods.
result Supervised Kernel Dimension Reduction is equivalent to supervised PCA.
This paper provides mathematical foundations for regression methods used in forward initial margin approximation.
problem Developing robust methods for approximating forward initial margin.
method Introduces mathematical rigor to show that regression methods are variations of approximating the conditional expectation function.
result Each regression method is a numerical estimation of the conditional expectation with a different functional form.
Develops fast approximations for conditional Shapley values in linear and polynomial models.
problem Estimating conditional Shapley values using regression models is computationally expensive.
method A new approximative estimation method for conditional Shapley values using linear and polynomial regression models.
result Our method significantly reduces computation time compared to existing methods.
Many problems in financial engineering involve the estimation of unknown conditional expectations across a time interval. Often Least Squares Monte Carlo techniques are used for the estimation. One method that can be combined with Least Squares Monte Carlo is the "Regress-Later" method. Unlike conventional methods wher…
This paper reviews SDR methods for multivariate response regression.
problem Handling sufficient dimension reduction for multivariate response regression.
method Characterizes SDR estimators as inverse or forward regression methods.
result Pooled marginal, projective resampling, distance-based, ordinary least squares, partial least squares, and semiparametric SDR estimators are discussed.
Paper develops methods for semi-supervised Fréchet regression.
problem High costs of obtaining non-Euclidean labels.
method Proposes semi-supervised NW Fréchet regression and semi-supervised kNN Fréchet regression.
result Demonstrates superior performance over supervised methods.
Probit regression was first proposed by Bliss in 1934 to study mortality rates of insects. Since then, an extensive body of work has analyzed and used probit or related binary regression methods (such as logistic regression) in numerous applications and fields. This paper provides a fresh angle to such well-established…
We propose Bayesian extensions of two nonparametric regression methods which are kernel and mutual k k k -nearest neighbor regression methods. Derived based on Gaussian process models for regression, the extensions provide distributions for target value estimates and the framework to select the hyperparameters. It is show…
Proposes methods to correct bias and missing data in regression models.
problem Nonignorable selection bias and missing response in regression models.
method Imputation-based and importance weighted regression methods, including repeated regression and doubly robust combination.
result Repeated regression can effectively correct bias and outperforms weighted regression in extrapolation.
Modal regression estimates the local modes of the distribution of Y Y Y given X = x X=x X = x , instead of the mean, as in the usual regression sense, and can hence reveal important structure missed by usual regression methods. We study a simple nonparametric method for modal regression, based on a kernel density estimate (KDE) of …
Paper introduces semi-supervised linear extremile regression for high-dimensional data.
problem Challenges in high-dimensional extremile regression due to data sparsity and overfitting.
method Proposes semi-supervised learning for linear extremile regression, achieving n \sqrt{n} n -consistency. result Demonstrates improved estimation efficiency and performance in high-dimensional settings.
A new optimizer, MVO, improves nonlinear regression performance.
problem Finding optimal coefficients in nonlinear regression models.
method Multi-Verse Optimizer (MVO) compared to Particle Swarm Optimizer (PSO).
result MVO statistically outperforms PSO in 10 nonlinear regression problems.
Tensor Regression tackles high-dimensional data analysis.
problem Challenges in traditional data representation methods for high-dimensional data.
method Systematic study and analysis of tensor-based regression models.
result Provides solutions for specific regression tasks with multiway data.
Efficiently performs robust and sparse kernel regression.
problem Robust and sparse kernel regression.
method Sign gradient descent and early stopping.
result Sign gradient descent achieves robust and sparse kernel regression efficiently.
Proposes a method for differentially private linear regression and synthetic data generation.
problem Lack of valid inference and synthetic data generation methods for small-scale datasets in privacy-aware settings.
method Gaussian differentially private linear regression with bias-corrected estimator and SDG procedure.
result Improves accuracy and provides valid confidence intervals for downstream tasks.
Developed shrinkage methods for Poisson regression models with experts to handle multicollinearity.
problem Multicollinearity in Poisson regression models with experts.
method Ridge and Liu-type shrinkage methods.
result Shrinkage methods offer more reliable estimates for coefficients in multicollinearity.
Combination of distributional regression algorithms improves uncertainty estimation of satellite precipitation products.
problem Uncertainty estimation in satellite precipitation products.
method Ensemble learning methods combining conditional zero-adjusted probability distributions estimated with GAMLSS, spline-based GAMLSS, and distributional regression forests.
result Stacking of methods outperformed individual methods in most quantile levels using the quantile loss function.
A fast method for LOOCV in k-NN regression reduces computation time.
problem Efficient computation of LOOCV for k-NN regression.
method Identical LOOCV estimate to (k+1)-NN MSE on training data.
result LOOCV computation can be done with (k+1)-NN regression once.
Flexible empirical Bayes for large-scale multiple linear regression.
problem Large-scale multiple linear regression with flexible priors and efficient computation.
method Adaptive shrinkage priors combined with variational approximations for hyperparameter estimation.
result The posterior mean from the empirical Bayes method solves a penalized regression problem.
Kernel regression is a popular non-parametric fitting technique. It aims at learning a function which estimates the targets for test inputs as precise as possible. Generally, the function value for a test input is estimated by a weighted average of the surrounding training examples. The weights are typically computed b…
Reduced-rank method improves least-squares regression under output regularity.
problem Least-squares regression with infinite dimensional outputs.
method Reduced-rank method for solving least-squares problems with output regularity assumptions.
result Learning bounds and improved statistical performance compared to full-rank method.
A new method combines experts' opinions to train regression models with noisy labels.
problem Training regression models with noisy labels from multiple experts.
method Estimate each labeler's expertise and combine opinions using learned weights.
result Empirically outperforms existing techniques on simulated and real data.
Proposes an online method for high-dimensional streaming data.
problem Increasing variable dimensions with sample size in online kernel sliced inverse regression.
method Introduces approximate linear dependence condition and dictionary variable sets to address the problem. Transforms into online generalized eigen-decomposition problem and uses stochastic optimization for updates.
result Achieves close performance to batch processing kernel sliced inverse regression.
Three privacy-preserving methods for median regression are proposed.
problem Protecting individual privacy in median regression analysis.
method Three privacy-preserving methods: finite smoothing, iterative, and greedy coordinate descent.
result Numerical results show varying performance across different sample sizes.
Proposes a method for fair regression using RKHS.
problem Ensuring fairness in regression models with multiple sensitive attributes.
method Uses reproducing kernel Hilbert space (RKHS) to construct a functional space that satisfies MP fairness.
result Derives a closed-form solution for fair regression that is efficient and interpretable.
Paper introduces methods to create fair and accurate regression models.
problem Creating fair and accurate regression models.
method Mixed-integer optimization methods, exact formulations, branch-and-bound algorithm, coordinate descent algorithm.
result Developed methods produce fair and accurate models with reduced training times.
Change detection in heterogeneous multitemporal satellite images is an emerging and challenging topic in remote sensing. In particular, one of the main challenges is to tackle the problem in an unsupervised manner. In this paper we propose an unsupervised framework for bitemporal heterogeneous change detection based on…
Paper studies ensemble probabilistic regression trees for smooth approximations.
problem Smooth approximations of regression functions.
method Ensemble versions of probabilistic regression trees.
result Ensemble probabilistic regression trees are consistent and perform well.
A new robust regression method handles outliers in high-dimensional data.
problem Outliers in high-dimensional data make conventional regression methods ineffective.
method Robust penalized least squares of depth trimmed residuals regression.
result The new method outperforms existing methods in estimation and prediction accuracy.
Regression learns Mori-Zwanzig operators for dynamical systems.
problem Learning Mori-Zwanzig operators for complex dynamical systems.
method Statistical regression to extract Markov and memory operators.
result Regression models improve learning of memory-dependent corrections.
Efficiently selects predictors in sparse regression without approximations.
problem High computational cost in subset selection for sparse regression.
method Conditional uncorrelation formula and efficient non-approximate method.
result Significant reduction in computational complexity for subset selection.
Optimizes binary regression models with gradient ascent-descent methods.
problem Regression problems with binary weights in quantized learning and digital communication.
method Maximin optimization using gradient ascent-descent methods.
result The approach is optimal in linear regression with low noise and robust regression with few outliers.
A debiasing method improves nonparametric regression's statistical properties.
problem Lack of theoretical guarantees for modern nonparametric regression methods.
method Model-free debiasing method incorporating a correction term.
result Debiased estimator satisfies pointwise and uniform risk convergence, asymptotic normality.
Paper develops a new method for distribution regression with indefinite kernels.
problem Distribution regression with indefinite kernels.
method Coefficient-based regularized distribution regression with two-stage sampling.
result Optimal learning rates derived for the algorithm under mild conditions.
Twin neural network regression predicts differences between two data points.
problem Traditional regression methods are inaccurate for certain data sets.
method TNN regression predicts differences between two data points and averages predictions from an ensemble of all training data points.
result TNN regression yields more accurate predictions compared to other methods.
A new tree method for tensor data improves regression accuracy.
problem Efficiently modeling tensor data for regression problems.
method Scalar-output regression tree models for scalar-on-tensor problems, and tensor-on-tensor problems using additive tree ensemble approaches.
result The tensor-input tree (TT) method outperforms tensor-input GP models in efficiency and accuracy.
The paper proposes a method to produce well-calibrated predictions in regression tasks using maximum mean discrepancy.
problem The need for accurate uncertainty quantification in machine learning predictions.
method The method uses maximum mean discrepancy to minimize the kernel embedding measure and calibrate predictions.
result The method produces well-calibrated and sharp prediction intervals, outperforming state-of-the-art methods.
This chapter reviews classic regression methods and their evolution to physics-informed approaches.
problem Finding relationships between variables using regression.
method Introduces traditional and physics-informed regression methods, linking them to computational science.
result Regression methods have evolved from purely statistical to incorporating physical knowledge.
Proposes a parametric modal regression method using the implicit function theorem.
problem Finding conditional modes for multi-modal conditional distributions.
method Uses the implicit function theorem to develop an objective function for learning a joint function over inputs and targets.
result Empirically demonstrates scalability and effectiveness in learning multi-valued functions and high-dimensional inputs.
Lecture notes on advanced linear regression methods.
problem Understanding the properties of linear regression estimators in high dimensions.
method Proposition-proof exploration of least squares, ridgeless, ridge, and lasso estimators.
result Detailed analysis of the existence, uniqueness, relations, computation, and non-asymptotic properties of these estimators.
SAR evaluates ML-based linear regression models for statistical significance.
problem Lack of formal statistical significance in ML-based regression models.
method Statistical Agnostic Regression (SAR) using concentration inequalities and worst-case scenario analysis.
result SAR provides a threshold for statistical significance without assuming underlying assumptions.
Private sketches protect linear regression data privacy.
problem Protecting sensitive information in linear regression.
method Release private sketches of datasets, compute approximate solutions.
result Private sketches maintain good approximation guarantees to the original problem.
We simplify complex regression coefficients using linearization and feature comparison.
problem Interpreting high-dimensional regression coefficients from nonlinear responses.
method Developed a linearization method to derive feature coefficients and compare them with regression coefficients.
result Shows how regression coefficients relate to linearized feature coefficients and how they change under regularization.
New fair regression method improves fairness in chronic kidney disease classification.
problem Mitigating societal bias in health care for multiple groups.
method Penalized fair regression framework for multiple groups, with penalties for true positive rate disparity.
result Achieves fairness-accuracy frontier beyond existing methods in simulations and real-world data.
We propose two nonlinear regression methods, named Adversarial Orthogonal Regression (AdOR) for additive noise models and Adversarial Orthogonal Structural Equation Model (AdOSE) for the general case of structural equation models. Both methods try to make the residual of regression independent from regressors while put…
Gradient descent outperforms ridge regression under certain covariance matrix decay conditions.
problem Comparing the performance of gradient descent and ridge regression in linear models.
method Investigated gradient descent and ridge regression for linear regression with random isotropic ground truth.
result Gradient descent outperforms ridge regression under specific covariance matrix decay conditions.
Paper proposes a new method for Bayesian linear regression using spike-and-slab priors.
problem Identifying predictors with similar relationships in linear regression models.
method Hierarchical Bayesian models with spike-and-slab priors and a Gibbs sampler.
result The proposed method outperforms previous methods in simulations and real data analysis.
Simple methods improve regression transferability estimation.
problem Estimating how well regression models transfer between tasks.
method Two simple, computationally efficient approaches based on negative regularized mean squared error.
result Significantly outperform existing methods in accuracy and efficiency.