Neural regression trees convert regression to classification more effectively.
problem Suboptimal approaches for regression via classification.
method Joint optimization framework for learning optimal discretization thresholds and feature selection in a neural regression tree.
result Empirically validated as state-of-the-art on challenging regression tasks.
Proposes a neural network for contextual regression.
problem Improving model efficiency and interpretability in regression with contextual features.
method Simple contextual neural network (SCtxtNN) that separates context identification from context-specific regression.
result SCtxtNN achieves lower excess mean squared error and more stable performance than feed-forward neural networks.
Deep neural nets can estimate regression with dependent data without the curse of dimensionality.
problem Regression with dependent data and structural assumptions on the regression function.
method Deep recurrent neural network estimate under suitable structural assumptions.
result Deep neural nets can circumvent the curse of dimensionality for regression with dependent data.
Paper develops neural network for distribution regression.
problem Regression with probability measures.
method Develops a novel fully connected neural network (FNN) for distribution inputs.
result Almost optimal learning rates for distribution regression derived.
Neural networks can be simplified to linear regression for easier understanding by statisticians.
problem Introducing neural networks to statisticians who are not familiar with them.
method Describing neural networks that approximate linear regression and discussing customizations.
result Statisticians can now understand neural networks by focusing on linear regression.
Analyzes neural networks using linear models to understand their behavior.
problem Understanding multi-layer neural networks through linear models.
method Recalls and reviews four models: linear regression with concentrated features, kernel ridge regression, random feature model, and neural tangent model.
result Highlights limitations of linear theory and discusses approaches to overcome them.
DRN outperforms conventional neural networks in distribution regression tasks.
problem Improving performance of distribution regression models.
method Theoretical analysis and comprehensive experiments on DRN compared to conventional neural networks.
result DRN consistently outperforms conventional neural networks in generalizability.
Tensor regression networks improve neural network compression and regularization.
problem Improving neural network compression and regularization with low-rank tensor approximations.
method Investigating various low-rank tensor approximations in tensor regression networks.
result Tensor regression networks with Global Average Pooling layer outperformed in deep CNNs, while shallow CNNs with tensor regression and dropout achieved lower test error.
This paper extends neural collapse to regression problems, revealing key features and structures.
problem Understanding the structure learned by deep neural networks in regression tasks.
method Established Neural Regression Collapse (NRC) across different models, analyzing feature and weight alignments.
result Deep neural regression models exhibit a collapsed feature space, aligning with target dimensions and covariances.
Neural networks improve Bermudan option pricing accuracy.
problem Pricing Bermudan options with conditional expectation challenges.
method Neural network approximations of conditional expectations.
result Longstaff and Schwartz algorithm convergence with neural networks.
DRE combines DNN with random feature regression for efficient neural network design.
problem Designing and training deep neural networks (DNN) efficiently and effectively.
method DRE architecture with two-layer neural networks, randomly drawn input and output weights trained with linear ridge regression.
result DRE outperforms state-of-the-art DNN in many data sets with lower computational cost.
Twin neural network regression predicts differences between two data points.
problem Traditional regression methods are inaccurate for certain data sets.
method TNN regression predicts differences between two data points and averages predictions from an ensemble of all training data points.
result TNN regression yields more accurate predictions compared to other methods.
Adapting robust statistics to neural networks, researchers found neural networks can be more robust with certain loss functions.
problem The robustness of neural networks in complex learning tasks.
method Adapting the regression breakdown point from robust statistics to neural networks and comparing different configurations and contamination settings.
result Neural networks can benefit from robust loss functions, as demonstrated in extensive simulations.
New neural networks combine additive regression with traditional architectures.
problem Performance limitations and high parameter requirements of traditional neural networks.
method Introduce hybrid deep additive neural networks with simpler activation and basis functions.
result Hybrid neural networks achieve better performance with fewer parameters.
DRIFT uses neural flows to replace distributional regression models.
problem Lack of neural network representations for distributional regression models.
method Inverse flow transformations (DRIFT) for distributional regression.
result Neural representations in DRIFT match classical statistical methods in performance.
Neural linear model performs well on simple regression tasks but requires tuning.
problem Characterizing the neural linear model's performance on simple regression tasks.
method Characterized the neural linear model on UCI and gap datasets.
result The neural linear model shows good performance but requires good hyperparameter tuning.
Bayesian neural network models improve uncertainty quantification in multivariate regression.
problem Uncertainty quantification in multivariate regression models with heteroscedastic noise.
method Proposes Bayesian Last Layer neural network models and EM algorithms for parameter learning.
result Capable of disentangling aleatoric and epistemic uncertainty.
Bayesian Additive Regression Networks use neural networks for regression tasks.
problem Regression tasks with small neural networks and ensemble learning.
method Bayesian Additive Regression Tree principles applied to small neural networks, Gibbs sampling for ensemble learning.
result BARN provides more consistent and often more accurate results than shallow neural networks, BART, and ordinary least squares.
Neural networks improve nonparametric regression with measurement errors.
problem Nonparametric regression with measurement errors.
method Proposes a neural network design using FNN, normalizing flow, and inference network.
result Neural network approach is more flexible and superior or comparable to classical methods.
Optimal rates for shallow ReLU networks in nonparametric regression.
problem Approximating smooth and non-smooth functions with shallow ReLU networks.
method Analysis of shallow ReLUk neural networks, using variation norms and deep learning theory. result Optimal approximation rates for shallow ReLU networks in nonparametric regression.
NTK neural networks are robust to adversarial attacks in nonparametric regression.
problem Adversarial robustness of neural networks in nonparametric regression.
method Gradient flow with early stopping for NTK neural networks, proving robustness in Sobolev spaces.
result NTK neural networks achieve optimal adversarial robustness rates in Sobolev spaces.
The neural tangent kernel equivalence theorem fails in practice.
problem Does the neural tangent kernel (NTK) equivalence theorem hold in practical neural network training?
method Rigorously derived NTK and conducted numerical experiments to evaluate the equivalence theorem.
result Adding a layer to a neural network and the corresponding updated NTK do not yield matching changes in predictor error.
A new neural network model for ordinal regression.
problem Ordinal regression with non-proportional odds.
method Interpretable neural network for both continuous and discrete responses, training a non-linear neural network as a coefficient function.
result N3POM preserves interpretability while offering flexibility. A mathematical framework connects neural networks and polynomial regression for better model understanding.
problem Neural networks are black boxes with challenges in dimensioning and prediction error evaluation.
method Developed a mathematical framework using Taylor expansion to relate neural networks and polynomial regression.
result Polynomial approximations from neural networks trained on polynomial data are accurate locally.
Proposes CORAL framework for consistent ordinal regression in neural networks.
problem Inconsistencies in ordinal regression with neural networks.
method Transforms ordinal targets into binary classification subtasks and applies CORAL framework for rank-monotonicity and consistent confidence scores.
result Reduction of prediction error in age prediction tasks.
A neural network solves logistic regression with ℓ1 regularization efficiently.
problem Efficiently solving logistic regression with ℓ1 regularization due to non-differentiability of ℓ1 norm. method A simple projection neural network that avoids auxiliary variables and smooth approximations.
result The neural network converges to a solution of the problem with any initial value and outperforms existing methods.
A new GGN method speeds up training of deep neural networks for regression tasks.
problem Training deep neural networks efficiently for regression problems.
method Proposes a Gram-Gauss-Newton (GGN) algorithm for overparameterized neural networks.
result For sufficiently wide neural networks, GGN achieves quadratic convergence rate.
CQNPs enhance predictive performance and distribution modeling using quantile regression.
problem Limited predictive likelihood of Gaussian models for complex distributions.
method Introducing Conditional Quantile Neural Processes (CQNPs) that focus on estimating informative quantiles.
result Significant improvements in predictive performance and better modeling of multimodal distributions.
The paper bounds neural networks' approximation error and applies it to regression and GANs.
problem Bounding the approximation error of norm-constrained neural networks.
method Proved upper and lower bounds on approximation error using Rademacher complexity.
result Obtained convergence rates for over-parameterized neural networks and optimal GAN learning rates.
Gradient descent trains neural networks to match kernel regression's sharp generalization rate.
problem Training over-parameterized neural networks for nonparametric regression.
method Gradient descent with early stopping on over-parameterized two-layer neural networks.
result Trained neural networks achieve sharp generalization rate of O(εn2). Tensor Neural Networks improve regression accuracy and efficiency.
problem Nonparametric regression problems with complex, high-dimensional functions.
method Integrates statistical regression and numerical integration within a tensor neural network framework.
result Superior performance in approximation accuracy and generalization capacity compared to FFNs and RBNs.
Enhances neural networks for regression tasks with minimal learning time increase.
problem Improving performance of neural networks in regression tasks.
method Extends the learning procedure of a neural network to improve its performance without changing the prediction.
result The modified model performs better than the original model with minimal learning time increase.
DRN compactly encodes functions for distribution regression.
problem Challenges in encoding functions compactly in neural networks.
method Designs a compact network representation to encode and propagate functions in single nodes.
result Achieves higher prediction accuracies with fewer parameters.
Deep neural networks with ReLU activation achieve optimal nonparametric regression rates.
problem Nonparametric regression with general composition assumptions.
method Sparsely connected deep neural networks with ReLU activation function.
result Achieve minimax rates of convergence under general composition assumption.
Deep neural networks estimate regression functions on manifolds.
problem Estimating regression functions on manifolds from data.
method Fully connected deep neural networks with ReLU activation, analyzing convergence rates.
result Estimates achieve a rate of convergence dependent on manifold dimension, not predictor dimension.
Neural Local Wasserstein Regression models distribution-on-distribution regression with flexible, localized transport maps.
problem Estimating distribution-on-distribution regression with global optimal transport maps or linearization limitations.
method Proposes Neural Local Wasserstein Regression, a flexible nonparametric framework using locally defined transport maps in Wasserstein space.
result Demonstrates effective capture of nonlinear and high-dimensional distributional relationships.
PSQRNN model forecasts electricity consumption in China by integrating neural networks and quantile regression.
problem Electricity forecasting in China due to regional economic, social, and natural conditions.
method PSQRNN combines neural networks and semiparametric quantile regression to model electricity consumption.
result PSQRNN model outperforms traditional methods in forecasting electricity consumption in China.
TSN improves sparse signal recovery with less complexity.
problem Sparse regression problem of recovering sparse signals from measurements.
method Tree search algorithm driven by deep neural network with pruning.
result TSN outperforms conventional methods in various sensing matrices.
This paper compares linear regression and neural networks for pricing swing options.
problem Pricing swing options using approximation methods.
method Linear regression and neural networks for approximating the continuation value and swing price.
result The approximation methods converge to the actual swing price as the number of functions or Monte Carlo samples increases.
Deep neural networks enforce non-crossing quantile regression curves.
problem Estimating quantile regression curves without crossing.
method Penalized deep ReQU neural networks with a non-crossing penalty.
result Established non-asymptotic risk and error bounds for the estimated QRP.
Deep neural networks with adversarial training achieve sup-norm convergence for nonparametric regression.
problem Achieving sup-norm convergence for deep neural network estimators in nonparametric regression.
method Developed an adversarial training scheme to address the sup-norm convergence issue.
result Deep neural network estimators achieve optimal sup-norm convergence with the proposed adversarial training.
Neural networks simplify SDR in regression tasks.
problem Sufficient dimension reduction in regression problems.
method Applying neural networks with rank regularization to estimate the central mean subspace.
result Neural networks effectively perform SDR, consistent with theoretical estimations.
Deep neural networks can learn smooth functions without parameters.
problem Learning smooth functions from shallow ReLU neural networks.
method Using over-parameterized shallow ReLU neural networks with norm constraints.
result Least squares estimators based on shallow neural networks are minimax optimal.
New deep learning method for real-time regression analysis.
problem Real-time regression analysis for time series data.
method Novel deep learning algorithms for real-time regression analysis.
result Demonstrated real-time regression analysis for time series data.
TRNN combines tensor geometry with neural network nonlinearity for HD data.
problem Modeling high-dimensional data with preserved tensor geometry and nonlinear interactions.
method Introduces TRNN that integrates tensor geometry and neural network nonlinearity.
result TRNN preserves tensor geometry while offering nonlinearity.
Nash integrates covariate-specific side info into sparse regression via neural networks.
problem Sparse linear regression struggles with covariates exhibiting structure or coming from heterogeneous sources.
method Neural Adaptive Shrinkage (Nash) framework that integrates side information into sparse regression via neural networks. Uses split variational empirical Bayes algorithm.
result Nash improves accuracy and adaptability over existing methods in real data experiments.
We analyzed optimism in linear and kernel regression models.
problem Understanding predictive complexity in regression models.
method Derived closed-form asymptotic optimism for linear and kernel regression models.
result Scaled optimism is a useful measure for model complexity.
R2T hybrid model improves robust regression for asymmetric noise.
problem Least-squares regression fails with asymmetric structured noise.
method Transformer encoder, compression NN, fixed symbolic equation.
result Median regression MSE of 6e-6 to 3.5e-5 on synthetic data.