Bayesian Additive Regression Networks use neural networks for regression tasks.
problem Regression tasks with small neural networks and ensemble learning.
method Bayesian Additive Regression Tree principles applied to small neural networks, Gibbs sampling for ensemble learning.
result BARN provides more consistent and often more accurate results than shallow neural networks, BART, and ordinary least squares.
Proposes a neural network for contextual regression.
problem Improving model efficiency and interpretability in regression with contextual features.
method Simple contextual neural network (SCtxtNN) that separates context identification from context-specific regression.
result SCtxtNN achieves lower excess mean squared error and more stable performance than feed-forward neural networks.
Neural networks can be simplified to linear regression for easier understanding by statisticians.
problem Introducing neural networks to statisticians who are not familiar with them.
method Describing neural networks that approximate linear regression and discussing customizations.
result Statisticians can now understand neural networks by focusing on linear regression.
Tensor Neural Networks improve regression accuracy and efficiency.
problem Nonparametric regression problems with complex, high-dimensional functions.
method Integrates statistical regression and numerical integration within a tensor neural network framework.
result Superior performance in approximation accuracy and generalization capacity compared to FFNs and RBNs.
Regression Prior Networks improve ensemble performance on regression tasks.
problem Improving ensemble performance on regression tasks.
method Extending Prior Networks and Ensemble Distribution Distillation (EnD2) to regression tasks using the Normal-Wishart distribution. result Regression Prior Networks yield performance competitive with ensemble approaches on regression tasks.
Paper develops neural network for distribution regression.
problem Regression with probability measures.
method Develops a novel fully connected neural network (FNN) for distribution inputs.
result Almost optimal learning rates for distribution regression derived.
Modified ReLU networks improve regression estimation rates.
problem Regression estimation with smooth functions.
method Using modified ReLU neural networks with specific weight modifications.
result Empirical risk minimizers achieve minimax rate of prediction.
New framework for network regression models accounting for community structure.
problem Inaccurate modeling of residual dependencies in network regression models.
method Modeling errors as community-based and exploiting exchangeability properties.
result Parsimonious standard errors for regression parameters.
Twin neural network regression predicts differences between two data points.
problem Traditional regression methods are inaccurate for certain data sets.
method TNN regression predicts differences between two data points and averages predictions from an ensemble of all training data points.
result TNN regression yields more accurate predictions compared to other methods.
Bayes-optimal learning of deep random networks with Gaussian weights is studied.
problem Learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights.
method Closed-form expressions for Bayes-optimal test error, ridge regression, kernel and random features regression are computed.
result Optimally regularized ridge regression and kernel regression achieve Bayes-optimal performances, while logistic loss yields a near-optimal test error for classification.
Tensor regression networks achieve high compression rate of neural networks while having slight impact on performances. They do so by imposing low tensor rank structure on the weight matrices of fully connected layers. In recent years, tensor regression networks have been investigated from the perspective of their comp…
Enhances kernel regression with network data for better predictions.
problem Improving predictive power in high-dimensional data.
method Combines kernel regression with network cohesion data to model nonlinearities.
result Significantly better predictive performances in high-dimensional data.
Adapting robust statistics to neural networks, researchers found neural networks can be more robust with certain loss functions.
problem The robustness of neural networks in complex learning tasks.
method Adapting the regression breakdown point from robust statistics to neural networks and comparing different configurations and contamination settings.
result Neural networks can benefit from robust loss functions, as demonstrated in extensive simulations.
Despite the superior performance of deep learning in many applications, challenges remain in the area of regression on function spaces. In particular, neural networks are unable to encode function inputs compactly as each node encodes just a real value. We propose a novel idea to address this shortcoming: to encode an …
Quantile regression with ReLU networks achieves minimax rates for various function types.
problem Estimating quantiles from covariates with neural networks.
method Quantile regression with rectified linear unit (ReLU) neural networks.
result ReLU networks achieve minimax rates for broad collections of function types.
Analyzes neural networks using linear models to understand their behavior.
problem Understanding multi-layer neural networks through linear models.
method Recalls and reviews four models: linear regression with concentrated features, kernel ridge regression, random feature model, and neural tangent model.
result Highlights limitations of linear theory and discusses approaches to overcome them.
New neural networks combine additive regression with traditional architectures.
problem Performance limitations and high parameter requirements of traditional neural networks.
method Introduce hybrid deep additive neural networks with simpler activation and basis functions.
result Hybrid neural networks achieve better performance with fewer parameters.
Neural networks perform differently when regression is treated as classification.
problem Understanding why neural networks perform better when regression is treated as classification.
method Analyzing two-layer ReLU networks and their feature spaces, focusing on the cross entropy loss vs. square loss.
result The support of the measure induced by the square loss differs from that of the cross entropy loss, indicating optimization difficulties.
Bayesian neural network models improve uncertainty quantification in multivariate regression.
problem Uncertainty quantification in multivariate regression models with heteroscedastic noise.
method Proposes Bayesian Last Layer neural network models and EM algorithms for parameter learning.
result Capable of disentangling aleatoric and epistemic uncertainty.
Neural networks improve nonparametric regression with measurement errors.
problem Nonparametric regression with measurement errors.
method Proposes a neural network design using FNN, normalizing flow, and inference network.
result Neural network approach is more flexible and superior or comparable to classical methods.
DRE combines DNN with random feature regression for efficient neural network design.
problem Designing and training deep neural networks (DNN) efficiently and effectively.
method DRE architecture with two-layer neural networks, randomly drawn input and output weights trained with linear ridge regression.
result DRE outperforms state-of-the-art DNN in many data sets with lower computational cost.
Optimal rates for shallow ReLU networks in nonparametric regression.
problem Approximating smooth and non-smooth functions with shallow ReLU networks.
method Analysis of shallow ReLUk neural networks, using variation norms and deep learning theory. result Optimal approximation rates for shallow ReLU networks in nonparametric regression.
NTK neural networks are robust to adversarial attacks in nonparametric regression.
problem Adversarial robustness of neural networks in nonparametric regression.
method Gradient flow with early stopping for NTK neural networks, proving robustness in Sobolev spaces.
result NTK neural networks achieve optimal adversarial robustness rates in Sobolev spaces.
A new method for network regression using optimal transport.
problem How network topology changes with Euclidean covariates.
method Optimal transport approach based on Wasserstein metric.
result The method improves prediction accuracy in real-world data.
Network-assisted regression uses conformal prediction for valid inference.
problem Predicting node attributes using network and conventional covariates with valid statistical inference.
method Network analog of conformal prediction under mild joint exchangeability assumption.
result Achieves finite sample validity and asymptotic conditional validity for various network covariates.
The paper bounds neural networks' approximation error and applies it to regression and GANs.
problem Bounding the approximation error of norm-constrained neural networks.
method Proved upper and lower bounds on approximation error using Rademacher complexity.
result Obtained convergence rates for over-parameterized neural networks and optimal GAN learning rates.
Gradient descent trains neural networks to match kernel regression's sharp generalization rate.
problem Training over-parameterized neural networks for nonparametric regression.
method Gradient descent with early stopping on over-parameterized two-layer neural networks.
result Trained neural networks achieve sharp generalization rate of O(εn2). A new neural network model for ordinal regression.
problem Ordinal regression with non-proportional odds.
method Interpretable neural network for both continuous and discrete responses, training a non-linear neural network as a coefficient function.
result N3POM preserves interpretability while offering flexibility. Deep neural networks enforce non-crossing quantile regression curves.
problem Estimating quantile regression curves without crossing.
method Penalized deep ReQU neural networks with a non-crossing penalty.
result Established non-asymptotic risk and error bounds for the estimated QRP.
There is emerging interest in performing regression between distributions. In contrast to prediction on single instances, these machine learning methods can be useful for population-based studies or on problems that are inherently statistical in nature. The recently proposed distribution regression network (DRN) has sh…
Proposes a novel network-based neighborhood regression for biological systems.
problem Lack of comprehensive analysis on biological modules using both global and local network data.
method Develops a community-wise least square optimization approach to analyze gene modules and their regulatory strength.
result Achieves exact minimax optimality and linear consistency in identifying gene module associations.
A mathematical framework connects neural networks and polynomial regression for better model understanding.
problem Neural networks are black boxes with challenges in dimensioning and prediction error evaluation.
method Developed a mathematical framework using Taylor expansion to relate neural networks and polynomial regression.
result Polynomial approximations from neural networks trained on polynomial data are accurate locally.
We consider the classical sparse regression problem of recovering a sparse signal x0 given a measurement vector y=Φx0+w. We propose a tree search algorithm driven by the deep neural network for sparse regression (TSN). TSN improves the signal reconstruction performance of the deep neural network designed for sp…
The network Lasso (nLasso) has been proposed recently as an efficient learning algorithm for massive networked data sets (big data over networks). It extends the well-known least absolute shrinkage and selection operator (Lasso) from learning sparse (generalized) linear models to network models. Efficient implementatio…
Deep neural networks estimate regression functions on manifolds.
problem Estimating regression functions on manifolds from data.
method Fully connected deep neural networks with ReLU activation, analyzing convergence rates.
result Estimates achieve a rate of convergence dependent on manifold dimension, not predictor dimension.
This paper compares linear regression and neural networks for pricing swing options.
problem Pricing swing options using approximation methods.
method Linear regression and neural networks for approximating the continuation value and swing price.
result The approximation methods converge to the actual swing price as the number of functions or Monte Carlo samples increases.
Deep neural nets can estimate regression with dependent data without the curse of dimensionality.
problem Regression with dependent data and structural assumptions on the regression function.
method Deep recurrent neural network estimate under suitable structural assumptions.
result Deep neural nets can circumvent the curse of dimensionality for regression with dependent data.
PSQRNN model forecasts electricity consumption in China by integrating neural networks and quantile regression.
problem Electricity forecasting in China due to regional economic, social, and natural conditions.
method PSQRNN combines neural networks and semiparametric quantile regression to model electricity consumption.
result PSQRNN model outperforms traditional methods in forecasting electricity consumption in China.
Deep single-index Fréchet regression for metric space-valued outputs
problem Predicting outputs in non-Euclidean spaces
method DeSI (Deep Single-Index Fréchet Regression)
result Interpretable index direction for inputs
We analyzed optimism in linear and kernel regression models.
problem Understanding predictive complexity in regression models.
method Derived closed-form asymptotic optimism for linear and kernel regression models.
result Scaled optimism is a useful measure for model complexity.
New method extracts aleatoric and epistemic uncertainties from regression-based neural networks.
problem Need for principled uncertainty reasoning in machine learning systems.
method Learning evidential distributions for aleatoric and epistemic uncertainties.
result Allows for the simultaneous extraction of both uncertainties without sampling or out-of-distribution data.
The pricing of Bermudan options amounts to solving a dynamic programming principle, in which the main difficulty, especially in high dimension, comes from the conditional expectation involved in the computation of the continuation value. These conditional expectations are classically computed by regression techniques o…
Wasserstein active regression improves estimation precision.
problem Improving regression model accuracy through active learning.
method Combines Wasserstein distance and GroupSort Neural Networks for uncertainty quantification.
result Wasserstein active regression often provides more precise estimations.
A neural network solves logistic regression with ℓ1 regularization efficiently.
problem Efficiently solving logistic regression with ℓ1 regularization due to non-differentiability of ℓ1 norm. method A simple projection neural network that avoids auxiliary variables and smooth approximations.
result The neural network converges to a solution of the problem with any initial value and outperforms existing methods.
Consider the multivariate nonparametric regression model. It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly chosen network architecture achieve the minimax rates of convergence (up to logn-factors) under a general composition assumption on the re…
Deep neural networks can learn smooth functions without parameters.
problem Learning smooth functions from shallow ReLU neural networks.
method Using over-parameterized shallow ReLU neural networks with norm constraints.
result Least squares estimators based on shallow neural networks are minimax optimal.
We apply network Lasso to semi-supervised regression problems involving network structured data. This approach lends quite naturally to highly scalable learning algorithms in the form of message passing over an empirical graph which represents the network structure of the data. By using a simple non-parametric regressi…
New results show neural networks generalize well due to polynomial regression, not just overparametrization.
problem Generalization of neural networks despite overparametrization.
method Teacher/Student model with polynomial regression.
result Student networks interpolating teacher-generated data generalize well with a minimal sample size.