arXiv research

Deep neural nets can estimate regression with dependent data without the curse of dimensionality.

Oracle inequality for sparse neural nets adapts to unknown structure.

Metrics assess uncertainty structure and distribution for regression models.

Obtaining accurate and reliable images from low-dose computed tomography (CT) is challenging. Regression convolutional neural network (CNN) models that are learned from training data are increasingly gaining attention in low-dose CT reconstruction. This paper modifies the architecture of an iterative regression CNN, BC…

Neural net reconstructs dark matter density from halo velocities.

Study assesses neural nets for optimization problems, highlighting SiLU's effectiveness.

Survival regression method improves log-likelihood scores.

Automatic debiasing for causal and policy effects using Neural Nets and Random Forests.

Electronic phenotyping is the task of ascertaining whether an individual has a medical condition of interest by analyzing their medical record and is foundational in clinical informatics. Increasingly, electronic phenotyping is performed via supervised learning. We investigate the effectiveness of multitask learning fo…

Recent research shows that the following two models are equivalent: (a) infinitely wide neural networks (NNs) trained under l2 loss by gradient descent with infinitesimally small learning rate (b) kernel regression with respect to so-called Neural Tangent Kernels (NTKs) (Jacot et al., 2018). An efficient algorithm to c…

Proposes EE-Net for neural exploration in contextual bandits.

PIVEN predicts both specific values and prediction intervals.

In this paper we present a new framework for time-series modeling that combines the best of traditional statistical models and neural networks. We focus on time-series with long-range dependencies, needed for monitoring fine granularity data (e.g. minutes, seconds, milliseconds), prevalent in operational use-cases. Tra…

Neural Additive Models combine neural nets with interpretable models for high stakes tasks.

A new deep neural network tackles nonlinear functional regression with improved dimensionality reduction.

Two models predict net loan losses using Bayesian and frequentist regression.

DFIV uses deep neural nets to learn nonlinear features in IV regression.

LALR adapts learning rate for faster convergence in regression and neural nets.

Paper proposes robust estimators for heavy-tailed data with infinite variance.

Neural-net-induced Gaussian process (NNGP) regression inherits both the high expressivity of deep neural networks (deep NNs) as well as the uncertainty quantification property of Gaussian processes (GPs). We generalize the current NNGP to first include a larger number of hyperparameters and subsequently train the model…

The paper develops a new simulation technique for estimating conditional expectations in financial models.

Neural Ordinary Differential Equations (N-ODEs) are a powerful building block for learning systems, which extend residual networks to a continuous-time dynamical system. We propose a Bayesian version of N-ODEs that enables well-calibrated quantification of prediction uncertainty, while maintaining the expressive power …

How well does a classic deep net architecture like AlexNet or VGG19 classify on a standard dataset such as CIFAR-10 when its width --- namely, number of channels in convolutional layers, and number of nodes in fully-connected internal layers --- is allowed to increase to infinity? Such questions have come to the forefr…

New algorithm optimizes tessellated kernels for larger datasets and improved performance.

Study on Transfer Elastic Net error bounds and grouping effect.

A new method predicts precipitation distributions from ensemble forecasts.

Enhances POU-Nets with probabilistic noise model for efficient spatial data clustering.

We study deep neural networks and their use in semiparametric inference. We establish novel rates of convergence for deep feedforward neural nets. Our new rates are sufficiently fast (in some cases minimax optimal) to allow us to establish valid second-step inference after first-step estimation with deep learning, a re…

Simplifies neural regression by combining two sub-networks for predictions and uncertainties.

Paper develops algorithms for sparse linear regression with generalized elastic net penalty.

We propose a new Bayesian Neural Net formulation that affords variational inference for which the evidence lower bound is analytically tractable subject to a tight approximation. We achieve this tractability by (i) decomposing ReLU nonlinearities into the product of an identity and a Heaviside step function, (ii) intro…

An important class of distance metrics proposed for training generative adversarial networks (GANs) is the integral probability metric (IPM), in which the neural net distance captures the practical GAN training via two neural networks. This paper investigates the minimax estimation problem of the neural net distance ba…

Convolutional nets require fewer samples than fully-connected nets for image classification.

ConquerNet smooths quantile regression for deep learning with minimax guarantees.

DER uses neural nets to better handle uncertainty in machine learning.

Effective training of deep neural networks suffers from two main issues. The first is that the parameter spaces of these models exhibit pathological curvature. Recent methods address this problem by using adaptive preconditioning for Stochastic Gradient Descent (SGD). These methods improve convergence by adapting to th…

We propose a robust elastic net (REN) model for high-dimensional sparse regression and give its performance guarantees (both the statistical error bound and the optimization bound). A simple idea of trimming the inner product is applied to the elastic net model. Specifically, we robustify the covariance matrix by trimm…

Despite the phenomenal success of deep learning in recent years, there remains a gap in understanding the fundamental mechanics of neural nets. More research is focussed on handcrafting complex and larger networks, and the design decisions are often ad-hoc and based on intuition. Some recent research has aimed to demys…

We develop a fast, tractable technique called Net-Trim for simplifying a trained neural network. The method is a convex post-processing module, which prunes (sparsifies) a trained network layer by layer, while preserving the internal responses. We present a comprehensive analysis of Net-Trim from both the algorithmic a…

In this paper, we introduce transformations of deep rectifier networks, enabling the conversion of deep rectifier networks into shallow rectifier networks. We subsequently prove that any rectifier net of any depth can be represented by a maximum of a number of functions that can be realized by a shallow network with a …

Within the framework of statistical learning theory we analyze in detail the so-called elastic-net regularization scheme proposed by Zou and Hastie for the selection of groups of correlated variables. To investigate on the statistical properties of this scheme and in particular on its consistency properties, we set up …

GIT-Net uses neural networks to approximate PDE operators efficiently.

pystacked combines machine learning models for improved predictions.

We study the problem of treatment effect estimation in randomized experiments with high-dimensional covariate information, and show that essentially any risk-consistent regression adjustment can be used to obtain efficient estimates of the average treatment effect. Our results considerably extend the range of settings …

Paper simplifies complex sports analytics models for better understanding.

DP-Net uses dynamic programming for efficient deep neural network compression.

GD with early stopping trains shallow neural nets for nonparametric regression robustly.

Proposes a new deep learning model for uncertainty quantification and propagation.