arXiv research

Proposes a new method to estimate Bayesian neural network depth.

Paper uses statistical depth to create DP estimators for regression.

Learning based methods have shown very promising results for the task of depth estimation in single images. However, most existing approaches treat depth prediction as a supervised regression problem and as a result, require vast quantities of corresponding ground truth depth data for training. Just recording quality d…

Uniform consistency proven for spatial distribution and depth estimators in any dimension.

Develops privacy-preserving multivariate median estimation methods.

Estimates neural network error approximating compact sets.

This paper studies robust regression in the settings of Huber's $ε$-contamination models. We consider estimators that are maximizers of multivariate regression depth functions. These estimators are shown to achieve minimax rates in the settings of $ε$-contamination models for various regression problems including nonpa…

New method improves reliability of depth estimation models.

One-shot neural architecture search allows joint learning of weights and network architecture, reducing computational cost. We limit our search space to the depth of residual networks and formulate an analytically tractable variational objective that allows for obtaining an unbiased approximate posterior over depths in…

Robust estimation under Huber's $ε$-contamination model has become an important topic in statistics and theoretical computer science. Statistically optimal procedures such as Tukey's median and other estimators based on depth functions are impractical because of their computational intractability. In this paper, we est…

Single-pass method estimates neural network uncertainty.

In the last decade, supervised deep learning approaches have been extensively employed in visual odometry (VO) applications, which is not feasible in environments where labelled data is not abundant. On the other hand, unsupervised deep learning approaches for localization and mapping in unknown environments from unlab…

Introduces Polar Depth for analyzing multivariate heavy-tailed data extremes.

We describe a method to infer dense depth from camera motion and sparse depth as estimated using a visual-inertial odometry system. Unlike other scenarios using point clouds from lidar or structured light sensors, we have few hundreds to few thousand points, insufficient to inform the topology of the scene. Our method …

Paper introduces MCSD, a method for uncertainty estimation in deep learning.

We present a self-supervised approach to training convolutional neural networks for dense depth estimation from monocular endoscopy data without a priori modeling of anatomy or shading. Our method only requires monocular endoscopic videos and a multi-view stereo method, e.g., structure from motion, to supervise learnin…

We develop a scalable method for Bayesian neural networks with stochastic differential equations.

Per-pixel ground-truth depth data is challenging to acquire at scale. To overcome this limitation, self-supervised learning has emerged as a promising alternative for training models to perform monocular depth estimation. In this paper, we propose a set of improvements, which together result in both quantitatively and …

We present an unsupervised approach for learning to estimate three dimensional (3D) facial structure from a single image while also predicting 3D viewpoint transformations that match a desired pose and facial geometry. We achieve this by inferring the depth of facial keypoints of an input image in an unsupervised manne…

Transformers can learn noisy linear systems with depth and IID data.

Study Transformer layers under cross-entropy training using mean field control.

This paper studies the expressive power of graph neural networks falling within the message-passing framework (GNNmp). Two results are presented. First, GNNmp are shown to be Turing universal under sufficient conditions on their depth, width, node attributes, and layer expressiveness. Second, it is discovered that GNNm…

Robust scatter estimation is a fundamental task in statistics. The recent discovery on the connection between robust estimation and generative adversarial nets (GANs) by Gao et al. (2018) suggests that it is possible to compute depth-like robust estimators using similar techniques that optimize GANs. In this paper, we …

Method combines LD and Fermat Distance for neural network uncertainty.

We consider the problem of estimating the conditional probability of a label in time O(log n), where n is the number of possible labels. We analyze a natural reduction of this problem to a set of binary regression problems organized in a tree structure, proving a regret bound that scales with the depth of the tree. Mot…

This paper establishes the (nearly) optimal approximation error characterization of deep rectified linear unit (ReLU) networks for smooth functions in terms of both width and depth simultaneously. To that end, we first prove that multivariate polynomials can be approximated by deep ReLU networks of width $\mathcal{O}(N…

Smooth activations enable optimal error rates in neural networks for Sobolev function classes.

Attention-only transformers learn from context via two stages of inference.

The theory of tunnel number 1 knots detailed in our previous paper, The tree of knot tunnels, provides a non-negative integer invariant called the depth of the tunnel. We give various results related to the depth invariant. Noting that it equals the minimum number of Goda-Scharlemann-Thompson tunnel moves needed to con…

Depth perception is a key component for autonomous systems that interact in the real world, such as delivery robots, warehouse robots, and self-driving cars. Tasks in autonomous robotics such as 3D object recognition, simultaneous localization and mapping (SLAM), path planning and navigation, require some form of 3D sp…

We present a model for the joint estimation of disparity and motion. The model is based on learning about the interrelations between images from multiple cameras, multiple frames in a video, or the combination of both. We show that learning depth and motion cues, as well as their combinations, from data is possible wit…

Deep networks improve by progressively refining approximations at each layer.

With the ubiquity of sensors in the IoT era, statistical observations are becoming increasingly available in the form of massive (multivariate) time-series. Formulated as unsupervised anomaly detection tasks, an abundance of applications like aviation safety management, the health monitoring of complex infrastructures …

This article concerns the expressive power of depth in neural nets with ReLU activations and bounded width. We are particularly interested in the following questions: what is the minimal width $w_{\text{min}}(d)$ so that ReLU nets of width $w_{\text{min}}(d)$ (and arbitrary depth) can approximate any continuous functio…

New algorithm quantifies uncertainty in regression models for complex data types.

Regression Prior Networks improve ensemble performance on regression tasks.

The study analyzes games and social hierarchies, incorporating luck and depth of competition.

Study reveals how depth of reasoning affects generalization in models.

A new depth measure for non-convex data supports, faster than halfspace depth.

The paper studies randomized approximations of Tukey's depth for log-concave isotropic data.

This paper connects functional data analysis with machine learning techniques.

This work generalizes bounds on the number of linear regions in CPWL NNs.

Self-attention models benefit equally from width and depth, but beyond a certain point, depth becomes less efficient.

A new depth measure based on optimal control theory captures multi-modal data.

Improved sample complexity for ReLU networks with norm constraints.

A new depth measure and median defined on Hadamard manifolds.

New approach uses loss functions to extend data depth for anomaly detection.

This article concerns the expressive power of depth in deep feed-forward neural nets with ReLU activations. Specifically, we answer the following question: for a fixed $d_{in}\geq 1,$ what is the minimal width $w$ so that neural nets with ReLU activations, input dimension $d_{in}$, hidden layer widths at most $w,$ and …