A new depth measure for non-convex data supports, faster than halfspace depth.
problem Non-convex data supports in multivariate statistics.
method Extending halfspace depth to Reproducing Kernel Hilbert Space (RKHS).
result The new depth measure is consistent and can be computed faster.
This paper connects functional data analysis with machine learning techniques.
problem Lack of theoretical analysis for functional depths.
method Viewing functional depths as kernel mean embeddings in machine learning.
result Facilitates answers to open questions about functional depths.
The paper studies randomized approximations of Tukey's depth for log-concave isotropic data.
problem The challenge of approximating Tukey's depth in high dimensions.
method The study examines randomized algorithms for approximating Tukey's depth for log-concave isotropic data.
result Randomized algorithms correctly approximate maximal depth and close to zero depths but not intermediate depths.
New approach uses loss functions to extend data depth for anomaly detection.
problem Anomaly detection in high-dimensional data.
method Introducing loss depths to generalize halfspace depth.
result New loss depths improve anomaly detection efficiency and interpretability.
Data depth aids in identifying anomalies in multivariate data.
problem Detecting abnormal observations in multivariate datasets.
method Using data depth to assign abnormality labels to observations with lower depth values.
result Data depth effectively identifies anomalies in multivariate settings.
A new depth measure based on optimal control theory captures multi-modal data.
problem Statistical depths for high-dimensional data.
method Eikonal equations and optimal control theory.
result The new depth measure is robust under adversarial models.
Following the seminal idea of Tukey, data depth is a function that measures how close an arbitrary point of the space is located to an implicitly defined center of a data cloud. Having undergone theoretical and computational developments, it is now employed in numerous applications with classification being the most po…
Learning based methods have shown very promising results for the task of depth estimation in single images. However, most existing approaches treat depth prediction as a supervised regression problem and as a result, require vast quantities of corresponding ground truth depth data for training. Just recording quality d…
Introduces Polar Depth for analyzing multivariate heavy-tailed data extremes.
problem Analyzing the behavior of extremes from multivariate heavy-tailed distributions.
method Introduces Polar Depth, a novel statistical depth function expressed in polar coordinates.
result The polar depth of the largest observations converges to the polar depth of the limiting distribution as the threshold increases.
Bayesian linear networks reveal optimal depth and width trade-offs.
problem Understanding how depth, width, and dataset size affect model quality in linear networks.
method Zero noise Bayesian inference with Gaussian weight priors and mean squared error.
result Optimal predictions at infinite depth and maximized Bayesian model evidence at infinite depth.
Extends partitioned local depth concept with probabilistic considerations.
problem Uncertain, variable, and conflicting information in data.
method Partitioned local depth with probabilistic concepts of local relevance and support division.
result Extends original ideas to handle uncertain data.
A new depth function improves multivariate data analysis by considering variability directions.
problem Developing a depth function that respects quantile properties and is affine-invariant.
method Integrating rank-weighted depth with affine-invariance and covariance matrices.
result The AI-IRW depth function provides accurate quantile estimates and is robust to data variability.
A new procedure, called DDa-procedure, is developed to solve the problem of classifying d-dimensional objects into q >= 2 classes. The procedure is completely nonparametric; it uses q-dimensional depth plots and a very efficient algorithm for discrimination analysis in the depth space [0,1]^q. Specifically, the depth i…
New depth function for partial orders helps compare machine learning algorithms.
problem Comparing machine learning algorithms using non-standard data types.
method Adapted simplicial depth to partial orders, using ufg depth for comparison.
result Demonstrates promising variety of analysis approaches based on ufg methods.
A new data-level recombination strategy improves RGB-D salient object detection.
problem RGB-D salient object detection struggles with depth information.
method Proposes a novel data-level recombination strategy to fuse RGB and depth data before feature extraction.
result Achieves a new state-of-the-art performance in RGB-D salient object detection.
Proposes a new method to estimate Bayesian neural network depth.
problem Estimating the depth of Bayesian neural networks.
method Uses a discrete truncated normal distribution to learn depth mean and variance, inferring posterior distributions by minimizing variational free energy.
result Improves test accuracy and reduces posterior depth variance on the spiral dataset.
New method for multi-modal depth prediction challenges.
problem Challenges in applying unimodal coreset selection to multi-modal data.
method Adapted state-of-the-art coreset selection technique for multimodal data.
result Challenges in extending unimodal algorithms to multi-modal scenarios.
UDN adapts depth to data complexity, outperforming standard neural networks.
problem Adapting neural network depth to data complexity.
method Variational inference for infinitely deep neural networks with a novel algorithm.
result UDN outperforms standard neural networks and other infinite-depth approaches.
Enhances SSL methods with depth cues for better image understanding.
problem Lack of depth cues in 2D image pixel maps limits SSL performance.
method Integrates depth signals from a pretrained monocular RGB-to-depth model into contrastive learning frameworks.
result Improves SSL methods' robustness and generalization with depth signals.
Transformers can learn noisy linear systems with depth and IID data.
problem Learning noisy linear dynamical systems with transformers.
method Theoretical analysis of multi-layer and single-layer transformers with respect to L2-testing loss. result Single-layer transformers have a non-diminishing lower bound on approximation error, suggesting depth separation.
A new pseudo-metric uses data depth to compare probability distributions.
problem Designing a metric between probability distributions for machine learning applications.
method Extension of univariate quantiles to multivariate spaces, using data depth and Hausdorff distance.
result The pseudo-metric is robust, factorizes translations, and has good behavior under transformations.
We prove the precise scaling, at finite depth and width, for the mean and variance of the neural tangent kernel (NTK) in a randomly initialized ReLU network. The standard deviation is exponential in the ratio of network depth to width. Thus, even in the limit of infinite overparameterization, the NTK is not determinist…
Study explores how dataset breadth and depth affect Siamese Neural Network performance.
problem Impact of dataset breadth and depth on Siamese Neural Network performance.
method Experiments with three keystroke datasets varying breadth and depth factors.
result Increasing dataset breadth improves model performance, while depth's impact varies by dataset type.
Scene recognition with RGB images has been extensively studied and has reached very remarkable recognition levels, thanks to convolutional neural networks (CNN) and large scene datasets. In contrast, current RGB-D scene data is much more limited, so often leverages RGB large datasets, by transferring pretrained RGB CNN…
Study how depth affects inference in deep Bayesian neural networks.
problem Understanding how depth impacts inference in overparameterized linear Bayesian neural networks.
method Interpreting finite deep linear Bayesian neural networks as scale mixtures of Gaussian process predictors.
result Advances analytical understanding of how depth affects inference in a simple class of Bayesian neural networks.
Paper uses statistical depth to create DP estimators for regression.
problem Creating differentially private estimators in high dimensions.
method Uses halfspace and regression depth to analyze maximum influence and construct DP estimators.
result New DP estimators for location and regression show favorable performance.
New algorithm quantifies uncertainty in regression models for complex data types.
problem Uncertainty quantification in regression models for complex data types.
method Model-free uncertainty quantification algorithm based on conditional depth measures and kernel mean embeddings.
result Provides faster convergence rates and non-asymptotic guarantees for prediction regions.
The paper explores how the depth of neural networks affects their ability to represent data accurately.
problem Understanding the implicit bias and rank of neural networks with large depth.
method Analyzing the convergence of representation cost to a notion of rank as network depth increases, and investigating conditions for recovering the true rank of data.
result There is a range of network depths where the true rank of data is recovered, and this affects the topology of class boundaries.
This paper introduces depth functions for ranking data, improving statistical summaries.
problem Lack of comprehensive statistical summaries for ranking data.
method Metric-based depth functions on symmetric group to define rankings, depths, and procedures.
result Novel depth functions provide a more informative summary of ranking data.
New insights into how depth and width affect in-context learning in deep models.
problem Understanding how various resources impact in-context learning in deep models.
method Analyzed linear regression in a deep linear self-attention model, varying resources like depth, width, context length, and training steps.
result Increasing depth improves in-context learning even at infinite context length, contrary to previous findings.
Deep RNNs excel at capturing long-term dependencies in sequential data.
problem Lack of a formal measure for RNNs' long-term memory capacity.
method Introduced a measure called Start-End separation rank to quantify RNNs' ability to model long-term dependencies.
result Deep RNNs support Start-End separation ranks that are combinatorially higher than shallow ones.
Wide neural networks converge to Gaussian processes, improving generalization.
problem Understanding the generalization of wide neural networks, especially deep equilibrium models.
method Investigation of deep equilibrium models (DEQs) with infinite-depth layers, focusing on their convergence to Gaussian processes as width and depth approach infinity.
result Wide DEQs converge to Gaussian processes, maintaining generalization performance.
This paper explores how neural network width and depth behave as they approach infinity.
problem Understanding the behavior of neural functions as width and depth go to infinity.
method Formal definition of commutativity framework, study of neural covariance kernel, novel proof techniques.
result Taking width and depth to infinity in a deep neural network with skip connections results in the same covariance structure, regardless of the order of taking limits.
Develops privacy-preserving multivariate median estimation methods.
problem Lack of rigorous privacy guarantees for robust multivariate location estimation.
method Novel finite-sample performance guarantees for differentially private multivariate depth-based medians.
result Sharp performance guarantees for multivariate depth-based medians under differential privacy.
Self-attention models benefit equally from width and depth, but beyond a certain point, depth becomes less efficient.
problem Understanding the optimal balance between depth and width in self-attention models.
method Theoretical predictions and empirical ablations on networks of varying depths and widths.
result An optimal width of 30K is recommended for a 1-Trillion parameter network, marking a significant width for self-attention models.
With the ubiquity of sensors in the IoT era, statistical observations are becoming increasingly available in the form of massive (multivariate) time-series. Formulated as unsupervised anomaly detection tasks, an abundance of applications like aviation safety management, the health monitoring of complex infrastructures …
A new depth measure and median defined on Hadamard manifolds.
problem Statistical depth and median on Hadamard manifolds.
method Horospherical depth and Busemann median defined using renormalized distance functions.
result The Busemann median exists for every Borel probability measure on Hadamard manifolds.
Proves depth 2 neural networks can't approximate certain functions as well as depth 3 networks.
problem Approximating functions with depth 2 networks in high dimensions.
method Lower bound proof using worst-to-average-case random self-reducibility.
result Proves depth 2 networks can't approximate certain functions as well as depth 3 networks, resolving an open problem.
In the last decade, supervised deep learning approaches have been extensively employed in visual odometry (VO) applications, which is not feasible in environments where labelled data is not abundant. On the other hand, unsupervised deep learning approaches for localization and mapping in unknown environments from unlab…
New findings on depth vs. width in neural networks, showing depth can improve learnability.
problem Understanding the role of depth in neural networks, especially when width is unbounded.
method Analyzing sample complexity for learnability in norm-controlled depth-2 and depth-3 ReLU networks.
result Depth can improve learnability of functions that are otherwise unlearnable with depth-2 networks.
Multimodal fusion frameworks for Human Action Recognition (HAR) using depth and inertial sensor data have been proposed over the years. In most of the existing works, fusion is performed at a single level (feature level or decision level), missing the opportunity to fuse rich mid-level features necessary for better cla…
DUNs improve active learning by dynamically adjusting model complexity.
problem Active learning datasets grow, leading to model bias or overfitting.
method Depth Uncertainty Networks (DUNs) infer model depth for flexibility and complexity.
result DUNs outperform other BNN variants on active learning tasks, reducing overfitting.
Study on feature learning dynamics in infinite-depth neural networks, focusing on ResNets.
problem Understanding how features evolve during training in deep neural networks, especially in the large-depth limit.
method Conditional Gaussian representations and SDE system with decoupled backward weights.
result Depth-induced suppression of forward-backward coupling in infinite-depth networks, leading to a decoupled forward-backward SDE system.
We address representational challenges in normalizing flows, particularly depth and conditioning issues.
problem Challenges in training normalizing flows, including vanishing/exploding gradients and poor conditioning.
method Analyzes representational aspects of depth and conditioning in normalizing flows, proving theoretical bounds and investigating phenomena.
result Proves that shallow affine coupling networks are universal approximators in Wasserstein distance if ill-conditioning is allowed.
The paper proves barriers to approximating functions with small weights and depth in neural networks.
problem Proving barriers to approximating functions with constant depth neural networks.
method Reduction to open problems and natural-proof barriers in circuit complexity, and a new approach to polynomially-bounded functions.
result There are fundamental barriers to proving results beyond depth 4 for constant-depth neural networks.
seMCD computes depth functions with statistical guarantees using sequential Monte Carlo.
problem Computing depth functions is computationally challenging, especially in high dimensions.
method Sequential Monte Carlo methodology with theoretical and empirical guarantees.
result The seMCD method provides accurate depth approximations with fewer samples than traditional methods.
Over-parameterized CNNs show U-shaped test risk with depth increase.
problem Understanding the impact of depth on test risk in over-parameterized CNNs.
method Empirical image classification experiments and linear regression framework.
result Test risk is U-shaped with increasing depth in over-parameterized CNNs.
AR-GANs learn depth and DoF from unlabeled images using aperture rendering and focus cues.
problem Learning depth and DoF from unlabeled natural images with diverse viewpoints and shapes.
method Aperture rendering and focus cues to learn depth and DoF from unlabeled images.
result AR-GANs effectively learn depth and DoF from various datasets, including flower, bird, and face images.