This paper studies robust regression in the settings of Huber's -contamination models. We consider estimators that are maximizers of multivariate regression depth functions. These estimators are shown to achieve minimax rates in the settings of -contamination models for various regression problems including nonpa…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “robust depth”
Data depth aids in identifying anomalies in multivariate data.
problem Detecting abnormal observations in multivariate datasets.
method Using data depth to assign abnormality labels to observations with lower depth values.
result Data depth effectively identifies anomalies in multivariate settings.
A new depth measure based on optimal control theory captures multi-modal data.
problem Statistical depths for high-dimensional data.
method Eikonal equations and optimal control theory.
result The new depth measure is robust under adversarial models.
CRAD clusters data with robust depth-based dissimilarity, outperforming existing methods.
problem Clustering data with varying densities and unknown true number of clusters.
method CRAD uses a robust data depth as dissimilarity measure and a novel neighbor searching function.
result CRAD outperforms DBSCAN, OPTICS, and DBCA in detecting clusters with varying densities.
Develops privacy-preserving multivariate median estimation methods.
problem Lack of rigorous privacy guarantees for robust multivariate location estimation.
method Novel finite-sample performance guarantees for differentially private multivariate depth-based medians.
result Sharp performance guarantees for multivariate depth-based medians under differential privacy.
Enhances SSL methods with depth cues for better image understanding.
problem Lack of depth cues in 2D image pixel maps limits SSL performance.
method Integrates depth signals from a pretrained monocular RGB-to-depth model into contrastive learning frameworks.
result Improves SSL methods' robustness and generalization with depth signals.
Robust estimation under Huber's -contamination model has become an important topic in statistics and theoretical computer science. Statistically optimal procedures such as Tukey's median and other estimators based on depth functions are impractical because of their computational intractability. In this paper, we est…
A new depth measure for non-convex data supports, faster than halfspace depth.
problem Non-convex data supports in multivariate statistics.
method Extending halfspace depth to Reproducing Kernel Hilbert Space (RKHS).
result The new depth measure is consistent and can be computed faster.
Inverse depth scaling found in LLMs due to similar layers averaging error.
problem Understanding how depth affects loss in large language models.
method Analysis of LLMs and toy residual networks.
result Loss scales inversely proportional to depth in LLMs.
AutoGrow automatically discovers optimal depth in DNNs.
problem Designing optimal depth in deep neural networks is difficult and time-consuming.
method AutoGrow grows new layers in a seed architecture if it improves accuracy; stops if no improvement. Robust policies generalize to different architectures and datasets.
result AutoGrow discovers near-optimal depth on various datasets, improving accuracy-computation trade-off in ResNets.
Decision trees perform well in complex interactions, even when interactions are not fully accounted for.
problem Interpreting complex interactions in machine learning models.
method Experiments on datasets and two methods for robust GLMs.
result Tree depth compensates for model misspecification, enhancing performance in complex scenarios.
Learning based methods have shown very promising results for the task of depth estimation in single images. However, most existing approaches treat depth prediction as a supervised regression problem and as a result, require vast quantities of corresponding ground truth depth data for training. Just recording quality d…
Looped Transformers improve robustness and expressivity in in-context learning for diverse tasks.
problem Improving robustness and expressivity in in-context learning for diverse tasks.
method Study in-context linear regression with diverse tasks, focusing on depth and looping.
result Looped Transformers exhibit similar expressive power and are provably robust under mild assumptions.
This paper investigates how network width and depth affect adversarially robust DNNs.
problem Understanding architectural configurations for adversarially robust DNNs.
method Comprehensive investigation on the impact of network width and depth on adversarial robustness.
result Optimal architectural configuration for adversarial robustness exists and can improve robustness.
New method uses GANs and proper scoring rules for robust scatter estimation.
problem Robust scatter estimation in statistics.
method General learning via classification framework based on proper scoring rules.
result Proposed robust scatter estimators achieve minimax rate under Huber's contamination model.
Robust penalized least squares of depth trimmed residuals regression for high-dimensional datastat.ML
A new robust regression method handles outliers in high-dimensional data.
problem Outliers in high-dimensional data make conventional regression methods ineffective.
method Robust penalized least squares of depth trimmed residuals regression.
result The new method outperforms existing methods in estimation and prediction accuracy.
A new depth measure and median defined on Hadamard manifolds.
problem Statistical depth and median on Hadamard manifolds.
method Horospherical depth and Busemann median defined using renormalized distance functions.
result The Busemann median exists for every Borel probability measure on Hadamard manifolds.
Study on Tukey depth in machine learning using Hamilton-Jacobi equations.
problem Understanding Tukey depth in machine learning applications.
method Derive necessary conditions for Tukey depth in continuum limit, formulating them as a Hamilton-Jacobi equation.
result Prove existence and uniqueness of viscosity solutions for the derived equation, which bounds Tukey depth.
Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margincs.LG
New margin measure improves deep learning generalization and robustness.
problem Unclear relationship between output margin and generalization for deep models.
method Introduced 'all-layer margin' for deep neural networks.
result Tighter generalization bounds for neural nets with no exponential depth dependency.
A new depth measure captures anomalies in time series data.
problem Anomaly detection in time series data.
method Convex hull area of sampled curves for functional depth.
result Captures gradual departures from centrality.
Single-pass method estimates neural network uncertainty.
problem Uncertainty estimation in deep learning requires multiple passes.
method Probabilistic reasoning over neural network depths.
result Single forward pass for uncertainty estimation.
Unified spectral framework for μP under joint width-depth scaling.
problem Challenges in stable feature learning and HP transfer for width-depth scaled models.
method Developed a simple and unified spectral framework for μP under joint width-depth scaling.
result Unified and generalized μP formulation for practical architectures with multi-transformation branches.
New method improves reliability of depth estimation models.
problem Uncertainty quantification in large-scale vision models.
method Parameter-efficient Bayesian neural networks with PEFT methods.
result Combining PEFT methods with Bayesian inference enhances predictive performance.
Unified learning-rate scale for CNNs and ResNets, avoiding depth imbalance.
problem Challenges in choosing an appropriate learning rate for deep networks, especially as depth increases.
method Introduces Arithmetic-Mean P (AM-P), constraining network-wide average pre-activation second moment to a constant scale, combined with residual-aware He fan-in initialization.
result Demonstrates a scaling law for learning rates across depths, enabling zero-shot learning-rate transfer.
DGPs enhanced with GVI improve robustness and uncertainty quantification.
problem Improving robustness and uncertainty quantification in deep Gaussian processes.
method Integrating Generalized Variational Inference (GVI) into Deep Gaussian Processes (DGPs).
result DGPs show significant benefits from GVI modifications in empirical tests.
The paper introduces a method to explain redundancy in deep CNNs using unit impulse response.
problem Redundancy in deep CNNs leads to unnecessary computations and increased cost.
method Empirical demonstration and unit impulse response analysis to identify and quantify redundancy across layers and depth.
result Identifies and quantifies redundancy in deep CNNs, providing better insights into their internal dynamics.
A new pseudo-metric uses data depth to compare probability distributions.
problem Designing a metric between probability distributions for machine learning applications.
method Extension of univariate quantiles to multivariate spaces, using data depth and Hausdorff distance.
result The pseudo-metric is robust, factorizes translations, and has good behavior under transformations.
Bayesian method learns neural network architecture parameters.
problem Estimating optimal neural network architecture parameters.
method Bayesian learning of concrete distributions over layer size and network depth.
result Regular networks with learnt structure generalize better on small datasets, while stochastic networks are more robust to initialisation.
Paper introduces MCSD, a method for uncertainty estimation in deep learning.
problem Need for reliable uncertainty quantification in deep neural networks.
method Theoretical connection to variational inference and empirical benchmarking of MCSD.
result MCSD achieves competitive predictive accuracy and improves uncertainty ranking.
Randomly initialized ReLU networks of depth two can approximate smooth functions well.
problem Approximation power of two-layer networks of random ReLUs.
method Harmonic analysis and ridgelet representation theory for upper bounds, dimensionality arguments for lower bounds.
result Near-matching upper and lower bounds for -approximation and Sobolev norms.
New framework improves robustness of implicit neural networks.
problem Ill-posedness and convergence instability in implicit neural networks.
method NEMON framework based on contraction theory for norm, including well-posedness condition, average iteration, and input-output Lipschitz constant regularization.
result Improved accuracy and robustness of implicit models with smaller input-output Lipschitz bounds.
Deeper models have a more favorable optimization landscape, making them more robust to noise.
problem Characterizing the effect of depth on the optimization landscape of linear regression models.
method Robust and over-parameterized setting, simple sub-gradient method.
result A simple sub-gradient method converges to a balanced solution that is close to the ground truth and enjoys a flat local landscape.
Develops a robust model for skewed and heavy-tailed data in periodontal studies.
problem Skewed and heavy-tailed data in periodontal pocket depth measurements.
method Flexible two-piece scale Student-t error distribution and deep neural network with monotonicity constraints.
result Robust mode-based estimation resistant to outliers with clinical interpretability.
The Effect of Prior Lipschitz Continuity on the Adversarial Robustness of Bayesian Neural Networksstat.ML
Study shows prior Lipschitz continuity can improve adversarial robustness of Bayesian Neural Networks.
problem Improving adversarial robustness of Bayesian Neural Networks.
method Analysis of i.i.d., zero-mean Gaussian priors and posteriors approximated via mean-field variational inference.
result Adversarial robustness is sensitive to the prior variance.
This research proposes a new distance metric using Isolation Forests.
problem Approximating spatial distance between data points.
method Isolation Forests for outlier detection, transforming separation depth into a distance metric.
result The method produces a distance metric invariant to variable scales and capable of handling non-linear relationships.
We present a generalization of the Cauchy/Lorentzian, Geman-McClure, Welsch/Leclerc, generalized Charbonnier, Charbonnier/pseudo-Huber/L1-L2, and L2 loss functions. By introducing robustness as a continuous parameter, our loss function allows algorithms built around robust loss minimization to be generalized, which imp…
Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?cs.LG
DBU models struggle with robust uncertainty estimates under adversarial attacks.
problem Robustness of DBU models in adversarial settings.
method Investigated robustness of DBU models under adversarial attacks; proposed median smoothing approach.
result DBU models are not robust in indicating correctly and wrongly classified samples, detecting adversarial examples, and distinguishing ID and OOD data.
The paper develops a method to create robust control policies for robots using information bottlenecks.
problem Robotic control policies are sensitive to task-irrelevant state and sensor changes.
method Derives a policy gradient algorithm that creates an information bottleneck between states and task-relevant representations.
result Task-driven policies are more robust to sensor noise and environmental changes.
This study improves scalability of randomized smoothing for certifying classifier robustness.
problem Certifying machine learning classifiers against adversarial attacks is challenging and scalable solutions are needed.
method The study reviews and explores randomized smoothing and its derivatives, focusing on scalability.
result The study provides theoretical guarantees and discusses scalability challenges of randomized smoothing.
Bayesian Deep Learning Hyperparameter Search for Robust Function Mapping to Polynomials with Noisecs.LG
This paper explores BDL hyperparameters for robust polynomial mapping with noise.
problem Designing BDL hyperparameters for robust function mapping with uncertainty quantification.
method Mapping Bayesian connectionist representations to polynomials of varying orders and noise types.
result Optimal network depth and ensemble size for prediction and uncertainty quantification.
STARK improves denoising of low-depth spatial transcriptomics images.
problem Denoising spatial transcriptomics images at ultra-low sequencing depths.
method Adaptive regularization with kernel ridge regression and graph Laplacian.
result STARK optimizes denoising performance over competing methods.
New algorithm improves Bayesian inference for complex models.
problem Poor performance of existing Bayesian approaches for simulators.
method Posterior bootstrap and maximum mean discrepancy estimators.
result Strong robustness and parallelizability of the new algorithm.
The paper studies randomized approximations of Tukey's depth for log-concave isotropic data.
problem The challenge of approximating Tukey's depth in high dimensions.
method The study examines randomized algorithms for approximating Tukey's depth for log-concave isotropic data.
result Randomized algorithms correctly approximate maximal depth and close to zero depths but not intermediate depths.
This paper connects functional data analysis with machine learning techniques.
problem Lack of theoretical analysis for functional depths.
method Viewing functional depths as kernel mean embeddings in machine learning.
result Facilitates answers to open questions about functional depths.
Self-attention models benefit equally from width and depth, but beyond a certain point, depth becomes less efficient.
problem Understanding the optimal balance between depth and width in self-attention models.
method Theoretical predictions and empirical ablations on networks of varying depths and widths.
result An optimal width of 30K is recommended for a 1-Trillion parameter network, marking a significant width for self-attention models.
GoTube verifies neural networks over time, scaling to large horizons.
problem Verifying the robustness of time-continuous neural networks.
method Solves Go problems to construct a conservative execution set.
result Substantially outperforms existing tools in size, speed, and scalability.
Stable Neural Flowscs.LG
Stable neural flows ensure robustness and efficiency in deep learning.
problem Ensuring robustness and stability in deep learning models.
method Introducing a stable variant of neural ODEs with a neural network parametrizing an energy functional, solving as an optimal control problem with adjoint sensitivity analysis.
result The proposed model provides robustness against input perturbations and low computational burden.
Data Depth as a Riskstat.ML
New approach uses loss functions to extend data depth for anomaly detection.
problem Anomaly detection in high-dimensional data.
method Introducing loss depths to generalize halfspace depth.
result New loss depths improve anomaly detection efficiency and interpretability.