arXiv research

A method for estimating the median of gradients in stochastic optimization.

Recently, there is a growing interest in the study of median-based algorithms for distributed non-convex optimization. Two prominent such algorithms include signSGD with majority vote, an effective approach for communication reduction via 1-bit compression on the local gradients, and medianSGD, an algorithm recently pr…

Paper shows how to use geometric median for robust SGD in high dimensions.

Empirical median performs well in estimating location with varying scales.

This paper introduces online algorithms to estimate robust geometric median in large data streams.

Improved median of means estimator with tighter bounds.

New estimator for symmetric kernel expectations, robust to missing data.

Paper proposes a novel method to improve matrix completion with median loss for large datasets.

This paper is a short summary of our recent work on the medians and means of probability measures in Riemannian manifolds. Firstly, the existence and uniqueness results of local medians are given. In order to compute medians in practical cases, we propose a subgradient algorithm and prove its convergence. After that, F…

Three methods for tuning HMC diagonal scale matrices compared.

We introduce a new sub-linear space sketch---the Weight-Median Sketch---for learning compressed linear classifiers over data streams while supporting the efficient recovery of large-magnitude weights in the model. This enables memory-limited execution of several statistical analyses over streams, including online featu…

New methods validate a hypothesis explaining how neural nets generalize well.

The consistency of Fréchet medians is proved for probability measures in proper metric spaces. In the context of Riemannian manifolds, assuming that the probability measure has more than a half mass lying in a convex ball and verifies some concentration conditions, the positions of its Fréchet medians are estimated. It…

Paper proposes a robust method for federated ICA with geometric median aggregation.

MFRDE uses medians of forest estimators to robustly estimate densities in noisy data.

Paper introduces MoM-KDE for robust density estimation robust to anomalous data.

Develops privacy-preserving multivariate median estimation methods.

We consider the non-parametric regression problem under Huber's $ε$-contamination model, in which an $ε$ fraction of observations are subject to arbitrary adversarial noise. We first show that a simple local binning median step can effectively remove the adversary noise and this median estimator is minimax optimal up t…

Improved private geometric median estimation with nearly-linear time complexity.

Paper improves statistical efficiency of median-of-means estimator for Byzantine robust distributed inference.

Distributed model training is vulnerable to byzantine system failures and adversarial compute nodes, i.e., nodes that use malicious updates to corrupt the global model stored at a parameter server (PS). To guarantee some form of robustness, recent work suggests using variants of the geometric median as an aggregation r…

A main goal of regression is to derive statistical conclusions on the conditional distribution of the output variable Y given the input values x. Two of the most important characteristics of a single distribution are location and scale. Support vector machines (SVMs) are well established to estimate location functions …

Evolutionary algorithms (EAs) are a sort of nature-inspired metaheuristics, which have wide applications in various practical optimization problems. In these problems, objective evaluations are usually inaccurate, because noise is almost inevitable in real world, and it is a crucial issue to weaken the negative effect …

We analyze the performance of the Tukey median estimator under total variation (TV) distance corruptions. Previous results show that under Huber's additive corruption model, the breakdown point is 1/3 for high-dimensional halfspace-symmetric distributions. We show that under TV corruptions, the breakdown point reduces …

We derive concentration inequalities for differentially private median and mean estimators building on the "Propose, Test, Release" (PTR) mechanism introduced by Dwork and Lei (2009). We introduce a new general version of the PTR mechanism that allows us to derive high probability error bounds for differentially privat…

New method improves mean estimation for heavy-tailed data.

Improved CountSketch method reduces variance for estimating vector coordinates.

New graph properties inherited by Frechet mean and median.

Clustering with fast algorithms large samples of high dimensional data is an important challenge in computational statistics. Borrowing ideas from MacQueen (1967) who introduced a sequential version of the $k$-means algorithm, a new class of recursive stochastic gradient algorithms designed for the $k$-medians loss cri…

In this paper, we define the geometric median of a probability measure on a Riemannian manifold, give its characterization and a natural condition to ensure its uniqueness. In order to calculate the median in practical cases, we also propose a subgradient algorithm and prove its convergence as well as estimating the er…

This paper investigates the phase retrieval problem, which aims to recover a signal from the magnitudes of its linear measurements. We develop statistically and computationally efficient algorithms for the situation when the measurements are corrupted by sparse outliers that can take arbitrary values. We propose a nove…

Stochastic gradient descent's long-term fluctuations are described by a diffusion limit.

Continuous Sweep improves binary quantifier performance.

We investigate existence and uniqueness of p-means and the median of a probability measure on a Finsler manifold, in relation with the convexity of the support of the measure. We prove that the p-mean is the limit point of a continuous time gradient flow. Under some additional condition which is always satisfied for la…

This work achieves exponential concentration in heavy-tailed data over CAT(κ) spaces using the Fréchet median.

New method explains survival analysis models using median-SHAP.

We propose a new clustering algorithm that is robust to the presence of outliers in the dataset. We perform Lloyd-type iterations with robust estimates of the centroids. More precisely, we build on the idea of median-of-means statistics to estimate the centroids, but allow for replacement while constructing the blocks.…

A new method for fast and robust sparsity learning over networks.

Paper explores using bootstrap methods to improve SGD's stability and robustness.

Improved manifold-adaptive dimension estimator for better data complexity assessment.

Accelerated optimization methods improve robustness and privacy in estimation.

This paper proposes methods to compute differentially private confidence intervals for the median.

For massive data sets, efficient computation commonly relies on distributed algorithms that store and process subsets of the data on different machines, minimizing communication costs. Our focus is on regression and classification problems involving many features. A variety of distributed algorithms have been proposed …

This work shows that Gaussian is the only prior for optimal linear estimation in $L^1$ loss.

Robustifies Markowitz portfolios to reduce transaction costs and improve performance.

Unified meta algorithms estimate various distribution functionals in infinite-armed bandits.

We consider the problem of sparsity-constrained $M$-estimation when both explanatory and response variables have heavy tails (bounded 4-th moments), or a fraction of arbitrary corruptions. We focus on the $k$-sparse, high-dimensional regime where the number of variables $d$ and the sample size $n$ are related through $…

We construct compactifications for median spaces with compact intervals, generalising Roller boundaries of ${\rm CAT}(0)$ cube complexes. Examples of median spaces with compact intervals include all finite rank median spaces and all proper median spaces of infinite rank. Our methods also work for general median algebra…