arXiv research

Confident learning estimates label quality by identifying and correcting noisy data.

We study the effect of imperfect training data labels on the performance of classification methods. In a general setting, where the probability that an observation in the training dataset is mislabelled may depend on both the feature vector and the true label, we bound the excess risk of an arbitrary classifier trained…

Study interprets neural network generalization and memorization on corrupted data.

Generative models improve image reconstruction and uncertainty quantification.

Noisy PN learning is the problem of binary classification when training examples may be mislabeled (flipped) uniformly with noise rate rho1 for positive examples and rho0 for negative examples. We propose Rank Pruning (RP) to solve noisy PN learning and the open problem of estimating the noise rates, i.e. the fraction …

Efficiently learns halfspaces with malicious noise, near-optimal label complexity.

In this paper, we consider batch supervised learning where an adversary is allowed to corrupt instances with arbitrarily large noise. The adversary is allowed to corrupt any $l$ features in each instance and the adversary can change their values in any way. This noise is introduced on test instances and the algorithm r…

In high dimensions, most machine learning methods are brittle to even a small fraction of structured outliers. To address this, we introduce a new meta-algorithm that can take in a base learner such as least squares or stochastic gradient descent, and harden the learner to be resistant to outliers. Our method, Sever, p…

Study on list learning with noisy data, showing limits and some learnable cases.

Singular Value Decomposition (and Principal Component Analysis) is one of the most widely used techniques for dimensionality reduction: successful and efficiently computable, it is nevertheless plagued by a well-known, well-documented sensitivity to outliers. Recent work has considered the setting where each point has …

DFMR improves robustness of learning finite mixture models in distributed settings.

Dropout training, originally designed for deep neural networks, has been successful on high-dimensional single-layer natural language tasks. This paper proposes a theoretical explanation for this phenomenon: we show that, under a generative Poisson topic model with long documents, dropout training improves the exponent…

Paper shows how to use geometric median for robust SGD in high dimensions.

We introduce a comprehensive and statistical framework in a model free setting for a complete treatment of localized data corruptions due to severe noise sources, e.g., an occluder in the case of a visual recording. Within this framework, we propose i) a novel algorithm to efficiently separate, i.e., detect and localiz…

A fast, robust AMP algorithm for quadratic optimization problems.

New robust estimators achieve subgaussian bounds using VC-dimension.

Manifold learning and dimensionality reduction techniques are ubiquitous in science and engineering, but can be computationally expensive procedures when applied to large data sets or when similarities are expensive to compute. To date, little work has been done to investigate the tradeoff between computational resourc…

The presence of data corruption in user-generated streaming data, such as social media, motivates a new fundamental problem that learns reliable regression coefficient when features are not accessible entirely at one time. Until now, several important challenges still cannot be handled concurrently: 1) corrupted data e…

Low-rank representation~(LRR) has been a significant method for segmenting data that are generated from a union of subspaces. It is, however, known that solving the LRR program is challenging in terms of time complexity and memory footprint, in that the size of the nuclear norm regularized matrix is $n$-by-$n$ (where $…

Robust VAE detects anomalies in corrupted data.

Dynamic robust PCA refers to the dynamic (time-varying) extension of robust PCA (RPCA). It assumes that the true (uncorrupted) data lies in a low-dimensional subspace that can change with time, albeit slowly. The goal is to track this changing subspace over time in the presence of sparse outliers. We develop and study …

Subspace clustering methods based on $\ell_1$, $\ell_2$ or nuclear norm regularization have become very popular due to their simplicity, theoretical guarantees and empirical success. However, the choice of the regularizer can greatly impact both theory and practice. For instance, $\ell_1$ regularization is guaranteed t…

Neural Empirical Bayes estimates source distributions from noisy simulations.

Simple AMP algorithm robust to adversarial corruption.

Paper proposes WGAIN for missing feature reconstruction.

Study learns linear utility functions from comparisons, showing learnability gaps between passive and active learning.

New algorithm reduces linear contextual bandit regret with adversarial corruption.

Convolutional networks can denoise images without training data.

Enhances deep learning robustness to noise without sacrificing clean data accuracy.

Advances in machine learning have led to broad deployment of systems with impressive performance on important problems. Nonetheless, these systems can be induced to make errors on data that are surprisingly similar to examples the learned system handles correctly. The existence of these errors raises a variety of quest…

We consider a model of robust learning in an adversarial environment. The learner gets uncorrupted training data with access to possible corruptions that may be affected by the adversary during testing. The learner's goal is to build a robust classifier, which will be tested on future adversarial examples. The adversar…

Proposes a new model for noisy labels considering multiple labelers and adversarial attacks.

Label smoothing improves model performance even with noisy labels.

Paper proposes a method to recover accurate labels from partially valid data in multi-label learning.

CbMLC improves multi-label classification with noisy labels.

Proposes ML-GCN for multi-label network node representation learning.

Proposes MGPLL for PL learning with non-random noise.

Logistic regression can handle noisy labels effectively when labels are imperfectly assigned by multiple experts.

FLAME auto-labels mobile data efficiently on diverse processors.

PML-LFC improves PML by estimating label confidence from both feature and label spaces.

An active learner is given a hypothesis class, a large set of unlabeled examples and the ability to interactively query labels to an oracle of a subset of these examples; the goal of the learner is to learn a hypothesis in the class that fits the data well by making as few label queries as possible. This work addresses…

Proposes methods to improve multi-label learning by addressing local label imbalance.

An important problem in multi-label classification is to capture label patterns or underlying structures that have an impact on such patterns. This paper addresses one such problem, namely how to exploit hierarchical structures over labels. We present a novel method to learn vector representations of a label space give…

New algorithm for XMC from aggregated labels.

Zero-shot learning transfers knowledge from seen classes to novel unseen classes to reduce human labor of labelling data for building new classifiers. Much effort on zero-shot learning however has focused on the standard multi-class setting, the more challenging multi-label zero-shot problem has received limited attent…

Enhances labels from unlabeled data using sample correlations.

Retraining with predicted labels improves model accuracy in noisy settings.

Multi-label classification aims to classify instances with discrete non-exclusive labels. Most approaches on multi-label classification focus on effective adaptation or transformation of existing binary and multi-class learning approaches but fail in modelling the joint probability of labels or do not preserve generali…