arXiv research

Simpler majority vote of three classifiers achieves optimal error bounds.

Study on Privileged ERM showing limitations and providing capacity analysis.

Empirical risk minimization (ERM), with proper loss function and regularization, is the common practice of supervised classification. In this paper, we study training arbitrary (from linear to deep) binary classifier from only unlabeled (U) data by ERM. We prove that it is impossible to estimate the risk of an arbitrar…

Multiple fairness constraints have been proposed in the literature, motivated by a range of concerns about how demographic groups might be treated unfairly by machine learning classifiers. In this work we consider a different motivation; learning from biased training data. We posit several ways in which training data m…

In many classification systems, sensing modalities have different acquisition costs. It is often {\it unnecessary} to use every modality to classify a majority of examples. We study a multi-stage system in a prediction time cost reduction setting, where the full data is available for training, but for a test example, m…

We consider the problem of adaptation to the margin and to complexity in binary classification. We suggest an exponential weighting aggregation scheme. We use this aggregation procedure to construct classifiers which adapt automatically to margin and complexity. Two main examples are worked out in which adaptivity is a…

Learning with noisy labels is a common challenge in supervised learning. Existing approaches often require practitioners to specify noise rates, i.e., a set of parameters controlling the severity of label noises in the problem, and the specifications are either assumed to be given or estimated using additional steps. I…

NAPP-ERM improves ERM with differential privacy guarantees by iteratively achieving target regularization and delivering strong convexity.

Entropy asymmetry affects regularization in ERM, leading to biased solutions.

Simple DP algorithms find approximate solutions for nonconvex ERM.

New dual formulation reduces generalization error for ERM-fDR.

Paper shows ERM's suboptimality due to bias, not variance.

New method for collaborative learning with multiple distributions, minimizing sample size.

New research shows SVM and related methods can overfit without harm in multiclass classification.

While many solutions for privacy-preserving convex empirical risk minimization (ERM) have been developed, privacy-preserving nonconvex ERM remains a challenge. We study nonconvex ERM, which takes the form of minimizing a finite-sum of nonconvex loss functions over a training set. We propose a new differentially private…

Empirical risk minimization (ERM) is ubiquitous in machine learning and underlies most supervised learning methods. While there has been a large body of work on algorithms for various ERM problems, the exact computational complexity of ERM is still not understood. We address this issue for multiple popular ERM problems…

Paper proposes a new method for SP with covariates using PADR and ERM.

Noise-ignorant empirical risk minimization achieves state-of-the-art performance on noisy data.

Study on tradeoffs between mistakes and ERM oracle calls in online and transductive learning.

This paper studies universal rates of ERM for binary classification under agnostic learning.

Default-ERM shortcut learning persists even without additional information.

Dual optimization connects ERM-fDR to normalization function.

Paper presents ERM with $f$-divergence regularization and its properties.

Develops a new learning framework for dynamic data.

ERM struggles with synthetic data, but some algorithms can still learn correctly.

Currently, machine learning plays an important role in the lives and individual activities of numerous people. Accordingly, it has become necessary to design machine learning algorithms to ensure that discrimination, biased views, or unfair treatment do not result from decision making or predictions made via machine le…

Paper explores universal rates of ERM in machine learning.

PAC learning, dating back to Valiant'84 and Vapnik and Chervonenkis'64,'74, is a classic model for studying supervised learning. In the agnostic setting, we have access to a hypothesis set $\mathcal{H}$ and a training set of labeled samples $(x_1,y_1),\dots,(x_n,y_n) \in \mathcal{X} \times \{-1,1\}$ drawn i.i.d. from a…

We show that the empirical risk minimization (ERM) problem for neural networks has no solution in general. Given a training set $s_1, \dots, s_n \in \mathbb{R}^p$ with corresponding responses $t_1,\dots,t_n \in \mathbb{R}^q$, fitting a $k$-layer neural network $ν_θ: \mathbb{R}^p \to \mathbb{R}^q$ involves estimation of…

Algorithm solves online binary classification and infinite games using ERM oracle.

Upper bounds and lower bounds show ERM outperforms DG methods in various settings.

We develop a new active learning algorithm for the streaming setting satisfying three important properties: 1) It provably works for any classifier representation and classification problem including those with severe noise. 2) It is efficiently implementable with an ERM oracle. 3) It is more aggressive than all previo…

MRCpy implements minimax risk classifiers with performance guarantees and distribution shift adaptability.

Study risk bounds for distributed ERM with general loss functions and hypothesis spaces.

ERM with square loss achieves sublinear error for learnable function classes with smoothed data.

Sharp bounds on ERM's minimal error in regression.

Deep models learn spurious features correlated with target, but can still perform well.

FeAT improves OOD generalization by learning richer features.

Online TERM improves robustness and fairness in streaming data.

New algorithm reduces ERM problem size while maintaining accuracy.

Distributed machine learning is an approach allowing different parties to learn a model over all data sets without disclosing their own data. In this paper, we propose a weighted distributed differential privacy (WD-DP) empirical risk minimization (ERM) method to train a model in distributed setting, considering differ…

Study identifies and analyzes spurious correlations in data-driven models.

A new DP algorithm for weighted ERM protects sensitive data in predictive models.

Recent work shows GRW approaches do not improve over ERM in distributional shift.

This guide simplifies high-probability regret bounds in empirical risk minimization.

ERM uses energy-based selection to improve recursive reasoning.

It is challenging to handle a large volume of labels in multi-label learning. However, existing approaches explicitly or implicitly assume that all the labels in the learning process are given, which could be easily violated in changing environments. In this paper, we define and study streaming label learning (SLL), i.…

Reweighting improves risk bounds in certain data regions.