The paper analyzes SMOTE for imbalanced classification, providing theoretical bounds and guidelines.
problem The challenge of imbalanced classification problems, especially with minority classes.
method Theoretical analysis of SMOTE and related oversampling techniques for minority classes.
result Derives concentration and excess risk bounds for SMOTE and kernel-based classifiers.
This research improves PAC-Bayesian bounds for classification tasks using convexified loss.
problem Deriving generalization bounds for classification tasks with non-convex loss functions.
method Shift focus to misclassification excess risk bounds for PAC-Bayesian classification using convex surrogate loss and leveraging PAC-Bayesian relative bounds in expectation.
result Improved PAC-Bayesian bounds for classification tasks with convex surrogate loss.
We consider a high dimensional binary classification problem and construct a classification procedure by minimizing the empirical misclassification risk with a penalty on the number of selected features. We derive non-asymptotic probability bounds on the estimated sparsity as well as on the excess misclassification ris…
Study excess risk in statistical inference with transformations.
problem Excess risk in estimating random variables from feature vectors and transformations.
method Characterize lossless transformations, develop test statistics, and information-theoretic bounds.
result Strongly consistent partitioning test statistic for lossless transformations.
The paper explores the information-theoretic nature of excess risk in machine learning.
problem Understanding the excess risk in machine learning models.
method Formulates the minimax excess risk as a zero-sum game and modifies it to allow swapping of the order of play.
result Proves that under certain conditions, the duality gap is zero, allowing for the application of Bayesian results to provide bounds on minimax excess risk.
The paper provides theoretical guarantees for neural network-based anomaly detection.
problem Theoretical guarantees for unsupervised neural network-based anomaly detection.
method Casting anomaly detection as a binary classification problem, establishing non-asymptotic upper bounds and convergence rates.
result The convergence rate on the excess risk matches the minimax optimal rate.
Paper provides optimal statistical guarantees for adversarial robustness in Gaussian classification.
problem Understanding statistical risks for adversarial robustness in Gaussian classification models.
method Established minimax lower bounds and designed an efficient estimator for excess risk.
result Optimal minimax guarantees for excess risk under Gaussian mixture model with AdvSNR.
Study non-asymptotic bounds for robust estimators under misspecified models.
problem Evaluate performance of robust estimators under adversarial conditions.
method Propose a general approach to adversarial risk analysis, including investigations on generalization and approximation errors.
result Establish non-asymptotic upper bounds for adversarial excess risk under Lipschitz loss functions.
We consider a standard binary classification problem. The performance of any binary classifier based on the training data is characterized by the excess risk. We study Bahadur's type exponential bounds on the minimax accuracy confidence function based on the excess risk. We study how this quantity depends on the comple…
We tackle imbalanced classification by weighting losses and derive robust risks.
problem Imbalanced classification where a label has low marginal probability.
method We examine convergence rates of weighted risks, define robust risks, and derive new robust risk problems.
result We show that particular weightings lead to conditional value at risk (CVaR) and derive new robust risk problems.
This paper examines error bounds for deep learning classifiers with noisy labels.
problem Understanding the performance of classifiers trained on noisy data.
method Derives error bounds for excess risk, decomposing it into statistical and approximation errors. Uses independent block construction for statistical dependencies and vector-valued setting for approximation error.
result Established theoretical results for error bounds in deep learning with noisy labels, mitigating the impact of high-dimensional input spaces.
This paper analyzes neural network classifiers' performance in binary classification.
problem Performance of neural network classifiers in binary classification problems.
method Plug-in classifiers based on neural networks, considering a more general function class and surrogate loss.
result Dimension-free, uniform rate of convergence for the excess risk of neural networks, showing minimax optimality.
New research shows that binary classification can be done with noisy data, but only if there are clean samples available.
problem Learning binary classification with instance and label dependent label noise.
method Theoretical analysis and empirical risk minimization.
result Empirical risk minimization achieves the optimal excess risk bound without additional assumptions.
The paper bounds the excess risk of deep neural networks for weakly dependent processes.
problem Learning with weakly dependent data using deep neural networks.
method Approximation of smooth functions by deep neural networks and a bound on excess risk.
result The excess risk bound for deep learning under weak dependence is close to O(n−1/2) for sufficiently smooth functions. Sparse-penalized deep neural networks improve performance in weakly dependent processes.
problem Nonparametric regression and classification under weak dependence.
method Sparse-penalized deep neural networks with oracle inequalities and convergence rates established.
result The proposed estimators outperform non-penalized ones in simulations.
Paper analyzes risk bounds for in-context learning in multiclass classification.
problem Risk bounds for in-context learning in multiclass classification.
method Formalizes tasks as sequences of labeled examples and queries, estimates conditional class probabilities, establishes oracle inequality for KL divergence.
result ICL achieves minimax optimal rate for conditional probability estimation.
Deep neural networks classify unbounded Gaussian mixture data without dimensionality issues.
problem Binary classification of unbounded Gaussian mixture data.
method Deep ReLU neural networks with non-asymptotic upper bounds and convergence rates.
result Deep ReLU networks can classify unbounded Gaussian mixture data without dimensionality constraints.
We consider the problem of binary classification where one can, for a particular cost, choose not to classify an observation. We present a simple proof for the oracle inequality for the excess risk of structural risk minimizers using a lasso type penalty.
Fine-tuning with pre-training data improves performance.
problem Limited training data for tasks.
method Theoretical analysis of excess risk bound and selection of pre-training data subset.
result Improvement in generalization performance with pre-training data.
The overarching goal of this paper is to derive excess risk bounds for learning from exp-concave loss functions in passive and sequential learning settings. Exp-concave loss functions encompass several fundamental problems in machine learning such as squared loss in linear regression, logistic loss in classification, a…
The paper sets information-theoretic lower bounds for neural networks' parameter recovery and excess risk.
problem Establishing sample complexity lower bounds for neural network parameters and excess risk.
method Using information-theoretic tools, the paper proves lower bounds by constructing a generative network.
result Proves information-theoretic lower bounds for exact parameter recovery and positive excess risk.
The study compares clustering risk in Hidden Markov and i.i.d. models, showing the Bayes classifier is nearly optimal.
problem Comparing clustering risk in Hidden Markov and i.i.d. models.
method Analysis of Bayes risk, theoretical bounds, and simulations.
result The Bayes classifier is nearly optimal for clustering in both Hidden Markov and i.i.d. models.
Paper shows similarity learning can lead to strong binary classification performance.
problem How similarity learning can lead to good classification performance.
method Product-type formulation of similarity learning is connected to binary classification through an excess risk bound.
result Similarity learning can directly elicit a decision boundary for binary classification.
Unified framework for fair classification with group-blindness/awareness guarantees.
problem Challenges in enforcing fairness and group-blindness in binary classification.
method Unified framework based on post-processing procedure, applicable to various group fairness notions.
result Minimax rate-optimality of the proposed algorithm with controlled excess risk.
This work analyzes label embedding for large multiclass classification problems.
problem Label embedding for large multiclass classification problems.
method Analysis of label embedding in extreme multiclass classification, presenting an excess risk bound and showing a trade-off between computational and statistical efficiency.
result The statistical penalty for label embedding vanishes with sufficiently low coherence under the Massart noise condition.
The paper analyzes kernel classifiers' performance in Sobolev spaces and proves their optimality.
problem Theoretical analysis of kernel classifiers' performance in Sobolev spaces.
method Deriving upper and lower bounds on classification excess risk using kernel regression theory and estimating interpolation smoothness.
result The proposed kernel classifier is optimal in Sobolev spaces, with theoretical bounds confirmed by real data.
We study the effect of imperfect training data labels on the performance of classification methods. In a general setting, where the probability that an observation in the training dataset is mislabelled may depend on both the feature vector and the true label, we bound the excess risk of an arbitrary classifier trained…
Deep neural network with l_1-regularization achieves nearly optimal risk bounds.
problem Achieving optimal risk bounds in deep learning.
method Empirical risk minimization with l_1-regularization.
result Adaptively nearly-minimax risk bound across various function classes.
Paper introduces a new method for classifying interval-valued time series.
problem Classification of interval-valued time series.
method Extends point-valued time series imaging methods to interval-valued scenarios using DK-distance and employs deep learning for classification. result Proposed method achieves superior classification performance compared to existing methods.
In statistical learning theory, convex surrogates of the 0-1 loss are highly preferred because of the computational and theoretical virtues that convexity brings in. This is of more importance if we consider smooth surrogates as witnessed by the fact that the smoothness is further beneficial both computationally- by at…
New tool detects 'fleeting modes' causing excess risk in financial markets.
problem Detecting portfolios with statistically significant excess risk in financial markets.
method Random Matrix Theory to identify 'fleeting modes' independent of underlying correlation structure.
result Fleeting modes exist in both futures and equity markets, and momentum is a source of excess risk.
Paper analyzes sparse aggregation in GLMs with Kullback-Leibler risk bounds.
problem Sparse aggregation in GLMs for parameter approximation.
method Exponential weighted aggregation scheme with Kullback-Leibler risk bounds.
result Sharp oracle inequality for Kullback-Leibler risk with leading constant 1 and minimax-optimal rate of aggregation.
The paper analyzes the excess risk of PCA and provides a precise characterization.
problem Understanding the excess risk of principal component analysis (PCA).
method Established a central limit theorem for PCA error and derived the excess risk distribution.
result Obtained a non-asymptotic upper bound on the excess risk of PCA.
This paper studies fairness and privacy in federated learning, proposing algorithms to balance both.
problem Joint impact of differential privacy and fairness in federated classification.
method Proposes FDP-Fair and CDP-Fair algorithms for demographic disparity constrained classification under federated differential privacy.
result Established theoretical guarantees on privacy, fairness, and excess risk control.
Paper studies fair classification of functional data.
problem Mitigating disparities in functional data classification.
method Unified framework for fairness-aware functional classification.
result Established theoretical guarantees on fairness and excess risk controls.
ConvResNets approximate Besov functions and classify on low-dimensional manifolds.
problem Lack of statistical theories for deep learning on high-dimensional data.
method Exploits low-dimensional geometric structures of real-world data sets using ConvResNets.
result ConvResNets can approximate Besov functions and learn classifiers with optimal excess risk.
We consider the problem of binary classification with abstention in the relatively less studied \emph{bounded-rate} setting. We begin by obtaining a characterization of the Bayes optimal classifier for an arbitrary input-label distribution PXY. Our result generalizes and provides an alternative proof for the resul…
The paper addresses classification imbalance by framing it as a transfer learning problem.
problem Classification imbalance where one class is much rarer than the other.
method The paper studies oversampling procedures to balance classes, focusing on SMOTE and bootstrapping.
result The excess risk decomposes into balanced training rate and transfer cost, with SMOTE having a higher transfer cost.
We consider the problem of learning convex aggregation of models, that is as good as the best convex aggregation, for the binary classification problem. Working in the stream based active learning setting, where the active learner has to make a decision on-the-fly, if it wants to query for the label of the point curren…
New method estimates tensors from noisy data with missing entries.
problem Tensor estimation from noisy observations with missing entries.
method Sign series representation for tensor completion, addressing low- and high-rank signals.
result Excess risk bounds, estimation error rates, and sample complexities established.
Paper improves clustering risk bounds for kernel k-means.
problem Improving clustering risk bounds for kernel k-means.
method Analyzes kernel k-means and Nyström approximation.
result Achieves nearly optimal excess clustering risk bound.
We consider the binary classification problem in a setup that preserves the privacy of the original sample. We provide a privacy mechanism that is locally differentially private and then construct a classifier based on the private sample that is universally consistent in Euclidean spaces. Under stronger assumptions, we…
New method estimates model risk without knowing function class.
problem Evaluating model risk for complex, opaque models.
method Wild refitting with Bregman losses and randomized symmetrization.
result Valid upper bound on excess risk for opaque models.
Develops a deep learning framework for various data types.
problem Handling nonparametric regression and classification across different data types.
method Introduces a general framework with two estimators: NPDNN and SPDNN, based on data satisfying generalized Bernstein-type inequalities.
result Both NPDNN and SPDNN estimators are minimax optimal in many classical settings.
DEUP directly predicts epistemic uncertainty, improving model optimization and exploration.
problem Existing measures of epistemic uncertainty do not account for model misspecification.
method Proposes a framework to estimate excess risk as a measure of epistemic uncertainty, using a secondary predictor for generalization error.
result DEUP improves sequential model optimization and exploration in interactive learning environments.
Local SGD proves efficient in overparameterized linear regression.
problem Efficiently learning overparameterized linear models in distributed settings.
method Distributed SGD (DSGD) with overparameterized models.
result Excess risk of SGD is smaller than ridge regression in the same sample complexity.
Study finds high cyber risk stocks generate significant excess returns.
problem Understanding and quantifying cyber risk's impact on stock returns.
method Machine learning algorithm measuring cyber risk proximity to a corpus.
result High cyber risk stocks generate an excess return of 18.72% p.a.
New metrics improve understanding of predictive system reliability.
problem Evaluating conditional coverage of predictive systems.
method Casting conditional coverage estimation as a classification problem, using excess risk of the target coverage (ERT) metrics.
result Modern classifiers provide higher statistical power for estimating conditional coverage.