JTT improves model worst-group accuracy without group annotations.
problem Low worst-group accuracy in standard ERM models with spurious correlations.
method Two-stage approach: first ERM, then upweight misclassified examples.
result JTT closes 75% of the gap in worst-group accuracy compared to group DRO.
Throwing away data can improve worst-group error in imbalanced datasets.
problem Improving worst-group accuracy in imbalanced datasets.
method Leveraging extreme value theory to analyze the tails of data distributions and their impact on classifier performance.
result Throwing away data restores geometric symmetry in classifiers, improving worst-group generalization.
Data augmentation methods improve worst-case model performance.
problem Ensuring fair predictions across subpopulations in large models.
method Linear last layer retraining with data augmentation techniques.
result Optimal worst-group accuracy achieved for Gaussian latent representation distribution.
Unified framework for fairness, robustness, and distribution shifts.
problem Diverse failure modes of machine learning systems.
method Formalizes biases as violations of conditional independence and proves equivalence conditions.
result Equivalent effects of biases in different failure modes under specific conditions.
Overparameterized neural networks can be highly accurate on average on an i.i.d. test set yet consistently fail on atypical groups of the data (e.g., by learning spurious correlations that hold on average but not in such groups). Distributionally robust optimization (DRO) allows us to learn models that instead minimize…
Improves overparameterized models' robustness to distribution shifts.
problem Accuracy drop on testing distributions different from training.
method Importance tempering to improve decision boundaries.
result State-of-the-art results on worst group classification tasks.
ISP improves DNN uncertainty for better subgroup accuracy.
problem Improving accuracy-group robustness in deep neural networks.
method Introspective Self-play (ISP) adds an introspection task to improve bias-awareness and uncertainty.
result ISP improves the accuracy-group robustness trade-off of AL methods.
RAD improves robustness to domain annotation noise without explicit domain annotations.
problem Robustness to domain annotation noise in training data.
method Regularized Annotation of Domains (RAD) for last layer retraining.
result RAD outperforms state-of-the-art methods even with 5% noise in training data.
Boosts test performance with importance sampling for subpopulation issues.
problem Limited ERM performance on data with spurious correlation or subpopulation.
method Importance sampling as a tool to solve subpopulation problems.
result Achieves state-of-the-art performance on benchmark datasets.
SkewSize detects model biases by analyzing mistakes across subgroups.
problem Benchmarking model performance in the presence of spurious correlations.
method Introducing SkewSize, a metric that captures bias from model mistakes.
result SkewSize highlights biases not captured by other metrics.
XRM discovers environments without human annotations for OOD methods.
problem Costly and biased manual annotations limit OOD methods.
method XRM trains twin networks to mimic mistakes, eliminating hyper-parameters.
result XRM achieves oracle worst-group-accuracy for OOD methods.
This paper improves model robustness to underrepresented groups using ranking metrics and reweighting.
problem Underrepresented groups suffer from low accuracy in models trained via ERM.
method Proposes Discounted Cumulative Gain (DCG) and Discounted Rank Upweighting (DRU) methods.
result Models trained with DRU show superior generalization to unseen groups.
A new approach to group fairness treats it as a bargaining problem.
problem Fairness in deploying predictors across subpopulations.
method Interpreting fairness as a bargaining problem and proposing relative improvement.
result Relative improvement provides axiomatic justification and finite-sample convergence guarantees.
ROME improves algorithmic fairness by learning latent group structure robustly.
problem Latent subgroup disparities and distribution shifts in machine learning models.
method ROME uses an Expectation-Maximization algorithm for linear models and a neural Mixture-of-Experts for nonlinear settings.
result ROME significantly improves fairness compared to standard methods while maintaining average performance.
Deep models learn spurious features correlated with target, but can still perform well.
problem Spurious correlations in feature learning.
method Empirical risk minimization and specialized group robustness training.
result Feature representations learned by ERM are competitive with specialized methods.
Overparameterized models can worsen minority group errors even when overall test error improves.
problem Overparameterization exacerbates spurious correlations, harming minority groups.
method Simulations and experiments on image datasets, theoretical analysis of linear models.
result Subsampling the majority group can achieve low minority error in overparameterized models.
This study examines how learning algorithms affect collective action in machine learning.
problem The impact of collective action on machine learning is limited when not considering the choice of learning algorithms.
method Focuses on distributionally robust optimization and stochastic gradient descent, analyzing their effects on collective success.
result The choice of learning algorithm significantly impacts the effective size and success of a collective in machine learning.
New method improves transfer and robustness of supervised contrastive learning.
problem Class collapse in supervised contrastive learning leads to poor representation quality.
method Adding a weighted class-conditional InfoNCE loss and a class-conditional autoencoder.
result Improves transfer and robustness on 5 standard datasets and 3 worst-group robustness datasets.
SemiNAS reduces NAS cost by predicting accuracy of unlabeled architectures.
problem Costly evaluation of architectures limits NAS efficiency.
method SemiNAS uses unlabeled architectures to train an accuracy predictor.
result SemiNAS achieves comparable accuracy with less data.
Proposes a new adversarial model to avoid accuracy vs. adversarial accuracy tradeoff.
problem Inherent tradeoff between accuracy and adversarial accuracy in existing adversarial robustness definitions.
method Introduces Voronoi-epsilon adversary that balances perturbation constraints.
result Voronoi-epsilon adversary avoids accuracy vs. adversarial accuracy tradeoff even with large ε. Selective classification can worsen accuracy disparities between groups.
problem Selective classification can magnify existing accuracy disparities between various groups.
method Study of margin distribution and distributionally-robust models.
result Selective classification can uniformly improve each group on distributionally-robust models.
Calibrated ensembles improve both ID and OOD accuracy in distribution shift.
problem Desired balance between in-distribution and out-of-distribution accuracy.
method Ensemble standard and robust models, calibrating on ID data only.
result ID-calibrated ensembles outperform state-of-the-art methods on multiple datasets.
Paper monitors DNN accuracy to enhance trustworthiness.
problem Varying DNN accuracy in practice and lack of ground truth labels.
method Post-hoc accuracy monitor model using Monte-Carlo dropout ensemble.
result Accuracy monitor provides close-to-true accuracy estimation.
Optimizes glmnet configuration for better accuracy and efficiency.
problem Inappropriate glmnet configuration leads to inaccurate solutions and increased computation time.
method Data-driven framework using neural networks to predict accuracy and computation time from dataset characteristics and configuration.
result Automatic selection of optimal configuration maximizing accuracy under a time constraint.
TRUST improves tree models' accuracy while maintaining interpretability.
problem Piecewise-constant regression trees lack in predictive accuracy compared to black-box models.
method Combines Random Forest accuracy with interpretability of shallow trees and sparsity of linear models, using LLMs for explanations.
result TRUST outperforms other interpretable models in predictive accuracy and matches Random Forest's accuracy.
FADE framework improves fairness and accuracy in ensemble learning.
problem Improving fairness in existing models without sacrificing accuracy.
method Flexible fair ensemble learning framework targeting multiple fairness criteria.
result Multiple unfairness measures can be minimized simultaneously with little impact on accuracy.
Paper explores tradeoff between standard and robust accuracy for latent models.
problem Tradeoff between standard accuracy and robust accuracy in adversarial training.
method Revisits adversarial training for latent models, considering Gaussian mixture and generalized linear models.
result Low-dimensional manifold structure mitigates the tradeoff between standard and robust accuracy.
New methods show robustness and accuracy can coexist.
problem Inevitability of robustness-accuracy tradeoff in deep learning.
method Prove robustness and accuracy achievable through locally Lipschitz functions; explore combining dropout with robust training methods.
result Achieving robustness and accuracy requires methods imposing local Lipschitzness and deep learning generalization techniques.
Improved forecast accuracy for Knitwear by 20% using adaptive AI/ML model.
problem Low accuracy in demand forecasts for Knitwear product category.
method Dynamic selection of the best algorithm from an algorithm rack based on performance and context.
result Increased forecast accuracy from 60% to 80% for Knitwear.
Proposes an accuracy-preserving calibration method for DNNs.
problem Calibration of deep neural networks (DNNs) to measure prediction reliability.
method Uses Concrete distribution on the probability simplex to calibrate DNNs without accuracy loss.
result The proposed method outperforms previous methods in accuracy-preserving calibration tasks.
Improves natural accuracy of deep learning models by combining robust predictions and features.
problem Maintaining natural accuracy while resisting adversarial attacks.
method Ensemble methods combining robust and standard models.
result Optimized natural accuracy through ensemble of robust models.
The paper studies and mitigates accuracy disparity in regression models.
problem Accuracy disparity between different demographic subgroups in high-stakes domains.
method Error decomposition theorem and distribution alignment algorithm.
result The proposed algorithm effectively mitigates accuracy disparity while maintaining predictive power.
New work shows limits of certifying neural network robustness.
problem Certified training improves robustness but decreases accuracy.
method Bayes error analysis to investigate robustness limits.
result Upper bound for certified robust accuracy established.
Project fair estimators while maintaining accuracy.
problem Making estimators fair without sacrificing accuracy.
method Optimal transport tools to find closest fair estimator.
result Efficiently constructs fair estimators with quantified cost.
The paper analyzes adversarial training effects on classification accuracy.
problem Understanding adversarial training's impact on standard and robust accuracy.
method Derived precise statistical analysis for binary classification problems with Gaussian data.
result Theoretical explanation of standard and robust accuracy trends for adversarial training.
Empirical law predicts accuracy of Google Translate's translation chains.
problem Predicting accuracy in machine translation with multiple hops.
method Empirical testing of Google Translate's sequential translation.
result Accuracy decreases with the number of translating hops, following a power law.
Learning ReLU networks to high uniform accuracy requires exponentially many samples.
problem Achieving high uniform accuracy on ReLU networks for security-critical applications.
method Quantified the number of training samples needed for any algorithm to guarantee uniform accuracy.
result The minimal number of training samples scales exponentially with network depth and input dimension.
Unhinged loss minimization fails to improve classifier accuracy for simple data.
problem Accuracy of classifiers minimizing the unhinged loss.
method Minimizing the unhinged loss function.
result Minimizing the unhinged loss yields classifiers with accuracy no better than random guessing for simple data.
The paper examines how adversarial robustness affects accuracy disparity across different classes.
problem Understanding the impact of adversarial robustness on accuracy disparity across different classes.
method Linear classifiers under a Gaussian mixture model, decomposing the impact into inherent and imbalance effects.
result Adversarial robustness consistently degrades standard accuracy in balanced classes, but the class imbalance ratio plays a different role in accuracy disparity.
APQ jointly optimizes neural architecture, pruning, and quantization for efficient inference.
problem Efficient deep learning inference on resource-constrained hardware.
method Joint optimization of neural architecture, pruning, and quantization policy using a quantization-aware accuracy predictor.
result Joint optimization leads to 2.3% higher ImageNet accuracy with reduced latency and energy consumption.
In this paper, we investigate a new form of automated curriculum learning based on adaptive selection of accuracy requirements, called accuracy-based curriculum learning. Using a reinforcement learning agent based on the Deep Deterministic Policy Gradient algorithm and addressing the Reacher environment, we first show …
Accuracy on in-distribution data correlates with out-of-distribution data when data is noisy or contains nuisance features.
problem Correlation between in-distribution and out-of-distribution accuracy in noisy or feature-rich data.
method Analyzes the impact of noise and nuisance features on model performance.
result Accuracy on in-distribution and out-of-distribution data can become negatively correlated in noisy or feature-rich data.
Paper analyzes trade-offs in top-k classification accuracies and proposes a new loss function.
problem CE loss does not always optimize top-k prediction, especially with complex data.
method Introduces a novel top-k transition loss to improve top-k accuracy.
result Our loss function improves top-k accuracy, especially for k > 10.
Linear trends in classifier accuracy observed under distribution shift.
problem Understanding why classifier accuracies show linear trends under distribution shift.
method Assumed model similarity and verified empirically.
result Linear trend in classifier accuracy occurs unless distribution shift is large.
Paper introduces a method to control early classification accuracy gaps.
problem Maintaining accuracy in early classification without full input processing.
method Statistical framework for a calibrated stopping rule.
result Reduces up to 94% of timesteps while controlling accuracy gaps.
Researchers study fairness-accuracy tradeoffs in predictive models for multiple groups.
problem Understanding the tradeoff between fairness and accuracy in models serving multiple demographic groups.
method Characterizing the fairness-accuracy (FA) Pareto frontier, approximating it from limited data, and bounding the worst-case gap.
result Derivation of worst-case-optimal estimators and uniform finite-sample bounds for the entire FA frontier.
TIP-Search optimizes market prediction accuracy and timeliness under uncertain load.
problem Real-time market prediction requires accurate predictions before a deadline.
method Filters feasible models, dispatches workers, trades accuracy for deadline risk.
result Optimized pool achieves 0.991 timely accuracy and 0.994 raw accuracy.
In this paper we present a comprehensive view of prominent causal discovery algorithms, categorized into two main categories (1) assuming acyclic and no latent variables, and (2) allowing both cycles and latent variables, along with experimental results comparing them from three perspectives: (a) structural accuracy, (…