Learning a classifier with control on the false-positive rate plays a critical role in many machine learning applications. Existing approaches either introduce prior knowledge dependent label cost or tune parameters based on traditional classifiers, which lack consistency in methodology because they do not strictly adh…
New method calibrates false detection rates in sequential change detection.
problem Challenges in setting time-invariant thresholds for false positives.
method Simulation-based approach to time-varying thresholds.
result Accurately targets desired expected runtime while keeping false positive rate constant.
Study controls error rates of binary classifiers using hypothesis testing.
problem Traditional binary classifiers have uncontrolled error rates.
method Combines binary classification with statistical hypothesis testing.
result Trained classifiers can be made to meet target error rate thresholds.
PatternLocal improves XAI for non-linear models by suppressing suppressor variables.
problem Suppressor variables cause false-positive feature attributions in non-linear models.
method PatternLocal uses locally linear surrogate models and transforms weights into a generative representation.
result PatternLocal reduces false-positive attributions and provides more reliable explanations.
A statistical test controls false positives in anomaly localization using diffusion models.
problem Uncertainty and bias in generative models for anomaly localization.
method Selective inference to quantify significance and control false positives.
result The method effectively controls false positive detection rates.
New method improves feature selection by integrating stability paths.
problem Improving feature selection with tighter false positive control.
method Integrating stability paths to strengthen theoretical bounds on E(FP).
result Significantly more true positives with same E(FP) control.
Nonparametric IPSS selects features with false discovery control.
problem Feature selection in high-dimensional data with theoretical false discovery control.
method Integrated Path Stability Selection (IPSS) applied to nonparametric feature importance scores.
result IPSS accurately controls false discovery rate and detects more true positives than existing methods.
We address the problem of non-parametric multiple model comparison: given l candidate models, decide whether each candidate is as good as the best one(s) or worse than it. We propose two statistical tests, each controlling a different notion of decision errors. The first test, building on the post selection inference…
Algorithm reconstructs triangle-free networks from data, certifying correctness.
problem Reconstructing triangle-free dynamic networks from observational data.
method Developed an algorithm for triangle-free networks, providing guarantees on correctness.
result Algorithm either certifies correctness or outputs a sparser graph with no false positives.
Framework uses human feedback to safely set OOD detection thresholds, reducing false positives.
problem Challenges in setting OOD detection thresholds for safety-critical applications.
method Mathematically grounded framework leveraging expert feedback to dynamically update thresholds.
result Guaranteed to meet FPR constraint while minimizing human feedback, maintaining FPR at most 5%.
In recent years, deep learning methods have outperformed other methods in image recognition. This has fostered imagination of potential application of deep learning technology including safety relevant applications like the interpretation of medical images or autonomous driving. The passage from assistance of a human d…
We propose an adaptive sampling approach for multiple testing which aims to maximize statistical power while ensuring anytime false discovery control. We consider n distributions whose means are partitioned by whether they are below or equal to a baseline (nulls), versus above the baseline (actual positives). In addi…
New algorithm for adaptive experimental design in scientific settings.
problem Identifying true positives while controlling false discoveries in adaptive experimental design.
method Provably sample efficient adaptive algorithm for FDR control.
result First provably sample efficient adaptive algorithm for adaptive experimental design.
In high dimensional settings where a small number of regressors are expected to be important, the Lasso estimator can be used to obtain a sparse solution vector with the expectation that most of the non-zero coefficients are associated with true signals. While several approaches have been developed to control the inclu…
Develops a new criterion for subgroup fairness in algorithmic decision support.
problem Identifying fair recommendations in algorithms despite group-level differences.
method IJDI criterion and IJDI-Scan approach to detect and mitigate disparities.
result Identifies significant disparities in recommendations across subpopulations.
Proposes cost-sensitive feature selection for SVMs.
problem Asymmetric misclassification costs in feature selection.
method Mathematical optimization-based approach for SVMs.
result Substantial reduction in feature count with desired error rates.
New algorithm balances user reward and statistical inference by mixing TS with UR based on difference size.
problem Combining statistical inference with user reward in adaptive experiments.
method TS-PostDiff algorithm that uses UR when differences are small and TS when large.
result TS-PostDiff reduces false positives and increases statistical power for small differences, while maximizing reward for large ones.
Benchmarking recursive collapse claims with a new framework under false-positive control.
problem Evaluating recursive systems for failure patterns and warning claims.
method Developed Loopzero framework for testing recursive failures, specified claim boundaries in Lean, evaluated under FP constraint, and compared with standard detectors.
result No standard detectors or Loopzero's pre-registered quantile detector achieved the required operating point under the false-positive contract.
Information systems have widely been the target of malware attacks. Traditional signature-based malicious program detection algorithms can only detect known malware and are prone to evasion techniques such as binary obfuscation, while behavior-based approaches highly rely on the malware training samples and incur prohi…
New method improves false-/true-positive-rate estimation in fraud detection with noisy labels.
problem Estimating FPR/TPR in fraud detection with class-conditional label noise.
method Directly cleaning model's validation data to de-correlate cleaning error with model scores.
result Improves accuracy of FPR/TPR estimates, especially in asymmetric label noise scenarios.
This work improved clustering methods by analyzing various datasets and dendrograms.
problem Avoiding false positives in clustering, especially for unimodal and bimodal data.
method Applied agglomerative clustering methods (single, average, median, complete, centroid, Ward's) to various datasets.
result Many methods detected two clusters in unimodal data, with single-linkage being more resilient.
Paper estimates FPR of Bayes classifier using soft labels.
problem Determining optimal classifier performance.
method Uses soft labels and denoising technique.
result Consistent and unbiased FPR estimator developed.
Pointwise localization allows more precise localization and accurate interpretability, compared to bounding box, in applications where objects are highly unstructured such as in medical domain. In this work, we focus on weakly supervised localization (WSL) where a model is trained to classify an image and localize regi…
Transformer learns representations from time series data for money laundering detection.
problem Detecting money laundering using structured time series data.
method Contrastive learning for representation learning, followed by scoring and thresholding.
result Transformer outperforms rule-based and LSTM methods in detecting money laundering with controlled false positives.
Algorithm detects concept drift and adapts models in streaming data.
problem Concept drift in streaming data renders models inaccurate.
method Adaptive learning algorithm that detects drifts and reacts to them.
result Risk competitive to an algorithm with perfect drift knowledge.
New findings control FDR for online testing methods under positive dependence.
problem Maintaining FDR control for online testing methods under positive dependence.
method Developed new methods to control FDR for online testing procedures under positive dependence.
result SAFFRON and LORD control FDR under positive dependence, not just conditional superuniformity.
Reduces false positives in classifying rare online platforms.
problem Challenges in accurately identifying rare online platforms with ML.
method Calibrated probabilities and ensembles to reduce bias.
result Significantly reduces false positives in rare event detection.
The paper examines how machine learning tools in justice settings can unfairly affect different racial groups.
problem Machine learning tools in justice settings can unfairly affect different racial groups.
method Exploring different ideas of racial equity and their computational trade-offs.
result Computation alone is unlikely to solve the unfairness in machine learning tools for justice settings.
Bottlenecks of binary classification from positive and unlabeled data (PU classification) are the requirements that given unlabeled patterns are drawn from the test marginal distribution, and the penalty of the false positive error is identical to the false negative error. However, such requirements are often not fulfi…
Generating large quantities of quality labeled data in medical imaging is very time consuming and expensive. The performance of supervised algorithms for various tasks on imaging has improved drastically over the years, however the availability of data to train these algorithms have become one of the main bottlenecks f…
A new metric, Weighted Regret, unifies FDR and power evaluation in online multiple testing.
problem The asymmetric costs of false positives and false negatives in automated pipelines.
method Introducing Weighted Regret and Decoupled-OMT (DOMT) to unify FDR and power evaluation.
result DOMT achieves an order-optimal sublinear mitigation of threshold depletion in bursty environments.
In regression settings where explanatory variables have very low correlations and there are relatively few effects, each of large magnitude, we expect the Lasso to find the important variables with few errors, if any. This paper shows that in a regime of linear sparsity---meaning that the fraction of variables with a n…
Private online FDR control for adaptive testing under differential privacy.
problem Controlling false discoveries in adaptive multiple hypothesis testing with privacy constraints.
method Private online algorithms based on non-private results, ensuring privacy and statistical performance.
result Strong guarantees for privacy and statistical performance in FDR and power.
Cheap permutation tests speed up distribution testing without sacrificing accuracy.
problem Efficiently testing distribution differences and independence.
method Group datapoints into bins and permute only these bins, using stored sufficient statistics.
result Cheap permutation tests maintain the accuracy and optimality of standard tests but are significantly faster.
We present a new approach for mitigating unfairness in learned classifiers. In particular, we focus on binary classification tasks over individuals from two populations, where, as our criterion for fairness, we wish to achieve similar false positive rates in both populations, and similar false negative rates in both po…
A genome-wide association study (GWAS) correlates marker variation with trait variation in a sample of individuals. Each study subject is genotyped at a multitude of SNPs (single nucleotide polymorphisms) spanning the genome. Here we assume that subjects are unrelated and collected at random and that trait values are n…
Collision avoidance is a critical task in many applications, such as ADAS (advanced driver-assistance systems), industrial automation and robotics. In an industrial automation setting, certain areas should be off limits to an automated vehicle for protection of people and high-valued assets. These areas can be quaranti…
Paper tackles MIAs vulnerability by controlling FDR, providing guarantees on false discoveries.
problem Vulnerability of deep learning models to membership inference attacks (MIAs).
method Designs a novel membership inference attack method that provides FDR guarantees.
result Demonstrates the effectiveness of the method in various settings.
Generative models create indistinguishable adversarial objects for object detection.
problem Creating unrestricted adversarial examples for object detection.
method Search over latent space of GAN for adversarial objects.
result Generated adversarial objects are indistinguishable from non-adversarial objects and transferable.
New research disproves a key conjecture in optimization.
problem Comparison of sampling methods in stochastic optimization.
method Reduction to noncommutative arithmetic-geometric mean inequality and application of noncommutative Positivstellensatz.
result The Recht-Ré conjecture is false for general n.
FairCal improves face verification accuracy while making results fairer.
problem Bias in face recognition models disproportionately affects minority groups.
method Post-training approach that builds fairer decision classifiers using pre-trained model features.
result State-of-the-art results with increased accuracy and fairness.
Simple bounds show most cross-sectional predictability findings are likely true.
problem Determining the validity of cross-sectional return predictability findings.
method Developed simple and intuitive bounds on the false discovery rate (FDR).
result Bounds show the FDR is small, indicating most findings are likely true.
The paper proposes a method to create efficient remote monitoring models.
problem Large and complex machine learning models are unsuitable for remote monitoring on edge devices.
method Decompose the model into a simple local monitoring function and a complex correction term evaluated on the server.
result The proposed framework learns monitoring models with significantly reduced complexity that maintain safety.
As video games attract more and more players, the major challenge for game studios is to retain them. We present a deep behavioral analysis of churn (game abandonment) and what we called "purchase churn" (the transition from paying to non-paying user). A series of churning behavior profiles are identified, which allows…
Machine learning detects subhalos in lensed images with high accuracy and low false positives.
problem Detecting substructure in strongly lensed images.
method Developed a neural network for image segmentation to locate and mass estimate subhalos.
result The network can detect subhalos with masses m≳108.5M⊙ and measure the subhalo mass function. Paper reduces false positives in lung nodule detection using deep learning on point clouds.
problem Reduces false positives in lung nodule detection.
method Uses deep learning models for point clouds to transform 3D CT scan data.
result Achieved 85.98 FROC compared to 77.26 FROC for baseline models.
With the rapid growth of crowdsourcing platforms it has become easy and relatively inexpensive to collect a dataset labeled by multiple annotators in a short time. However due to the lack of control over the quality of the annotators, some abnormal annotators may be affected by position bias which can potentially degra…
New method controls false discoveries in structured hypothesis spaces.
problem Controlling false discoveries in large-scale, interconnected hypothesis spaces.
method Reproducing Kernel Hilbert Space (RKHS) optimization for structured FDR control.
result Unified framework for continuous domains, graphs, and hierarchies.