Learning a classifier with control on the false-positive rate plays a critical role in many machine learning applications. Existing approaches either introduce prior knowledge dependent label cost or tune parameters based on traditional classifiers, which lack consistency in methodology because they do not strictly adh…
PatternLocal improves XAI for non-linear models by suppressing suppressor variables.
problem Suppressor variables cause false-positive feature attributions in non-linear models.
method PatternLocal uses locally linear surrogate models and transforms weights into a generative representation.
result PatternLocal reduces false-positive attributions and provides more reliable explanations.
A statistical test controls false positives in anomaly localization using diffusion models.
problem Uncertainty and bias in generative models for anomaly localization.
method Selective inference to quantify significance and control false positives.
result The method effectively controls false positive detection rates.
New method improves feature selection by integrating stability paths.
problem Improving feature selection with tighter false positive control.
method Integrating stability paths to strengthen theoretical bounds on E(FP).
result Significantly more true positives with same E(FP) control.
Framework uses human feedback to safely set OOD detection thresholds, reducing false positives.
problem Challenges in setting OOD detection thresholds for safety-critical applications.
method Mathematically grounded framework leveraging expert feedback to dynamically update thresholds.
result Guaranteed to meet FPR constraint while minimizing human feedback, maintaining FPR at most 5%.
Benchmarking recursive collapse claims with a new framework under false-positive control.
problem Evaluating recursive systems for failure patterns and warning claims.
method Developed Loopzero framework for testing recursive failures, specified claim boundaries in Lean, evaluated under FP constraint, and compared with standard detectors.
result No standard detectors or Loopzero's pre-registered quantile detector achieved the required operating point under the false-positive contract.
This work improved clustering methods by analyzing various datasets and dendrograms.
problem Avoiding false positives in clustering, especially for unimodal and bimodal data.
method Applied agglomerative clustering methods (single, average, median, complete, centroid, Ward's) to various datasets.
result Many methods detected two clusters in unimodal data, with single-linkage being more resilient.
The paper reviews techniques for detecting errors in semantic segmentation models.
problem Detecting false positives and false negatives in semantic segmentation models.
method Uncertainty quantification techniques applied to semantic segmentation.
result Techniques for detecting false positives and false negatives are proposed and discussed.
Pointwise localization allows more precise localization and accurate interpretability, compared to bounding box, in applications where objects are highly unstructured such as in medical domain. In this work, we focus on weakly supervised localization (WSL) where a model is trained to classify an image and localize regi…
Transformer learns representations from time series data for money laundering detection.
problem Detecting money laundering using structured time series data.
method Contrastive learning for representation learning, followed by scoring and thresholding.
result Transformer outperforms rule-based and LSTM methods in detecting money laundering with controlled false positives.
Algorithm detects concept drift and adapts models in streaming data.
problem Concept drift in streaming data renders models inaccurate.
method Adaptive learning algorithm that detects drifts and reacts to them.
result Risk competitive to an algorithm with perfect drift knowledge.
Reduces false positives in classifying rare online platforms.
problem Challenges in accurately identifying rare online platforms with ML.
method Calibrated probabilities and ensembles to reduce bias.
result Significantly reduces false positives in rare event detection.
In high dimensional settings where a small number of regressors are expected to be important, the Lasso estimator can be used to obtain a sparse solution vector with the expectation that most of the non-zero coefficients are associated with true signals. While several approaches have been developed to control the inclu…
New method calibrates false detection rates in sequential change detection.
problem Challenges in setting time-invariant thresholds for false positives.
method Simulation-based approach to time-varying thresholds.
result Accurately targets desired expected runtime while keeping false positive rate constant.
Generating large quantities of quality labeled data in medical imaging is very time consuming and expensive. The performance of supervised algorithms for various tasks on imaging has improved drastically over the years, however the availability of data to train these algorithms have become one of the main bottlenecks f…
Cheap permutation tests speed up distribution testing without sacrificing accuracy.
problem Efficiently testing distribution differences and independence.
method Group datapoints into bins and permute only these bins, using stored sufficient statistics.
result Cheap permutation tests maintain the accuracy and optimality of standard tests but are significantly faster.
Study controls error rates of binary classifiers using hypothesis testing.
problem Traditional binary classifiers have uncontrolled error rates.
method Combines binary classification with statistical hypothesis testing.
result Trained classifiers can be made to meet target error rate thresholds.
Paper estimates FPR of Bayes classifier using soft labels.
problem Determining optimal classifier performance.
method Uses soft labels and denoising technique.
result Consistent and unbiased FPR estimator developed.
Collision avoidance is a critical task in many applications, such as ADAS (advanced driver-assistance systems), industrial automation and robotics. In an industrial automation setting, certain areas should be off limits to an automated vehicle for protection of people and high-valued assets. These areas can be quaranti…
Generative models create indistinguishable adversarial objects for object detection.
problem Creating unrestricted adversarial examples for object detection.
method Search over latent space of GAN for adversarial objects.
result Generated adversarial objects are indistinguishable from non-adversarial objects and transferable.
FairCal improves face verification accuracy while making results fairer.
problem Bias in face recognition models disproportionately affects minority groups.
method Post-training approach that builds fairer decision classifiers using pre-trained model features.
result State-of-the-art results with increased accuracy and fairness.
Algorithm reconstructs triangle-free networks from data, certifying correctness.
problem Reconstructing triangle-free dynamic networks from observational data.
method Developed an algorithm for triangle-free networks, providing guarantees on correctness.
result Algorithm either certifies correctness or outputs a sparser graph with no false positives.
New algorithm balances user reward and statistical inference by mixing TS with UR based on difference size.
problem Combining statistical inference with user reward in adaptive experiments.
method TS-PostDiff algorithm that uses UR when differences are small and TS when large.
result TS-PostDiff reduces false positives and increases statistical power for small differences, while maximizing reward for large ones.
Information systems have widely been the target of malware attacks. Traditional signature-based malicious program detection algorithms can only detect known malware and are prone to evasion techniques such as binary obfuscation, while behavior-based approaches highly rely on the malware training samples and incur prohi…
Proposes cost-sensitive feature selection for SVMs.
problem Asymmetric misclassification costs in feature selection.
method Mathematical optimization-based approach for SVMs.
result Substantial reduction in feature count with desired error rates.
Paper reduces false positives in lung nodule detection using deep learning on point clouds.
problem Reduces false positives in lung nodule detection.
method Uses deep learning models for point clouds to transform 3D CT scan data.
result Achieved 85.98 FROC compared to 77.26 FROC for baseline models.
The problem of multiple hypothesis testing arises when there are more than one hypothesis to be tested simultaneously for statistical significance. This is a very common situation in many data mining applications. For instance, assessing simultaneously the significance of all frequent itemsets of a single dataset entai…
In variable or graph selection problems, finding a right-sized model or controlling the number of false positives is notoriously difficult. Recently, a meta-algorithm called Stability Selection was proposed that can provide reliable finite-sample control of the number of false positives. Its benefits were demonstrated …
Generative Adversarial Network purifies images from steganography without degrading quality.
problem Destruction of image steganography while maintaining visual quality.
method Generative Adversarial Network (GAN) optimized for steganography destruction.
result High rate of steganographic content destruction with minimal visual quality degradation.
Develops a new criterion for subgroup fairness in algorithmic decision support.
problem Identifying fair recommendations in algorithms despite group-level differences.
method IJDI criterion and IJDI-Scan approach to detect and mitigate disparities.
result Identifies significant disparities in recommendations across subpopulations.
A neural network for online NP classification with reduced complexity.
problem Online nonlinear Neyman-Pearson classification.
method Single hidden layer feedforward neural network (SLFN) initialized with random Fourier features (RFFs). Uses stochastic gradient descent for sequential learning.
result Expedited online adaptation and powerful nonlinear Neyman-Pearson modeling.
We address the problem of non-parametric multiple model comparison: given l candidate models, decide whether each candidate is as good as the best one(s) or worse than it. We propose two statistical tests, each controlling a different notion of decision errors. The first test, building on the post selection inference…
In this work, we present a comparison of a shallow and a deep learning architecture for the automated segmentation of white matter lesions in MR images of multiple sclerosis patients. In particular, we train and test both methods on early stage disease patients, to verify their performance in challenging conditions, mo…
Improved change point detection using matched filters for non-parametric tests.
problem False positives and localization ambiguity in non-parametric two-sample tests.
method Derived and applied matched filters for various two-sample tests.
result Matched filters reduce false positives and improve test precision.
A new method assesses algorithmic fairness using game theory.
problem Evaluating algorithmic fairness without proprietary data.
method Cohort Shapley value, a game-theoretic approach.
result Identifies individual impact of protected attributes.
Machine learning detects subhalos in lensed images with high accuracy and low false positives.
problem Detecting substructure in strongly lensed images.
method Developed a neural network for image segmentation to locate and mass estimate subhalos.
result The network can detect subhalos with masses m≳108.5M⊙ and measure the subhalo mass function. New method quantifies deep kNN anomaly detection significance.
problem Lack of uncertainty quantification in deep kNN AD.
method Selective Inference for anomaly scoring.
result Validates AD reliability with controlled false positives.
Probabilistic graphical models are graphical representations of probability distributions. Graphical models have applications in many fields including biology, social sciences, linguistic, neuroscience. In this paper, we propose directed acyclic graphs (DAGs) learning via bootstrap aggregating. The proposed procedure i…
Active learning improves inspection systems by using weakly labeled data.
problem Rapidly updating machine vision inspection systems in evolving manufacturing processes.
method Developed a methodology for active learning from weakly labeled data, addressing covariate shift with domain-adversarial training.
result Demonstrated that active learning can accelerate the annotation process and reduce false positives.
New approach detects adversarial samples with certifiable guarantees.
problem Adversarial samples can trick CNNs, posing a threat.
method Certifiable Taboo Trap (CTT) approach to detect adversarial inputs.
result CTT outperforms existing defenses on various lp norms. Study identifies high-density anomalies in normal data regions.
problem Detecting anomalies in normal data regions.
method Introduces non-parametric algorithmic frameworks for unsupervised detection.
result IPP framework yields the best detection results.
Anomaly detectors are often used to produce a ranked list of statistical anomalies, which are examined by human analysts in order to extract the actual anomalies of interest. Unfortunately, in realworld applications, this process can be exceedingly difficult for the analyst since a large fraction of high-ranking anomal…
Paper proposes a statistical test for feature selection pipelines using selective inference.
problem Assessing the significance of feature selection pipelines in data analysis.
method Selective inference technique applied to feature selection pipelines composed of various algorithms.
result The proposed statistical test controls false positive feature selection probabilities.
Study proposes a statistical test for Vision Transformer's attention mechanisms.
problem ViT's attention mechanisms may focus on irrelevant regions, leading to unreliable evidence.
method Selective inference framework to quantify statistical significance of attentions as p-values.
result Proposed method enables reliable quantification of false positive detection probability of attentions.
The paper examines how machine learning tools in justice settings can unfairly affect different racial groups.
problem Machine learning tools in justice settings can unfairly affect different racial groups.
method Exploring different ideas of racial equity and their computational trade-offs.
result Computation alone is unlikely to solve the unfairness in machine learning tools for justice settings.
Simple methods combine statistical tests for out-of-distribution detection.
problem Detecting data points not following the training distribution.
method Combining classical parametric tests (Rao's score test) and a typicality test.
result Combining Fisher's method of test statistics improves out-of-distribution detection accuracy.
Paper introduces PTL-SI for statistical inference in TL-HDR, controlling FPR.
problem Quantifying statistical significance in TL-HDR with limited data.
method PTL-SI framework for valid p-values in TL-HDR feature selection. result Valid p-values and controlled FPR in TL-HDR feature selection. AutoYara generates effective Yara rules faster than humans.
problem Developing high-quality Yara rules for malware families is labor-intensive.
method Leverages biclustering on large n-grams to automate Yara rule generation.
result AutoYara reduces analyst workload by 44-86% and matches human performance.