Study evaluates machine learning methods for large-scale network reliability, revealing ANN's and PR's performance.
problem Tackles the NP-hard problem of approximating binary-state network reliability for large-scale systems.
method Compares 20 machine learning methods across three reliability regimes and evaluates their performance on large-scale networks.
result Large-scale networks with arc reliability ≥ 0.9 exhibit near-unity system reliability, enabling computational simplifications.
Although the threshold network is one of the most used tools to characterize the underlying structure of a stock market, the identification of the optimal threshold to construct a reliable stock network remains challenging. In this paper, the concept of dynamic consistence between the threshold network and the stock ma…
Improved reliability of machine learning predictions using variational auto-encoders.
problem Individual unreliability of machine learning models.
method Modified variational auto-encoders to identify a low-dimensional space for reliable classification.
result Improved reliability of predictions and robust identification of adversarial samples.
New method calculates sensitivity of system failure probability.
problem Difficulty in computing sensitivity of failure probability.
method Monte Carlo strategy using response gradient and kernel smoothing.
result Single Monte Carlo run for sensitivity estimates.
Fault-tolerant neural networks inspired by biological error correction codes.
problem Achieving reliable computation with unreliable neurons.
method Using biological error correction codes from grid cells in the mammalian cortex to develop a fault-tolerant neural network.
result Noisy biological neurons operate below a fault-tolerance threshold, suggesting a mechanism for reliable computation in the brain.
INCAD clusters and detects anomalies in streaming data without thresholds.
problem Clustering and anomaly detection for streaming data with unknown clusters and thresholds.
method Probabilistic clustering and anomaly detection in a streaming model.
result More reliable definition of normal vs abnormal behavior in streaming data.
New method improves reliability of selecting individuals based on predicted treatment effects.
problem Reliability of selecting individuals based on predicted conditional average treatment effects (CATE) is unreliable.
method Denoised Conformal Alignment, combining proxy errors, variance estimation, and Benjamini-Hochberg selection.
result Significantly improved power in selecting individuals while maintaining false discovery rate control.
New method calibrates uncertainty estimates for image classifiers without labeled data.
problem Uncertainty estimates for modern classifiers are unreliable without labeled calibration data.
method Calibrates uncertainty estimates using unlabeled examples for distribution shifts.
result Proposes a method that provides excellent uncertainty estimates under natural distribution shifts.
Fairness audits fail under missing protected labels, especially at zero access.
problem Understanding the reliability of fairness audits with incomplete protected-label data.
method Introduced a seed-calibrated stress test to separate missingness effects from seed-to-seed movement.
result Missing protected labels do not significantly alter fairness mitigation methods, but they can lead to harmful intersectional outcomes.
New metrics CWSA and CWSA+ improve model evaluation under confidence thresholds.
problem Lack of metrics capturing model reliability under confidence thresholds.
method Introducing CWSA and CWSA+ metrics that reward confident accuracy and penalize overconfident mistakes.
result CWSA and CWSA+ outperform classical metrics in trust-sensitive tests.
Proposes a method to choose thresholds for LLM evaluation metrics.
problem Ensuring reliable large language models (LLMs) with correct threshold selection.
method Identify risks, stakeholders' risk tolerance, and use ground-truth data to determine thresholds.
result Demonstrates a concrete example with the Faithfulness metric and HaluBench dataset.
The study analyzes when Bayesian averaging over decision trees is reliable.
problem When do Bayesian model averaging weights over decision trees provide reliable information?
method Closed-form solution for Bayesian decision trees with Catalan-exponential priors.
result Established a complete non-asymptotic theory of rational commitment thresholds.
Improved AI lung ultrasound segmentation using expert confidence values.
problem Label uncertainty in lung ultrasound due to subjective interpretation by radiologists.
method Designing a data annotation protocol capturing expert confidence, training AI on binarized labels with confidence thresholds.
result Improved AI segmentation and better clinical outcomes (e.g., S/F oxygenation ratio estimation, patient readmission prediction).
We define the information threshold in Bayesian decision-making.
problem Understanding the optimal amount of information for reliable classification.
method Defining the information threshold as the point of maximum curvature in the prior vs. posterior curve.
result At the information threshold, additional evidence does not significantly improve posterior probability.
In this work, a heuristic as operational tool to estimate the lactate threshold and to facilitate its integration into the training process of recreational runners is proposed. To do so, we formalize the principles for the lactate threshold estimation from empirical data and an iterative methodology that enables experi…
Develops a statistical test for IV, improving feature selection reliability.
problem Lack of statistical justification in conventional IV-based feature selection.
method Establishes connection with Jeffreys divergence and proposes a nonparametric hypothesis test.
result The J-Divergence test provides rigorous guarantees and is more reliable than traditional IV thresholds.
We present a novel distribution-free approach, the data-driven threshold machine (DTM), for a fundamental problem at the core of many learning tasks: choose a threshold for a given pre-specified level that bounds the tail probability of the maximum of a (possibly dependent but stationary) random sequence. We do not ass…
A framework for detecting out-of-distribution data in RL using uncertainty-based classification.
problem Detecting out-of-distribution data in deep reinforcement learning systems.
method A one-class classification problem approach based on epistemic uncertainty reduction.
result The proposed UBOOD framework reliably detects out-of-distribution situations when combined with ensemble-based uncertainty estimators.
We consider the problem of aligning a pair of databases with jointly Gaussian features. We consider two algorithms, complete database alignment via MAP estimation among all possible database alignments, and partial alignment via a thresholding approach of log likelihood ratios. We derive conditions on mutual informatio…
High-dimensional models become unstable when sample size falls below a critical level, leading to a phase transition.
problem Instability in high-dimensional learning models when sample size is insufficient.
method Proved the necessity of a Fisher eigenvalue threshold for stability, introduced Fisher floor for verification.
result A sharp phase transition between reliable concentration and inevitable failure in high-dimensional learning.
Proposes a statistical test for VAE-based anomaly detection reliability.
problem Ensuring reliability of anomaly detection in high-stakes applications.
method Variance Autoencoder (VAE) Test based on selective inference.
result Validates VAE-based anomaly detection with p-values controlling false detection probability.
Statistical guarantees for hyperparameter selection
problem Hyperparameter selection in AI systems
method Learn-then-test framework
result Provable reliability and safety
New study shows diversification can increase risk for heavy-tailed losses.
problem Diversification can increase tail risk for heavy-tailed losses.
method Comparison of diversified portfolio to a 'one-basket' benchmark.
result Diversified portfolio has larger tail probabilities than a 'one-basket' benchmark for all thresholds.
Much of modern practice in financial forecasting relies on technicals, an umbrella term for several heuristics applying visual pattern recognition to price charts. Despite its ubiquity in financial media, the reliability of its signals remains a contentious and highly subjective form of 'domain knowledge'. We investiga…
The paper improves A/B testing for non-Gaussian data, ensuring reliable results with large sample sizes.
problem Inaccurate A/B testing results due to non-normal data and unequal sample sizes.
method Derives explicit formulas for minimum sample size and introduces an Edgeworth-based correction.
result Corrected method improves reliability of A/B testing in real-world conditions.
CTI produces efficient prediction intervals with guaranteed coverage.
problem Efficient and reliable uncertainty quantification in regression.
method CTI estimates conditional density for interval length, then thresholds intervals based on this density.
result CTI achieves smaller prediction sets with guaranteed coverage compared to existing methods.
The paper proposes a method to improve random forest classification accuracy by weighting trees based on their decision path reliability.
problem Random forests' uniform voting fails to correct errors in regions where incorrect tree representations outnumber correct ones.
method The paper introduces using the structural pattern of each tree's decision path as an instance-adaptive reliability signal to identify and weight more reliable trees.
result Using the proposed method yields a statistically significant accuracy improvement over RF on 36 binary classification benchmarks.
The paper proposes a method to assess when automated predictions are reliable.
problem Ensuring reliability and safety of automated decision-making in machine learning.
method Clustering to measure distances between outputs and class centroids, defining a safety threshold based on these distances.
result The proposed metric can efficiently determine when automated predictions are acceptable and when they should be deferred.
Paper tackles machine performance testing under uncertain inputs.
problem Guarantee machine performance under input uncertainty.
method Formulates as IU-rLSE problem, proposes active learning method.
result Efficient algorithm for reliable level set estimation.
Although many successful ensemble clustering approaches have been developed in recent years, there are still two limitations to most of the existing approaches. First, they mostly overlook the issue of uncertain links, which may mislead the overall consensus process. Second, they generally lack the ability to incorpora…
This paper calibrates Gaussian process predictive distributions for Bayesian optimization to improve sampling decisions.
problem Lower-tail miscalibration in GP predictive distributions affects BO sampling decisions.
method Introduces goal-oriented calibration for GP predictive distributions below a threshold t. result Post-hoc method tcGP improves lower-tail calibration and BO performance.
In this paper, a novel joint transmit power and resource allocation approach for enabling ultra-reliable low-latency communication (URLLC) in vehicular networks is proposed. The objective is to minimize the network-wide power consumption of vehicular users (VUEs) while ensuring high reliability in terms of probabilisti…
Bayesian methods detect clusters in noisy data more reliably.
problem Noisy data distorts traditional clustering methods, leading to unreliable results.
method Bayesian community detection using Minimum Description Length principle.
result Bayesian methods identify more robust clusters in noisy data.
Multiple hypothesis testing is a significant problem in nearly all neuroimaging studies. In order to correct for this phenomena, we require a reliable estimate of the Family-Wise Error Rate (FWER). The well known Bonferroni correction method, while simple to implement, is quite conservative, and can substantially under…
Detects jumps in financial asset prices with U-shape volatility.
problem Identifying jumps in financial asset prices with varying volatility.
method Threshold method applied to five-minute log-returns.
result Visualized jumps and volatility patterns for Apple Inc. (AAPL) stock.
Develops a continuous compliance index for Islamic equity screening.
problem Binary rulebooks lead to inconsistent compliance assessment of firms.
method Integrates six leading financial and business activity standards into a single continuous index.
result Firms with the same pass/fail label can differ significantly in compliance strength.
Estimating the leading principal components of data, assuming they are sparse, is a central task in modern high-dimensional statistics. Many algorithms were developed for this sparse PCA problem, from simple diagonal thresholding to sophisticated semidefinite programming (SDP) methods. A key theoretical question is und…
Contagion maps detect network structure in noisy data.
problem Detecting underlying manifold structure in noisy data.
method Using activation times in threshold contagions to map network nodes to high-dimensional space.
result Contagion maps reliably detect manifold structure in noisy data, while Isomap fails.
Adaptive BO improves solder joint reliability by 3% with half the computational cost.
problem Improving solder joint reliability under thermomechanical loading.
method Adaptive Bayesian optimization with Gaussian process regression.
result Adaptive BO outperforms regular BO by 3% on average at any given computational budget.
Self-paced learning and hard example mining re-weight training instances to improve learning accuracy. This paper presents two improved alternatives based on lightweight estimates of sample uncertainty in stochastic gradient descent (SGD): the variance in predicted probability of the correct class across iterations of …
Locus scores predictions for risk, reducing large-loss events.
problem Deployment cost from inaccurate predictions, especially large losses.
method Distribution-free loss-scale reliability score using any predictive distribution.
result Reduces large-loss frequency compared to standard heuristics.
The paper studies early stopping methods in linear contextual bandits.
problem Minimizing in-experiment regret and conducting robust post-experiment inferences in contextual bandits.
method The study proposes early stopping rules based on the Opportunity Cost and Threshold Method, using variances of estimators to quantify upper regret bounds.
result The proposed method provides a systematic approach to minimize in-experiment regret and conduct robust post-experiment inferences.
Paper presents a probabilistic model to improve LLM cascade performance.
problem Complexity of LLM cascades and their interaction error rates.
method Probabilistic model for joint performance distribution of LLMs.
result Improves area under the error-cost curve by 4.3% on average for cascades with k≥3 models.
Since the events of the Arab Spring, there has been increased interest in using social media to anticipate social unrest. While efforts have been made toward automated unrest prediction, we focus on filtering the vast volume of tweets to identify tweets relevant to unrest, which can be provided to downstream users for …
Study detects signals in spiked Wigner models using log likelihood ratio.
problem Detecting signals in rank-one spiked Wigner models with non-Gaussian noise.
method Proved asymptotic normality of log likelihood ratio and computed error thresholds.
result Optimal signal-to-noise ratio threshold for reliable detection.
New method provides calibrated feature importance explanations for regression models.
problem Lack of uncertainty quantification in existing local explanation methods.
method Extension of Calibrated Explanations method to support regression and probabilistic regression.
result Calibrated Explanations for regression provides quantified uncertainty and robust explanations.
Develops a minimax optimal estimator for system stability under distribution shift.
problem Ensuring system reliability under changes in the underlying environment.
method Minimax optimal estimation of stability defined in terms of acceptable performance degradation.
result Characterizes the minimax convergence rate and demonstrates practical utility.
The paper proposes a method to reliably select design algorithms for machine learning-guided design tasks.
problem Choosing the right design algorithm for machine learning-guided design tasks.
method Combining designs' predicted property values with held-out labeled data to reliably forecast characteristics of the label distributions produced by different design algorithms.
result The method is guaranteed to return design algorithms that yield successful label distributions.