The paper proposes using entropy to assess model confidence.
problem Discarded information in probability distributions hinders model confidence assessment.
method Entropy methods applied to assess model confidence.
result Theoretical explanation of confidence degradation in Naive Bayes model.
We study the tradeoff between computational effort and classification accuracy in a cascade of deep neural networks. During inference, the user sets the acceptable accuracy degradation which then automatically determines confidence thresholds for the intermediate classifiers. As soon as the confidence threshold is met,…
This work examines how adversarial robustness regularization affects neural network performance.
problem Adversarial robustness regularization leads to performance degradation on natural examples.
method Empirical analysis of neural network generalization and regularization effects.
result Adversarial robustness regularization causes neural networks to become less confident, leading to worse standard performance.
Risk monitoring detects when TTA models degrade at test time.
problem Detecting when TTA models degrade at test time.
method Extended risk monitoring tools based on sequential testing with confidence sequences.
result Demonstrated effectiveness of TTA monitoring framework across various datasets and methods.
Statistical test detects model degradations in optimized language models.
problem Detecting model degradations in optimized language models.
method Statistical hypothesis testing framework based on McNemar's test.
result Even small accuracy degradations (0.3%) can be attributed to actual degradations, not noise.
LLMs generate answers under incomplete context, and their uncertainty should scale with missing information.
problem Evaluating the quality of LLM answers under incomplete context.
method A controlled framework with varying context availability, and two uncertainty measures (sampling-based confidence and response entropy) evaluated on SQuAD.
result Response entropy increases with context removal and explains more variance in accuracy than confidence, suggesting it is a more responsive uncertainty measure.
SelectMix improves deep learning robustness against noisy labels.
problem Deep neural networks memorize noisy labels, degrading performance.
method Confidence-guided targeted sample mixing with soft labels.
result SelectMix consistently outperforms baseline methods on noisy label datasets.
RegMixMatch optimizes Mixup for semi-supervised learning by integrating high- and low-confidence samples.
problem Mixup degrades SSL performance by compromising artificial labels purity.
method RegMixMatch integrates high- and low-confidence samples, uses class-aware Mixup, and mitigates confirmation bias.
result RegMixMatch achieves state-of-the-art performance in SSL benchmarks.
A theorem for debiasing machine learning with finite sample guarantees.
problem Calculating confidence intervals for machine learning functionals.
method Debiased machine learning based on bias correction and sample splitting.
result Nonasymptotic debiased machine learning theorem with finite sample guarantees.
The paper develops adaptive confidence intervals for Efron's Gaussian two-groups model with unknown contamination.
problem Developing robust uncertainty quantification for Efron's Gaussian two-groups model with unknown contamination fraction.
method The approach involves Fourier-based certification procedures to find minimax-optimal adaptive confidence intervals.
result The minimax-optimal length of adaptive confidence intervals is polynomially worse than when contamination fraction is known.
Detects harmful distribution shifts in deployed models without false alarms.
problem Detecting harmful distribution shifts in deployed models without false alarms.
method Sequential tools for testing if the difference between source and target distributions leads to a significant increase in a risk function.
result Demonstrated the efficacy of the proposed framework through extensive empirical studies.
The study examines label smoothing to improve confidence calibration in fine-tuned LLMs.
problem Improving confidence calibration in fine-tuned large language models (LLMs) after instruction tuning.
method Examine various open-sourced LLMs, label smoothing, and custom kernel design.
result Label smoothing is effective in maintaining confidence calibration but faces challenges in large vocabulary LLMs.
Enhances deep networks robustness with data mollification and label smoothing.
problem Improving deep neural networks' robustness against corruptions.
method Coupling data mollification (image noising and blurring) with label smoothing.
result Improved robustness and uncertainty quantification on corrupted image benchmarks.
The US Census Bureau corrupts data to protect privacy, but we show how to clean and analyze it effectively.
problem Analyzing Census data with intentional corruption to maintain privacy.
method Formulated a semiparametric model, proposed data cleaning, estimation, and inference procedures.
result Demonstrated that data cleaning can maintain precision and provided theoretical and empirical support.
InfoSFT improves LLMs by focusing on informative, medium-confidence tokens.
problem Overfitting to unlikely samples and degradation of prior capabilities in SFT.
method InfoSFT uses a principled weighting scheme to concentrate learning signals on medium-confidence tokens.
result InfoSFT improves generalization and preserves pre-existing capabilities over vanilla SFT and likelihood-weighted baselines.
The genomic profile underlying an individual tumor can be highly informative in the creation of a personalized cancer treatment strategy for a given patient; a practice known as precision oncology. This involves next generation sequencing of a tumor sample and the subsequent identification of genomic aberrations, such …
Researchers show how to secretly train models with hidden data, detect usage with high confidence.
problem Protecting training data from traceability in large language models.
method Gradient-based optimization to learn secret sequences absent from training data.
result Secret sequences can be learned by models without performance degradation, detectable with high confidence.
PAC-MCTS addresses biased search in LLM-guided planning by dynamically pruning.
problem Systematic biases in LLMs lead to inefficient and unsafe search in deep planning tasks.
method Formulates node expansion as BAI under bounded bias, derives sample complexity bounds, and proposes PAC-MCTS for dynamic confidence bounds.
result PAC-MCTS improves robustness and efficiency by up to 78% fewer API evaluations and 3x higher sample efficiency.
Multi-expert L2D underfits more severely, requiring new methods.
problem Underfitting in multi-expert L2D settings.
method PiCCE (Pick the Confident and Correct Expert), a surrogate-based method.
result PiCCE effectively reduces multi-expert L2D to a single-expert-like problem, resolving underfitting.
Measures DNA quality degradation effects.
problem Identifying degraded DNA sequence data.
method Novel quality quantification based on intentional degradation effects.
result Quantified measures of degradation can be used for multiple purposes.
Modeling coating degradation with fewer data points.
problem Efficiently forecasting material degradation from high-frequency sensor data.
method Discrete degradation events using Hawkes processes.
result Forecasting future coating failure with superior performance.
Deep learning improves Hurst parameter estimation for fractional processes.
problem Estimating the Hurst parameter in fractional stochastic processes.
method Training Long Short-Term Memory (LSTM) networks on extensive datasets of fBm, fOU, and lfsm processes.
result LSTM outperforms traditional methods in fBm and fOU processes but has limited accuracy on lfsm.
Paper uses deep learning to model systems with degrading behavior.
problem Modeling systems with degrading hysteretic behavior and uncertainty.
method Uses low-fidelity data to train a deep operator network (DeepONet).
result Improves prediction error in degrading hysteretic systems with uncertainty.
Lifelong learning with deep neural networks is well-known to suffer from catastrophic forgetting: the performance on previous tasks drastically degrades when learning a new task. To alleviate this effect, we propose to leverage a large stream of unlabeled data easily obtainable in the wild. In particular, we design a n…
BNCR-GAN improves GANs to generate clean images from degraded inputs.
problem Generating clean images from blurred, noisy, and compressed degraded inputs.
method Multiple-generator model with image, blur-kernel, noise, and quality-factor generators, using masking architectures and adaptive consistency losses.
result BNCR-GAN effectively learns clean image generators from degraded images without degradation parameters.
Active learning suffers from biased non-response, which this paper addresses.
problem Active learning's effectiveness is compromised by biased non-response in real-world contexts.
method Proposes a cost-based correction to the sampling strategy, UCB-EU, to mitigate the impact of biased non-response.
result UCB-EU successfully reduces the harm from labelling non-response in many settings.
New method certifies risks of LLM outputs, improving accuracy and reliability.
problem Uncertain and incorrect outputs from large language models.
method Information-lift certificates using PAC-Bayes bounds and skeleton design.
result Achieves 77.0% coverage at 2% risk, outperforming baselines.
Adaptive AI delegation framework for dynamic decision authority allocation.
problem Dynamic allocation of decision authority to AI-generated recommendations under evolving evidence quality and uncertainty.
method Formulated as a Governance-Aware POMDP, using Bayesian inference for informational state estimation and sequential optimization for authority allocation.
result Sequential Bayesian governance provides the strongest general-purpose policy across AI-quality regimes, adapting to evolving evidence.
Framework optimizes battery storage for markets by separating long-term degradation from short-term market dynamics.
problem Intractable computation due to timescale mismatch between battery degradation and market dynamics.
method Approximate dynamic programming with value function approximation and pseudo-time encoding.
result Policy outperforms benchmarks in real-time market scenarios.
Unseen data conditions can inflict serious performance degradation on systems relying on supervised machine learning algorithms. Because data can often be unseen, and because traditional machine learning algorithms are trained in a supervised manner, unsupervised adaptation techniques must be used to adapt the model to…
Training a neural network for a classification task typically assumes that the data to train are given from the beginning. However, in the real world, additional data accumulate gradually and the model requires additional training without accessing the old training data. This usually leads to the catastrophic forgettin…
Paper develops a hybrid DNN approach for RUL prediction with adaptive drift.
problem RUL estimation challenges in practice, especially online update and uncertainty quantification.
method Hybrid DNN approach with Wiener-based-degradation model and adaptive drift. LSTM-CNN for trajectory prediction and Bayesian inference for adaptive drift.
result Superior accuracy in RUL prediction demonstrated on turbofan engines data.
SVGP KAN integrates uncertainty quantification into Kolmogorov-Arnold networks.
problem Uncertainty quantification in scientific machine learning models.
method Sparse variational Gaussian process inference with Kolmogorov-Arnold topology.
result Demonstrated ability to distinguish aleatoric and epistemic uncertainty in various scientific applications.
A method for inferring ground-truth signals from degraded sensor data.
problem Inferring ground-truth signals from multiple degraded sensor signals.
method Iterative correction of degraded signals using a Bayesian multi-sensor data fusion method.
result The method effectively infers ground-truth signals from noisy and degraded sensor data.
Paper uses neural networks to predict NOx emissions from gas turbines.
problem Predicting NOx emissions from degrading gas turbines.
method Applied neural network algorithm to model NOx emissions from nine process variables.
result Neural network model optimizes process variables for minimal NOx emissions.
Paper predicts bearing degradation stages for pharmaceutical industry maintenance.
problem Predicting when to maintain specific parts of production machines.
method AutoEncoder-based k-means segmentation of high-frequency vibration data.
result Framework generates reliable predictions for bearing degradation stages.
Paper proposes a new regularization method to prevent model degradation under distribution shifts.
problem Model performance degrades under distribution shifts.
method Supervised contrastive learning with heterogeneous similarity.
result The proposed method outperforms existing regularization methods on benchmark datasets.
Analyzes layer-wise quantization effects in neural networks.
problem Identifying and fixing degradation in quantized neural networks.
method Layer-wise quantization analysis framework.
result Local fixes can significantly reduce quantization degradation.
To discover powerful yet compact models is an important goal of neural architecture search. Previous two-stage one-shot approaches are limited by search space with a fixed depth. It seems handy to include an additional skip connection in the search space to make depths variable. However, it creates a large range of per…
The paper proposes a method to detect relevant model degradations without over-alerting.
problem Detecting meaningful changes in machine learning model performance over time.
method Sequential monitoring scheme accounting for temporal dependence and multiple testing issues.
result The proposed method outperforms benchmark methods in detecting relevant changes in model quality.
The literature on optimal reinsurance does not deal with how much the effectiveness of such solutions is degraded by errors in parameters and models. The issue is investigated through both asymptotics and numerical studies. It is shown that the rate of degradation is often O(1/n) as the sample size n of historical …
Deep learning predicts RNA degradation from crowdsourced data.
problem Predicting RNA degradation to improve thermostability.
method Crowdsourced machine learning competition on Kaggle.
result 41% of predictions matched experimental data, and models generalized to longer RNA molecules.
Proposes a federated learning approach for RUL prediction from nonparametric degradation and failure signals.
problem Cost-effective RUL prediction from limited, non-shared CM signals with unknown parametric forms.
method Joint modeling of nonlinear degradation signals and failure events using federated learning.
result Superior RUL prediction compared to alternatives, validated through simulations and real data.
Paper presents a probabilistic diagnostic model for identifying and treating supervised learning degradation issues.
problem Degradation problems in supervised learning, including class imbalance, overlapping, small-disjuncts, noisy labels, and sparseness.
method Develops a novel probabilistic diagnostic model to identify and treat degradation issues in supervised learning.
result Early and correct diagnosis of degradation issues allows for selecting appropriate remediation treatments and unbiased performance metrics.
Study optimizes HTL-free PSCs with MWCNTs, improving efficiency and stability.
problem Optimizing efficiency and degradation in HTL-free perovskite solar cells.
method Machine learning-driven framework integrating experimental validation and numerical simulations.
result Achieved RMSEs of 0.0179 and 0.0117 for efficiency and degradation, respectively.
Online monitor detects classifier drift and adapts predictions.
problem Silent degradation of classifier accuracy under distributional shift.
method Sliding-window KS statistic with calibrated alarm thresholds.
result 86.6% valid detection across various shift conditions.
This research improves asset life prediction by integrating deep learning with mixture distributions.
problem Predicting residual useful life for assets with multiple failure modes.
method Integrates mixture (log)-location-scale distribution with deep learning.
result Proposed models outperform existing methods in predicting residual useful life.
CRC improves multivariate forecasting accuracy without risking performance degradation.
problem Systematic errors and lack of guarantees in multivariate forecasters.
method CRC uses a causality-inspired encoder and hybrid corrector with a safety mechanism.
result CRC consistently improves accuracy and ensures high non-degradation rates.