SJS model predicts label shifts in multinomial datasets.
problem Predicting label shifts in multinomial datasets.
method Sparse joint shift model for dataset shift.
result Valid predictions and class prior probabilities estimates.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “prior probability shift”
Friedman's method performs well for estimating class distributions.
problem Estimating prior class probabilities without label observations.
method Friedman's method and DeBias method for designing linear equation systems.
result Friedman's method performs well for binary and multi-class quantification.
We introduce Fisher consistency in the sense of unbiasedness as a desirable property for estimators of class prior probabilities. Lack of Fisher consistency could be used as a criterion to dismiss estimators that are unlikely to deliver precise estimates in test datasets under prior probability and more general dataset…
Reduces quantifier variance with accuracy optimization of base classifier.
problem Minimizing quantifier variance under prior probability shift.
method Optimizes the Brier score of a base classifier for training data.
result Optimizing Brier score on training data reduces quantifier variance on test data.
The quantification problem consists of determining the prevalence of a given label in a target population. However, one often has access to the labels in a sample from the training population but not in the target population. A common assumption in this situation is that of prior probability shift, that is, once the la…
FJS method improves multinomial classification accuracy.
problem Improving multinomial classification accuracy under dataset shift.
method Derive FJS representation and propose alternative methods.
result Factorizable joint shift is not fully identifiable without additional assumptions.
Label shift refers to the phenomenon where the prior class probability p(y) changes between the training and test distributions, while the conditional probability p(x|y) stays fixed. Label shift arises in settings like medical diagnosis, where a classifier trained to predict disease given symptoms must be adapted to sc…
New CPS model tackles conditional probability shift in machine learning.
problem Discrepancy between source and target distributions in machine learning.
method Conditional Probability Shift Model (CPSM) using multinomial regression and EM algorithm.
result Superior balanced classification accuracy on target data compared to existing methods.
Proposes robust ITRs integrating multiple datasets to handle posterior shift.
problem Posterior shift in conditional outcome distributions between source and target populations.
method Distributionally robust approach with closed-form solution and adaptive uncertainty tuning.
result Achieves superior performance compared to existing methods in simulations and real-data applications.
Extends FJS analysis to general label spaces, including classification and regression.
problem Distribution shift in general label spaces, including covariate and label shifts.
method Proposes a framework for analyzing FJS in general label spaces and generalizes existing results.
result Generalizes FJS analysis to general label spaces, including classification and regression.
Estimates class prior for unlabeled data using kernel embedding.
problem Estimating class prior in PU learning scenario where only positive and full population samples are available.
method Direct estimator based on distribution matching and kernel embedding in Reproducing Kernel Hilbert Space.
result Asymptotic consistency and explicit deviation bound for the estimator.
The paper analyzes recalibration methods for binary classifiers under distribution shift.
problem Recalibrating binary classifiers to match a target prior probability.
method Analysis of distribution shift assumptions and proposal of new recalibration methods.
result QMM methods provide conservative results for risk weights functions.
Simple method improves uncertainty estimation for distribution shifts.
problem Improving uncertainty estimation in deep image classification under distribution shifts.
method Exposing original model to corrupted images and performing simple statistical calibration.
result Superior performance on various distribution shifts and unsupervised domain adaptation tasks.
Framework improves classifier calibration under differential privacy for domain shift.
problem Improving classifier calibration under domain shift with privacy constraints.
method Differential privacy framework for adapting recalibration algorithms.
result Novel accuracy temperature scaling algorithm outperforms existing methods on private datasets.
Improves model robustness to shifts in subpopulations.
problem Poor performance of ML models under data distribution shifts.
method Develops group-aware priors (GAP) over neural network parameters.
result Training with GAP yields state-of-the-art performance.
In safety-critical applications of machine learning, it is often important to abstain from making predictions on low confidence examples. Standard abstention methods tend to be focused on optimizing top-k accuracy, but in many applications, accuracy is not the metric of interest. Further, label shift (a shift in class …
New method needed for class prior estimation when covariates are reduced.
problem Class prior estimation fails under covariate shift when covariates are reduced.
method Propose a probing algorithm for class prior estimation.
result Provable transformations preserving covariate shift are necessary for class prior estimation.
Linear trends in classifier accuracy observed under distribution shift.
problem Understanding why classifier accuracies show linear trends under distribution shift.
method Assumed model similarity and verified empirically.
result Linear trend in classifier accuracy occurs unless distribution shift is large.
Estimates calibration error under label shift without labels.
problem Ensuring model reliability in the face of dataset shift without access to labels.
method Importance re-weighting of the labeled source distribution to estimate calibration error under label shift.
result Effective and reliable CE estimation with respect to the shifted target distribution.
CBDL uses credal sets to improve uncertainty quantification in deep learning.
problem Uncertainty in predictions and robustness to distribution shifts in deep learning.
method Train an infinite ensemble of Bayesian Neural Networks using credal sets.
result CBDL distinguishes between aleatoric and epistemic uncertainties and quantifies them better than single BNNs.
A new method for adapting to label shifts using class probability matching.
problem Adapting to label shifts where class probabilities differ between source and target domains.
method Class Probability Matching using Kernel Methods (CPMKM) framework.
result CPMKM outperforms existing methods on real datasets.
GS-BSE improves label shift estimation by smoothing priors on a graph.
problem Label shift adaptation when source and target distributions share conditional but not marginal probabilities.
method Graph-Smoothed Bayesian Black-Box Shift Estimator (GS-BSE) places Laplacian-Gaussian priors on log-priors and confusion-matrix columns tied by a label-similarity graph.
result GS-BSE produces a tractable posterior with HMC or Newton-CG schemes, proving identifiability, contraction, and robustness.
Bayesian framework estimates label shift for improved classifier performance.
problem Label shift in supervised learning leading to degraded classifier performance.
method Bayesian framework with dynamic Dirichlet priors and online EM algorithms.
result Significant improvements in classifier accuracy over state-of-the-art methods.
Bayesian method adapts to unknown distribution shifts in online learning.
problem Online learning with unknown and irregular distribution shifts.
method Bayesian inference with change-point detection and beam search.
result Improves adaptation to new data distributions over state-of-the-art methods.
Efficiently estimates mean in contaminated Gaussian data with near-optimal sample complexity.
problem Robust mean estimation in the presence of mean-shift contamination.
method First computationally efficient algorithm with near-optimal sample complexity and polynomial-time running.
result Approximates the target mean to any desired accuracy with constant fraction of outliers tolerated.
Point estimation of class prevalences in the presence of data set shift has been a popular research topic for more than two decades. Less attention has been paid to the construction of confidence and prediction intervals for estimates of class prevalences. One little considered question is whether or not it is necessar…
Bayesian framework improves uncertainty estimates under covariate shifts.
problem Neural networks' unreliable uncertainty estimates under covariate shifts.
method Adaptive prior conditioned on training and new covariates, amortized variational inference.
result Significantly improved uncertainty estimates under distribution shifts.
WR-CP reduces prediction set size and coverage gap under distribution shift.
problem Guaranteed coverage under distribution shift not achievable with i.i.d. assumption.
method Wasserstein distance, probability measure pushforwards, importance weighting, regularized representation learning.
result Reduces coverage gap to 3.2% across different confidence levels.
New method for estimating class proportions in open-set label shift data.
problem Estimating class proportions and distributions when test data includes novel classes.
method Semiparametric density ratio model framework with maximum empirical likelihood estimators and confidence intervals.
result Improved estimation accuracy and classification performance compared to existing methods.
Theory for RLHF generalization under reward shift and clipped KL.
problem Theoretical understanding of RLHF generalization, especially with reward shift and clipped KL.
method Developed generalization theory for RLHF, accounting for reward shift and clipped KL.
result Presented generalization bounds for RLHF, suggesting generalization error from sampling, reward shift, and KL clipping.
Training models to prefer certain responses can unintentionally shift probability to harmful ones.
problem Likelihood displacement in DPO models, leading to unintended unalignment.
method Characterized and mitigated likelihood displacement using CHES score.
result Training models to prefer certain responses can unintentionally shift probability mass to harmful responses.
Proposes a FoE prior for improving CNN performance in distribution shifts.
problem Improving CNN performance in image analysis tasks with distribution shifts.
method Uses a field-of-experts (FoE) prior to match feature distributions of test and training images.
result Outperforms previous TTA methods in lesion segmentation and most healthy tissue segmentation tasks.
MDL principle aids in learning neural network-based causal structures.
problem Learning causal relationships from observations with neural networks.
method Prequential minimum description length (MDL) principle.
result Competitive results on synthetic and real-world data, often recovering correct structure.
This paper establishes non-asymptotic learning bounds for the DR covariate shift adaptation.
problem Distribution shift between training and test domains in machine learning.
method Doubly-robust (DR) estimator combining density ratio estimation and pilot regression model.
result First non-asymptotic learning bounds for DR covariate shift adaptation.
Drug Discovery under Covariate Shift with Domain-Informed Prior Distributions over Functionsq-bio.BM
Q-SAVI model improves drug discovery accuracy with prior knowledge of chemical space.
problem Challenges in drug discovery due to covariate shift and limited labeled data.
method Probabilistic model with domain-informed prior distributions over functions.
result Q-SAVI outperforms state-of-the-art techniques in predictive accuracy and calibration.
Bottlenecks of binary classification from positive and unlabeled data (PU classification) are the requirements that given unlabeled patterns are drawn from the test marginal distribution, and the penalty of the false positive error is identical to the false negative error. However, such requirements are often not fulfi…
New algorithm identifies best arm in semiparametric bandits with near optimal efficiency.
problem Fixed-confidence Best Arm Identification in semiparametric bandits with unknown baseline shift.
method Phase-elimination algorithm based on orthogonalized regression design.
result Nearly optimal high-probability sample-complexity upper bound established.
New method adapts to structural shifts in graph data for better label prevalence estimation.
problem Structural shifts in graph data affect label prevalence estimation.
method Importance sampling variant of KDEy quantification approach.
result Adapts to structural shifts and outperforms standard approaches.
Detects changes in classifier scores to identify shifts in class priors.
problem Label shift changes in classification data.
method Sequential changepoint detection of classifier scores.
result Outperforms other detection procedures in label shift settings.
Study evaluates methods for improving model robustness to various real-world distribution shifts.
problem Improving model robustness to real-world distribution shifts like geographic changes.
method Introduced new datasets and evaluated existing methods on four types of shifts (style, blurriness, location, camera operation).
result Data augmentations and larger models can improve robustness on real-world distribution shifts, contrary to prior claims.
Generative models help make decisions under changing data distributions.
problem Making decisions based on historical data when the actual data distribution changes.
method Flow- and score-based generative models to represent and transform distributions.
result Generative models can learn nominal uncertainty, create stressed distributions, and produce conditional distributions.
New method learns models to adapt to domain shifts at test time.
problem Learning models robust to distribution shifts in practical applications.
method Adaptive Risk Minimization (ARM) framework.
result Performance gains of 1-4% on image classification problems.
Drift-Resilient TabPFN learns to adapt to changing data distributions.
problem Real-world data often shifts over time, degrading model performance.
method In-Context Learning with a Prior-Data Fitted Network, using structural causal models.
result Significant performance improvements across various datasets.
CtrlNS learns latent factors and distribution shifts from sparse transitions without prior knowledge.
problem Lack of prior knowledge of domain variables limits causal temporal representation learning.
method Sparse transition assumption and identifiability results from theoretical perspective.
result Effective in identifying distribution shifts and latent factors without prior knowledge.
New method gives provable error bounds for neural nets under distribution shift.
problem Proving reliable error bounds for neural networks under distribution shift.
method Optimizing a classifier to disagree with another, using a new 'disagreement loss'.
result Valid error bounds with comparable accuracy to competitive methods.
In real supervised learning scenarios, it is not uncommon that the training and test sample follow different probability distributions, thus rendering the necessity to correct the sampling bias. Focusing on a particular covariate shift problem, we derive high probability confidence bounds for the kernel mean matching (…
ShapShift explains shifts in model predictions due to data distribution changes.
problem Prediction shifts caused by changes in input distribution.
method Subgroup Conditional Shapley Values applied to decision trees and ensembles.
result Simple, faithful, and near-complete explanations of prediction shifts across model classes.
Adaptive algorithm identifies best arm with abstention, showing phase transition from polynomial to exponential error probability.
problem Bayesian best-arm identification with abstention to reduce undetected error.
method Adaptive algorithm PGWS that optimally uses abstention budget.
result Introducing any positive abstention budget induces an exponential decay in undetected error probability.