Unified framework for robust A/B testing under model misspecification.
problem Improving sample efficiency in A/B testing with model misspecification.
method Unified framework for contextual bandit and dynamic settings, proving worst-case mean squared error bounds.
result Empirically validated approach using synthetic and real-world datasets.
New method improves experimental design under model misspecification.
problem Improving experimental design under time and budget constraints with model misspecification.
method Developed a new acquisition function (R-IDeA) that targets representativeness, informativeness, and de-amplification.
result The new method outperforms methods focusing on representativeness or informativeness alone.
This paper introduces novel backtests for the risk measure Expected Shortfall (ES) following the testing idea of Mincer and Zarnowitz (1969). Estimating a regression framework for the ES stand-alone is infeasible, and thus, our tests are based on a joint regression for the Value at Risk and the ES, which allows for dif…
Paper proposes efficient method for evaluating Bayesian models in imaging.
problem Evaluation of Bayesian models in imaging when ground truth is unavailable.
method Novel combination of Bayesian cross-validation and data fission for unsupervised model selection and misspecification detection.
result Achieved excellent selection and detection accuracy with low computational cost.
For many machine learning algorithms, two main assumptions are required to guarantee performance. One is that the test data are drawn from the same distribution as the training data, and the other is that the model is correctly specified. In real applications, however, we often have little prior knowledge on the test d…
Paper proposes kernel-based tests for model misspecification.
problem Determining if a model is misspecified.
method Minimum distance estimators based on MMD and KSD.
result Correct test level maintained without data splitting.
Study reveals model misspecification significantly impacts neural SBI algorithms.
problem Impact of model misspecification on neural SBI algorithms.
method Comprehensive study of neural SBI algorithms under various forms of model misspecification.
result Misspecification profoundly deleterious to performance of neural SBI algorithms.
New algorithm mitigates misspecification amplification in regression models with covariate shift.
problem Distribution shift and model misspecification in regression models.
method Developed a new algorithm inspired by robust optimization to avoid misspecification amplification.
result No misspecification amplification while still achieving optimal statistical rates.
Paper introduces RVNP to improve SBI in misspecified models.
problem Misspecification in simulation-based inference leads to unreliable posterior estimation.
method RVNP uses variational inference and error modeling to bridge the simulation-to-reality gap.
result RVNP can recover robust posterior inference without hyperparameters or priors.
New methods for CI testing under model misspecification.
problem Challenges in CI testing with misspecified models.
method Proposes new approximations and upper bounds for testing errors of regression-based CI tests.
result Introduces the Rao-Blackwellized Predictor Test (RBPT) robust against misspecified inductive biases.
UDA improves ABI robustness but fails under certain prior misspecifications.
problem Robustness of ABI in noisy real-world data.
method Systematic evaluation of UDA across various misspecification scenarios.
result UDA aligns summary spaces but can fail under prior misspecifications.
The paper examines skill estimation and variance under model misspecification in IRT.
problem Underestimation and overestimation of skills when non-compensatory model is misspecified as compensatory.
method Theoretical approach to analyze underestimation and overestimation of skills and variance.
result Overestimation of skills occurs around the origin and asymptotic variance differs under model misspecification.
DP models misspecify LF dependencies, leading to significant performance errors.
problem Misspecification of LF dependencies in DP models.
method Theoretical bounds and empirical analysis of modeling errors.
result Modeling errors can be substantial, even with sensible LF structures.
We introduce new forecast encompassing tests for the risk measure Expected Shortfall (ES). The ES currently receives much attention through its introduction into the Basel III Accords, which stipulate its use as the primary market risk measure for the international banking regulation. We utilize joint loss functions fo…
Alternative to likelihood-based LSNM model selection, residual independence testing is more robust to noise misspecification.
problem Cause-effect inference in location-scale noise models with misspecified noise distributions.
method Residual independence testing as an alternative to likelihood-based model selection.
result Residual independence testing is more robust to noise misspecification.
New testing method for robust actor-critic bandit algorithms.
problem Balancing data collection for app performance and user adherence.
method Modified actor-critic algorithm and novel testing procedure.
result Testing procedure is robust to critic misspecification.
The paper highlights the importance of model misspecification in uncertainty estimation.
problem The reliability of uncertainty estimates in machine learning models under model misspecification.
method Thought experiments and literature review.
result Model misspecification should be given more attention in uncertainty estimation.
We propose a nonparametric sequential test that aims to address two practical problems pertinent to online randomized experiments: (i) how to do a hypothesis test for complex metrics; (ii) how to prevent type 1 error inflation under continuous monitoring. The proposed test does not require knowledge of the underlying…
The paper shows that causal identification is not essential for efficient portfolios, focusing on geometric sufficiency conditions.
problem The necessity of causal identification for efficient portfolios.
method Re-examination of predictive signals and their impact on portfolio efficiency under structural misspecification.
result Efficiency is governed by geometric sufficiency conditions (directional alignment, ranking preservation, and calibration) rather than causal identification.
Optimized α-posteriors reduce KL divergence from true posterior in parametric misspecification.
problem Reduction of KL divergence from true posterior in parametric model misspecification.
method Derivation of Bernstein-von Mises theorem and optimization of α-posteriors. result Optimized α-posteriors minimize KL divergence from true posterior, especially in severe misspecification. Paper tackles SBI under model misspecification, presenting robust strategies.
problem Challenges in SBI under model misspecification.
method Three key strategies: robust summary statistics, generalised Bayesian inference, and error modelling.
result Empirical results show vulnerabilities of SBI and effectiveness of misspecification-robust alternatives.
Adapts to misspecification in contextual bandits using offline regression.
problem Unexpected regret due to misspecified reward models.
method Adapts to misspecification by reverting to a safe policy when necessary.
result Regret guarantees degrade gracefully with misspecification level.
Study robust hypothesis testing under Hellinger distance, proving lower bounds and providing tests.
problem Testing close variants of specified distributions robustly to Hellinger distance.
method Lower bound on slack factor, testing with Hellinger balls, symmetric chi-squared distance analysis.
result Lower bound on slack factor quantifies robustness under misspecification.
The paper investigates model misspecification in Bayesian inference using neural networks.
problem Detecting model misspecification in Bayesian inference with neural networks.
method Conceptualized types of model misspecification and proposed an augmented optimization objective with MMD.
result MMD can detect potentially catastrophic misspecifications in Bayesian inference.
FedGVI improves FL robustness to model misspecification.
problem Limited robustness in FL approaches to model misspecification.
method Probabilistic Federated Learning framework that generalizes previous methods.
result FedGVI provides robust and calibrated predictions under model misspecification.
New algorithms adapt to model misspecification in contextual bandits.
problem Design efficient algorithms for contextual bandits that handle model misspecification gracefully.
method Oracle-efficient algorithms for ε-misspecified contextual bandits using square loss regression.
result First algorithm achieving optimal regret bound for unknown misspecification level in linear contextual bandits.
Detects model misspecifications in causal models using observational data.
problem Identifying predictor variables with causal effects in misspecified models.
method Develops a general framework based on observational data distribution and proposes an algorithm for finite sample data.
result Identifies predictor variables for causal effects even in misspecified models.
Study improves convergence rates for GVI under prior misspecification.
problem Improving convergence rates for GVI under prior misspecification.
method Proves rates of convergence and robustness to prior misspecification in GVI framework.
result Establishes sufficient conditions for existence and uniqueness of GVI posteriors.
The paper tackles misspecification in contextual bandits by incorporating arm-specific variables.
problem Misspecification in contextual bandits due to unexplained inter-arm heterogeneity.
method Develops robust contextual bandit algorithms (RoLinUCB and RoLinTS) that incorporate arm-specific variables to address misspecification.
result The developed algorithms bound the n-round Bayes regret and show superior performance in various misspecification scenarios. New insights into bias-variance tradeoff for data-driven optimization under local misspecification.
problem Understanding the relative performance of SAA, IEO, and ETO under local misspecification.
method Developed a local misspecification perspective using contiguity theory in statistics.
result Explicit expressions for decision bias and geometric understanding of variance.
The paper improves SBI for BHMs by diagnosing misspecification and inferring parameters.
problem Model misspecification in Bayesian hierarchical models.
method Two-step framework: latent function diagnosis followed by SBI of target parameters.
result Improved simulation-based inference for complex models without explicit model checking.
RoPE framework calibrates misspecified simulators for reliable inference.
problem Misspecification compromises reliability of simulation-based inference.
method Data-driven calibration using optimal transport and a small calibration set.
result RoPE framework improves inference accuracy and uncertainty calibration.
Improved guarantees for misspecified kernelized bandit optimization.
problem Misspecification in kernelized bandit optimization.
method Localization and domain splitting techniques.
result Logarithmic or polylogarithmic growth of misspecification amplification.
Algorithm mitigates performance loss in constrained reinforcement learning with model misspecification.
problem Performance loss in reinforcement learning policies due to model misspecification in constrained control systems.
method Proposes an algorithm to handle constrained model misspecification in continuous control systems.
result Algorithm successfully mitigates performance loss in real-world reinforcement learning tasks.
Enhances predictive models against misspecification and outliers.
problem Suboptimal generalization under misspecification and outliers.
method Combines PACm ensemble bounds with a generalized logarithm score function. result Produces predictive distributions resistant to both misspecification and outliers.
Model misspecification is a long-standing enigma of the Bayesian inference framework as posteriors tend to get overly concentrated on ill-informed parameter values towards the large sample limit. Tempering of the likelihood has been established as a safer way to do updates from prior to posterior in the presence of mod…
Improved A/B testing by leveraging system similarities.
problem Traditional A/B testing ignores potential system similarities.
method Off-policy estimation to exploit system propensities.
result Improved A/B testing estimators achieve better accuracy.
Optimizes signal detection in particle physics by decorrelating classifiers.
problem Systematic errors in background models can mislead signal detection.
method Use optimal transport to decorrelate classifiers from protected variables, then apply semiparametric mixture model.
result Decorrelation and signal enrichment improve the stability, robustness, and power of signal detection tests.
Proposes a method to improve SBI under model misspecification.
problem Unreliable inference from SBI methods under model misspecification.
method Introduces a regularized loss function to penalize statistics that increase model-data mismatch.
result Demonstrates superior performance and robust inference in misspecified scenarios.
Calibration without labels in multiple testing
problem Interpretable error probabilities in large-scale hypothesis testing
method Constructing pseudo-labels from spacings of ordered p-values result Finding that q-value can be severely miscalibrated Unified framework for corruption-robust linear bandits with optimal gap-dependent misspecification bounds.
problem Effective learning in linear bandits with corrupted rewards across different corruption models.
method Unified framework for analyzing strong and weak corruption, connection to gap-dependent misspecification, and specialized algorithm.
result Optimal bounds for gap-dependent misspecification in linear bandits.
SBI provides more accurate pole positions than chi-squared minimization in model misspecification.
problem Accurate pole position estimation in pi-pi scattering models.
method Simulation Based Inference (SBI) method compared to chi-squared minimization.
result SBI leads to more robust predictions of pole positions in models of pi-pi scattering.
Optimal sequential testing for Markovian data with lower and upper bounds.
problem Sequential hypothesis testing for Markovian data.
method Non-asymptotic lower bounds and optimal test design.
result Optimal test matches lower bound asymptotically.
Variational Bayes (VB) is a scalable alternative to Markov chain Monte Carlo (MCMC) for Bayesian posterior inference. Though popular, VB comes with few theoretical guarantees, most of which focus on well-specified models. However, models are rarely well-specified in practice. In this work, we study VB under model missp…
New method improves simulation-based inference by avoiding model misspecification.
problem Inefficient parameter estimation for models with intractable likelihoods.
method Proposes a robust SNL method with additional adjustment parameters.
result Demonstrates more accurate point estimates and uncertainty quantification.
The paper examines how kernel approximations affect Gaussian process regression in large data applications.
problem Effect of kernel approximations on Gaussian process regression in large data applications.
method Unified framework to analyze Gaussian process regression under computational and epistemic misspecification.
result Theoretical analysis of Gaussian process regression under various misspecifications.
Improves estimation under model misspecification with fake features.
problem Model misspecification with fake features.
method Proposes a framework to decompose output error into underlying, fake, and missing features.
result Fake features can significantly improve estimation performance, even when not correlated with underlying features.
Bayesian regression underestimates parameter uncertainties in noisy models.
problem Parameter uncertainties are underestimated in Bayesian regression for imperfect models.
method Analyzed and designed an ansatz to correct for misspecification in near-deterministic surrogate models.
result Posterior distributions must cover all training points to avoid divergent generalization error.