Proposes FIPO-BC for efficient online calibration of complex models.
problem Efficiently calibrating computationally expensive models with large datasets.
method Fixed inducing points online Bayesian calibration (FIPO-BC) algorithm.
result FIPO-BC is at least ten times faster than standard methods and enables online updates.
Better boosting with bandits improves probability estimation in online learning.
problem Poor calibration of probability estimates from boosting ensembles in online learning.
method Use bandit optimization to decide between updating ensemble or calibrator parameters.
result Superior performance in probability estimation compared to uncalibrated and naively-calibrated online boosting.
Study online monotone density estimation with expert aggregation and log-optimal calibration.
problem Online monotone density estimation and log-optimal calibration.
method Proposed two online estimators: Grenander estimator and expert aggregation estimator.
result Online estimators achieve O ( n 1 / 3 ) O(n^{1/3}) O ( n 1/3 ) cumulative log-likelihood gap and n log n \sqrt{n\log{n}} n log n pathwise regret bound. MCNet improves uncertainty calibration in online advertising by modeling complex relations and balancing performance.
problem Lack of effective calibration for complex relations and context features in online advertising.
method Introduces MCNet with MCF, order-preserving, and field-balance regularizers.
result Superior performance in generating well-calibrated probability predictions on public and industrial datasets.
New algorithm achieves online calibration in polynomial time for high-dimensional problems.
problem Online calibration of high-dimensional probability distributions over many days.
method Randomly selects among sub-forecasters, each predicting empirical outcome frequency over recent time windows.
result Achieves asymptotically calibrated strategies after polynomial number of rounds, resolving open questions.
BRPC online Bayesian calibration handles gradual and abrupt system changes.
problem Aligning model outputs with field observations in evolving systems.
method Bayesian Recursive Projected Calibration (BRPC) for streaming data under simulator mismatch and nonstationarity.
result Improves calibration accuracy under gradual changes and robustness under abrupt regime shifts.
The paper addresses errors in online selective conformal prediction and proposes new strategies to ensure valid inference.
problem Online selective conformal prediction's exchangeability issues and false coverage rate control problems.
method Evaluation and correction of existing calibration selection strategies, proposing new ones that preserve exchangeability.
result Novel calibration selection strategies ensure both selection-conditional coverage and FCR control.
The paper proposes calibration to improve algorithm performance using machine learning predictions.
problem Improving real-world performance of online algorithms with machine learning predictions.
method Calibration as a tool to bridge the gap between prediction uncertainty and algorithm design.
result Calibrated advice leads to more effective guidance in high-variance settings and significant performance improvements in real-world data.
Paper proposes real-time VaR estimation using quantile regression forest with conformal calibration.
problem Real-time estimation of Value at Risk (VaR) in rapidly changing markets.
method Quantile regression forest trained offline, real-time VaR estimates via observed risk factors, conformalized estimator for reliability.
result The proposed method provides reliable real-time VaR estimates.
Post-processing predictors reduces calibration errors for decision-making.
problem Predictors with low calibration error for machine learning may have high error for decision-making.
method Post-processing with ε distance to calibration adds noise to make predictions differentially private.
result Post-processing achieves O(√ε) ECE and CDL, asymptotically optimal.
Online Platt Scaling adapts to varying data distributions.
problem Adapting Platt scaling to non-i.i.d. settings with distribution drift.
method Combines Platt scaling with online logistic regression and calibeating.
result OPS+calibeating method is guaranteed to be calibrated for adversarial outcomes.
New decision-theoretic calibration error metric improves prediction reliability.
problem Improving the reliability of predictions for decision-making.
method Proposed Calibration Decision Loss (CDL) and an efficient algorithm to achieve near-optimal CDL.
result Near-optimal CDL guarantees vanishing payoff loss from miscalibration.
Simple algorithm achieves distance to calibration error of at most 2√T+1.
problem Achieving distance to calibration error of O(√T) in adversarial setting.
method An extremely simple, efficient, deterministic algorithm.
result Obtains distance to calibration error at most 2√T+1.
C-PP-COAD detects anomalies with limited real data, reducing dependency on real calibration data.
problem Limited real calibration data for online anomaly detection.
method Context-aware prediction-powered conformal online anomaly detection (C-PP-COAD).
result Significantly reduces dependency on real calibration data without compromising FDR control.
The paper improves Bayesian optimization by calibrating uncertainty estimates.
problem Improper uncertainty estimates in Bayesian optimization when data is non-stationary.
method Proposes online learning algorithms to maintain calibration on non-i.i.d. data and integrates them into Bayesian optimization.
result Calibrated Bayesian optimization converges to better optima in fewer steps.
Paper proves tight lower bounds for online multicalibration, separating it from marginal calibration.
problem Proving lower bounds for online multicalibration in relation to marginal calibration.
method Information-theoretic approach, constructing group families from orthonormal bases.
result Establishes tight lower bounds for online multicalibration, matching upper bounds up to logarithmic factors.
Optimal multiclass U-calibration error found to be Θ(√KT).
problem Online multiclass U-calibration with low regret for all bounded proper losses.
method Follow-the-Perturbed-Leader algorithm and lower bound construction.
result Optimal U-calibration error is Θ(√KT).
CAP algorithm controls FCR in online selective prediction.
problem Online predictive tasks with temporal multiplicity and FCR control.
method CAP framework with adaptive pick rule and calibration set construction.
result CAP achieves exact selection-conditional coverage guarantee and FCR control.
Smooth calibration improves forecast reliability even with leaked information.
problem Improving forecast reliability with leaked information.
method Combining nearby forecasts to ensure smooth calibration, which can be guaranteed by deterministic procedures.
result Smooth calibration can be guaranteed by deterministic procedures even with leaked forecasts, and it yields uncoupled finite-memory dynamics in games.
A new method calibrates forecasts without sacrificing expertise.
problem Forecasters' calibration scores can be manipulated to appear expert.
method Deterministic and stochastic online procedures to calibrate forecasts.
result Calibration can be achieved without losing expertise.
New algorithm calibrates forecasts in high dimensions with minimal regret.
problem Online calibration of multi-dimensional forecasts over convex sets.
method Connects to external regret minimization for online linear optimization, using swap regret minimization.
result Obtains ε-calibrated forecasts with minimal rounds and without expert knowledge.
New method achieves faster calibration without randomization.
problem Calibrating probabilistic forecasts in adversarial settings.
method Using interval forecasts and the power of two choices.
result Achieves O ( 1 / T ) O(1/T) O ( 1/ T ) calibration error rate without randomization. Survey of algorithms to correct past mistakes in prediction.
problem Improving prediction accuracy by correcting past errors.
method Defensive Forecasting as a sequential game theory approach to minimize prediction metrics.
result Simple, near-optimal algorithms for various prediction tasks.
Optimizes predictions by recalibrating online forecasts with minimal error.
problem Tackles the challenge of recalibrating online predictions to be more accurate.
method Uses an imbalanced extension of the Blackwell approachability reduction framework to achieve ( ε , ε 2 ) (\varepsilon, \varepsilon^2) ( ε , ε 2 ) -recalibration. result Achieves ( ε , ε 2 ) (\varepsilon, \varepsilon^2) ( ε , ε 2 ) -recalibration for Lipschitz proper losses in T ≈ ε − 3 T \approx \varepsilon^{-3} T ≈ ε − 3 rounds. New algorithms adaptively calibrate predictions in non-stationary environments, matching optimal rates.
problem Designing online prediction algorithms that adapt to varying levels of non-stationarity.
method Epoch-based scheduling and non-uniform partitioning of the prediction space.
result Achieves adaptive calibration guarantees under multiple measures with optimal rates.
New method calibrates probabilistic linear solver for online coverage guarantees.
problem Uncertainty in probabilistic linear solver solutions without coverage guarantees.
method Online conformal prediction-PLS (OCP-PLS) method to calibrate uncertainty thresholds.
result Validates online calibration of uncertainty thresholds via online conformal prediction.
Reduces false positives in classifying rare online platforms.
problem Challenges in accurately identifying rare online platforms with ML.
method Calibrated probabilities and ensembles to reduce bias.
result Significantly reduces false positives in rare event detection.
Study shows sample complexity for multicalibration is Θ(ε^-3) with polylogarithmic factors.
problem Minimizing Expected Calibration Error (ECE) for predictors with respect to a family of groups.
method Proved necessary and sufficient sample complexity of Θ(ε^-3) for multicalibration, using online-to-batch reduction and lower bounds.
result Sample complexity of multicalibration is Θ(ε^-3) with polylogarithmic factors, distinguishing it from marginal calibration.
We introduce a local volatility model for the valuation of options on commodity futures by using European vanilla option prices. The corresponding calibration problem is addressed within an online framework, allowing the use of multiple price surfaces. Since uncertainty in the observation of the underlying future price…
Improved online algorithm for convex losses with near-optimal swap regret.
problem Minimizing swap regret for convex losses.
method Randomized online algorithm using multi-scale binning.
result Near-optimal O ~ ( T ) \widetilde O(\sqrt T) O ( T ) expected swap regret. Recently, a novel family of biologically plausible online algorithms for reducing the dimensionality of streaming data has been derived from the similarity matching principle. In these algorithms, the number of output dimensions can be determined adaptively by thresholding the singular values of the input data matrix. …
Develops a framework to control risk in online learning models.
problem Rigorous uncertainty quantification for online learning models.
method A framework for constructing uncertainty sets that provably control risk.
result Guarantees risk control at any user-specified level even with distribution shifts.
We present a universal algorithm for online trading in Stock Market which performs asymptotically at least as good as any stationary trading strategy that computes the investment at each step using a fixed function of the side information that belongs to a given RKHS (Reproducing Kernel Hilbert Space). Using a universa…
Develops a method to ensure accurate quantile forecasts across multiple levels.
problem Ensuring accurate quantile forecasts at multiple levels, even under distribution shifts.
method Multi-level quantile tracker (MultiQT) wraps around any forecaster to produce calibrated forecasts.
result Guaranteed calibration of quantile forecasts at multiple levels, even against adversarial shifts.
A new method for online prediction uncertainty quantification in non-exchangeable panel data.
problem Challenges in quantifying predictive uncertainty for non-exchangeable panel data.
method Online conformal prediction framework for non-exchangeable panel data, using similarity weights and adaptive miscoverage levels.
result Improves coverage on worst-covered target units through adaptive interval-width allocation.
New insights link no-regret learning to online conformal prediction in adversarial settings.
problem Understanding the relationship between no-regret learning and online conformal prediction in adversarial environments.
method Analysis of existing algorithms and new connections between no-regret learning and conformal prediction.
result No-regret learning algorithms can provide group-conditional coverage guarantees in adversarial settings.
Objective: Using traditional approaches, a Brain-Computer Interface (BCI) requires the collection of calibration data for new subjects prior to online use. Calibration time can be reduced or eliminated e.g.~by transfer of a pre-trained classifier or unsupervised adaptive classification methods which learn from scratch …
Improved conversion rate prediction in online advertising using self-supervised pre-training.
problem Data sparsity and calibration issues in predicting conversions given clicks.
method Self-supervised pre-training on all conversion events to enrich CVR prediction model without compromising calibration.
result Improvements in offline training and online A/B tests, with full deployment to Yahoo native advertising system.
WamOL uses PINNs to efficiently calibrate IVS from sparse data.
problem Calibrating time-dependent IVS from sparse market data.
method Physics-Informed Neural Networks (PINNs) with adaptive reweighting.
result WamOL outperforms in calibrating intraday IVS from uneven data.
Midicoth compresses online probability estimates by correcting prior smoothing biases.
problem Compression inefficiency due to prior smoothing in adaptive models.
method Micro-diffusion denoising applied in a bitwise tree hierarchy.
result Significant compression improvement with reliable calibration.
Improved upper bound for online calibrated forecasting of binary sequences.
problem Online calibrated forecasting of binary sequences.
method Introducing a variant of Qiao & Valiant's sign preservation game called sign preservation with reuse (SPR) and proving its equivalence to calibrated forecasting.
result Improved upper bound of O ( T 2 / 3 − ε ) O(T^{2/3 - \varepsilon}) O ( T 2/3 − ε ) for calibrated forecasting, improving the O ( T 2 / 3 ) O(T^{2/3}) O ( T 2/3 ) bound of Foster & Vohra. ORCA calibrates LLMs for efficient, generalizable reasoning.
problem Miscalibration of large language models leading to inefficiencies.
method Online Reasoning Calibration (ORCA) using conformal prediction and test-time training.
result ORCA provides higher efficiency and generalization across different reasoning tasks.
Paper proposes a method to predict MOBA game winners with calibrated confidence.
problem Predicting MOBA game winners with noisy data and uncertain noise.
method A novel confidence-calibration method considering data uncertainty.
result Achieves outstanding expected calibration error (ECE) of 0.57%.
LatentTrack generates model parameters online for nonstationary data.
problem Online probabilistic prediction under nonstationary dynamics.
method Sequential neural architecture with latent filtering and amortized inference.
result Consistently lower negative log-likelihood and mean squared error than baselines.
Parity calibration aims to predict increase-decrease events, not values.
problem Forecasting future increase-decrease events rather than exact values.
method Online binary calibration method to achieve parity calibration.
result Online binary calibration achieves parity calibration effectively.
Unified approach to fair online learning with stochastic contexts.
problem Fairness in online learning with unknown sensitive contexts.
method Adapting Blackwell's approachability theory to handle unknown contexts' distributions.
result Characterization of optimal trade-off between fairness and performance objectives.
Simple online monitor detects unsafe LLM outputs.
problem LLMs generate unsafe outputs despite training.
method Thresholding external verifier signal to decide alarms.
result Simple design competitive with advanced methods.
Paper shows equivalence between two alignment methods and introduces a new algorithm.
problem Ensuring human alignment of large language models for useful, safe, and pleasant user experience.
method Introduces IPO-MD algorithm, showing equivalence between IPO and Nash-MD methods.
result Equivalence between IPO and Nash-MD methods proven when considering online version of IPO.