Detects changes in classifier scores to identify shifts in class priors.
problem Label shift changes in classification data.
method Sequential changepoint detection of classifier scores.
result Outperforms other detection procedures in label shift settings.
M-FISHER detects and adapts to streaming data shifts with statistical validity and stability.
problem Detecting and adapting to distributional shifts in streaming data.
method Constructs an exponential martingale from non-conformity scores and applies Ville's inequality for detection. Fisher-preconditioned updates for adaptation.
result Establishes M-FISHER as a principled approach for robust, anytime-valid detection and geometrically stable adaptation.
This paper tackles sequential distribution shifts in representation learning.
problem Learning meaningful representations in a sequence of distribution shifts.
method Nonlinear Independent Component Analysis (ICA) framework for continual causal representation learning.
result The method achieves performance comparable to joint training on multiple offline distributions and shows no benefit from the incoming new distribution on all latent variables.
In this paper, we investigate the multi-variate sequence classification problem from a multi-instance learning perspective. Real-world sequential data commonly show discriminative patterns only at specific time periods. For instance, we can identify a cropland during its growing season, but it looks similar to a barren…
Detects harmful distribution shifts in deployed models without false alarms.
problem Detecting harmful distribution shifts in deployed models without false alarms.
method Sequential tools for testing if the difference between source and target distributions leads to a significant increase in a risk function.
result Demonstrated the efficacy of the proposed framework through extensive empirical studies.
Proposes real-time risk monitoring for machine learning systems under unknown shifts.
problem Dynamic distribution shifts challenge real-world machine learning systems' risk assurances.
method Sequential hypothesis testing with 'testing by betting' to detect risk violations.
result Effective real-time risk monitoring under various unknown shifts.
This paper examines how adversarial perturbations affect model performance and equilibrium learning.
problem Adversarial perturbations and covariate shifts impact model performance and equilibrium learning.
method Characterizes the extrapolation region in regression and classification, analyzes dynamics of adversarial learning games.
result Establishes two directional convergence results: a blessing in regression and a curse in classification.
Paper proves conformal prediction works for any data distribution.
problem Quantifying risk in AI systems with non-exchangeable data.
method Developed a method to extend conformal prediction to any data distribution.
result Valid conformal prediction guarantees for any data distribution.
FADE adapts machine learning models to evolving data efficiently.
problem Sequential covariate shift in dynamic environments.
method FADE uses Fisher information geometry for robust learning under SCS.
result FADE achieves up to 19% higher accuracy under severe shifts.
This work tackles sequential data learning challenges by improving neural network robustness to non-iid distribution shifts.
problem Sequential data learning challenges, particularly non-iid distribution shifts across batches.
method Cramér-Rao-based regularization using Fisher Information Matrix to adapt to sequential covariate shifts.
result Achieves 19% accuracy improvement over state-of-the-art methods.
Detects harmful shifts without labels for model performance.
problem Detecting distribution shifts without access to labels.
method Uses a proxy derived from predictions of an error estimator.
result High power and false alarm control under various shifts.
Online monitoring system for safety classifiers with shift detection and conformal adaptation
problem Detecting and adapting to distributional shifts in deployed safety classifiers
method Calibrated sequential statistics for online monitoring, conformal abstention for adaptation
result 86.6% valid detection with mean latency of 39.5 steps
New approach for semi-supervised learning under covariate shifts.
problem Semi-supervised learning under covariate shifts where labeled and unlabeled data distributions differ.
method Information-theoretical approach, addressing covariate shifts.
result Improved performance compared to previous methods.
New framework TDRL identifies latent causal variables from sequential data.
problem Identify latent causal variables from sequential data.
method Proposes TDRL framework to recover time-delayed latent causal variables and identify their relations from measured sequential data.
result Identifies latent causal variables reliably from sequential data.
Risk monitoring detects when TTA models degrade at test time.
problem Detecting when TTA models degrade at test time.
method Extended risk monitoring tools based on sequential testing with confidence sequences.
result Demonstrated effectiveness of TTA monitoring framework across various datasets and methods.
Anytime-valid confirmation of label-shift corrections
problem Small-batch scientific deployments with scarce labeled outcomes
method Conditional e-value and martingale-based rule
result Nonnegative martingale and anytime-valid confirmation rule
This paper presents the construction of a particle filter, which incorporates elements inspired by genetic algorithms, in order to achieve accelerated adaptation of the estimated posterior distribution to changes in model parameters. Specifically, the filter is designed for the situation where the subsequent data in on…
A new method detects distribution shifts faster than existing CTMs.
problem Detecting distribution shifts in data streams with contamination issues.
method Uses a fixed reference dataset to compare each new sample, avoiding contamination.
result Detects distribution shifts faster and more reliably than standard CTMs.
Model change points in time-series data with neural SDEs and variational autoencoders.
problem Modeling change points in time-series data with neural stochastic differential equations.
method Proposes a novel model formulation and training procedure based on the variational autoencoder framework, alternating between updating neural SDE parameters and change points.
result Demonstrates the expressive power of the proposed model in modeling both classical parametric SDEs and real datasets with distribution shifts.
This paper tackles continuous covariate shift by adaptively training predictors.
problem Continuous covariate shift where input distributions change over time.
method Online density ratio estimation method to adaptively train predictors.
result Excess risk guarantee for the predictor through dynamic regret bound.
Adaptive monitoring for AI systems detects and diagnoses shifts in data distribution.
problem Continuous monitoring of AI systems to detect and address unsafe behavior.
method Weighted-conformal martingales (WCTMs) for online monitoring of AI systems.
result Improved performance over state-of-the-art baselines on real-world datasets.
This research improves online learning by correcting for target shift in machine learning.
problem Online learning struggles with distributional shift, especially in target values.
method Derives closed-form expressions for online and offline learning, and target correction.
result Online kernel-based learning can learn the same predictor as offline learning with target correction.
Gaussian processes help in modeling complex, nonlinear relationships in signal processing.
problem Modeling complex, nonlinear relationships in signal processing.
method Sequential inference for Gaussian processes.
result Gaussian processes enable efficient and accurate modeling of complex relationships.
Paper analyzes impact of PRM on binary random variables and distribution shifts.
problem Impact of performative risk minimization on binary random variables and distribution shifts.
method Formulated two measures of impact, derived explicit formulas for full information, and provided estimators for partial information.
result PRM can have amplified side effects compared to methods that do not model data shift.
Sequential Kernel-based Conditional Independence Testing via Adaptive Betting
problem Testing conditional independence
method Testing-by-betting on an adaptively optimized Kernel Conditional Independence statistic
result Significantly reduces Type I error inflation while preserving high power
Efficiently audits model fairness with continuous monitoring and flexible data collection.
problem Continuous monitoring and flexible data collection for fairness auditing.
method Sequential, anytime-valid inference and game-theoretic statistics.
result Demonstrated efficacy on three fairness datasets.
We develop the setting of sequential prediction based on shifting experts and on a "smooth" version of the method of specialized experts. To aggregate experts predictions, we use the AdaHedge algorithm, which is a version of the Hedge algorithm with adaptive learning rate, and extend it by the meta-algorithm Fixed Shar…
Novel framework synthesizes stochastic trajectories with anticipated structural breaks.
problem Synthesizing forward-looking, time-evolving stochastic trajectories with anticipated structural breaks.
method Anticipatory Neural Jump-Diffusion (ANJD) flow, AVNSG for dynamic spectral whitening.
result The framework effectively captures non-commutative moments and high-order stochastic texture.
LatentTrack generates model parameters online for nonstationary data.
problem Online probabilistic prediction under nonstationary dynamics.
method Sequential neural architecture with latent filtering and amortized inference.
result Consistently lower negative log-likelihood and mean squared error than baselines.
This paper discovers classification models from sequential data without prior knowledge.
problem Lack of prior knowledge in defining kernels for online classification.
method Adapts GP-based time-series structure discovery with SMC to learn new features from sequential data.
result Improves classification accuracy by 10% on real-world data.
We address challenges in collaborative black-box optimization through three frameworks.
problem Challenges in distributed experimentation, heterogeneity, and privacy in black-box optimization.
method Three unifying frameworks: global, local, and predictive.
result Shift from descriptive/predictive to prescriptive federated learning in black-box optimization.
Single-pass method estimates neural network uncertainty.
problem Uncertainty estimation in deep learning requires multiple passes.
method Probabilistic reasoning over neural network depths.
result Single forward pass for uncertainty estimation.
Single model estimates uncertainty via biased data shifts.
problem Estimating uncertainties in deep neural networks.
method Trivial input transformation to approximate ensemble behavior.
result Single model uncertainty estimates are superior to current methods.
C-kNN-LSH identifies similar patient histories for causal inference in longitudinal data.
problem Estimating causal effects from longitudinal trajectories with high-dimensional confounding.
method C-kNN-LSH uses locality-sensitive hashing to find clinical twins and estimate treatment effects.
result C-kNN-LSH outperforms existing methods in capturing recovery heterogeneity and estimating policy values.
Develops locally private methods for nonparametric contextual bandits.
problem Privacy concerns in sequential decision-making on sensitive data.
method Uniform-confidence-bound-type estimator and jump-start scheme.
result Minimax optimality of proposed methods supported by lower bounds.
A sequential learning framework improves domain generalization performance.
problem Training models robust to domain shift across multiple domains.
method Inspired by lifelong learning, a sequential training approach optimizing for all following domains.
result Improves performance on DG benchmarks with a simple, fast algorithm.
Optimizes mobile notifications for multiple objectives using reinforcement learning.
problem Optimizing mobile notification systems for multiple objectives.
method End-to-end offline reinforcement learning with Double Deep Q-network and Conservative Q-learning.
result Demonstrates improved performance and benefits of the proposed approach.
In this work, we propose a novel probabilistic sequence model that excels at capturing high variability in time series data, both across sequences and within an individual sequence. Our method uses temporal latent variables to capture information about the underlying data pattern and dynamically decodes the latent info…
New method improves ABI for sequential data, reducing forgetting and improving accuracy.
problem Performance degradation of ABI under model misspecification and distribution shifts.
method Decouples simulation-based pre-training from unsupervised SC fine-tuning, using memory buffer and elastic weight consolidation.
result Significant mitigation of forgetting and improved posterior estimates compared to standard simulation-based training.
New framework uses OR to ensure AI systems make safe decisions.
problem Ensuring generative AI systems make safe decisions as they gain autonomy.
method Developed a conceptual framework combining flow-based models and adversarial robustness.
result Increased autonomy requires new OR approaches for feasibility, robustness, and stress testing.
Framework predicts melt pool geometry with AI, improving manufacturing quality.
problem Achieving consistent product quality in Metal Additive Manufacturing.
method Surprise-guided sequential learning framework integrating CTGAN for limited data.
result Enhanced predictive accuracy for melt pool dimensions.
Combines experimental and historical data for robust policy evaluation.
problem Policy evaluation with mixed data sources, especially experimental vs historical.
method Linear integration of estimators from experimental and historical data, optimized for MSE minimization.
result Proposed estimators outperform traditional methods in ridesharing company data.
This paper improves change-point detection for complex data streams using denoising score matching.
problem Timely identification of distributional shifts in high-dimensional, complex data streams.
method Score-based CUSUM change-point detection with denoising score matching.
result Denoising score matching enhances detection power by effectively controlling noise scale.
Deep learning complements OR/MS for decision-making under uncertainty.
problem Sequential decision-making in uncertain environments.
method Integration of deep learning and OR/MS frameworks.
result Deep learning enhances adaptability and scalability in decision systems.
A new framework detects forecast model inadequacies using online monitoring of forecast errors.
problem Inaccurate forecasts lead to poor decision-making in complex models.
method Sequential changepoint techniques on forecast errors for real-time identification of process changes.
result The framework identifies shifts in forecast errors faster than in the original models, indicating process changes.
In this paper, we propose a robust sequential learning strategy for training large-scale Recommender Systems (RS) over implicit feedback mainly in the form of clicks. Our approach relies on the minimization of a pairwise ranking loss over blocks of consecutive items constituted by a sequence of non-clicked items follow…
RAN model recognizes multiple activities from unlabeled sensor data.
problem Handling weakly labeled multi-activity data from wearable sensors.
method Recurrent Attention Networks (RAN) for sequential multi-activity recognition and localization.
result RAN model can infer multiple activities and determine activity locations from unlabeled data.
Unified framework for semi-supervised regression with misspecified models.
problem Estimating regression coefficients in conditional mean models with unlabeled data.
method Developed an augmented inverse probability weighted (AIPW) method using regularized calibrated estimators for PS and OR nuisance models.
result The proposed estimator is consistent, asymptotically normal, and provides valid confidence intervals even with misspecified OR models and high-dimensional data.