New similarity measure for covariate shift improves nonparametric regression rates.
problem Improving nonparametric regression under covariate shift.
method Introducing a new similarity measure based on probability ratios.
result Shows a sharper rate of convergence compared to transfer exponent.
Proposes a new measure to evaluate stability of statistical parameters under distributional shifts.
problem Difficulty in transferring knowledge across data sets due to distributional changes.
method Introduces a measure of instability quantifying sensitivity of statistical parameters to Kullback-Leibler divergence and directional shifts.
result The proposed measure can elucidate the type of shifts a parameter is sensitive to and improve estimation accuracy under shifted distributions.
Fairness measures fail in predictive settings that intentionally shift outcomes.
problem Fairness measures fail in performative prediction settings.
method Formalized concept shift and counterfactual outcomes.
result Predictors that are fair during training become unfair during deployment.
Causal Imitation Learning handles noisy measurements and distribution shifts.
problem Learning from noisy state observations and distributional shifts.
method Causal inference framework and adversarial RKHS learning.
result Improved robustness to distribution shifts compared to standard methods.
A new policy for contextual bandits adapts to reward vector shifts.
problem Learning under reward vector shifts with ordered rewards.
method Adaptive-discretization and optimistic elimination policy.
result Established upper bounds on preference-based regret.
Paper proposes a new regularization method to prevent model degradation under distribution shifts.
problem Model performance degrades under distribution shifts.
method Supervised contrastive learning with heterogeneous similarity.
result The proposed method outperforms existing regularization methods on benchmark datasets.
Develops a minimax optimal estimator for system stability under distribution shift.
problem Ensuring system reliability under changes in the underlying environment.
method Minimax optimal estimation of stability defined in terms of acceptable performance degradation.
result Characterizes the minimax convergence rate and demonstrates practical utility.
Paper introduces a novel measure to analyze excess error in classification under covariate shift.
problem Analyzing excess error in classification under covariate shift.
method Utilizes vicinity information to characterize excess error.
result Faster or competitive convergence rates compared to previous techniques.
In real supervised learning scenarios, it is not uncommon that the training and test sample follow different probability distributions, thus rendering the necessity to correct the sampling bias. Focusing on a particular covariate shift problem, we derive high probability confidence bounds for the kernel mean matching (…
Improved vehicle motion prediction with uncertainty estimation.
problem Robust motion prediction for autonomous vehicles, especially under distributional shift.
method Presented an approach significantly improving the benchmark and taking 2nd place on the leaderboard.
result Significantly improved motion prediction and uncertainty measurement.
BREEDS benchmarks assess model robustness to subpopulation shifts.
problem Measuring model robustness to novel subpopulation shifts.
method Controlled synthesis of realistic distribution shifts using class structure.
result Validated model sensitivity and effectiveness of robustness interventions.
FLUXtrapolation benchmarks machine learning for extrapolating ecosystem fluxes under distribution shifts.
problem Machine learning challenges in extrapolating ecosystem fluxes under distribution shifts.
method Defined temporal, spatial, and temperature-based extrapolation scenarios; evaluated performance across domains, temporal aggregations, and tail errors.
result Baselines perform similarly under median hourly RMSE but differ under tail-focused and multi-scale evaluations.
Study quantifies distribution shifts and uncertainties to improve machine learning model robustness.
problem Distribution shifts between training and test datasets impact model generalization and robustness.
method Synthetic data generation and quantitative measures (KL divergence, JS distance, Mahalanobis distance) to assess data similarity and model uncertainty.
result Utilizing statistical measures like Mahalanobis distance helps assess distribution shift and model uncertainty.
FSL-Net detects and localizes feature shifts in large, high-dimensional datasets.
problem Feature shifts between data sources lead to erroneous features in various applications.
method FSL-Net is a neural network trained on multiple datasets to localize feature shifts.
result FSL-Net accurately localizes feature shifts from unseen datasets without re-training.
Extend CPS to non-exchangeable settings with observation-specific permutation weights
problem Calibrated predictive bands under distributional shifts
method Encoding distributional shifts through observation-specific permutation weights
result Shift-aware predictive systems remain valid
Study shows current image classification models lack robustness to real-world dataset shifts.
problem Robustness of current image classification models to natural distribution shifts in real datasets.
method Evaluation of 204 ImageNet models in 213 different test conditions.
result Little to no transfer of robustness from synthetic to natural distribution shifts.
A new protocol corrects confounding effects to measure alignment-induced activation shifts accurately.
problem Confounding effects in measuring alignment-induced activation shifts using naive methods.
method Introduces a four-variant decomposition to separate alignment shift from template effects.
result Correctly measures alignment-induced activation shifts, recovering behaviorally active subspace.
Detects domain shifts in datasets using interpretable feature subspaces.
problem Detecting subtle differences in dataset probability distributions.
method Localised density anomaly detection in high-dimensional feature spaces.
result Extracts interpretable feature subspaces for domain shifts.
Paper analyzes impact of PRM on binary random variables and distribution shifts.
problem Impact of performative risk minimization on binary random variables and distribution shifts.
method Formulated two measures of impact, derived explicit formulas for full information, and provided estimators for partial information.
result PRM can have amplified side effects compared to methods that do not model data shift.
This paper introduces a novel clustering algorithm for heteroscedastic Gaussian data without needing to know the number of clusters.
problem Clustering heteroscedastic Gaussian data without prior knowledge of the number of clusters.
method Introduces a novel cost function and fixed-point analysis to estimate centroids, introduces Wald kernel for measurement plausibility, and derives CENTRE-X algorithm.
result CENTRE-X algorithm can estimate centroids without prior knowledge of the number of clusters and performs comparably to standard algorithms K-means and Mean-Shift.
Bayes-consistent disagreement discrepancy loss improves model robustness.
problem Distribution shift in real-world neural network deployment.
method Introducing a novel disagreement loss that is Bayes consistent.
result Proves existing surrogates for disagreement discrepancy are not Bayes consistent.
Paper develops a new method to improve model calibration under distribution shifts.
problem Challenges in uncertainty quantification with different training and test distributions.
method Develops multi-domain temperature scaling to handle distribution shifts.
result Outperforms existing methods on in-distribution and out-of-distribution test sets.
Simple method improves uncertainty estimation for distribution shifts.
problem Improving uncertainty estimation in deep image classification under distribution shifts.
method Exposing original model to corrupted images and performing simple statistical calibration.
result Superior performance on various distribution shifts and unsupervised domain adaptation tasks.
The paper analyzes covariate shift in nonparametric regression with Markovian data.
problem Covariate shift in regression problems with Markovian data.
method Extension of nonparametric convergence rates to Markovian dependence structures, using Hölder smoothness assumptions and similarity measures.
result Precise convergence rates for Nadaraya-Watson kernel estimators under specific Markovian conditions.
The paper proposes a method to assess when automated predictions are reliable.
problem Ensuring reliability and safety of automated decision-making in machine learning.
method Clustering to measure distances between outputs and class centroids, defining a safety threshold based on these distances.
result The proposed metric can efficiently determine when automated predictions are acceptable and when they should be deferred.
CPATTA uses conformal prediction for efficient test-time adaptation.
problem Low data selection efficiency in existing ATTA methods.
method Conformal Prediction, online weight-update algorithm, domain-shift detector, staged update scheme.
result CPATTA consistently outperforms state-of-the-art methods by 5% in accuracy.
New method restores source features for SFDA without source data.
problem Domain adaptation without access to source data.
method Feature Restoration (FR) and Bottom-Up Feature Restoration (BUFR).
result BUFR outperforms existing SFDA methods in accuracy, calibration, and data efficiency.
Machine learning and data mining techniques have been used extensively in order to detect credit card frauds. However purchase behaviour and fraudster strategies may change over time. This phenomenon is named dataset shift or concept drift in the domain of fraud detection. In this paper, we present a method to quantify…
Study evaluates how well question-answering models generalize to new data types.
problem Generalization of question-answering models to new data types.
method Constructed new test sets from different domains and evaluated models' performance.
result Models show significant performance drops when tested on new data types.
MAGDiff detects data shifts in neural networks without retraining.
problem Neural networks' sensitivity to data distribution shifts.
method Extracts MAGDiff representations from neural networks to detect shifts.
result MAGDiff representations improve data set shift detection.
New measures quantify dependence between variables without distribution estimation.
problem Measuring dependence between variables in arbitrary dimensions.
method Proposed matrix-based normalized total correlation and dual total correlation measures.
result Measures are differentiable and statistically more powerful than existing methods.
The paper addresses instability in CNNs' first layer by proving max pooling's shift invariance.
problem Instability in CNNs' first layer, leading to sensitivity to small input shifts.
method Establishing conditions for max pooling's shift invariance and deriving a measure of stability.
result Max pooling approximates a nearly shift-invariant complex modulus under certain conditions.
KNF uses Koopman theory to forecast time series with changing dynamics.
problem Temporal distributional shifts in time series data.
method KNF combines DNNs with Koopman theory to learn dynamic operators.
result KNF outperforms alternatives on time series datasets with distributional shifts.
WR-CP reduces prediction set size and coverage gap under distribution shift.
problem Guaranteed coverage under distribution shift not achievable with i.i.d. assumption.
method Wasserstein distance, probability measure pushforwards, importance weighting, regularized representation learning.
result Reduces coverage gap to 3.2% across different confidence levels.
New techniques identify shifts in financial market sectors.
problem Identifying shifts in financial market structure and composition.
method Developed new mathematical techniques to identify nonlinear shifts in market sectors.
result Identified meaningful sector-to-sector mappings and optimal portfolio styles.
New framework to test neural network representation similarity measures.
problem Disagreements among dissimilarity measures in neural networks.
method Statistical testing framework to evaluate measures based on functional behavior.
result Current metrics have different weaknesses; a classical baseline performs surprisingly well.
Framework LiLY recovers latent causal variables from time-series data under distribution shifts.
problem Learning and correcting models under unknown distribution shifts in time-series data.
method LiLY framework that recovers latent causal variables and identifies their relations from temporal data under different distribution shifts.
result The framework reliably identifies time-delayed latent causal influences from observed variables under different distribution changes.
The paper tackles uncertainty quantification for classification under label shift without assuming i.i.d. data.
problem Uncertainty quantification for classification under label shift in non-i.i.d. settings.
method The paper uses conformal prediction and post-hoc binning for distribution-free UQ, and reweights these methods for label shift.
result The reweighted methods improve UQ performance under label shift, preserving coverage and calibration.
Validates composite systems using discrepancy propagation.
problem Validation of industrial systems with costly real-world tests.
method Propagates bounds on distributional discrepancy measures through a composite system.
result Derives upper bound on real system failure probability from simulations.
The paper introduces a method to decompose variance in twin networks for better treatment effect estimation.
problem Accurate treatment effect estimation requires reliable uncertainty measures to locate model failures.
method Layer-wise variance decomposition using Monte Carlo Dropout in twin networks.
result The encoder component dominates under distributional shift, providing a practical diagnostic for data collection.
TRACE analyzes risk changes in models trained on shifted data.
problem Understanding performance changes when a model trained on shifted data is used.
method TRACE framework decomposes risk change into four factors: generalization gaps, model change penalty, and covariate shift penalty.
result TRACE provides a diagnostic tool to understand and quantify risk changes due to covariate shift.
Unified framework for fairness, robustness, and distribution shifts.
problem Diverse failure modes of machine learning systems.
method Formalizes biases as violations of conditional independence and proves equivalence conditions.
result Equivalent effects of biases in different failure modes under specific conditions.
The paper analyzes how machine learning models perform under covariate shift, especially when the feature shift in x is larger than that in y.
problem Performance of machine learning models under covariate shift with heterogeneous feature changes.
method Empirical risk minimization (ERM) over functions f+g, fit on a training distribution, evaluated on a test distribution with covariate shift. result ERM is more resilient to heterogeneous covariate shifts when the class F is simpler than G. Proposes methods to aggregate prediction intervals for domain shift uncertainty.
problem Uncertainty quantification in distribution shifts.
method Aggregates prediction intervals for minimal width and adequate coverage.
result Effective methodologies for unsupervised domain shift under labeled source and unlabeled target.
Identifies directed graphs from node measurements using polynomial filters.
problem Inferring directed network topology from nodal measurements.
method System identification of graph convolutional filter followed by topology inference.
result Effective recovery of directed graphs from measurements.
Estimates proxy-based inference adjustments for distribution shifts.
problem Imperfect proxy data leads to biased inference.
method Empirical calibration of proxy-primary metric discrepancy as a random effect.
result Empowers inference without individual-level response data.
Prediction-time batch normalization improves model robustness under covariate shift.
problem Dealing with covariate shift in deep learning models.
method Prediction-time batch normalization, a simple but effective method.
result Significantly improves model accuracy and calibration under covariate shift.
CATS adapts multivariate time series models by addressing correlation shift.
problem Correlation differences across domains in multivariate time series data.
method CATS introduces correlation shift to measure domain differences, and uses a graph attention module and temporal convolution to align target correlations with source correlations.
result CATS increases over 10% average accuracy compared to vanilla Transformer-based models with minimal additional parameters.