VCoTTA uses variational Bayesian methods to adapt models under continuous domain shifts.
problem Error accumulation in continual test-time adaptation.
method VCoTTA employs variational Bayesian techniques to update a Bayesian Neural Network (BNN) during testing, combining priors from source and teacher models.
result VCoTTA effectively mitigates error accumulation in CTTA, as shown by experimental results on three datasets.
Risk monitoring detects when TTA models degrade at test time.
problem Detecting when TTA models degrade at test time.
method Extended risk monitoring tools based on sequential testing with confidence sequences.
result Demonstrated effectiveness of TTA monitoring framework across various datasets and methods.
PETAL adapts models to changing target domains over time.
problem Lifelong test-time adaptation in changing target domains.
method Probabilistic framework with student-teacher model and data-driven parameter restoration.
result PETAL achieves better results than state-of-the-art for online lifelong test-time adaptation.
In-Place TTT enhances LLMs with dynamic parameter updates at inference time.
problem Static training limits LLMs from adapting to new information.
method In-Place TTT updates a subset of model parameters (fast weights) at inference time.
result In-Place TTT enables 4B-parameter models to outperform on tasks with up to 128k contexts.
A probabilistic framework for online test-time adaptation
problem Adapting models to new data under distributional shift
method State-space modelling architecture
result Characterizing parameter learning, time evolution, prior tuning, and prediction
This paper improves test-time adaptation for distribution shifts using confidence maximization and input transformation.
problem Improving deep networks' performance on data shifted from the training distribution.
method Proposes a novel loss function combining confidence maximization and batch-wise entropy maximization with an input transformation module.
result Significantly improves robustness of pretrained networks to corruptions on benchmarks like ImageNet-C.
COME replaces entropy minimization to prevent model collapse.
problem Overconfidence in entropy minimization leads to model collapse.
method COME explicitly models uncertainty with a Dirichlet prior distribution.
result COME achieves state-of-the-art performance on various TTA settings.
Paper tackles test-time adaptation for tabular data.
problem Performance degradation due to distribution shifts in testing.
method Proposes FTAT for robustly adapting tabular models during testing.
result FTAT outperforms state-of-the-art methods on benchmark datasets.
RG-TTA adapts neural forecasters to streaming time series shifts by modulating adaptation intensity.
problem Adapting neural forecasters to distribution shifts in streaming time series data.
method RG-TTA uses a meta-controller that continuously modulates adaptation intensity based on distributional similarity.
result RG-TTA achieves the lowest MSE in 156 of 224 seed-averaged experiments, reducing MSE by 5.7% vs TTA.
Tent adapts models during testing by minimizing entropy of predictions.
problem Adapting models to new data during testing with limited information.
method Test entropy minimization (tent) and online channel-wise affine transformations.
result Reduces generalization error on various datasets and benchmarks.
The goal of this paper is to design image classification systems that, after an initial multi-task training phase, can automatically adapt to new tasks encountered at test time. We introduce a conditional neural process based approach to the multi-task classification setting for this purpose, and establish connections …
CPATTA uses conformal prediction for efficient test-time adaptation.
problem Low data selection efficiency in existing ATTA methods.
method Conformal Prediction, online weight-update algorithm, domain-shift detector, staged update scheme.
result CPATTA consistently outperforms state-of-the-art methods by 5% in accuracy.
A new approach for test-time adaptation detects and reacts to distribution shifts.
problem Improving test-time accuracy under distribution shifts.
method Online self-training with a detection tool based on entropy values and betting martingales.
result The classifier's entropy values match those of the source domain, building invariance to distribution shifts.
Method adapts frozen models for few-shot tasks without training.
problem Deployment constraints limit model updates, necessitating new adaptation methods.
method Exponential tilting of latent distribution for inference.
result Method outperforms parameter-update methods across benchmarks.
Adapts CNN for robust medical image segmentation across different scanners and protocols.
problem Performance degradation of CNNs in medical image segmentation due to mismatch between training and test images.
method Designs a segmentation CNN as a concatenation of a shallow normalization CNN and a deep CNN. At test time, adapts the normalization sub-network for each test image using a denoising autoencoder.
result Consistently improves performance on multi-center MRI datasets of brain, heart, and prostate.
Enhances image quality to improve test-time adaptation accuracy.
problem Reducing accuracy loss due to distribution shift in deep networks.
method Integrates image enhancement with TTA methods to reduce prediction uncertainty.
result TECA method increases accuracy of TTA methods without hyperparameters.
M-L2O adapts fast to new tasks by self-adapting during test-time.
problem L2O optimizers struggle with out-of-distribution tasks.
method Meta-training an L2O optimizer to adapt quickly to new tasks.
result M-L2O converges significantly faster than vanilla L2O with only 5 steps of adaptation.
Traditional model-based RL relies on hand-specified or learned models of transition dynamics of the environment. These methods are sample efficient and facilitate learning in the real world but fail to generalize to subtle variations in the underlying dynamics, e.g., due to differences in mass, friction, or actuators a…
We show that several popular few-shot learning benchmarks can be solved with varying degrees of success without using support set Labels at Test-time (LT). To this end, we introduce a new baseline called Centroid Networks, a modification of Prototypical Networks in which the support set labels are hidden from the metho…
This paper tackles over-certainty in test-time adaptation models, proposing a solution to improve calibration.
problem Over-certainty in predictions caused by domain shifts, leading to misplaced trust.
method Introduces a certainty regularizer that dynamically adjusts pseudo-label confidence based on backbone entropy and logit norm.
result Achieves state-of-the-art performance in terms of Expected Calibration Error and Negative Log Likelihood, while maintaining accuracy.
STAD adapts models to evolving time-based data shifts.
problem Gradual distribution shifts over time challenge existing test-time adaptation methods.
method Bayesian filtering method that learns time-varying dynamics in hidden features.
result STAD excels in handling small batch sizes and label shift on real-world data.
TTLSA adapts models to label shifts across domains with nuisance factors.
problem Adapting models to changes in label distributions with nuisance factors.
method TTLSA uses EM on unlabeled samples to adapt a trained model to new label distributions.
result TTLSA improves model performance over invariance methods and baseline methods.
AdapTable adapts tabular models to shifts without source data, improving HELOC performance.
problem Distribution shifts in tabular data threaten model performance.
method Shift-aware uncertainty calibrator and label distribution handler.
result Up to 16% improvement on HELOC dataset.
Test-time training adapts a pretrained model to each prompt via parameter updates, improving accuracy under pretraining-to-test distribution shifts.
problem Improving accuracy of pretrained models under distribution shifts.
method Explaining TTT behavior through a decision-theoretic lens.
result TTT reduces prediction error when updates are spectrally matched to the prompt's signal-to-noise ratio and aligned with query-relevant eigen-directions.
New method learns models to adapt to domain shifts at test time.
problem Learning models robust to distribution shifts in practical applications.
method Adaptive Risk Minimization (ARM) framework.
result Performance gains of 1-4% on image classification problems.
Best-of-∞ improves LLM performance by efficiently allocating inference-time computation.
problem Achieving optimal performance in test-time LLM ensembling with infinite budget.
method Adaptive generation scheme and weighted ensembles of LLMs, formulated as mixed-integer linear program.
result Optimal ensemble weighting improves performance over individual models.
PriorGuide adapts diffusion models to new priors at test time.
problem Limited applicability of prior distributions in diffusion-based inference.
method PriorGuide uses a guidance approximation to adapt diffusion models to new priors at test time.
result Enhances the versatility of pre-trained inference models by allowing flexible adaptation to new priors.
New framework improves model reliability under distribution shifts.
problem Lack of formal guarantees connecting shift magnitude to prediction reliability in TTA methods.
method Develops a PAC-Bayesian framework interpreting MMD-balls as credal sets.
result Establishes generalization bounds and provides epistemic uncertainty quantification.
Adaptive compute allocation improves model performance by prioritizing harder queries.
problem Inefficiency in allocating test-time compute uniformly across all queries.
method Formulated as a bandit learning problem, proposed adaptive algorithms that estimate query difficulty and allocate compute accordingly.
result Achieved up to 15.29% relative performance improvement on various benchmarks.
Paper tackles online adaptation to changing label distributions.
problem Adapting machine learning models to changing label distributions in real-world settings.
method Leverages novel analysis to show estimation of expected test loss is possible without true labels. Proposes adaptation algorithms inspired by classical online learning techniques.
result Empirically verified that OGD is particularly effective and robust to various label shift scenarios.
Study improves BN TTA under distribution shift using higher-order asymptotics.
problem Improving BN TTA for changing data distributions.
method Integrates Edgeworth expansion and saddlepoint approximation with one-step M-estimation.
result Derives optimal weighting parameter for minimized mean-squared error.
Empirical study shows consistent meta-RL algorithms adapt to OOD tasks.
problem Theoretical consistency of meta-RL algorithms and its practical implications.
method Empirical investigation of representative meta-RL algorithms, focusing on consistency and adaptation to out-of-distribution tasks.
result Theoretical consistent algorithms can adapt to OOD tasks, while inconsistent ones cannot, but can still fail for poor exploration.
Study compares data-driven vs model-based MRS quantification strategies, focusing on resilience to out-of-distribution effects.
problem Resilience to out-of-distribution effects in data-driven MRS quantification.
method Compared three data-driven strategies (supervised regression, self-supervised learning, test-time adaptation) against model-based fitting tools.
result Test-time adaptation proved most resilient to out-of-distribution effects, while self-supervised learning achieved intermediate performance.
A TTA framework improves forecasting accuracy in non-stationary time series.
problem Improving forecasting accuracy in non-stationary time series.
method Normalization-based test-time adaptation for causal timeseries forecasting and direction classification.
result Normalization-based TTA improves forecasting error in synthetic gradual drift and can even hurt in aggressive norm-only adaptation in financial markets.
Plug-in robust NPE method adapts summaries independently of pretrained NPE.
problem Misspecification of neural posterior estimators under test data distribution.
method Minimum-distance summaries using maximum mean discrepancy (MMD).
result Substantial robustness gains with minimal additional overhead.
Step-DAD improves BED by periodically updating a design policy during experiments.
problem Improving flexibility and robustness in Bayesian experimental design.
method Semi-amortized, policy-based approach that updates a design policy during data collection.
result Consistently superior decision-making and robustness compared to current BED methods.
Continual learning models allow to learn and adapt to new changes and tasks over time. However, in continual and sequential learning scenarios in which the models are trained using different data with various distributions, neural networks tend to forget the previously learned knowledge. This phenomenon is often referr…
Graph transformation framework improves graph neural network performance.
problem Data quality issues in graph neural networks.
method Test-time graph transformation framework (GTrans).
result Significant performance improvements (up to 3.8%) across various datasets.
DAD++ improves test-time adversarial defense without access to training data.
problem Improving adversarial robustness of pre-trained models without access to training data.
method Test-time Data-free Adversarial Defense (DAD++) with detection and correction schemes.
result DAD++ significantly improves adversarial robustness with minimal clean accuracy drop.
We study the problem of structured prediction under test-time budget constraints. We propose a novel approach applicable to a wide range of structured prediction problems in computer vision and natural language processing. Our approach seeks to adaptively generate computationally costly features during test-time in ord…
DINo forecasts PDEs with flexible extrapolation and adaptability.
problem Fixed discretizations limit real-world PDE forecasting.
method DINo uses implicit neural representations for continuous-time dynamics.
result DINo outperforms other neural PDE forecasters.
The safety and robustness of learning-based decision-making systems are under threats from adversarial examples, as imperceptible perturbations can mislead neural networks to completely different outputs. In this paper, we present an adaptive view of the issue via evaluating various test-time smoothing defense against …
We learn sensor trees from training data to minimize sensor acquisition costs during test time. Our system adaptively selects sensors at each stage if necessary to make a confident classification. We pose the problem as empirical risk minimization over the choice of trees and node decision rules. We decompose the probl…
TTT improves model adaptation to test data, especially for nonlinear models.
problem Improving model performance in adapting to test data, especially for nonlinear models.
method Combining Test-time Training (TTT) with In-context Learning (ICL) for nonlinear models.
result TTT enables models to adapt to both feature vector and link function shifts, improving performance.
Study on neural networks forgetting in continual learning.
problem Forgetting in neural networks during continual learning.
method Gradient descent analysis on XOR-cluster datasets.
result Explicit bounds on forgetting rate and generalization gap.
Continual learning aims to learn new tasks without forgetting previously learned ones. This is especially challenging when one cannot access data from previous tasks and when the model has a fixed capacity. Current regularization-based continual learning algorithms need an external representation and extra computation …
We propose a novel adaptive approximation approach for test-time resource-constrained prediction. Given an input instance at test-time, a gating function identifies a prediction model for the input among a collection of models. Our objective is to minimize overall average cost without sacrificing accuracy. We learn gat…
Transfer and adaptation to new unknown environmental dynamics is a key challenge for reinforcement learning (RL). An even greater challenge is performing near-optimally in a single attempt at test time, possibly without access to dense rewards, which is not addressed by current methods that require multiple experience …