CTRF combines logged data and randomized experiments for robust prediction.
problem Robust prediction models to handle distributional shifts between training and testing data.
method CTRF uses existing training data and a small amount of randomized experiment data to train a robust model.
result CTRF produces robust predictions and outperforms baseline methods in the presence of feature shifts.
New method robust to random distributional shifts in prediction.
problem Random distributional shifts in real-world settings.
method Hybrid approach combining long-term and proxy outcomes.
result Hybrid approach yields lower mean-squared error than current methods.
New insights into how high-dimensional models handle covariate shifts.
problem Covariate shift in high-dimensional random feature regression.
method Exact high-dimensional asymptotics of random feature regression under covariate shift.
result Overparameterized models exhibit enhanced robustness to covariate shift.
This paper improves computational efficiency in kernel ridge regression under covariate shift.
problem Covariate shift in nonparametric regression.
method Random projections in RKHS to reduce computational demands.
result Significant computational savings can be achieved without compromising learning performance under covariate shift.
A method to remove mean-shift noise from PCA using knockoffs.
problem High sensitivity of PCA to mean-shift contamination in high-dimensional data.
method Introducing knockoff mean-shift perturbation to separate and remove mean-shift components from PCA.
result The mean-shift spikes are spectrally separable from stable eigenvalues, allowing for robust PCA.
Develops a new random forest method for clustered data with improved prediction and inference.
problem Improving prediction and inference accuracy for clustered data with within-cluster dependence.
method Clustered Random Forests, using weighted least squares estimators for leaf predictions.
result Optimal prediction and inference weights vary under covariate shift, necessitating user-chosen weights.
This work uses adversarial learning to detect and correct feature shifts in various datasets.
problem Detecting and correcting feature shifts in real-world datasets.
method Adversarial learning applied to multiple discriminators to detect and correct feature shifts.
result Mainstream classifiers can effectively localize and correct feature shifts, outperforming existing techniques.
Paper analyzes impact of PRM on binary random variables and distribution shifts.
problem Impact of performative risk minimization on binary random variables and distribution shifts.
method Formulated two measures of impact, derived explicit formulas for full information, and provided estimators for partial information.
result PRM can have amplified side effects compared to methods that do not model data shift.
Method improves treatment effect prediction robust to unknown covariate shifts.
problem Estimating heterogeneous treatment effects for different populations.
method Post-processing CATE T-learners with multi-accurate predictors to handle unknown covariate shifts.
result Improves bias and mean squared error in simulations with covariate shifts.
This paper investigates how machine learning APIs change over time and proposes an efficient method to monitor these changes.
problem Understanding and assessing changes in machine learning APIs over time.
method Systematic investigation of ML API shifts, proposing a principled adaptive sampling algorithm (MASA) for efficient estimation of confusion matrix shifts.
result MASA can accurately estimate confusion matrix shifts using up to 90% fewer samples compared to random sampling.
New method tackles MNAR missingness in domain adaptation.
problem Handling missingness in both source and target data.
method Reduces MNAR missingness to imputation problem, leveraging recent MNAR imputation methods.
result Developed a novel domain adaptation procedure for MNAR missingness shift.
This paper improves GP-UCB by using a shifted exponential distribution for confidence parameters.
problem Theoretical confidence parameter in GP-UCB increases with iterations, leading to large values.
method Introduced IRGP-UCB, a randomized variant of GP-UCB using a shifted exponential distribution for confidence parameters.
result IRGP-UCB achieves sub-linear regret without increasing the confidence parameter.
Express Wavenet is an improved optical diffractive neural network. At each layer, it uses wavelet-like pattern to modulate the phase of optical waves. For input image with n2 pixels, express wavenet reduce parameter number from O(n2) to O(n). Only need one percent of the parameters, and the accuracy is still very high.…
We extend the randomized singular value decomposition (SVD) algorithm \citep{Halko2011finding} to estimate the SVD of a shifted data matrix without explicitly constructing the matrix in the memory. With no loss in the accuracy of the original algorithm, the extended algorithm provides for a more efficient way of matrix…
FedForest adapts RF for federated learning, improving performance and efficiency.
problem Adapting RF for federated learning with heterogeneous data.
method FedForest uses a novel splitting procedure to aggregate client statistics, allowing non-parametric personalization.
result FedForest's federated RF achieves performance close to centralized models while being communication-efficient.
Optimizes sample reweighting to match laws under covariate shift using Wasserstein distance.
problem Matching laws of samples with different distributions under covariate shift.
method Minimizes Wasserstein distance between empirical measures of samples using Nearest Neighbors weights.
result Consistent reweighting leads to asymptotic convergence of empirical measures.
A method to reduce bias in model-based policy evaluation by shifting operators.
problem Bias in value function computation from noisy estimated models.
method Operator shifting method to reduce the residual norm error.
result The shifting factor is always positive and upper bounded by $1+O\left(1/n
ight)$.
The paper analyzes how combining samples from two tasks can improve performance, especially in high dimensions.
problem Understanding when combining samples from two related tasks outperforms learning with one task alone.
method Applying random matrix theory to high-dimensional linear regression, focusing on proportional sample size increases.
result Precise high-dimensional asymptotics for bias and variance of HPS estimator, showing phase transitions in transfer performance.
Estimates proxy-based inference adjustments for distribution shifts.
problem Imperfect proxy data leads to biased inference.
method Empirical calibration of proxy-primary metric discrepancy as a random effect.
result Empowers inference without individual-level response data.
New method for identifying graph shift operators using vertex-time autoregressive models.
problem Identifying graph shift operators from graph signals.
method Online optimization using vertex-time autoregressive model and stochastic gradient projection.
result Successful recovery of graph shift operators from graph signals.
Task shift from classification to regression is possible in overparameterized linear models with limited additional data.
problem Transferability of latent knowledge from classification to regression in overparameterized linear models.
method Investigation of task shift in overparameterized linear regression, zero-shot and few-shot cases, with a focus on minimum-norm interpolation.
result Minimum-norm interpolators can transfer latent knowledge from classification to regression with limited additional data.
We show that removing sigmoid transformation in the skip-gram with negative sampling (SGNS) objective does not harm the quality of word vectors significantly and at the same time is related to factorizing a squashed shifted PMI matrix which, in turn, can be treated as a connection probabilities matrix of a random graph…
Paper quantifies label shift with robustness guarantees using distribution feature matching.
problem Estimating target label distribution under label shift.
method Distribution feature matching (DFM) framework and robustness analysis.
result General performance bound and robustness analysis in misspecified settings.
A novel variational inference based resampling framework is proposed to evaluate the robustness and generalization capability of deep learning models with respect to distribution shift. We use Auto Encoding Variational Bayes to find a latent representation of the data, on which a Variational Gaussian Mixture Model is a…
The study reveals a linear relationship between source and target domain classification errors based on disagreement.
problem Evaluating model performance under distribution shift with limited labeled data.
method Developed a theoretical foundation for analyzing disagreement in high-dimensional random features regression.
result The disagreement-on-the-line phenomenon occurs when classification error under the source domain is a linear function of the target domain.
Derives explicit investment strategy with random endowment.
problem Optimal investment with random endowment in a market.
method Duality arguments to derive explicit expression for optimal strategy.
result Explicit expression for optimal trading strategy exists.
Paper tackles missing data shifts in domains, showing how to adapt models.
problem Missing data mechanisms differ between source and target domains.
method Introduced DAMS, showing covariate shift under certain conditions, and providing adjustments for missingness.
result Simple adjustments can improve target model performance even when missingness rates vary.
Method uses aggregate crop statistics to improve satellite-based crop type mapping.
problem Limited field-level crop labels for training satellite-based maps.
method Corrects classifier by accounting for shifts in crop type composition and feature means.
result Substantial improvements in overall classification accuracy, reducing misclassifications by 21.9% on average.
Improved covariate shift handling with node-based Bayesian neural networks.
problem Improving generalization under covariate shift in neural networks.
method Introduced node-based Bayesian neural networks that learn latent noise variables to represent input corruptions.
result Node-based BNNs perform well under covariate shift due to input perturbations, improving uncertainty estimation and robustness.
Transformers learn linear models in-context without updates.
problem Understanding how transformers mimic linear models in-context.
method Gradient flow on linear regression tasks with random initialization.
result Transformers achieve prediction error competitive with best linear predictors.
Randomly biased data makes complex models as easy to learn as simple ones.
problem Learning complex models like multi-index and sparse Boolean functions.
method Introducing a small random shift in the first moment of the data distribution.
result Randomly biased data makes Gaussian single index models and sparse Boolean functions as easy to learn as linear functions.
The paper analyzes how machine learning models perform under covariate shift, especially when the feature shift in x is larger than that in y.
problem Performance of machine learning models under covariate shift with heterogeneous feature changes.
method Empirical risk minimization (ERM) over functions f+g, fit on a training distribution, evaluated on a test distribution with covariate shift. result ERM is more resilient to heterogeneous covariate shifts when the class F is simpler than G. Developed a random walk analog of geodesic flow on hyperbolic groups.
problem Geodesic flow on hyperbolic groups due to non-uniqueness of geodesics.
method Introduced a new framework using random walks and bi-infinite trajectories.
result Established ergodicity of the randomized geodesic flow and exponential mixing.
New framework to test neural network representation similarity measures.
problem Disagreements among dissimilarity measures in neural networks.
method Statistical testing framework to evaluate measures based on functional behavior.
result Current metrics have different weaknesses; a classical baseline performs surprisingly well.
We derive the exact form of the eigenvalue spectra of correlation matrices derived from a set of time-shifted, finite Brownian random walks (time-series). These matrices can be seen as random, real, asymmetric matrices with a special structure superimposed due to the time-shift. We demonstrate that the associated eigen…
Proposes a method to correct for covariate shift in meta-analysis of randomized trials.
problem Invalidation of standard IPD meta-analysis due to covariate shift across studies.
method Placebo-anchored transport framework that treats source-trial outcomes as proxy signals and target-trial placebo outcomes as gold labels.
result Yields target-identified effect estimates in connected targets and a principled screen--then--transport procedure in disconnected targets.
Over-parameterized models reduce Out-of-Distribution (OOD) generalization loss.
problem Understanding how over-parameterized models handle non-trivial distributional shifts.
method Investigating random feature models and examining non-trivial natural distributional shifts.
result Increasing model parameterization reduces OOD loss.
This study improves scalability of randomized smoothing for certifying classifier robustness.
problem Certifying machine learning classifiers against adversarial attacks is challenging and scalable solutions are needed.
method The study reviews and explores randomized smoothing and its derivatives, focusing on scalability.
result The study provides theoretical guarantees and discusses scalability challenges of randomized smoothing.
We consider the problem of improving the efficiency of randomized Fourier feature maps to accelerate training and testing speed of kernel methods on large datasets. These approximate feature maps arise as Monte Carlo approximations to integral representations of shift-invariant kernel functions (e.g., Gaussian kernel).…
Transformers learn to adapt to different task difficulties and resist distribution shifts.
problem Understanding and optimizing a Transformer's performance across various task difficulties and distribution shifts.
method Analyzing a pretrained Transformer on a mixture distribution of tasks, proving optimal convergence rates.
result Transformers achieve optimal convergence rates on tasks of specific difficulty levels, robust to distribution shifts.
New benchmark protocol evaluates neural network optimizers for efficiency and data shift sensitivity.
problem Benchmarking neural network optimizers with hyperparameter complexity and data shift sensitivity.
method Proposed a new evaluation protocol combining end-to-end and data-addition training efficiency, using bandit hyperparameter tuning and human study validation.
result No clear winner across all tasks, highlighting the complexity of optimizer performance.
New method creates diverse neural ensembles for better uncertainty estimation and robustness.
problem Creating more robust neural networks for uncertainty estimation and dataset shift.
method Automatically constructing ensembles with varying architectures.
result Ensembles with varying architectures outperform deep ensembles in accuracy, uncertainty calibration, and robustness.
Estimates disease prevalence using non-ignorable missing data in health surveys.
problem Estimating disease prevalence in non-representative samples with non-ignorable missing data.
method Connects auxiliary proxy variable framework to label shift setting, uses high-dimensional covariates without generative models.
result Fails to account for non-ignorable missingness can lead to significant misestimations.
Efficiently estimates mean in contaminated Gaussian data with near-optimal sample complexity.
problem Robust mean estimation in the presence of mean-shift contamination.
method First computationally efficient algorithm with near-optimal sample complexity and polynomial-time running.
result Approximates the target mean to any desired accuracy with constant fraction of outliers tolerated.
New spectral mixture representation for isotropic kernels simplifies random Fourier features.
problem Applying Random Fourier Features to complex kernels.
method Decompose isotropic kernels into scale mixtures of α-stable random vectors.
result Constructive spectral sampling formula for various kernels.
The paper analyzes the generalization error of min-norm interpolators in transfer learning with limited test samples.
problem Characterizing the generalization error of min-norm interpolators in transfer learning with limited test samples.
method Characterizes the bias and variance of pooled min-ℓ2-norm interpolation under covariate shift and model shift. result Shows that adding data can hurt when SNR is low and is beneficial at higher SNR levels under certain conditions.
Optimizer choice affects neural scaling laws, changing the exponent α.
problem The exponent α in neural scaling laws L(N)∝N−α varies with the optimizer used. method Controlled random-feature regression experiments with five optimizer variants and six spectral conditions.
result Preconditioned optimizers yield steeper scaling (larger α), with the α-shift increasing across most of the tested spectral range. Linear models can be poisoned by shifting a fraction of one class's data, revealing scaling laws and weight alignment.
problem Understanding and quantifying data poisoning in linear models.
method Analysis of ridge least squares with an unpenalized intercept, using resolvent techniques and random matrix theory.
result Closed-form limits for the poisoned score, revealing scaling laws and weight alignment with the poisoning direction.