PAMA learns covariate importance for better matching in observational studies.
problem Poor performance of conventional matching methods when covariates differ in relevance.
method PAMA is a semi-supervised framework that learns covariate importance from paired data and optimizes a weighted quadratic score.
result PAMA outperforms standard methods, particularly in high-dimensional settings and under model misspecification.
This paper improves error estimation in covariate shift by incorporating target information.
problem Error estimation is inaccurate in covariate shift scenarios.
method Proposes a redefinition of importance using target information for better error estimation.
result Incorporating target information leads to more accurate error estimation, especially with KLIEP.
Proposes a method to quantify uncertainty in predictions under covariate shift.
problem Uncertainty quantification challenges in machine learning with covariate shifts.
method Constructs PAC prediction sets with given importance weights and confidence intervals for weights.
result Algorithm gives prediction sets with the smallest average normalized size.
The paper proves a new method to improve generalization in covariate-shift scenarios.
problem Improving performance on test distributions that differ from training distributions.
method Independence-driven importance weighting algorithms for feature selection.
result Theoretical proof that these algorithms can identify optimal variables for covariate-shift generalization.
A new method identifies class-specific covariates in multi-class prediction tasks.
problem Identifying covariates specifically associated with one or more outcome classes in multi-class prediction tasks.
method Introducing multi forests (MuFs) with multi-way and binary splits to measure class-associated discriminatory ability.
result The multi-class VIM specifically ranks class-associated covariates highly, unlike conventional VIMs.
Proposes a new method for handling domain shift in samples with biases in both covariates and labels.
problem Domain shift in samples with biases in both covariates and labels.
method Factorizable Joint Shift (FJS) and Joint Importance Aligning (JIA).
result Our method can handle co-existence of sampling bias in covariates and labels.
A new method uses nearest neighbors for importance weighting.
problem Data covariate shift problems in machine learning.
method Nearest neighbor classification scheme for determining importance weights.
result Demonstrated effectiveness through comparative experiments on various classification tasks.
The paper proposes AIS for Bayesian inversion of multioutput signals with covariance estimation.
problem Performing uncertainty analysis of covariance matrices in Bayesian inversion problems for multioutput signals.
method Adaptive Importance Sampling (AIS) scheme, split variables, frequentist approach for noise covariance, prior density over covariance matrix.
result Estimation of model parameters and covariance matrix of noise.
Algorithm calibrates predictions for covariate shift using domain adaptation.
problem Uncertainty estimates overestimate certainty when real-world data differs from training data.
method Uses importance weighting and learns a feature map to equalize distributions.
result Outperforms existing approaches in calibrated prediction when covariate shift occurs.
Proposes a new method to estimate variable importance in black box models, mitigating correlation effects.
problem Correlation between covariates affects the interpretation of variable importance parameters.
method Develops a modified LOCO (Leave Out COvariates) method and uses semiparametric models for estimation.
result Shows how to estimate a modified LOCO method that mitigates correlation effects.
Paper tackles unbounded density ratio estimation for covariate shift adaptation.
problem Understudied challenge in statistical learning: unbounded density ratios.
method Three-step estimation method: relative density ratio, truncation, and transformation.
result Established rigorous convergence guarantees for density ratio and regression estimators.
Proposes a robust method for predicting missing outcomes in covariate shift adaptation.
problem Predicting missing outcomes in test data with covariate shift.
method Doubly robust estimator for covariate shift adaptation via importance weighting, incorporating an additional estimator for the regression function.
result Shows robustness against density-ratio estimation errors, maintaining consistency if either estimator is consistent.
New method improves variable importance in random forests.
problem MDA's inconsistency in random forests.
method Theoretical analysis and development of Sobol-MDA.
result Sobol-MDA consistently estimates forest accuracy decrease.
New method calibrates models under covariate shifts.
problem Calibration of models can be lost under covariate shifts.
method Importance sampling based approach.
result Efficacy demonstrated on real-world and synthetic datasets.
The paper examines when importance weighting is needed for nonparametric and misspecified models.
problem When is importance weighting correction needed for covariate shift adaptation?
method Analysis of IW-corrected kernel ridge regression in various settings.
result The importance weighting correction is needed for nonparametric and misspecified models to obtain the best approximation of the true unknown function.
Constructs covariant derivatives for Ehresmann connections.
problem Developing a method for covariant derivatives in fibre bundles.
method Introducing a vertical endomorphism to construct covariant derivatives on vertical and horizontal distributions.
result Covariant derivatives can be constructed separately on vertical and horizontal distributions and then glued together.
Variable selection in sparse regression models is an important task as applications ranging from biomedical research to econometrics have shown. Especially for higher dimensional regression problems, for which the link function between response and covariates cannot be directly detected, the selection of informative va…
Nonsingular estimation of high dimensional covariance matrices is an important step in many statistical procedures like classification, clustering, variable selection an future extraction. After a review of the essential background material, this paper introduces a technique we call slicing for obtaining a nonsingular …
Bayesian method selects important covariates in modal regression.
problem Bayesian modal regression with heavy-tailed responses.
method Expectation-maximization algorithm for parameter estimation; test statistic for variable selection.
result Efficacy of the proposed method in identifying important covariates.
DFI maps covariates to latent representations for feature importance.
problem Feature importance when predictors are statistically dependent.
method Disentangled Feature Importance (DFI) using entropic optimal transport.
result DFI yields stable, interpretable, uncertainty-quantified attributions of shared predictive signal.
The paper introduces Shapley curves for measuring variable importance in nonparametric settings.
problem Limited statistical understanding of Shapley values as variable importance measures.
method Introduces Shapley curves based on conditional expectation and covariate distribution; derives convergence rates and normality; proposes a novel bootstrap procedure.
result Validates theoretical findings with numerical studies and analyzes vehicle prices determinants.
CovRegRF estimates covariance matrix from covariates using random forests.
problem Estimating conditional covariances or correlations among multivariate responses.
method Random forest trees with a custom splitting rule to maximize covariance difference.
result Accurate covariance matrix estimates and controlled Type-1 error.
A new one-step method for covariate shift adaptation.
problem Real-world data often violates the assumption of same distribution for training and test samples.
method Proposes a one-step optimization approach to jointly learn the model and weights.
result The proposed method achieves a generalization error bound and is empirically effective.
The paper addresses portfolio allocation with uncertain covariance matrices, finding a logarithmic risk dependence.
problem Portfolio allocation with uncertain covariance matrices.
method Calculates the expected value of CARA utility function over a distribution of covariance matrices, considering uncertainty in future returns and covariances.
result Marginalization introduces a logarithmic dependence on risk, leading to lower allocation levels for higher uncertainties.
New method optimizes model selection in high-dimensional regression models.
problem Model selection in high-dimensional misspecified regression models with covariate shift.
method Importance-weighted orthogonal greedy algorithm (IWOGA) and high-dimensional importance-weighted information criterion (HDIWIC).
result IWOGA + HDIWIC achieves optimal convergence rates in terms of prediction error.
The paper derives theoretical foundations for two common machine learning variable importance measures.
problem Understanding variable importance in machine learning problems.
method The paper derives closed-form expressions for Permute-and-Predict (PaP) and Leave-One-Covariate-Out (LOCO) methods.
result Theoretical derivations explain the behavior of PaP and LOCO under collinearity, linking them to coefficients and predictor variability.
CPI overcomes limitations of permutation importance by providing accurate variable selection.
problem Misidentification of unimportant variables in complex models due to covariate correlations.
method Developed a model agnostic and computationally lean Conditional Permutation Importance (CPI) approach.
result CPI provides accurate type-I error control and more parsimonious variable selection.
Robust and reliable covariance estimates play a decisive role in financial and many other applications. An important class of estimators is based on Factor models. Here, we show by extensive Monte Carlo simulations that covariance matrices derived from the statistical Factor Analysis model exhibit a systematic error, w…
The paper compares LOCO and Shapley values for feature importance, highlighting their limitations and suggesting improvements.
problem Quantifying feature importance in the presence of feature correlation.
method LOCO and Shapley Values, critiquing their axioms and proposing new measures.
result Shapley values do not eliminate feature correlation, and a modified LOCO is recommended.
Current methods for covariate-shift adaptation use unlabelled data to compute importance weights or domain-invariant features, while the final model is trained on labelled data only. Here, we consider a particular case of covariate shift which allows us also to learn from unlabelled data, that is, combining adaptation …
In the covariate shift learning scenario, the training and test covariate distributions differ, so that a predictor's average loss over the training and test distributions also differ. In this work, we explore the potential of extreme dimension reduction, i.e. to very low dimensions, in improving the performance of imp…
Sparse Inverse Covariance Estimation (SICE) is useful in many practical data analyses. Recovering the connectivity, non-connectivity graph of covariates is classified amongst the most important data mining and learning problems. In this paper, we introduce a novel SICE approach using adaptive thresholding. Our method i…
Covariate shift relaxes the widely-employed independent and identically distributed (IID) assumption by allowing different training and testing input distributions. Unfortunately, common methods for addressing covariate shift by trying to remove the bias between training and testing distributions using importance weigh…
Study forecasts volatility and risk in electricity markets using matrix-HAR models.
problem Forecasting volatility and risk in electricity markets.
method Constructed a parsimonious matrix-HAR type model to estimate realized covariation and risk premia in electricity markets.
result Inclusion of longer time horizons and renewable generation information improves forecasts.
Network-assisted regression uses conformal prediction for valid inference.
problem Predicting node attributes using network and conventional covariates with valid statistical inference.
method Network analog of conformal prediction under mild joint exchangeability assumption.
result Achieves finite sample validity and asymptotic conditional validity for various network covariates.
New estimator handles covariate shift with closed-form solution and super-efficiency.
problem Handling covariate shift in missing data and causal inference problems.
method Minimum Wasserstein distance estimation framework.
result Closed-form expression and super-efficiency relative to semiparametric efficient estimator.
A new algorithm speeds up rerandomization for better experiment balance.
problem Achieving optimal covariate balance in randomized experiments.
method Metropolis-Hastings framework with sampling-importance resampling.
result PSRSRR achieves significant speedups while maintaining statistical guarantees.
We propose a novel calibration method for computer simulators, dealing with the problem of covariate shift. Covariate shift is the situation where input distributions for training and test are different, and ubiquitous in applications of simulations. Our approach is based on Bayesian inference with kernel mean embeddin…
Anomalies and outliers are common in real-world data, and they can arise from many sources, such as sensor faults. Accordingly, anomaly detection is important both for analyzing the anomalies themselves and for cleaning the data for further analysis of its ambient structure. Nonetheless, a precise definition of anomali…
Study tackles variable selection with missing covariates and outcomes using machine learning and imputation.
problem Missing data in both covariates and outcomes complicates variable selection in health studies.
method Exploits machine learning flexibility and bootstrap imputation for variable selection, comparing multiple methods.
result XGBoost and BART perform best in variable selection with bootstrap imputation, achieving high F1 scores and low Type I errors. Paper analyzes spectral algorithms under covariate shift, providing convergence rates.
problem Addressing distributional mismatch in regression models.
method Incorporates importance weights into spectral algorithms in RKHS.
result Establishes minimax-optimal convergence rates for misspecified cases.
Method tackles missing covariates in large-scale datasets.
problem Cross-population missing data problem in large-scale datasets.
method Augmented transfer regression learning method combining importance-weighted estimating equations and imputation terms.
result Estimator is n1/2-consistent and asymptotically normal, attaining semiparametric efficiency bound under correct specification. Enhanced Transformer models predict ETF portfolio performance by optimizing covariance and semi-covariance matrices.
problem Static covariance estimates fail to capture dynamic market fluctuations and non-linear correlations.
method Transformer-based models for real-time covariance and semi-covariance predictions.
result Portfolios optimized with semi-covariance matrix outperform those with standard covariance matrix, especially in volatile conditions.
There has been a lot of work fitting Ising models to multivariate binary data in order to understand the conditional dependency relationships between the variables. However, additional covariates are frequently recorded together with the binary data, and may influence the dependence relationships. Motivated by such a d…
A new efficient test addresses limitations of knockoffs for conditional independence testing.
problem Testing conditional independence under model-X assumptions.
method Leave-One-Covariate-Out Conditional Randomization Test (LOCO-CRT)
result LOCO-CRT produces valid p-values for familywise error rate control with minimal variability. This paper explores how effective sample size, dimensionality, and model performance are related in covariate shift adaptation.
problem Understanding the relationship between effective sample size, dimensionality, and generalization in covariate shift adaptation.
method Building a unified theory connecting effective sample size, data dimensionality, and generalization in the context of covariate shift adaptation.
result Dimensionality reduction or feature selection can increase effective sample size, supporting the practice of reducing dimensionality before covariate shift adaptation.
This paper rethinks confidence calibration under covariate shifts.
problem Calibration methods struggle with covariate shifts and unstable importance weighting.
method Derives Expectation consistency condition and proposes Expectation consistency loss (ECL).
result ECL loss is compatible with various types of calibration and has the same sample complexity as ECE.
Proposes a method to recover sparse tensors with covariate info.
problem Sparse tensor with high missing entries and many zeros.
method Covariate-assisted Sparse Tensor Completion (COSTCO) using latent components.
result 23% accuracy improvement over baseline in advertisement dataset.