Optimal transfer learning for missing not-at-random matrix completion using source data.
problem Matrix completion in a Missing Not-at-Random setting with incomplete and noisy source data.
method Active sampling of rows and columns, feature shift in latent space, minimax lower bounds, computationally efficient estimation framework.
result Achieves minimax lower bound for active sampling setting, avoiding incoherence assumptions.
New estimator for survival function with missing not at random censoring indicators.
problem Estimating survival function with missing not at random censoring indicators.
method Proposes a new estimator based on a conditional copula model for the missingness mechanism.
result Provides a new method for estimating conditional survival function with MNAR censoring indicators.
CVIB uses information theory to learn counterfactuals from MNAR data without RCTs.
problem Debiasing learning from missing-not-at-random (MNAR) data in recommendation systems.
method CVIB, a variational information bottleneck, separates task-aware mutual information into factual and counterfactual parts.
result CVIB significantly enhances both shallow and deep models in recommendation systems.
StableDR stabilizes doubly robust learning for biased recommendation data.
problem Data missing not at random in recommender systems.
method StableDR, a stabilized doubly robust learning approach.
result StableDR achieves bounded bias, variance, and generalization error.
This work addresses missing data imputation for MNAR scenarios with identifiable deep generative models.
problem Missing data with complex missingness mechanisms (MNAR) leading to biased imputation results.
method Systematic analysis and proposal of an identifiable deep generative model.
result Proposed model provides identifiability guarantees under mild assumptions for various MNAR mechanisms.
Proposes methods to handle missing data in clustering models.
problem Missing data, especially MNAR, hinders model-based clustering.
method Developed a mixture model for different types of data, including MNAR, using Expectation Maximization algorithm.
result The proposed MNARz model simplifies inference and enables clustering with MNAR data.
Study improves unbiased recommender learning by addressing missing-reward bias.
problem Data bias caused by missing-reward observations in recommender systems.
method Proposes a novel estimator using propensity scores to mitigate both position and reward bias.
result The proposed estimator outperforms other methods, even with increased reward observation bias.
New methods for identifying and estimating missing data under complex mechanisms.
problem Missing data mechanisms dependent on missing values themselves.
method Developed a new MNAR model and proposed semiparametric estimation methods.
result Established sufficient conditions for identifying complete-data distribution and missingness mechanism.
Proposes a Latent Block Model for analyzing missing data.
problem Missing data can lead to misleading conclusions if not properly accounted for.
method Co-clustering model based on Latent Block Model, with variational EM algorithm for inference and model selection criterion.
result The proposed model reveals meaningful groups and insights from non-voters in French Parliament voting records.
New method corrects bias in density ratio estimation for missing data.
problem Missing data bias in density ratio estimation.
method Adapted KLIEP method (M-KLIEP) for MNAR data.
result M-KLIEP restores consistency and minimax optimality.
A novel k-means method for MNAR data improves clustering accuracy.
problem Improving k-means clustering for data missing not at random.
method A magnitude-decaying MNAR scenario-based k-means method with size constraints.
result The method reduces bias in estimated cluster centers and improves clustering accuracy.
New algorithms handle missing outcomes in MAB, reducing regret.
problem Missing outcomes in real-world MAB scenarios lead to biased estimates and linear regret.
method Introduced algorithms for MAR and MNAR missingness mechanisms in MAB.
result Significant improvements in decision-making by accounting for missingness.
A new tensor completion method handles missing data with missing not at random entries.
problem Handling missing data in tensors where the probability of observation depends on other entries.
method Estimate propensities using convex relaxation, then use higher-order SVD with inverse propensities weights.
result Finite-sample error bounds on the completed tensor are provided.
Bounds and sensitivity analysis for causal effects with MNAR confounders.
problem Estimating causal effects with missing outcome data.
method Assumption-free bounds and sensitivity analysis for outcome-independent MNAR.
result Valid bounds and sensitivity analysis methods for causal effect estimation.
Off-policy evaluation for MNAR rewards in MDPs
problem Off-policy evaluation in MDPs with MNAR rewards
method Formalizing a reward-dependent propensity model and using future states as shadow variables
result Proposed an Fitted-Q-Evaluation-style estimator that propagates recovered rewards while allowing target policies to depend on past missingness indicators
Proposes a deep latent variable model for MNAR data.
problem Missing data leading to biased results in MAR assumptions.
method Deep latent variable models with conditional no self-censoring.
result Establishes identifiability of MNAR data distribution.
Unified framework for variable selection in model-based clustering with missing data.
problem Challenges in identifying relevant variables and handling missing data in model-based clustering.
method Unified framework incorporating a data-driven penalty matrix and a mechanism for missingness modeling.
result Achieves both asymptotic consistency and selection consistency in the presence of missing data.
Missing values challenge data analysis because many supervised and unsupervised learning methods cannot be applied directly to incomplete data. Matrix completion based on low-rank assumptions are very powerful solution for dealing with missing values. However, existing methods do not consider the case of informative mi…
Paper tackles bias-variance trade-off in missing data, proposing a dynamic framework.
problem Missing data in practical applications deteriorates model performance.
method Develops a fine-grained dynamic learning framework to jointly optimize bias and variance.
result Theoretical and empirical validation of joint bias-variance optimization.
Estimates missing distributions using nearest neighbors with kernel methods.
problem Missing data and unobserved confounding in multivariate distributions.
method Distributional matrix completion framework with kernel nearest neighbors.
result Consistent recovery of underlying distributions with missing data.
In most real-world recommender systems, the observed rating data are subject to selection bias, and the data are thus missing-not-at-random. Developing a method to facilitate the learning of a recommender with biased feedback is one of the most challenging problems, as it is widely known that naive approaches under sel…
RMT-Net tackles biased credit scoring data by learning from both default/non-default and rejection/approval tasks.
problem Missing-not-at-random selection bias in financial credit scoring data.
method Reject-aware Multi-Task Network (RMT-Net) that leverages the correlation between default/non-default and rejection/approval tasks.
result RMT-Net improves credit scoring models by learning from both default/non-default and rejection/approval tasks.
Recommender systems widely use implicit feedback such as click data because of its general availability. Although the presence of clicks signals the users' preference to some extent, the lack of such clicks does not necessarily indicate a negative response from the users, as it is possible that the users were not expos…
Proposes a framework to handle missing data in traffic forecasting with sensor blackouts.
problem Missing data in traffic forecasting due to sensor blackouts, especially when correlated with traffic conditions.
method Latent state-space framework that models traffic dynamics and sensor dropout probabilities.
result Improves traffic forecasting by reducing blackout imputation RMSE from 7.02 to 4.23, with MNAR modeling providing additional gains.
Matrix completion is often applied to data with entries missing not at random (MNAR). For example, consider a recommendation system where users tend to only reveal ratings for items they like. In this case, a matrix completion method that relies on entries being revealed at uniformly sampled row and column indices can …
A deep generative model improves imputation of MNAR data by treating missing and complete data equally.
problem Missing Not At Random (MNAR) data in analysis.
method A generative model-specific joint probability decomposition method (conjunction model) and a deep generative imputation model (GNR).
result GNR surpasses state-of-the-art MNAR baselines with significant margins in RMSE and better mask reconstruction.
Confounding bias, missing data, and selection bias are three common obstacles to valid causal inference in the data sciences. Covariate adjustment is the most pervasive technique for recovering casual effects from confounding bias. In this paper, we introduce a covariate adjustment formulation for controlling confoundi…
Emputation learns imputation models guided by missingness assumptions.
problem Learning imputation models for missing data given observed data.
method Guided by specific missingness assumptions, Emputation trains a deep generative model to learn the extrapolation distribution of missing variables.
result The population minimizer of the emputation risk recovers the target extrapolation distribution under various identification assumptions.
MissNODAG learns cyclic causal graphs from incomplete data.
problem Causal discovery in systems with feedback loops and missing data.
method Differentiable framework integrating additive noise model and expectation-maximization.
result MissNODAG uncovers cyclic structures and missingness mechanisms from partially observed data.
Observed events in recommendation are consequence of the decisions made by a policy, thus they are usually selectively labeled, namely the data are Missing Not At Random (MNAR), which often causes large bias to the estimate of true outcomes risk. A general approach to correct MNAR bias is performing small Randomized Co…
Proposes a method to evaluate classifiers with missing labels using multiple imputation.
problem Missing labels during model evaluation can introduce bias, especially in Missing Not At Random (MNAR) data.
method Develops a multiple imputation technique to estimate and provide predictive distributions for metrics like precision, recall, and ROC-AUC.
result The predictive distribution's location and shape are generally correct, even in the MNAR regime.
The paper introduces tests for missing data models based on graph assumptions.
problem Verification of assumptions in missing data models is insufficiently addressed.
method The paper explores three classes of missing data models and designs goodness-of-fit tests.
result The paper provides new insights and tests for missing data graphical models.
Paper tackles informative labels in semi-supervised learning, proposing debiasing methods.
problem Informative labels can bias semi-supervised learning models, especially when some classes are more likely to be labeled.
method Estimates missing-data mechanism and uses inverse propensity weighting to debias SSL algorithms.
result Proposed methods improve SSL performance, demonstrated on various datasets including medical ones.
Robust Lasso-Zero handles missing covariates and sparse corruptions.
problem Sparse corruptions and missing covariates in sparse linear models.
method Extension of Lasso-Zero to handle sparse corruptions, with theoretical guarantees on sign recovery.
result Robust Lasso-Zero can handle missing values without specifying a parametric model.
Unified framework for robust causal directionality in quantum systems under MNAR observation.
problem Determining causal directionality in quantum systems under MNAR observation.
method Integrates CVAE-based latent constraints, MNAR-aware selection models, GEE-stabilized regression, penalized empirical likelihood, and Bayesian optimization.
result Achieves lower bias and variance, near-nominal coverage, and superior quantum-specific diagnostics.
Random forest (RF) missing data algorithms are an attractive approach for dealing with missing data. They have the desirable properties of being able to handle mixed types of missing data, they are adaptive to interactions and nonlinearity, and they have the potential to scale to big data settings. Currently there are …
CACTI improves tabular data imputation by leveraging missingness patterns and contextual information.
problem Tabular data imputation with improved accuracy and robustness.
method Masked autoencoding approach with median truncated copy masking and contextual information.
result Average R2 gain of 7.8% over the next best method across various datasets and missingness conditions. Deep learning for missing data with flexible modelling.
problem Missing data with missingness dependent on the missing values.
method Deep latent variable models with a neural network for missingness pattern modeling, importance-weighted variational inference, and stochastic gradients.
result Explicitly modelling missingness improves inference on various data sets.
Develops a VAE model for datasets with missing data.
problem Applying VAEs to datasets with missing data.
method A novel latent variable model of a corruption process generating missing data, with a tractable ELBO.
result Improved marginal log-likelihood and better missing data imputation compared to existing approaches.
We study offline recommender learning from explicit rating feedback in the presence of selection bias. A current promising solution for the bias is the inverse propensity score (IPS) estimation. However, the performance of existing propensity-based methods can suffer significantly from the propensity estimation bias. I…
New method tackles MNAR missingness in domain adaptation.
problem Handling missingness in both source and target data.
method Reduces MNAR missingness to imputation problem, leveraging recent MNAR imputation methods.
result Developed a novel domain adaptation procedure for MNAR missingness shift.
New DL model handles missing data in biomedical datasets.
problem Handling missing data in modern biomedical datasets.
method Proposed a new DL architecture, dlglm, for generalized linear models.
result Outperforms existing methods in MNAR missingness scenarios.
Develop conformal prediction for dyadic regression under complex missingness.
problem Conformal prediction for dyadic regression under complex missingness mechanisms.
method Developing general technical tools and conformal prediction procedures for dyadic regression under complex missingness.
result Establishing asymptotic validity of weighted conformal prediction under a nonparametric graphon model for missingness mechanism.
Many real datasets contain values missing not at random (MNAR). In this scenario, investigators often perform list-wise deletion, or delete samples with any missing values, before applying causal discovery algorithms. List-wise deletion is a sound and general strategy when paired with algorithms such as FCI and RFCI, b…
Study linear regression with missing or corrupted data, showing error bounds.
problem Linear regression under missing or corrupted data.
method Information-theoretic lower bounds and efficient algorithms.
result Error bounds match in missing and corruption settings.
PbP strategy improves logistic model prediction with missing values.
problem Predicting with missing inputs in logistic models.
method Pattern-by-Pattern (PbP) strategy for logistic models with missing values.
result PbP accurately approximates Bayes probabilities under GPMM across various missing data scenarios.
New method improves reliability of recommender systems in face of fake accounts and manipulation.
problem Improving reliability of recommender systems in the presence of fake accounts and manipulation.
method Robust Discrete Matrix Completion (RDMC) method designed to handle sparse rating data and manipulation.
result Evaluations show RDMC offers a statistically-sound blueprint for future studies on recommender systems.
Missing data are ubiquitous in many domains including healthcare. When these data entries are not missing completely at random, the (conditional) independence relations in the observed data may be different from those in the complete data generated by the underlying causal process. Consequently, simply applying existin…