Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

209419628837 · Jun 202019922001200920172026
48 results for missing at random

Missing data are ubiquitous in many domains including healthcare. When these data entries are not missing completely at random, the (conditional) independence relations in the observed data may be different from those in the complete data generated by the underlying causal process. Consequently, simply applying existin…

2018-07-11abs ↗pdf ↗

A new tensor completion method handles missing data with missing not at random entries.

problem Handling missing data in tensors where the probability of observation depends on other entries.
method Estimate propensities using convex relaxation, then use higher-order SVD with inverse propensities weights.
result Finite-sample error bounds on the completed tensor are provided.

CVIB uses information theory to learn counterfactuals from MNAR data without RCTs.

problem Debiasing learning from missing-not-at-random (MNAR) data in recommendation systems.
method CVIB, a variational information bottleneck, separates task-aware mutual information into factual and counterfactual parts.
result CVIB significantly enhances both shallow and deep models in recommendation systems.

New estimator for survival function with missing not at random censoring indicators.

problem Estimating survival function with missing not at random censoring indicators.
method Proposes a new estimator based on a conditional copula model for the missingness mechanism.
result Provides a new method for estimating conditional survival function with MNAR censoring indicators.

Proposes methods to handle missing data in clustering models.

problem Missing data, especially MNAR, hinders model-based clustering.
method Developed a mixture model for different types of data, including MNAR, using Expectation Maximization algorithm.
result The proposed MNARz model simplifies inference and enables clustering with MNAR data.

Proposes a method to make statistical inferences robust in spatially dependent settings with missing at random labels.

problem Statistical inference challenges with missing at random labels and spatial dependence.
method Doubly robust estimator with cross-fit nuisances and jackknife spatial HAC variance correction.
result Asymptotically valid confidence intervals with improved finite-sample calibration.

The paper analyzes kk-means clustering for missing data, proving statistical guarantees under MCAR.

problem Statistical guarantees for kk-means clustering with missing data, especially under Missing Completely at Random (MCAR).
method Established n\sqrt{n}-excess risk bound and consistency of cluster centers under general missing mechanisms; derived n\sqrt{n}-convergence rate and asymptotic normality for MCAR.
result Achieving n\sqrt{n}-rate and converging to true cluster centers requires distinct true cluster centers in every dimension under MCAR.

Rating prediction is an important application, and a popular research topic in collaborative filtering. However, both the validity of learning algorithms, and the validity of standard testing procedures rest on the assumption that missing ratings are missing at random (MAR). In this paper we present the results of a us…

2012-06-20abs ↗pdf ↗

New methods for identifying and estimating missing data under complex mechanisms.

problem Missing data mechanisms dependent on missing values themselves.
method Developed a new MNAR model and proposed semiparametric estimation methods.
result Established sufficient conditions for identifying complete-data distribution and missingness mechanism.

The paper introduces tests for missing data models based on graph assumptions.

problem Verification of assumptions in missing data models is insufficiently addressed.
method The paper explores three classes of missing data models and designs goodness-of-fit tests.
result The paper provides new insights and tests for missing data graphical models.

A new method for semi-supervised learning with missing data using GMM and margin confidence.

problem Handling missing data in semi-supervised learning with classification uncertainty.
method Explicitly models missingness mechanism, uses margin confidence and Aranda Ordaz function, develops ECM algorithm.
result Effective reduction of bias and robustness in semi-supervised learning with substantial missing labels.

This work addresses missing data imputation for MNAR scenarios with identifiable deep generative models.

problem Missing data with complex missingness mechanisms (MNAR) leading to biased imputation results.
method Systematic analysis and proposal of an identifiable deep generative model.
result Proposed model provides identifiability guarantees under mild assumptions for various MNAR mechanisms.

Proposes a Latent Block Model for analyzing missing data.

problem Missing data can lead to misleading conclusions if not properly accounted for.
method Co-clustering model based on Latent Block Model, with variational EM algorithm for inference and model selection criterion.
result The proposed model reveals meaningful groups and insights from non-voters in French Parliament voting records.

A deep generative model improves imputation of MNAR data by treating missing and complete data equally.

problem Missing Not At Random (MNAR) data in analysis.
method A generative model-specific joint probability decomposition method (conjunction model) and a deep generative imputation model (GNR).
result GNR surpasses state-of-the-art MNAR baselines with significant margins in RMSE and better mask reconstruction.

A novel k-means method for MNAR data improves clustering accuracy.

problem Improving k-means clustering for data missing not at random.
method A magnitude-decaying MNAR scenario-based k-means method with size constraints.
result The method reduces bias in estimated cluster centers and improves clustering accuracy.

Autoencoder neural network is implemented to estimate the missing data. Genetic algorithm is implemented for network optimization and estimating the missing data. Missing data is treated as Missing At Random mechanism by implementing maximum likelihood algorithm. The network performance is determined by calculating the…

2008-12-09abs ↗pdf ↗

Optimal transfer learning for missing not-at-random matrix completion using source data.

problem Matrix completion in a Missing Not-at-Random setting with incomplete and noisy source data.
method Active sampling of rows and columns, feature shift in latent space, minimax lower bounds, computationally efficient estimation framework.
result Achieves minimax lower bound for active sampling setting, avoiding incoherence assumptions.

Random forest (RF) missing data algorithms are an attractive approach for dealing with missing data. They have the desirable properties of being able to handle mixed types of missing data, they are adaptive to interactions and nonlinearity, and they have the potential to scale to big data settings. Currently there are …

2017-01-19abs ↗pdf ↗

We present a method for incorporating missing data in non-parametric statistical learning without the need for imputation. We focus on a tree-based method, Bayesian Additive Regression Trees (BART), enhanced with "Missingness Incorporated in Attributes," an approach recently proposed incorporating missingness into deci…

2013-06-03abs ↗pdf ↗

Study improves unbiased recommender learning by addressing missing-reward bias.

problem Data bias caused by missing-reward observations in recommender systems.
method Proposes a novel estimator using propensity scores to mitigate both position and reward bias.
result The proposed estimator outperforms other methods, even with increased reward observation bias.

Missing values frequently arise in modern biomedical studies due to various reasons, including missing tests or complex profiling technologies for different omics measurements. Missing values can complicate the application of clustering algorithms, whose goals are to group points based on some similarity criterion. A c…

2019-02-26abs ↗pdf ↗

Missing values challenge data analysis because many supervised and unsupervised learning methods cannot be applied directly to incomplete data. Matrix completion based on low-rank assumptions are very powerful solution for dealing with missing values. However, existing methods do not consider the case of informative mi…

2018-12-29abs ↗pdf ↗

This paper tackles time series imputation by identifying and modeling different missing mechanisms.

problem Different types of missing mechanisms (MAR, MNAR) in time series data.
method Proposes a framework for time series imputation by analyzing data generation processes and modeling latent variables via variational inference and normalizing flow.
result Establishes identifiability results for latent variables under nonlinear independent component analysis, showing that latent variables are identifiable.

Kernel ridge regression for causal inference with missing data.

problem Estimating treatment effects with missing data in selected samples.
method Kernel ridge regression estimators for nonparametric dose response curves and semiparametric treatment effects.
result Uniform consistency and finite sample rates for continuous treatment, root-n consistency for discrete treatment.

Random imputation is surprisingly effective for linear predictors in missing data scenarios.

problem The effectiveness of naive imputation in missing data scenarios for linear predictors.
method A unique random features model framework to study predictive performances.
result Naive imputation is negligible in bias for linear predictors under MCAR assumption.

Proposes a framework to handle missing data in traffic forecasting with sensor blackouts.

problem Missing data in traffic forecasting due to sensor blackouts, especially when correlated with traffic conditions.
method Latent state-space framework that models traffic dynamics and sensor dropout probabilities.
result Improves traffic forecasting by reducing blackout imputation RMSE from 7.02 to 4.23, with MNAR modeling providing additional gains.

A new imputation method MissARF uses adversarial random forests for fast and accurate missing value imputation.

problem Handling missing values in biostatistical analyses.
method Adversarial Random Forests (ARF) for density estimation and data synthesis.
result MissARF performs comparably to state-of-the-art methods in imputation quality and runtime.

NeuMiss networks tackle supervised learning with missing values, offering efficient and robust predictions.

problem Challenges in supervised learning with missing values, especially when the response is a linear function of the complete data.
method Derive analytical form of optimal predictor under linearity assumption and various missing data mechanisms. Propose NeuMiss networks using multiplication by missingness indicator.
result Upper bound on Bayes risk and good predictive accuracy with independent complexity of missing data patterns.

Study compares traditional and machine learning methods for handling missing data in longitudinal studies.

problem Handling missing Not at Random (MNAR) and nonnormal data in longitudinal research.
method Monte Carlo simulations to assess six missing data techniques.
result FIML is most effective for MNAR data, while TSRE excels for MAR data.

A probabilistic query may not be estimable from observed data corrupted by missing values if the data are not missing at random (MAR). It is therefore of theoretical interest and practical importance to determine in principle whether a probabilistic query is estimable from missing data or not when the data are not MAR.…

2016-11-15abs ↗pdf ↗

PLS-SVD struggles with missing data in multimodal datasets, showing a phase transition in performance.

problem Missing data in PLS-SVD for multimodal datasets.
method Replica-symmetric analysis of spiked rectangular random matrices with missing entries.
result PLS-SVD performance transitions from uninformative to informative singular vectors at a critical signal-to-noise threshold.

A very simple interpretation of matrix completion problem is introduced based on statistical models. Combined with the well-known results from missing data analysis, such interpretation indicates that matrix completion is still a valid and principled estimation procedure even without the missing completely at random (M…

2016-05-10abs ↗pdf ↗

Unified framework for variable selection in model-based clustering with missing data.

problem Challenges in identifying relevant variables and handling missing data in model-based clustering.
method Unified framework incorporating a data-driven penalty matrix and a mechanism for missingness modeling.
result Achieves both asymptotic consistency and selection consistency in the presence of missing data.