Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

3.8%7.6%11.4%15.2% · Mar 199719922001200920172026
48 results for missing-data mechanism

Paper explores Bayes rule for Gaussian mixtures with missing data, outperforming supervised classifiers.

problem Improving classification accuracy in partially classified samples with missing data.
method Generative model framework with missing-data mechanism, Bayes rule allocation.
result Bayes rule classifier with missing-data mechanism outperforms fully supervised classifiers in various conditions.

Study on missing data mechanisms and simple imputation methods in fairness of machine learning algorithms.

problem Impact of missing data mechanisms and simple imputation methods on fairness of machine learning algorithms.
method Three popular datasets for classification fairness were used. Missing values were generated using three missing data mechanisms. Various missing data handling techniques (listwise deletion, mean imputation, mode imputation, multiple imputation) were applied to the datasets. Fairness was assessed using classification algorithms (random forests).
result Missing data mechanism does not significantly impact fairness; listwise deletion gives highest fairness on average.

We present a general framework, the coupled compound Poisson factorization (CCPF), to capture the missing-data mechanism in extremely sparse data sets by coupling a hierarchical Poisson factorization with an arbitrary data-generating model. We derive a stochastic variational inference algorithm for the resulting model …

2017-01-09abs ↗pdf ↗

Paper tackles informative labels in semi-supervised learning, proposing debiasing methods.

problem Informative labels can bias semi-supervised learning models, especially when some classes are more likely to be labeled.
method Estimates missing-data mechanism and uses inverse propensity weighting to debias SSL algorithms.
result Proposed methods improve SSL performance, demonstrated on various datasets including medical ones.

Autoencoder neural network is implemented to estimate the missing data. Genetic algorithm is implemented for network optimization and estimating the missing data. Missing data is treated as Missing At Random mechanism by implementing maximum likelihood algorithm. The network performance is determined by calculating the…

2008-12-09abs ↗pdf ↗

New model handles missing data effectively in autoregressive models.

problem Handling missing data in autoregressive models.
method Reinterpret existing models through missing data lens, introduce principled framework for incomplete datasets, active information acquisition.
result MO-ARM consistently outperforms imputation baselines across real-world benchmarks.

TMLE improves causal effect estimation in missing data scenarios with various positivity violations.

problem Estimating causal effects in studies with missing data and positivity violations.
method Targeted Maximum Likelihood Estimation (TMLE) with various missing data methods.
result Complete cases with TMLE incorporating an outcome-missingness model exhibit lower bias and greater robustness against positivity violations.

As systems are getting more autonomous with the development of artificial intelligence, it is important to discover the causal knowledge from observational sensory inputs. By encoding a series of cause-effect relations between events, causal networks can facilitate the prediction of effects from a given action and anal…

2020-01-15abs ↗pdf ↗

MissBGM uses AI and Bayesian modeling for better missing data imputation.

problem Missing data imputation in data science, especially with uncertainty quantification.
method AI-powered Bayesian generative modeling with explicit modeling of missingness mechanisms.
result MissBGM provides principled posterior uncertainty over imputations and superior performance.

The paper analyzes kk-means clustering for missing data, proving statistical guarantees under MCAR.

problem Statistical guarantees for kk-means clustering with missing data, especially under Missing Completely at Random (MCAR).
method Established n\sqrt{n}-excess risk bound and consistency of cluster centers under general missing mechanisms; derived n\sqrt{n}-convergence rate and asymptotic normality for MCAR.
result Achieving n\sqrt{n}-rate and converging to true cluster centers requires distinct true cluster centers in every dimension under MCAR.

New methods for identifying and estimating missing data under complex mechanisms.

problem Missing data mechanisms dependent on missing values themselves.
method Developed a new MNAR model and proposed semiparametric estimation methods.
result Established sufficient conditions for identifying complete-data distribution and missingness mechanism.

Algorithm identifies missing data distributions in graphical models.

problem Identifying missing data distributions in graphical models with interventionist perspective.
method Tree-based identification algorithm that tracks selection bias and admissible intervention strategies.
result Valid estimating equations for missingness mechanism and complete data distribution.

Adapts conformal prediction for missing data, ensuring valid coverage.

problem Uncertainty quantification with missing covariates.
method Proposes a reweighted conformal prediction procedure for handling missing values.
result Guaranteed Marginal Coverage and Mask-Conditional Validity for general missing data mechanisms.

Matrix completion is a modern missing data problem where both the missing structure and the underlying parameter are high dimensional. Although missing structure is a key component to any missing data problems, existing matrix completion methods often assume a simple uniform missing mechanism. In this work, we study ma…

2018-12-19abs ↗pdf ↗

Proposes methods to handle missing data in clustering models.

problem Missing data, especially MNAR, hinders model-based clustering.
method Developed a mixture model for different types of data, including MNAR, using Expectation Maximization algorithm.
result The proposed MNARz model simplifies inference and enables clustering with MNAR data.

Random forest (RF) missing data algorithms are an attractive approach for dealing with missing data. They have the desirable properties of being able to handle mixed types of missing data, they are adaptive to interactions and nonlinearity, and they have the potential to scale to big data settings. Currently there are …

2017-01-19abs ↗pdf ↗

Two methods use BART to model missing data in leaf photosynthetic trait data.

problem Handling missing data in multivariate outcomes with non-ignorable mechanisms.
method Bayesian Additive Regression Trees (BART) for joint modeling of data and missingness indicators.
result Both methods effectively recover various missingness mechanisms and outperform existing approaches.

We propose a general, theoretically justified mechanism for processing missing data by neural networks. Our idea is to replace typical neuron's response in the first hidden layer by its expected value. This approach can be applied for various types of networks at minimal cost in their modification. Moreover, in contras…

2018-05-18abs ↗pdf ↗

We focus on an interpolation method referred to Bayesian reconstruction in this paper. Whereas in standard interpolation methods missing data are interpolated deterministically, in Bayesian reconstruction, missing data are interpolated probabilistically using a Bayesian treatment. In this paper, we address the framewor…

2014-04-23abs ↗pdf ↗

Study compares traditional and machine learning methods for handling missing data in longitudinal studies.

problem Handling missing Not at Random (MNAR) and nonnormal data in longitudinal research.
method Monte Carlo simulations to assess six missing data techniques.
result FIML is most effective for MNAR data, while TSRE excels for MAR data.

A new method for semi-supervised learning with missing data using GMM and margin confidence.

problem Handling missing data in semi-supervised learning with classification uncertainty.
method Explicitly models missingness mechanism, uses margin confidence and Aranda Ordaz function, develops ECM algorithm.
result Effective reduction of bias and robustness in semi-supervised learning with substantial missing labels.

Estimates disease prevalence using non-ignorable missing data in health surveys.

problem Estimating disease prevalence in non-representative samples with non-ignorable missing data.
method Connects auxiliary proxy variable framework to label shift setting, uses high-dimensional covariates without generative models.
result Fails to account for non-ignorable missingness can lead to significant misestimations.

Bayesian approach models nonignorable missing data using copulas and marginal quantiles.

problem Nonignorable missing data in lead exposure and test score analysis.
method Gaussian copula model with auxiliary marginal quantiles for missingness indicators and study variables.
result Efficient MCMC algorithm estimates copula correlation and marginal distributions consistently.

New MMD estimators detect differences in missing paired data.

problem Handling missing data in matched pairs with complex distributions.
method Maximum mean discrepancy (MMD) estimators for complex data with missing values.
result Valid and consistent estimators detect differences in data distributions.

A probabilistic query may not be estimable from observed data corrupted by missing values if the data are not missing at random (MAR). It is therefore of theoretical interest and practical importance to determine in principle whether a probabilistic query is estimable from missing data or not when the data are not MAR.…

2016-11-15abs ↗pdf ↗

Estimation under missing data shows computational and statistical limits for Gaussian data.

problem Estimation of population parameters in high-dimensional data with missing data.
method Analysis of computational and statistical limits for mean and covariance estimation, and linear regression.
result Statistical-computational gaps exist for mean and covariance estimation, but not for linear regression.

Ranked data appear in many different applications, including voting and consumer surveys. There often exhibits a situation in which data are partially ranked. Partially ranked data is thought of as missing data. This paper addresses parameter estimation for partially ranked data under a (possibly) non-ignorable missing…

2019-02-28abs ↗pdf ↗

Proposes methods to correct bias and missing data in regression models.

problem Nonignorable selection bias and missing response in regression models.
method Imputation-based and importance weighted regression methods, including repeated regression and doubly robust combination.
result Repeated regression can effectively correct bias and outperforms weighted regression in extrapolation.

NeuMiss networks tackle supervised learning with missing values, offering efficient and robust predictions.

problem Challenges in supervised learning with missing values, especially when the response is a linear function of the complete data.
method Derive analytical form of optimal predictor under linearity assumption and various missing data mechanisms. Propose NeuMiss networks using multiplication by missingness indicator.
result Upper bound on Bayes risk and good predictive accuracy with independent complexity of missing data patterns.

Unified framework for variable selection in model-based clustering with missing data.

problem Challenges in identifying relevant variables and handling missing data in model-based clustering.
method Unified framework incorporating a data-driven penalty matrix and a mechanism for missingness modeling.
result Achieves both asymptotic consistency and selection consistency in the presence of missing data.

We present a method for incorporating missing data in non-parametric statistical learning without the need for imputation. We focus on a tree-based method, Bayesian Additive Regression Trees (BART), enhanced with "Missingness Incorporated in Attributes," an approach recently proposed incorporating missingness into deci…

2013-06-03abs ↗pdf ↗

Novel imputation method for EHRs with structured and sporadic missingness.

problem Missing data in integrated EHR datasets for clinical applications.
method Macomss, a novel imputation framework for structurally and heterogeneously missing data.
result Macomss outperforms existing methods in imputation and downstream prediction accuracy.

This work addresses missing data imputation for MNAR scenarios with identifiable deep generative models.

problem Missing data with complex missingness mechanisms (MNAR) leading to biased imputation results.
method Systematic analysis and proposal of an identifiable deep generative model.
result Proposed model provides identifiability guarantees under mild assumptions for various MNAR mechanisms.

Improved traffic forecasting model handles missing data.

problem Short-term traffic forecasting with missing values.
method Proposed SBU-LSTM architecture with bidirectional and unidirectional LSTM.
result Superior performance in accuracy and robustness for network-wide traffic prediction.