Framework generates precise synthetic populations for scalable modeling.
problem Generating accurate synthetic populations without personal data.
method Constraint-programming framework encoding aggregated statistics and structural relations.
result Exact control of demographic profiles without requiring microdata.
New method constructs synthetic treatment groups without mean exchangeability assumption.
problem Violations of mean exchangeability assumption in randomized controlled trials.
method Weighted mixture of treatment groups from source populations, minimizing conditional maximum mean discrepancy.
result Asymptotic normality of synthetic treatment group estimator established.
New method generates geolocated synthetic populations from real data.
problem Generating synthetic populations with explicit geographic coordinates.
method Mapping coordinates into a latent space using Normalizing Flows (NF), then combining with other features in a Variational Autoencoder (VAE).
result NF+VAE architecture outperforms existing methods in generating geolocated synthetic populations.
Copula-based method generates synthetic populations from marginal distributions.
problem Generating realistic synthetic populations from limited data.
method Copula-based framework for population synthesis.
result Copula framework enhances transferability and realism of synthetic populations.
A new model synthesizes population with fewer structural and sampling zeros.
problem Synthesizing a feasible and diverse synthetic population from limited data.
method A deep generative model with two regularizations to minimize structural zeros and preserve sampling zeros.
result The model significantly improves feasibility and diversity of synthetic populations.
CTGAN synthesizes population data for travel behavior simulation.
problem Synthesizing population data for agent-based transportation modeling.
method Composite Travel Generative Adversarial Network (CTGAN).
result Consistent and accurate generation of synthetic populations with tabular and sequential mobility data.
Population synthesis is concerned with the generation of synthetic yet realistic representations of populations. It is a fundamental problem in the modeling of transport where the synthetic populations of micro-agents represent a key input to most agent-based models. In this paper, a new methodological framework for ho…
PolicySynth improves synthetic data alignment with real data for better campaign decisions.
problem Synthetic data used in decision support systems often leads to incorrect decisions.
method PolicySynth framework that conditions synthetic data on churn scorer to align with real data decisions.
result PolicySynth achieves high strategy simulation fidelity (0.923-0.960) on churn and acquisition datasets.
Syntax designs adaptive trials for subpopulations with potential benefits.
problem Identifying subpopulations with positive treatment effects in diverse patient populations.
method Adaptive patient recruitment and synthetic control estimation.
result Syntax outperforms conventional trial designs in identifying beneficial subpopulations.
A new method combines synthetic data analysis and DP generation to produce accurate uncertainty estimates.
problem Invalid inferences from DP synthetic data analysis.
method Combining synthetic data analysis techniques from MI and NA Bayesian modeling with a novel noise-aware synthetic data generation algorithm.
result Accurate confidence intervals from DP synthetic data are produced, wider with tighter privacy.
New method for valid prediction intervals in counterfactual outcomes with runtime confounding.
problem Valid prediction intervals for counterfactual outcomes under runtime confounding.
method Debiased machine learning framework grounded in semiparametric efficiency theory.
result Prediction intervals achieve desired coverage rates with faster convergence compared to standard methods.
Synthetic population generation is the process of combining multiple socioeconomic and demographic datasets from different sources and/or granularity levels, and downscaling them to an individual level. Although it is a fundamental step for many data science tasks, an efficient and standard framework is absent. In this…
Synthetic data augmentation can improve imbalanced classification metrics.
problem Improving imbalanced classification metrics
method Developing a framework for analyzing the effects of synthetic data augmentation on score-based classification
result Augmentation can improve AUROC, AUPRC, balanced accuracy, and F1 score
In many applications, different populations are compared using data that are sampled in a biased manner. Under sampling biases, standard methods that estimate the difference between the population means yield unreliable inferences. Here we develop an inference method that is resilient to sampling biases and is able to …
New framework for adaptive clinical trials to address real-world challenges.
problem Real-world challenges in post-regulatory clinical trials.
method RFAN framework integrating regulatory constraints and treatment policy value.
result Empirical evaluation of RFAN's performance.
New method infers dynamical systems from population data.
problem Inferring dynamical systems from population data.
method Deducing and estimating Fokker-Planck equation, projecting to test functions, sparse inference.
result Induces driving forces of dynamical systems.
Paper tackles estimating individual treatment effects from observational data.
problem Estimating the difference between outcomes with and without treatment from single observation.
method Formulated as inference from hidden variables, uses a model of four causal populations, proposes ECM algorithm.
result ECM algorithm provides better performance compared to baseline methods on synthetic and real-world data.
Active learning method for neural population dynamics using optogenetics.
problem Efficiently selecting neurons to stimulate for identifying neural population dynamics.
method Developed active learning procedure for low-rank regression to determine informative photostimulation patterns.
result Demonstrated a two-fold reduction in data required for predictive power using low-rank linear dynamical systems model.
Uncertainty sampling, a popular active learning algorithm, is used to reduce the amount of data required to learn a classifier, but it has been observed in practice to converge to different parameters depending on the initialization and sometimes to even better parameters than standard training on all the data. In this…
Semi-supervised GAN creates synthetic genetic data for disease prediction.
problem Expensive and time-consuming to build large labeled genetic databases.
method Semi-supervised Genetic Generative Adversarial Network (gGAN).
result Model achieved satisfactory results with real genetic data.
We use multiple measures of graph complexity to evaluate the realism of synthetically-generated networks of human activity, in comparison with several stylized network models as well as a collection of empirical networks from the literature. The synthetic networks are generated by integrating data about human populatio…
A new method identifies sub-populations in unlabelled heterogeneous data by accounting for co-features.
problem Estimating sub-populations in unlabelled heterogeneous data with co-features.
method Mixture of Conditional Gaussian Graphical Models (CGGM) with penalized EM algorithm.
result The method successfully identifies sub-populations disrupted by co-features.
Analyzes how bias evolves in SGD training across different data sub-populations.
problem Understanding bias formation during machine learning training.
method Analytical description of SGD dynamics in a teacher-student setup with Gaussian-mixture model.
result Different sub-populations influence bias at different timescales, revealing shifting classifier preferences.
Kernel measures similarity of nonlinear causal structures in heterogeneous populations.
problem Learning causal structure in populations with diverse underlying structures.
method Distance covariance-based kernel for measuring similarity of causal structures.
result Kernel enables clustering of homogeneous subpopulations for causal structure learning.
Evaluating the computational reproducibility of data analysis pipelines has become a critical issue. It is, however, a cumbersome process for analyses that involve data from large populations of subjects, due to their computational and storage requirements. We present a method to predict the computational reproducibili…
New method predicts model performance under selection bias in healthcare.
problem Selection bias limits model generalizability in healthcare.
method Proposes a novel upper bound method for estimating model performance.
result Validates and demonstrates the practical utility of the method.
We consider continuous time Markovian processes where populations of individual agents interact stochastically according to kinetic rules. Despite the increasing prominence of such models in fields ranging from biology to smart cities, Bayesian inference for such systems remains challenging, as these are continuous tim…
New method estimates treatment effects across different populations.
problem Estimating treatment effects across populations with changing distributions.
method SBRL-HAP framework combining balancing and independence regularizers with hierarchical attention.
result Significant improvement in HTE estimation across out-of-distribution populations.
We investigate the problems of identity and closeness testing over a discrete population from random samples. Our goal is to develop efficient testers while guaranteeing Differential Privacy to the individuals of the population. We describe an approach that yields sample-efficient differentially private testers for the…
In this work, we consider hypothesis testing and anomaly detection on datasets where each observation is a weighted network. Examples of such data include brain connectivity networks from fMRI flow data, or word co-occurrence counts for populations of individuals. Current approaches to hypothesis testing for weighted n…
The paper analyzes SMOTE for imbalanced classification, providing theoretical bounds and guidelines.
problem The challenge of imbalanced classification problems, especially with minority classes.
method Theoretical analysis of SMOTE and related oversampling techniques for minority classes.
result Derives concentration and excess risk bounds for SMOTE and kernel-based classifiers.
This work generates synthetic EHRs with privacy guarantees for machine learning tasks.
problem Privacy concerns and heterogeneity in EHR data limit their use in machine learning.
method Generative Adversarial Networks (GANs) with differential privacy (DP) for synthetic data generation.
result Synthetic EHRs maintain performance close to real data, even with DP applied.
Framework evaluates quality of synthetic data generated with differential privacy.
problem Ensuring synthetic data retains statistical quality after applying differential privacy.
method Developed a framework to evaluate synthetic data quality from a practical researcher's viewpoint.
result Synthetic data can be evaluated against training data or underlying populations, and for specific tasks like inference or prediction.
NPE trains neural networks to approximate posterior distributions in SIR models from final outcome data.
problem Computational challenges in Bayesian inference for SIR models with final outcome data.
method Neural posterior estimation (NPE) using a logNormal posterior approximated by a neural network.
result NPE accurately recovers reference posteriors across various population sizes and transmission regimes.
Bayesian method identifies causal sets across populations without graph knowledge.
problem Transporting causal information across populations without causal graph knowledge.
method Combines observational and experimental data to identify s-admissible backdoor sets.
result Proves asymptotic convergence and corrects transportability bias in simulations.
Estimates class prior for unlabeled data using kernel embedding.
problem Estimating class prior in PU learning scenario where only positive and full population samples are available.
method Direct estimator based on distribution matching and kernel embedding in Reproducing Kernel Hilbert Space.
result Asymptotic consistency and explicit deviation bound for the estimator.
A new algorithm reconstructs population dynamics from coarse samples.
problem Reconstructing population dynamics from unlabeled samples at coarse time intervals.
method Deep Momentum Multi-Marginal Schrödinger Bridge (DMSB) framework.
result Significantly outperforms baselines in synthetic and real-world datasets.
Proposes efficient data acquisition for personalized treatment effects from observational data.
problem Efficiently acquiring outcomes for personalized treatment effects in observational studies.
method Introduces causal, Bayesian acquisition functions to select points with overlapping support.
result Demonstrates improved sample efficiency and accuracy in learning personalized treatment effects.
Synthetic augmentation improves financial machine learning performance in variance-dominant regimes.
problem Data scarcity in financial machine learning.
method Formalized synthetic augmentation, introduced size-matched null augmentation, and developed a non-parametric block permutation test.
result Synthetic augmentation is beneficial only in variance-dominant regimes, such as persistent volatility forecasting.
Clustering and community detection with multiple graphs have typically focused on aligned graphs, where there is a mapping between nodes across the graphs (e.g., multi-view, multi-layer, temporal graphs). However, there are numerous application areas with multiple graphs that are only partially aligned, or even unalign…
We lay theoretical foundations for new database release mechanisms that allow third-parties to construct consistent estimators of population statistics, while ensuring that the privacy of each individual contributing to the database is protected. The proposed framework rests on two main ideas. First, releasing (an esti…
Paper explores how unsupervised learning can be understood through linear algebra concepts.
problem Understanding unsupervised learning through linear algebra concepts.
method Introducing the concept of linearly independent populations and using them to solve for prevalence values.
result Unsupervised learning can be realized as a generalization of supervised learning.
In increasingly many settings, data sets consist of multiple samples from a population of networks, with vertices aligned across these networks. For example, brain connectivity networks in neuroscience consist of measures of interaction between brain regions that have been aligned to a common template. We consider the …
We introduce an evolutionary algorithm called recombinator-k-means for optimizing the highly non-convex kmeans problem. Its defining feature is that its crossover step involves all the members of the current generation, stochastically recombining them with a repurposed variant of the k-means++ seeding algorithm. Th…
Improving Bayesian filtering with strictly proper scoring rules
problem Bayesian filtering of partially and noisily observed dynamical systems
method Proper scoring ensemble filter (PSEF)
result Accurate approximation of challenging filtering distributions
Proposes a method to correct for covariate shift in meta-analysis of randomized trials.
problem Invalidation of standard IPD meta-analysis due to covariate shift across studies.
method Placebo-anchored transport framework that treats source-trial outcomes as proxy signals and target-trial placebo outcomes as gold labels.
result Yields target-identified effect estimates in connected targets and a principled screen--then--transport procedure in disconnected targets.
This paper is based on a previous publication [29]. Our work extends exception mining and outlier detection to the case of object-relational data. Object-relational data represent a complex heterogeneous network [12], which comprises objects of different types, links among these objects, also of different types, and at…
The paper studies stability of generative models trained on mixed data.
problem Training generative models on mixed datasets (real and synthetic data).
method Developed a framework to rigorously study stability under specific conditions.
result Proved the stability of iterative training under certain conditions.