New data accumulation prevents model collapse in generative models.
problem Model collapse in generative models trained on their own outputs.
method Empirical study of language models, diffusion models, and variational autoencoders; analytically tractable framework for linear models.
result Accumulating synthetic data alongside real data avoids model collapse, preventing performance degradation.
Study identifies negative data externalities affecting model performance on specific groups.
problem Negative data externalities on group performance in machine learning models.
method Characterized and detected data-model inefficiencies, focusing on specific types of externalities.
result Negative data externalities can lower model performance on specific sub-groups, even with larger datasets.
CoFinDiff generates synthetic financial data capturing stylized facts and meeting specified conditions.
problem Limited data availability and difficulty in controlling synthetic financial data generation.
method Conditional diffusion model with cross-attention to incorporate conditions derived from price data.
result Synthetic data generated by CoFinDiff accurately meets specified conditions for trends and volatility.
Picket guards against corrupted data in machine learning models.
problem Data corruption biases models and invalidates predictions.
method PicketNet detects corrupted data using self-supervised deep learning; flags corrupted queries online.
result Picket consistently protects models from corrupted data during training and deployment.
Two methods use BART to model missing data in leaf photosynthetic trait data.
problem Handling missing data in multivariate outcomes with non-ignorable mechanisms.
method Bayesian Additive Regression Trees (BART) for joint modeling of data and missingness indicators.
result Both methods effectively recover various missingness mechanisms and outperform existing approaches.
Prevents sensitive data generation in diffusion models using labeled and unlabeled data.
problem Generating sensitive data in diffusion models using unlabeled data.
method Positive-Unlabeled Diffusion Models, approximating ELBO with labeled and unlabeled data.
result Prevents the generation of sensitive data without compromising image quality.
Extract synthetic data from pretrained models for tasks without training data.
problem Lack of training data for tasks requiring model initialization.
method Extract 'Data Impressions' from pretrained deep models' parameters.
result Data Impressions enable various tasks like unsupervised domain adaptation and continual learning.
Breiman's data analysis dichotomy is outdated, offering a third approach: mechanistic models.
problem Data analysis dichotomy between data modelers and algorithmic modelers.
method Interpolating between simple interpretable models and flexible function approximations using mechanistic models.
result Flexible, interpretable, and scientifically-informed hybrids can provide accurate and robust predictions.
Framework uses synthetic data from pretrained models to improve predictive modeling.
problem Limited effectiveness of synthetic data from generative models for improving predictive performance.
method Proposes an end-to-end framework that generates and filters synthetic data through domain-specific statistical methods.
result Consistent improvements in predictive performance across various settings.
In-Run Data Shapley offers efficient data attribution for large-scale models.
problem Existing data attribution methods are computationally intensive and cannot target specific models.
method In-Run Data Shapley, which efficiently attributes data contributions to a specific model without re-training.
result In-Run Data Shapley achieves significant efficiency, enabling data attribution for pretraining models.
New method reveals how training data influence diffusion model outputs.
problem Difficulty in assessing training data impact on diffusion model outputs.
method Use of ensembles trained on carefully engineered splits of training data to identify influential training examples.
result Demonstrated the viability of ensembles as generative models and validity of assessing influence.
Bayesian data sketching speeds up inference for large functional data.
problem Slow posterior computations in Bayesian varying coefficient models for large data.
method Compress functional response and predictor matrix using random linear transformation.
result Fully model-based Bayesian inference on compressed data.
PUMA augments models to remove unique data points without performance loss.
problem Preserving model performance while removing unique training data points.
method Explicitly models data influence, reweights remaining data optimally.
result PUMA effectively removes unique data points without performance degradation.
Synthetic tabular data improves privacy while maintaining model performance.
problem Protecting privacy in synthetic data generation for machine learning.
method Deep generative models for tabular data, emphasizing privacy and model performance.
result Deep generative models enhance synthetic data generation for tabular datasets.
Method infers dynamics from incomplete time series data.
problem Challenges in inferring stochastic dynamics from time series with missing data.
method Expectation Maximization (EM) algorithm that iterates between E-step and M-step.
result The EM algorithm effectively recovers missing data points and infers underlying network models from real neuronal activities.
Generative models learn distributions of continuous functions.
problem Training generative models on discretized grids limits model size and data type.
method Parameterize data points by continuous functions, learn distributions over these functions.
result Models can learn rich distributions of functions independently of data type and resolution.
Framework reuses pre-trained models for data-free transfer learning.
problem Challenges in retrieving source data for model training.
method Model Recycling Framework for parameter-efficient training.
result Makes multi-source data-free supervised transfer learning possible.
FinDiff generates synthetic financial data for regulatory tasks.
problem Sharing microdata for research due to privacy regulations.
method Diffusion model using embedding encodings for mixed modality financial data.
result FinDiff excels in generating high-fidelity, privacy-preserving synthetic financial data.
We propose a probabilistic model for refining coarse-grained spatial data by utilizing auxiliary spatial data sets. Existing methods require that the spatial granularities of the auxiliary data sets are the same as the desired granularity of target data. The proposed model can effectively make use of auxiliary data set…
New method estimates model parameters from incomplete data.
problem Estimating model parameters from incomplete data.
method Variational Gibbs Inference (VGI)
result Competitive or better performance compared to existing methods.
We present a general framework, the coupled compound Poisson factorization (CCPF), to capture the missing-data mechanism in extremely sparse data sets by coupling a hierarchical Poisson factorization with an arbitrary data-generating model. We derive a stochastic variational inference algorithm for the resulting model …
The paper tackles imbalance in production data by proposing sampling methods to improve model performance on underrepresented observations.
problem Imbalance in production data negatively impacts model predictive performance on underrepresented observations.
method Three sampling approaches are investigated to adjust for imbalance in training data and improve model performance.
result Fitting a model using sampled data yields a small reduction in overall predictive performance but a better performance on underrepresented observations.
The paper studies stability of generative models trained on mixed data.
problem Training generative models on mixed datasets (real and synthetic data).
method Developed a framework to rigorously study stability under specific conditions.
result Proved the stability of iterative training under certain conditions.
Paper proposes methods to learn accurate models from incomplete data without imputation.
problem Learning accurate models from datasets with missing values.
method Unified approach for checking data imputation necessity and efficient algorithms.
result Significant reduction in time and effort needed for data imputation.
Enhances generative model for clinical data privacy and accuracy.
problem Data privacy in electronic patient records.
method Improves a time-series generative model with privacy safeguards.
result DP-TimeGAN achieves a mean authenticity of 0.778 on the CKD dataset.
We propose a simple yet powerful framework for modeling integer-valued data, such as counts, scores, and rounded data. The data-generating process is defined by Simultaneously Transforming and Rounding (STAR) a continuous-valued process, which produces a flexible family of integer-valued distributions capable of modeli…
Foundation models alter medical data science workflow, challenging veridical data science principles.
problem Foundation models disrupt traditional data science practices in medicine.
method Critically examined the medical foundation model lifecycle and its deviation from veridical data science principles.
result Foundation models challenge veridical data science principles of predictability, computability, and stability.
Paper shows Cox model optimisation leaks patient data in distributed learning.
problem Risk of patient data leakage in distributed learning models.
method Optimisation of Cox survival model using federated learning.
result Optimisation process can leak patient data, necessitating new validation methods.
Good data stewardship requires removal of data at the request of the data's owner. This raises the question if and how a trained machine-learning model, which implicitly stores information about its training data, should be affected by such a removal request. Is it possible to "remove" data from a machine-learning mode…
Deep learning models trained using massive amounts of data tend to capture one view of the data and its associated mapping. Different deep learning models built on the same training data may capture different views of the data based on the underlying techniques used. For explaining the decisions arrived by blackbox dee…
Paper establishes utility theory for synthetic data generation.
problem Lack of theoretical understanding in synthetic data utility.
method Statistical learning framework with two utility metrics: generalization and model ranking.
result Theoretical bounds for synthetic data utility metrics ensure comparable generalization and consistent model comparison.
Dividing deep learning models for consistent anomaly detection in changing log data.
problem Anomaly detection methods fail when log data types change, leading to false negatives.
method Divide deep learning models based on log data correlation and extract correlations.
result Continues anomaly detection accuracy even when log data changes.
SUMER updates deployed models with new data, improving performance.
problem Updating deployed machine learning models with new data is difficult.
method SUMER uses semi-supervised learning and noise remediation to iteratively retrain models.
result SUMER improves model performance, especially with limited initial training data.
MMM model clusters mixed-type longitudinal data efficiently.
problem Challenges in clustering multivariate longitudinal mixed-type data.
method MMM model reorganizes data into a three-way structure, using a mixture of matrix-variate normal distributions.
result MMM model handles various data types (continuous, ordinal, binary, nominal, count) and temporal dependence.
This dissertation automates deep learning pipelines and uses meta-learning for better model selection and data augmentation.
problem Challenges in selecting and fine-tuning deep learning pipelines for new datasets.
method Meta-learning for DL pipeline selection and data augmentation, using synthetic data.
result Meta-learned approaches outperform traditional methods in automated DL pipeline selection and data augmentation.
Adding data can sometimes hurt model performance in multi-source healthcare tasks.
problem Identifying when adding more data helps or hinders model outcomes in multi-source healthcare tasks.
method Identified the Data Addition Dilemma, demonstrated empirically observed trade-offs, introduced distribution shift heuristics.
result Adding data can sometimes reduce model performance due to distribution shift.
Develops geometric causal models for causal inference from dependent data.
problem Causal inference from structured, dependent data (e.g., spatial, network, molecular).
method Geometric causal models (GCMs) exploiting symmetries of data generating process, combining group theory, ergodic theory, and Bayesian inference.
result Establishes identification and estimation of causal effects from dependent data.
Diffusion models generalize better with hierarchical data structure and regularization.
problem Understanding generalization in diffusion models with finite data.
method Analyzing diffusion models through data covariance spectra and developing a theoretical framework based on linear neural networks.
result Generalization in diffusion models improves with hierarchical data structure and regularization.
Paper tackles model collapse in recursive generative models using a weighted training scheme.
problem Model collapse in recursive generative models trained on synthetic data.
method Iteratively trains models on real and synthetic data, evaluates weighted training schemes.
result Optimal weighting scheme for synthetic data follows a unified expression, revealing a trade-off with model performance.
Mixture of Experts (MoE) is a popular framework for modeling heterogeneity in data for regression, classification, and clustering. For regression and cluster analyses of continuous data, MoE usually use normal experts following the Gaussian distribution. However, for a set of data containing a group or groups of observ…
Mixture of Experts (MoE) is a popular framework in the fields of statistics and machine learning for modeling heterogeneity in data for regression, classification and clustering. MoE for continuous data are usually based on the normal distribution. However, it is known that for data with asymmetric behavior, heavy tail…
Diffusion models reveal latent hierarchical structure in data.
problem Quantitative measurement of data's hierarchical structure.
method Forward-backward experiments in diffusion models.
result Changes in latent variables manifest as correlated chunks in data.
We present a new model-based integrative method for clustering objects given both vectorial data, which describes the feature of each object, and network data, which indicates the similarity of connected objects. The proposed general model is able to cluster the two types of data simultaneously within one integrative p…
AutoDiff combines auto-encoder and diffusion model for realistic tabular data synthesis.
problem Generating realistic synthetic tabular data with heterogeneous features.
method Employing auto-encoder architecture to handle tabular data's complexity.
result Synthetic tables from AutoDiff show good statistical fidelity and perform well in machine learning tasks.
Formula adjusts steady-state models for control confounding.
problem Learning steady-state models from operational data can be flawed due to control confounding.
method Derives a formula to adjust for control confounding using structural dynamical causal models.
result Estimates a causal steady-state model from closed-loop operational data.
In scientific inference problems, the underlying statistical modeling assumptions have a crucial impact on the end results. There exist, however, only a few automatic means for validating these fundamental modelling assumptions. The contribution in this paper is a general criterion to evaluate the consistency of a set …
Big data sets must be carefully partitioned into statistically similar data subsets that can be used as representative samples for big data analysis tasks. In this paper, we propose the random sample partition (RSP) data model to represent a big data set as a set of non-overlapping data subsets, called RSP data blocks,…
DL models can outperform regionalized models in hydrology by pooling diverse data.
problem Traditional wisdom in hydrology suggests regionalization improves model performance, but DL models can unify data for better performance.
method Used DL models on pooled data from different regions, showing improved performance compared to regionalized models.
result DL models can improve performance by pooling diverse data, highlighting the 'data synergy' effect.