Develops a method to control model behavior with synthetic data.
problem Controlling language model behavior via synthetic training data.
method Dataset Policy Gradient (DPG) to optimize synthetic data generators.
result Can cause target models to embed specific patterns or rephrase inputs.
New approach generates better synthetic data for neural program synthesis.
problem Current approaches to neural program synthesis generalize poorly to real data.
method Adversarial approach to control synthetic data distributions.
result Proposed method outperforms current approaches.
AdaDPSyn generates synthetic examples to protect private data in ICL.
problem Protecting private data in in-context learning with large language models.
method Data-adaptive differentially private algorithm that dynamically adjusts noise level based on data properties.
result AdaDPSyn outperforms existing methods in preserving high ICL accuracy while maintaining differential privacy.
Example of spacetime with causal bubbling, splitting into timelike and spacelike parts.
problem Understanding causal bubbling in spacetimes.
method Example of a globally hyperbolic spacetime with a continuous metric, splitting orthogonally into timelike and spacelike parts.
result The synthetic timelike curvature-dimension (TCD) condition does not prevent causal bubbling.
New algorithms for clustering and synthetic data generation of heterogeneous tabular datasets.
problem Clustering and generating synthetic data from heterogeneous tabular datasets with hidden cluster structure.
method Developed MMM and MMMsynth algorithms for clustering and synthetic data generation.
result MMMsynth algorithm outperforms other literature tabular-data generators and approaches real data performance.
Synthetic Ricci flows defined for metric measure spaces.
problem Characterizing Ricci flows for non-smooth spaces.
method Heat flow, optimal transport, volume asymptotics.
result Equivalent characterizations of weighted Ricci flow.
Synthetic datasets help study bias in ML, overcoming data scarcity.
problem Lack of relevant datasets for bias research in ML.
method Presented a family of synthetic datasets with adjustable bias levels.
result Demonstrated an experiment using synthetic data to study bias.
Persistent homology can recognize knotting in curves.
problem Recognizing knotting in curves
method Compute one-dimensional persistent homology, extract cycle representatives, and assign a hypergraph curvature-based score.
result Systematic differences between knotted and unknotted structures are revealed.
Fine-tuning LLMs on privacy-sensitive data introduces privacy risk, and synthetic data audits can quantify this risk.
problem Fine-tuning LLMs on privacy-sensitive data introduces privacy risk.
method Generate synthetic canaries via high-temperature sampling from LLMs.
result Synthetic canaries are high-influence outliers that ensure strong audits.
Labels distilled from images improve model training efficiency and flexibility.
problem Creating synthetic labels for a small set of real images to train models effectively.
method Introduce a more robust and flexible meta-learning algorithm for distillation and an effective first-order strategy based on convex optimization layers.
result Label distillation leads to improved results and greater flexibility in neural architectures.
Data augmentation is rapidly gaining attention in machine learning. Synthetic data can be generated by simple transformations or through the data distribution. In the latter case, the main challenge is to estimate the label associated to new synthetic patterns. This paper studies the effect of generating synthetic data…
A new method learns from synthetic data without needing real-world examples.
problem Learning robust classifiers from limited real-world data.
method A novel setting and algorithm exploiting synthetic data independence.
result Robust classifiers trained on synthetic data generalize well to real-world domains.
Programming by example is the problem of synthesizing a program from a small set of input / output pairs. Recent works applying machine learning methods to this task show promise, but are typically reliant on generating synthetic examples for training. A particular challenge lies in generating meaningful sets of inputs…
A new method combines synthetic data analysis and DP generation to produce accurate uncertainty estimates.
problem Invalid inferences from DP synthetic data analysis.
method Combining synthetic data analysis techniques from MI and NA Bayesian modeling with a novel noise-aware synthetic data generation algorithm.
result Accurate confidence intervals from DP synthetic data are produced, wider with tighter privacy.
ERM struggles with synthetic data, but some algorithms can still learn correctly.
problem Learning from a mix of natural and synthetic data.
method Modeling the scenario as a sequence of learning tasks with oblivious algorithms, studying ERM and its limitations.
result ERM converges to the true mean but is outperformed by weighted algorithms. ERM does not always converge in the PAC setting, but there are algorithms capable of learning the correct hypothesis.
This work shows synthetic gradients can outperform backpropagation in sample efficiency.
problem The efficiency of backpropagation in training neural networks.
method Unified vectorized feedback framework for loss-based and reward-based learning, introducing synthetic gradients.
result Synthetic gradients can achieve lower gradient-estimation mean squared error than backpropagation under certain conditions.
One of the challenges of using machine learning techniques with medical data is the frequent dearth of source image data on which to train. A representative example is automated lung cancer diagnosis, where nodule images need to be classified as suspicious or benign. In this work we propose an automatic synthetic lung …
Improves synthetic-to-real generalization without real data.
problem Synthetic models struggle with real data generalization.
method Encourages similar ImageNet representations and automates learning rates.
result Significant improvement in synthetic-to-real generalization.
The paper develops methods to create private synthetic spatial point patterns.
problem Generating private synthetic spatial point patterns.
method Developed differentially private Poisson and Cox point synthesizers.
result The synthesizers effectively maintain privacy and utility of synthetic data.
The paper analyzes SMOTE for imbalanced classification, providing theoretical bounds and guidelines.
problem The challenge of imbalanced classification problems, especially with minority classes.
method Theoretical analysis of SMOTE and related oversampling techniques for minority classes.
result Derives concentration and excess risk bounds for SMOTE and kernel-based classifiers.
Optimal transport explored on a specific geometric space.
problem Optimal transport problem in sub-Lorentzian Heisenberg group.
method Synthetic metric spacetime structure analysis and sub-Lorentzian version of Brenier's theorem.
result Established sub-Lorentzian version of Brenier's theorem and derived Monge-Ampère equation.
Synthetic data can be used to ask more questions and accelerate discovery with provable validity guarantees.
problem Valid inference with synthetic data
method Task exchangeability
result Provable validity guarantees for synthetic data inference
This paper considers the quantification of the prediction performance in Gaussian process regression. The standard approach is to base the prediction error bars on the theoretical predictive variance, which is a lower bound on the mean square-error (MSE). This approach, however, does not take into account that the stat…
Develops optimal transport in Lorentzian spaces with synthetic curvature bounds.
problem Synthetic curvature bounds for Lorentzian spaces.
method Optimal transport, convexity analysis of entropy functionals.
result Synthetic notion of timelike Ricci curvature lower bounds.
New method minimizes experiment cost while maintaining accuracy.
problem Minimizing cost in experiments with interference or other concerns.
method Synthetically Controlled Thompson Sampling (SCTS).
result Minimizes regret and maintains inferential ability.
SSMBA generates synthetic data to improve robustness in natural language tasks.
problem Improving out-of-domain generalization of models trained on natural language data.
method SSMBA uses corruption and reconstruction functions to generate synthetic data points on the manifold assumption.
result SSMBA consistently outperforms existing methods on robustness benchmarks across multiple tasks and datasets.
This paper tackles imbalanced data in binary classification problems.
problem Imbalanced data leads to skewed results in classification problems.
method Synthetic Minority Oversampling Technique (SMOTE) and Adaptive Synthetic (ADASYN) Sampling Approach.
result Synthetic data points enhance understanding of oversampling techniques.
Deep learning methods, and in particular convolutional neural networks (CNNs), have led to an enormous breakthrough in a wide range of computer vision tasks, primarily by using large-scale annotated datasets. However, obtaining such datasets in the medical domain remains a challenge. In this paper, we present methods f…
Inherent risk scoring is an important function in anti-money laundering, used for determining the riskiness of an individual during onboarding before fraudulent transactions occur. It is, however, often fraught with two challenges: (1) inconsistent notions of what constitutes as high or low risk by experts a…
Interactive learning is a process in which a machine learning algorithm is provided with meaningful, well-chosen examples as opposed to randomly chosen examples typical in standard supervised learning. In this paper, we propose a new method for interactive learning from multiple noisy labels where we exploit the disagr…
The paper shows averaging gradients leads to memorization, proposing an alternative algorithm to focus on invariances.
problem The principle that 'good explanations are hard to vary' in deep learning is investigated.
method Formalizing consistency for loss surface minima, proposing an alternative algorithm based on logical AND.
result The alternative algorithm prevents memorization and focuses on invariances.
This paper presents a method to generate synthetic data with differential privacy to protect user privacy while maintaining data trends.
problem Protecting user privacy while using real data for AI applications.
method Develops a practical guide to generating synthetic data using differential privacy.
result Synthetic data can be generated to preserve trends of real data while ensuring strong privacy protections.
Foundation models leak sensitive data in synthetic tabular data generation, especially LLaMA 3.3 70B.
problem Privacy leakage in synthetic tabular data generation using foundation models.
method Benchmarked three foundation models (GPT-4o-mini, LLaMA 3.3 70B, TabPFN v2) against four baselines on 35 real-world tables.
result Foundation models, especially LLaMA 3.3 70B, have the highest privacy risk in synthetic tabular data generation.
Study shows current image classification models lack robustness to real-world dataset shifts.
problem Robustness of current image classification models to natural distribution shifts in real datasets.
method Evaluation of 204 ImageNet models in 213 different test conditions.
result Little to no transfer of robustness from synthetic to natural distribution shifts.
Dataset distillation is a method for reducing dataset sizes by learning a small number of synthetic samples containing all the information of a large dataset. This has several benefits like speeding up model training, reducing energy consumption, and reducing required storage space. Currently, each synthetic sample is …
The estimation of unknown values of parameters (or hidden variables, control variables) that characterise a physical system often relies on the comparison of measured data with synthetic data produced by some numerical simulator of the system as the parameter values are varied. This process often encounters two major d…
We consider the optimization of an uncertain objective over continuous and multi-dimensional decision spaces in problems in which we are only provided with observational data. We propose a novel algorithmic framework that is tractable, asymptotically consistent, and superior to comparable methods on example problems. O…
FinDiff generates synthetic financial data for regulatory tasks.
problem Sharing microdata for research due to privacy regulations.
method Diffusion model using embedding encodings for mixed modality financial data.
result FinDiff excels in generating high-fidelity, privacy-preserving synthetic financial data.
We present a windowed technique to learn parsimonious time-varying autoregressive models from multivariate timeseries. This unsupervised method uncovers interpretable spatiotemporal structure in data via non-smooth and non-convex optimization. In each time window, we assume the data follow a linear model parameterized …
Meta-learning framework for few-shot one-class classification using order-equivariant networks.
problem Few labeled examples for positive class in one-class classification tasks.
method Order-equivariant networks for meta-learning a binary classifier conditioned on positive examples.
result Meta-learning framework outperforms baselines on unseen synthetic streams.
Synthetic theory defines orbifolds as microlinear types with finite identifications.
problem Defining orbifolds in traditional set-level foundations with internal symmetries.
method Synthetic differential cohesive homotopy type theory, microlinearity, finite identifications.
result Proper étale groupoids are orbifolds in synthetic theory.
The paper introduces CPICFs for better counterfactual explanations in high-dimensional spaces.
problem Creating useful counterfactual explanations for complex machine learning models.
method Modeling individual knowledge and using conformal prediction intervals to identify informative counterfactuals.
result CPICFs provide more informative counterfactuals by considering individual knowledge and prediction uncertainty.
Instance- and Label-dependent label Noise (ILN) widely exists in real-world datasets but has been rarely studied. In this paper, we focus on Bounded Instance- and Label-dependent label Noise (BILN), a particular case of ILN where the label noise rates -- the probabilities that the true labels of examples flip into the …
Machine learning algorithms can misrepresent training data, study finds.
problem Misrepresentation of training data in machine learning algorithms.
method Demonstrated through underestimation of training data due to irreducible error, regularization, and class imbalance.
result Careful management of synthetic counterfactuals can mitigate underestimation bias.
Obtaining deep networks that are robust against adversarial examples and generalize well is an open problem. A recent hypothesis even states that both robust and accurate models are impossible, i.e., adversarial robustness and generalization are conflicting goals. In an effort to clarify the relationship between robust…
The goal of program synthesis is to automatically generate programs in a particular language from corresponding specifications, e.g. input-output behavior. Many current approaches achieve impressive results after training on randomly generated I/O examples in limited domain-specific languages (DSLs), as with string tra…
In this work we propose a model that can manipulate individual visual attributes of objects in a real scene using examples of how respective attribute manipulations affect the output of a simulation. As an example, we train our model to manipulate the expression of a human face using nonphotorealistic 3D renders of a f…
Modified CTGAN-Plus-Features method optimizes asset allocation with CVaR constraint.
problem Optimizing portfolio weights in asset allocation problems.
method Combines synthetic data generation with CVaR-constraint optimization.
result Synthetic data captures key characteristics of original data and outperforms conventional strategies.