Paper tackles source attribution for LLM-generated texts.
problem Identify the data provider of synthetic texts generated by LLMs.
method Watermarking to embed information about data providers in synthetic texts.
result Proposes and evaluates a source attribution framework that achieves effective source attribution.
The paper proposes a new framework to generate synthetic data with human-like imperfections to prevent model collapse.
problem Model collapse due to statistical optimization of synthetic data.
method Introduces Prompt-driven Cognitive Computing Framework (PMCSF) with Cognitive State Decoder (CSD) and Cognitive Text Encoder (CTE).
result The framework generates text with cognitive imperfections, reducing maximum drawdown and delivering defensive alpha.
Estimates watermarked content proportions in mixed-source texts.
problem Optimally estimating the proportion of watermarked content in texts with mixed sources.
method Casting the problem as estimating a proportion parameter in a mixture model based on pivotal statistics.
result Proposes efficient estimators for watermark proportion and shows their accuracy through evaluations.
Bayesian Topic Regression models causal inference with text and numerical data.
problem Causal inference using observational text data with both text and numerical confounders.
method Combines supervised Bayesian topic model with Bayesian regression framework, respecting the Frisch-Waugh-Lovell theorem.
result Joint approach recovers ground truth with lower bias than benchmarks, superior prediction results compared to separate approaches.
The Generative Adversarial Network (GAN) has achieved great success in generating realistic (real-valued) synthetic data. However, convergence issues and difficulties dealing with discrete data hinder the applicability of GAN to text. We propose a framework for generating realistic text via adversarial training. We emp…
A variety of methods existing for generating synthetic electronic health records (EHRs), but they are not capable of generating unstructured text, like emergency department (ED) chief complaints, history of present illness or progress notes. Here, we use the encoder-decoder model, a deep learning algorithm that feature…
FRAPPE estimates tensor canonical rank without CPD computation.
problem Estimating the canonical rank of tensors efficiently.
method Generates synthetic data matching input tensor's size and sparsity, trains a regression model to estimate rank.
result 24 times faster than best baseline, 10% improvement in MAPE on synthetic dataset.
Proposes neural network for causal inference with multimodal data.
problem Estimating causal effects with text and image data as confounders.
method Double machine learning framework adapted to partially linear models, semi-synthetic dataset generation.
result Improved performance in causal effect estimation with multimodal data.
EASTER improves OCR efficiency and scalability.
problem Efficient and scalable Optical Character Recognition (OCR) for machine printed and handwritten text.
method 1-D convolutional layers without recurrence, parallel training, synthetic dataset generation.
result EASTER achieves comparable performance to complex RNN models with less data and outperforms them on benchmark datasets.
Dataset distillation is a method for reducing dataset sizes by learning a small number of synthetic samples containing all the information of a large dataset. This has several benefits like speeding up model training, reducing energy consumption, and reducing required storage space. Currently, each synthetic sample is …
New method detects text changes under dependencies, outperforming baselines.
problem Detecting structural changes in m-dependent text data. method Kernel change-point detection under m-dependence. result Consistent and weakly consistent detection of change points in m-dependent text. Synthetic continued pretraining enhances model performance with synthetic data.
problem Data inefficiency in pretrained models when adapting to domain-specific documents.
method Synthetic data augmentation using EntiGraph to create a large synthetic corpus.
result Language models can answer questions and follow instructions without access to domain-specific documents.
SHMM models human mobility from GPS and text data, overcoming text sparsity.
problem Modeling human mobility from semantic trace data, especially addressing text sparsity.
method SHMM is a multi-modal spherical hidden Markov model that jointly models location, time, and text embeddings on a unit sphere using vMF distribution.
result SHMM outperforms state-of-the-art models in next location prediction and has lower training cost.
Paper develops heavy-tailed embeddings for better text classification and augmentation.
problem Improving text classification, especially for extreme values.
method Develops heavy-tailed embeddings using multivariate extreme value theory and introduces a scale-invariant classifier.
result The classifier outperforms baselines and generates meaningful augmented text.
CATR rationalizes text data to stabilize causal effect estimation.
problem Observational positivity violation in high-dimensional text data.
method Confounding-Aware Token Rationalization (CATR) selects necessary subset of tokens.
result CATR yields more accurate and stable causal effect estimates.
VEC-SBM detects communities using side information like texts and images.
problem Community detection in social networks with side information.
method Proposes a novel algorithm based on iterative refinement techniques.
result Optimally recovers latent communities with side information.
Modeling lead-lag relationship between two text corpora for improved topic modeling.
problem Recognizing the relationship between multiple text corpora for better topic modeling.
method Proposed a jointly dynamic topic model and embedding extension for large-scale text corpus.
result The proposed model can well recognize the lead-lag relationship between two text corpora and improve topic learning.
Paper tackles treatment leakage in text-based causal inference, proposing methods to mitigate bias.
problem Treatment leakage in text-as-confounder applications introduces bias in causal estimates.
method Formal definitions, four text distillation methods (passage removal, classification, salient feature removal, nullspace projection).
result Moderate distillation optimally balances bias reduction against confounder retention.
Data availability is a bottleneck during early stages of development of new capabilities for intelligent artificial agents. We investigate the use of text generation techniques to augment the training data of a popular commercial artificial agent across categories of functionality, with the goal of faster development o…
Does adding a theorem to a paper affect its chance of acceptance? Does labeling a post with the author's gender affect the post popularity? This paper develops a method to estimate such causal effects from observational text data, adjusting for confounding features of the text such as the subject or writing quality. We…
Proposes a new method to adapt to covariate shifts in supervised learning.
problem Covariate shift in training and testing samples with different marginal distributions.
method Minimax risk classification (MRC) approach that weights both training and testing samples.
result Significantly enhanced classification performance in synthetic and empirical experiments.
AMI framework improves text generation by optimizing mutual information between source and target.
problem Previous MI approaches ignored the backward network, leading to loose variational bounds.
method AMI is a saddle point optimization framework that iteratively promotes and demotes generated instances.
result AMI significantly outperforms baselines on various text generation tasks.
Proposes CLRS-Text, a new benchmark for evaluating LM reasoning capabilities.
problem Lack of transferable benchmarks for evaluating reasoning capabilities of language models.
method Developed a textual version of the CLRS benchmark, generating diverse algorithmic tasks.
result Demonstrates a novel challenge for the LM reasoning community and validates prior work.
A new method improves text generation quality and diversity.
problem Exposure bias in Maximum Likelihood Estimation for text generation.
method ψ-MLE, a new training scheme based on density ratio estimation.
result ψ-MLE outperforms Maximum Likelihood Estimation and other models in text generation quality and diversity.
RNNs classify text by accumulating evidence on a low-dimensional manifold.
problem Understanding how RNNs solve text classification tasks.
method Dynamical systems analysis applied to trained RNNs.
result RNNs use a low-dimensional attractor manifold to classify text.
Embed-KCPD segments text without labels, outperforming baselines.
problem Unsupervised text segmentation is challenging due to lack of boundary labels.
method Embed-KCPD represents sentences as embeddings and estimates boundaries via penalized KCPD.
result Embed-KCPD outperforms strong unsupervised baselines across benchmarks.
Improved text classification performance through conformal transformations of kernels.
problem Text document categorization in high-dimensional spaces.
method Introduced new Gaussian Cosine kernel and two conformal transformations.
result Conformal transformations significantly improve kernel performance, especially for sub-optimal kernels.
When the available data of a target speaker is insufficient to train a high quality speaker-dependent neural text-to-speech (TTS) system, we can combine data from multiple speakers and train a multi-speaker TTS model instead. Many studies have shown that neural multi-speaker TTS model trained with a small amount data f…
The Kalman filter (KF) is used in a variety of applications for computing the posterior distribution of latent states in a state space model. The model requires a linear relationship between states and observations. Extensions to the Kalman filter have been proposed that incorporate linear approximations to nonlinear m…
We consider the problem of making machine translation more robust to character-level variation at the source side, such as typos. Existing methods achieve greater coverage by applying subword models such as byte-pair encoding (BPE) and character-level encoders, but these methods are highly sensitive to spelling mistake…
As electronically stored data grow in daily life, obtaining novel and relevant information becomes challenging in text mining. Thus people have sought statistical methods based on term frequency, matrix algebra, or topic modeling for text mining. Popular topic models have centered on one single text collection, which i…
New method detects watermarks in LLM-generated text with human edits.
problem Dilution of watermark signals by human edits on LLM-generated text.
method Truncated goodness-of-fit test (Tr-GoF) for robust detection.
result Tr-GoF achieves optimality in robust detection of Gumbel-max watermark.
Forums play an important role in providing a platform for community interaction. The introduction of irrelevant content or spam by individuals for commercial and social gains tends to degrade the professional experience presented to the forum users. Automated moderation of the relevancy of posted content is desired. Ma…
This paper presents an algorithm for the unsupervised learning of latent variable models from unlabeled sets of data. We base our technique on spectral decomposition, providing a technique that proves to be robust both in theory and in practice. We also describe how to use this algorithm to learn the parameters of two …
Synthetic Combinations learns unit-specific causal outcomes for combinatorial interventions.
problem Estimating unit-specific causal outcomes for all combinations of p interventions in a heterogeneous setting. method Latent factor model with Fourier expansion sparsity, imposing structure across units and interventions.
result Synthetic Combinations provides consistent estimation with poly(r) * (N + s^2p) observations, outperforming previous methods.
The paper assesses text classification robustness through maximal safe radius computation.
problem Vulnerability of neural network models to small input modifications.
method Maximal safe radius computation, Monte Carlo Tree Search, syntactic filtering, linear bounding techniques.
result Approximation methods for computing upper and lower bounds of maximal safe radius.
ASK-NN detects distribution drifts in LLM-generated text.
problem Hallucinations and artificial text in LLM-generated outputs.
method Asymmetric two-sample test based on directed k-nearest-neighbor graph.
result ASK-NN is competitive with baselines on various benchmarks.
This work uses image generation models to find vision model bugs.
problem Automatically discovering failures in vision models.
method Conditional text-to-image generation and captioning models.
result Demonstrated utility of large-scale generative models to find vision model bugs.
Adapts causal inference for high-dimensional treatments like text strings.
problem Predicting effects of interventions with many possible variations.
method Adapts classical causal estimators to high-dimensional treatment spaces, balancing moment errors.
result Shows high-dimensional treatment spaces can be addressed with a single model.
Efficiently distills pretrained text-to-image models without real data, improving FID and CLIP scores.
problem Slow iterative refinement process of diffusion-based text-to-image models.
method Guided Score identity Distillation with Long and Short Classifier-Free Guidance.
result Achieves state-of-the-art FID performance with competitive CLIP score.
Privacy preserving networks can be modelled as decentralized networks (e.g., sensors, connected objects, smartphones), where communication between nodes of the network is not controlled by an all-knowing, central node. For this type of networks, the main issue is to gather/learn global information on the network (e.g.,…
CTSyn generates high-quality synthetic tabular data.
problem Challenges in generating high-quality synthetic tabular data.
method Diffusion-based generative foundation model with autoencoder and conditional latent diffusion.
result CTSyn outperforms existing table synthesizers on standard benchmarks.
LLMs encode latent topic distributions, suggesting Bayesian inference.
problem Capturing topic structure from large language models.
method Connecting LLM optimization to implicit Bayesian inference and de Finetti's theorem.
result LLMs recover latent topic distributions, matching LDA-generated topics.
Estimates sparse topic models with improved efficiency and adaptability.
problem Estimating sparse topic models with unknown sparsity and number of topics.
method Proposes a new algorithm for efficient estimation of sparse topic models with non-negative matrices.
result Upper bound matches minimax lower bound, demonstrating optimal performance.
Proposes a framework for generating explainable AI exemplars.
problem Need to explain decisions of complex deep learning models.
method Generative model with evolutionary strategy to synthesize exemplars.
result Framework is generic and model-agnostic for various data types.
Generative models simulate S&P 500 returns for financial analysis.
problem Modeling the joint distribution of S&P 500 equities.
method Conditional importance weighted autoencoders and conditional normalizing flows.
result Generative models accurately capture the complex joint distribution of S&P 500 returns.
This paper presents a method to generate synthetic data with differential privacy to protect user privacy while maintaining data trends.
problem Protecting user privacy while using real data for AI applications.
method Develops a practical guide to generating synthetic data using differential privacy.
result Synthetic data can be generated to preserve trends of real data while ensuring strong privacy protections.
In recent years, stochastic gradient descent (SGD) methods and randomized linear algebra (RLA) algorithms have been applied to many large-scale problems in machine learning and data analysis. We aim to bridge the gap between these two methods in solving constrained overdetermined linear regression problems---e.g., $\el…