Improved Gumbel watermark detection method.
problem Detecting Gumbel watermarking with high accuracy.
method A near-optimal model-agnostic detection mechanism.
result Demonstrates near-optimal performance under i.i.d. assumptions.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Gumbel watermarking”
Study on estimating Gumbel--Max watermark proportions in edited documents.
problem Estimating the proportion of a document generated from a watermarked LLM.
method Comparison of full observation and pivotal reduction observation regimes; development of estimators and information-theoretic lower bounds.
result Full observation yields a substantially smaller sample complexity compared to pivotal reduction.
Estimates proportions of LLM-generated text in mixed documents.
problem Estimating the proportion of text generated by a pre-specified LLM in mixed documents.
method Developed estimators for two observation regimes: full observation and pivotal reduction, and established sample complexity bounds.
result Full observation estimators require fewer samples than pivotal reduction estimators.
New framework improves text watermark detection under imperfect pseudorandomness.
problem Structured dependence in generated text from language models causes Type I error control issues.
method Hierarchical two-layer partition, minimal units, non-asymptotic efficiency measure, minimax hypothesis testing.
result Closed-form optimal rules for watermark detection under imperfect pseudorandomness.
New method detects watermarks in LLM-generated text with human edits.
problem Dilution of watermark signals by human edits on LLM-generated text.
method Truncated goodness-of-fit test (Tr-GoF) for robust detection.
result Tr-GoF achieves optimality in robust detection of Gumbel-max watermark.
Simple attack bypasses state-of-the-art DNN watermarking.
problem Protecting DNN models from watermark removal attacks.
method Combining imperceptible pattern embedding and spatial-level transformations for a simple yet effective watermark removal.
result Our attack bypasses state-of-the-art watermarking solutions with high success rates.
Watermarking of deep neural networks (DNN) can enable their tracing once released by a data owner. In this paper, we generalize white-box watermarking algorithms for DNNs, where the data owner needs white-box access to the model to extract the watermark. White-box watermarking algorithms have the advantage that they do…
As companies continue to invest heavily in larger, more accurate and more robust deep learning models, they are exploring approaches to monetize their models while protecting their intellectual property. Model licensing is promising, but requires a robust tool for owners to claim ownership of models, i.e. a watermark. …
Combining watermark and non-watermark detectors improves LLM detection.
problem Challenges in detecting watermarked language models due to limited entropy.
method Investigates hybrid schemes combining watermark and non-watermark detectors.
result Hybrid schemes outperform either class of detector under various conditions.
Study evaluates Tree-Ring Watermarking in rectified flow-based models, revealing detection and separability limitations.
problem Detecting and separating Tree-Ring Watermarks in rectified flow-based models.
method Extensive experimentation comparing SD 2.1 and FLUX.1-dev models with various text guidance configurations and augmentation attacks.
result Inversion limitations affect watermark recovery and statistical separation.
Unified framework for online LLM watermark detection using e-processes.
problem Detecting AI-generated text from human-written content in online settings.
method Unified framework based on e-processes for anytime-valid hypothesis testing on independence.
result Proposed methods achieve competitive performance in watermark detection.
Estimates watermarked content proportions in mixed-source texts.
problem Optimally estimating the proportion of watermarked content in texts with mixed sources.
method Casting the problem as estimating a proportion parameter in a mixture model based on pivotal statistics.
result Proposes efficient estimators for watermark proportion and shows their accuracy through evaluations.
Obtaining the state of the art performance of deep learning models imposes a high cost to model generators, due to the tedious data preparation and the substantial processing requirements. To protect the model from unauthorized re-distribution, watermarking approaches have been introduced in the past couple of years. W…
Adaptive testing segments watermarked text from LLMs.
problem Distinguishing LLM-generated text from human-written content.
method Generalized likelihood-based detection method adapted to inverse transform sampling, removing prompt estimation sensitivity.
result Effective and robust method for segmenting watermarked text.
Global watermark for diffusion language models decouples detection from local contexts.
problem Watermarking in diffusion language models is challenging due to joint sampling of distributions over many unresolved positions.
method Proposes a global vector-valued sketch representation to control watermarking in masked diffusion language models.
result The method decouples detection from local contexts, resulting in an order-agnostic statistic and robustness.
WISER detects watermarked segments in text via epidemic change-point analysis.
problem Identifying watermarked segments in mixed-source texts.
method Epidemic change-point perspective, WISER algorithm.
result WISER outperforms state-of-the-art methods in accuracy and speed.
The paper optimizes watermarking of LLMs, improving detection ability.
problem Optimizing watermarking of large language models (LLMs) while balancing model distortion and detection ability.
method Formulated as a constrained optimization problem, developed an online dual gradient ascent algorithm proving asymptotic Pareto optimality.
result Guarantees an increased green list probability and detection ability.
A new watermarking method corrects bias in language models using maximal coupling.
problem Correcting bias in language model token distributions.
method Maximal coupling to balance bias correction and text quality.
result Outperforms prior techniques in preserving text quality and detectability.
Method detects if text is generated by a language model with watermarks.
problem Detecting if text is generated by a language model with watermarks.
method Randomization tests and change point detection techniques.
result Method ensures Type I and Type II error control and accurately identifies watermarked sub-strings.
Algorithm removes backdoor watermarks from neural networks robustly.
problem Protecting neural networks from backdoor attacks.
method Neural network laundering algorithm.
result Effective removal of backdoor watermarks with high test accuracy.
Develops a power-calibrated framework for LLM watermarking, optimizing tradeoffs between detectability and distortion.
problem The trade-off between detectability and semantic distortion in logit-based watermarking.
method Power-calibrated statistical framework for watermark hyperparameters, establishing explicit relationships.
result Derives practical parameter selection procedures achieving optimal tradeoffs under constraints.
ADD embeds a 48-bit message into images, achieving high accuracy and speed.
problem Embedding high-fidelity messages into images to detect authenticity and source.
method Two-stage process: linear combination and addition of watermark to image, followed by decoding.
result ADD achieves 100% decoding accuracy for 48-bit watermarking, with minimal performance drop under various distortions.
Paper introduces a statistical framework for watermarking LLM-generated text.
problem Detecting LLM-generated text from human-written text with statistical signals.
method Hypothesis testing formulation, pivotal statistic selection, secret key, closed-form expression of false negative rate, minimax optimization.
result Derives optimal detection rules for LLM-generated text, demonstrating higher power than existing methods.
Entangled watermarks improve model defense against extraction attacks.
problem Model extraction attacks risk leaking valuable intellectual property.
method Entangled Watermarking Embeddings (EWE) leverages model capacity to overfit to outlier input-output pairs.
result The approach successfully claims model ownership with high confidence and minimal performance loss.
Proposes a general deep neural network method for digital watermarking.
problem Protecting intellectual content in a massive, IoT-acquired image dataset.
method Train a neural network on an image set and use it to protect distinct test images in bulk.
result Demonstrates the robustness and practicality of the proposed method.
FaceSigns embeds a secret watermark in images to authenticate and detect deepfakes.
problem Realistic image and video manipulation threats, especially deepfakes.
method Semi-fragile watermarking using neural networks, robust to face-swapping but fragile to deepfake manipulations.
result FaceSigns can reliably detect deepfake content with high accuracy.
Training machine learning (ML) models is expensive in terms of computational power, amounts of labeled data and human expertise. Thus, ML models constitute intellectual property (IP) and business value for their owners. Embedding digital watermarks during model training allows a model owner to later identify their mode…
We formulate statistical watermarking as hypothesis testing and establish near-optimal bounds.
problem Statistical watermarking in the context of hypothesis testing.
method Formulated as a hypothesis testing problem, using coupling of output tokens and rejection regions.
result Established nearly matching upper and lower bounds on the number of i.i.d. tokens required for small Type I and Type II errors.
This paper proposes a novel scheme for the watermarking of Deep Reinforcement Learning (DRL) policies. This scheme provides a mechanism for the integration of a unique identifier within the policy in the form of its response to a designated sequence of state transitions, while incurring minimal impact on the nominal pe…
WAFFLE embeds watermarks in federated learning models without access to training data.
problem Model theft in federated learning due to lack of centralized control.
method Retraining step at server after model aggregation, generating backdoor as watermark.
result Efficient embedding of resilient watermark with negligible accuracy loss.
Due to the rapid growth of machine learning tools and specifically deep networks in various computer vision and image processing areas, application of Convolutional Neural Networks for watermarking have recently emerged. In this paper, we propose a deep end-to-end diffusion watermarking framework (ReDMark) which can be…
Paper develops a new watermarking framework for LLMs.
problem Efficiently distinguishing machine-generated from human text.
method Develops Anchored E-Watermarking framework for anytime-valid inference.
result Framework enables valid, anytime-inference with reduced sample efficiency.
The commercialization of deep learning creates a compelling need for intellectual property (IP) protection. Deep neural network (DNN) watermarking has been proposed as a promising tool to help model owners prove ownership and fight piracy. A popular approach of watermarking is to train a DNN to recognize images with ce…
Ideal attribution mechanisms track model interactions for faithful watermarks.
problem Ensuring models provide transparent and fair attribution decisions.
method Introducing ideal attribution mechanisms and a ledger for tracking model interactions.
result A unified framework for evaluating watermarking schemes, clarifying attainable guarantees.
A new algorithm FastGM speeds up generating Gumbel-Max variables.
problem Efficiently generating multiple Gumbel-Max variables from high-dimensional vectors.
method FastGM reduces time complexity from to by generating variables in descending order.
result Significantly reduces computation time for generating Gumbel-Max variables.
A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learningcs.LG
The Gumbel-max trick and its extensions simplify sampling from categorical distributions in machine learning.
problem Sampling from categorical distributions with unnormalized probabilities.
method Extensions of the Gumbel-max trick for various applications.
result Simplified and efficient methods for sampling and gradient estimation.
The Gumbel trick is a method to sample from a discrete probability distribution, or to estimate its normalizing partition function. The method relies on repeatedly applying a random perturbation to the distribution in a particular way, each time solving for the most likely configuration. We derive an entire family of r…
Researchers analyze backdoor data poisoning attacks and identify a memorization capacity parameter.
problem Understanding and mitigating backdoor data poisoning attacks in machine learning models.
method Formal theoretical framework, statistical and computational analysis, explicit constructions, and algorithm design.
result Identified a memorization capacity parameter to assess vulnerability to backdoor attacks and developed algorithms to detect and mitigate them.
Paper tackles source attribution for LLM-generated texts.
problem Identify the data provider of synthetic texts generated by LLMs.
method Watermarking to embed information about data providers in synthetic texts.
result Proposes and evaluates a source attribution framework that achieves effective source attribution.
NTL protects AI models by restricting their generalization ability to specific domains.
problem Protecting AI models as intellectual property in a secure and robust manner.
method Non-Transferable Learning (NTL) captures exclusive data representation and restricts model generalization ability.
result NTL provides robust resistance to watermark removal and data-centric protection for usage authorization.
Paper proposes a new estimator for generic discrete distributions.
problem Estimating gradients for stochastic nodes in deep generative models.
method Generalized Gumbel-Softmax estimator using truncation, Gumbel-Softmax trick, and linear transformation.
result Efficacy and practical value demonstrated in synthetic examples and topic models.
Paper improves Gumbel-Softmax estimator variance reduction.
problem Challenges in gradient estimation for models with discrete latent variables.
method Rao-Blackwellization applied to straight-through Gumbel-Softmax estimator.
result Reduces mean squared error and variance of Gumbel-Softmax estimator.
Proposes a new method for estimating counterfactual treatment effects.
problem Uncertainty in identifying causal mechanisms from observational data.
method Introduces a parameterized family of causal mechanisms that generalize Gumbel-max, trained to minimize counterfactual effect variance.
result Trained mechanisms yield lower variance estimates of counterfactual treatment effects.
Generative Adversarial Networks (GAN) have limitations when the goal is to generate sequences of discrete elements. The reason for this is that samples from a distribution on discrete objects such as the multinomial are not differentiable with respect to the distribution parameters. This problem can be avoided by using…
The paper tackles data misappropriation in LLMs by embedding watermarks and testing for their presence.
problem Detecting data misappropriation in LLMs trained on copyrighted data.
method Embedding watermarks, formulating as hypothesis testing, developing statistical framework, constructing test statistics, determining optimal thresholds, controlling errors, establishing asymptotic optimality.
result The proposed statistical testing framework effectively detects data misappropriation in LLMs.
Categorical variables are a natural choice for representing discrete structure in the world. However, stochastic neural networks rarely use categorical latent variables due to the inability to backpropagate through samples. In this work, we present an efficient gradient estimator that replaces the non-differentiable sa…
A* Samplingstat.CO
The problem of drawing samples from a discrete distribution can be converted into a discrete optimization problem. In this work, we show how sampling from a continuous distribution can be converted into an optimization problem over continuous space. Central to the method is a stochastic process recently described in ma…
Investigates statistical properties of perturb-softmax and perturb-argmax distributions.
problem Underexplored statistical properties of Gumbel-Softmax and Gumbel-Argmax distributions.
method Investigates convexity and differentiability to determine completeness and minimality of these distributions.
result Identifies parameters that admit complete and minimal representation of probability distributions.