GRPO optimizes LLMs with verifiable rewards, amplifying policy success.
problem Improving LLMs' reasoning under verifiable binary rewards.
method Introduces GRPO, analyzes variants of reward normalization and regularization.
result GRPO amplifies policy success, converging to a fixed point exceeding the reference.
Offline RL with pre-trained features amplifies errors even under mild shifts.
problem Sample-efficient offline RL with pre-trained features under mild distribution shift.
method Empirical study of offline RL with pre-trained neural representations.
result Substantial error amplification occurs even with pre-trained features, requiring stronger conditions for successful offline RL.
Feedback loops amplify dataset biases, affecting future model performance.
problem Feedback loops amplify biases in datasets, risking future model reliability.
method Formalized system where model interactions are recorded and reused, analyzed for bias amplification.
result Models that behave like samples from the training distribution are more stable and calibrated.
New method amplifies hidden structure in complex networks.
problem Difficulty in uncovering hidden structure in complex networks.
method Iterative weakening of dominant structure through randomization.
result Theoretical support for the effectiveness of structure amplification.
Success conditioning optimizes policies by imitating successful trajectories, solving a trust-region optimization problem.
problem Improving policies through random actions that lead to desired outcomes.
method Success conditioning, which involves collecting and updating policies based on successful trajectories.
result Success conditioning solves a trust-region optimization problem, maximizing policy improvement with a χ2 divergence constraint. Behavior cloning training instabilities amplified by SGD noise over long horizons.
problem Training instabilities in behavior cloning with deep neural networks.
method Empirical dissection of minibatch SGD updates and their effects on long-horizon rewards.
result Exponential moving average (EMA) of iterates effectively mitigates gradient variance amplification (GVA).
Proposes a new framework to manage venture capital portfolio risk by focusing on deal-level correlations.
problem Managing venture capital portfolio risk, especially extreme outcomes.
method Gaussian-copula-based framework that learns deal-level dependence from observed joint success frequencies.
result Correlation amplifies extreme upside outcomes, shifting portfolio distribution toward heavier right tails.
Improves privacy amplification by shuffling for differential privacy.
problem Enhancing privacy guarantees in systems with anonymous data contributions.
method Theoretical and numerical analysis of Rényi differential privacy parameters and privacy amplification by shuffling.
result First asymptotically optimal analysis of Rényi differential privacy parameters for shuffled outputs.
Efficient algorithm reduces control system regret to sqrt(t).
problem Control of linear quadratic systems with guaranteed anytime performance.
method Combines SDP-based framework with regularization and perturbation mechanisms.
result Achieves anytime regret of order sqrt(t) with explicit dependence on system dimensions.
We consider the core reinforcement-learning problem of on-policy value function approximation from a batch of trajectory data, and focus on various issues of Temporal Difference (TD) learning and Monte Carlo (MC) policy evaluation. The two methods are known to achieve complementary bias-variance trade-off properties, w…
We study the phenomenon of bias amplification in classifiers, wherein a machine learning model learns to predict classes with a greater disparity than the underlying ground truth. We demonstrate that bias amplification can arise via an inductive bias in gradient descent methods that results in the overestimation of the…
Mitigates gender bias amplification in model predictions.
problem Gender bias amplification in model predictions.
method Posterior regularization to mitigate bias.
result Almost removes gender bias amplification in model predictions.
Gradient amplification boosts deep learning model performance without increasing training time.
problem Vanishing gradients in deep neural networks.
method Gradient amplification approach to prevent vanishing gradients and training strategy to enable/disable across epochs.
result Improves performance of deep learning models with reduced training time.
Privacy is enhanced by synthetic data release even with unlimited data.
problem Improving privacy guarantees for synthetic data release.
method Analyzing a bounded-parameter assumption to show privacy amplification persists with unlimited synthetic records.
result Privacy amplification is possible even with an unbounded number of synthetic records.
New method amplifies privacy in decentralized learning without centralized communication.
problem Privacy amplification in decentralized federated learning.
method Random check-in protocol for DP-SGD in FL.
result Privacy/accuracy trade-offs similar to subsampling/shuffling, but without server-initiated communication.
The paper introduces SuccessProbaMax to optimize policy success probability in online advertising.
problem Optimizing policy success probability in online advertising systems.
method SuccessProbaMax algorithm that optimizes for the probability of success rather than expected value.
result SuccessProbaMax outperforms conventional algorithms in terms of success rate.
Study off-policy evaluation and learning in dynamic pricing with context.
problem Dynamic personalized pricing and operations management problems with high-dimensional user types.
method Formalize causal structure, leverage single time-step evaluation, estimate marginal MDP.
result Improved out-of-sample policy performance in dynamic and capacitated pricing.
DG separates successes and failures by gating updates with advantage and surprisal.
problem Negative learning from surprising data in distributed reinforcement learning.
method DG gates each update with the product of advantage and surprisal, suppressing failures and preserving successes.
result DG outperforms other methods in various challenging reinforcement learning tasks.
Structured subsampling improves privacy in deep time series forecasting.
problem Incompatible privacy guarantees with time series forecasting.
method Structured subsampling of sequential data for privacy amplification.
result Structured subsampling enables training with strong privacy guarantees.
Latent-state environments with long horizons, such as those faced by recommender systems, pose significant challenges for reinforcement learning (RL). In this work, we identify and analyze several key hurdles for RL in such environments, including belief state error and small action advantage. We develop a general prin…
Domain randomization (DR) is a successful technique for learning robust policies for robot systems, when the dynamics of the target robot system are unknown. The success of policies trained with domain randomization however, is highly dependent on the correct selection of the randomization distribution. The majority of…
A fundamental result in differential privacy states that the privacy guarantees of a mechanism are preserved by any post-processing of its output. In this paper we investigate under what conditions stochastic post-processing can amplify the privacy of a mechanism. By interpreting post-processing as the application of a…
Many real world learning tasks involve complex or hard-to-specify objectives, and using an easier-to-specify proxy can lead to poor performance or misaligned behavior. One solution is to have humans provide a training signal by demonstrating or judging performance, but this approach fails if the task is too complicated…
New method for privacy amplification without sampling for matrix factorization.
problem Privacy amplification for differentially private model training with matrix factorization.
method Sampling-free bounds based on Rényi divergence and conditional composition.
result Stronger privacy guarantees for small ε, applicable to various matrices.
Unified framework for subsampling mechanisms with tighter privacy guarantees.
problem Improving privacy in machine learning models through subsampling.
method Conditional optimal transport for deriving mechanism-specific subsampling guarantees.
result Tighter privacy bounds for subsampled mechanisms compared to traditional methods.
Reinforcement learning (RL) algorithms still suffer from high sample complexity despite outstanding recent successes. The need for intensive interactions with the environment is especially observed in many widely popular policy gradient algorithms that perform updates using on-policy samples. The price of such ineffici…
Transforming sparse outcomes into dense process rewards for efficient reinforcement learning.
problem Training RL policies to maximize sparse outcomes.
method Incentivizing policy matching state-action visitations of successful episodes.
result Significantly faster RL finetuning performance.
The paper develops a theory explaining how machine learning models can amplify biases.
problem Understanding and mitigating bias in machine learning models.
method Analytical theory of ridge regression with and without random projections.
result Observations and predictions align with empirical data on machine learning bias.
Differential privacy comes equipped with multiple analytical tools for the design of private data analyses. One important tool is the so-called "privacy amplification by subsampling" principle, which ensures that a differentially private mechanism run on a random subsample of a population provides higher privacy guaran…
New sampling scheme improves privacy in DP-SGD without sacrificing utility.
problem Suboptimal privacy amplification due to participation variance in Poisson subsampling.
method Balanced Iteration Subsampling (BIS) with structured randomness.
result BIS achieves stronger privacy amplification than Poisson subsampling and is optimal at both extremes of noise spectrum.
We mathematically compare four competing definitions of group-level nondiscrimination: demographic parity, equalized odds, predictive parity, and calibration. Using the theoretical framework of Friedler et al., we study the properties of each definition under various worldviews, which are assumptions about how, if at a…
FLAME improves privacy in federated learning without trusted parties.
problem Ensuring privacy in federated learning without trusted parties.
method FLAME uses the shuffle model of differential privacy to achieve better accuracy and privacy.
result FLAME protocols improve testing accuracy by 60.7% compared to local model FL.
This study quantifies systemic importance in global banks using a continuous framework that amplifies localized shocks.
problem Analyzing financial contagion and systemic risk in global banks.
method Developed a continuous framework incorporating geographic proximity and interbank network linkages, using a master equation and Feynman-Kac representation.
result The amplification factor correctly identifies systemically important institutions and predicts crisis outcomes.
We consider a multi-armed bandit problem in a setting where each arm produces a noisy reward realization which depends on an observable random covariate. As opposed to the traditional static multi-armed bandit problem, this setting allows for dynamically changing rewards that better describe applications where side inf…
Missing data enhances privacy in differential privacy.
problem Privacy preservation in datasets with missing values.
method Formalized missing data as a privacy amplification mechanism within differential privacy.
result Incomplete data can yield privacy amplification for differentially private algorithms.
Sharp pseudospectral bounds prevent transient amplification in coupled gradient descent.
problem Transient amplification in coupled gradient descent systems.
method Developed a sharp pseudospectral theory for block-triangular Jacobians, proving Kreiss constant bounds and matching minimax lower bounds.
result Obtained a finite-horizon iteration-complexity bound of O(K(J)2log(1/δ)) for stochastic coupled descent. The study explains how market-makers' hedging affects stock volatility during gamma-squeeze events.
problem Endogenous volatility amplification in option markets during gamma-squeeze events.
method Developed a theoretical framework linking hedging behavior and market turbulence, incorporating beta-normalized volatility.
result Low-beta stocks amplify volatility more during gamma-squeeze events.
Recently, reinforcement learning (RL) algorithms have demonstrated remarkable success in learning complicated behaviors from minimally processed input. However, most of this success is limited to simulation. While there are promising successes in applying RL algorithms directly on real systems, their performance on mor…
Most of the econometric and econophysics models have been borrowed from the statistical physics, and as a cosequence, a new interdisciplinary science called econophysics has emerged. In this paper we planned to extend the analogy between different economic processes or phenomena and processes and phenomena from differe…
Synthetic data can amplify privacy in linear regression models.
problem Understanding how synthetic data can enhance privacy in linear regression models.
method Investigated through the linear regression framework, analyzing synthetic data generated from random inputs and controlled inputs.
result Releasing a limited number of synthetic data points amplifies privacy beyond the model's inherent guarantees when inputs are random, but not when inputs are controlled by an adversary.
Paper analyzes tech adoption in financial networks, finding key leadership and diffusion dynamics.
problem Understanding technology adoption and network effects in financial systems.
method Developed a spatial-network framework with a master equation and Feynman-Kac representation.
result Found strong support for two-regime adoption dynamics and significant leadership in network central banks.
We investigate the framework of privacy amplification by iteration, recently proposed by Feldman et al., from an information-theoretic lens. We demonstrate that differential privacy guarantees of iterative mappings can be determined by a direct application of contraction coefficients derived from strong data processing…
We propose a new objective, the counterfactual objective, unifying existing objectives for off-policy policy gradient algorithms in the continuing reinforcement learning (RL) setting. Compared to the commonly used excursion objective, which can be misleading about the performance of the target policy when deployed, our…
The paper analyzes and improves privacy in machine learning through importance sampling.
problem Ensuring privacy in machine learning while maintaining utility and efficiency.
method Individualized privacy analysis of importance sampling, proposing two approaches for constructing sampling distributions.
result Proposed approaches optimize privacy-efficiency trade-off and outperform uniform sampling.
Enhances privacy in machine learning through Rényi Pufferfish mechanisms.
problem Designing general and efficient Pufferfish mechanisms that maintain privacy and utility.
method Introduces a Rényi divergence-based variant of Pufferfish, generalizes the Wasserstein mechanism, and proves privacy amplification results.
result Extends the applicability of Pufferfish framework and provides stronger privacy guarantees.
AdaPrivate-TS: A differentially private Thompson Sampling algorithm for contextual bandits
problem Private Thompson Sampling for Contextual Bandits
method Combining Thompson Sampling with batched zCDP composition
result Achieves 93-99% of non-private performance at ε ∈ [0.5, 5] with logarithmic privacy cost
Policy gradient based reinforcement learning algorithms coupled with neural networks have shown success in learning complex policies in the model free continuous action space control setting. However, explicitly parameterized policies are limited by the scope of the chosen parametric probability distribution. We show t…
We consider the problem of learning from sparse and underspecified rewards, where an agent receives a complex input, such as a natural language instruction, and needs to generate a complex response, such as an action sequence, while only receiving binary success-failure feedback. Such success-failure rewards are often …