Method tackles uncertainty in reward models for LLMs from heterogeneous human feedback.
problem Uncertainty in reward models for LLMs from heterogeneous human feedback.
method Heterogeneous preference framework and alternating gradient descent algorithm.
result Established theoretical guarantees for estimator convergence and asymptotic distribution.
LoCo-RLHF models diverse human feedback with contextual information.
problem Heterogeneous human feedback from diverse contexts and preferences.
method Low-rank contextual preference model, PRS policy.
result LoCo-RLHF achieves tighter sub-optimality gap than existing methods.
Model shows how confidence feedback can lead to different crisis outcomes.
problem Characterizing the impact of economic recessions on different social strata.
method A self-reflexive DSGE model with heterogeneous households, varying parameters to analyze crisis typologies.
result Crisis propagation can be confined to high or low income households, depending on social network structure and income inequality.
New method learns from subgroup feedback in complex systems.
problem Optimizing complex systems with heterogeneous components.
method Decomposed Gaussian Process (GP) regression and optimization algorithm.
result Proved lower variance and improved accuracy in subgroup feedback.
SA-PEF improves federated learning efficiency by correcting gradient mismatches.
problem Slow decay of residual error in federated learning under non-IID data.
method Integrates step-ahead correction with partial error feedback.
result Achieves faster convergence and target accuracy compared to standard EF.
New algorithm tackles heterogeneous curvature in online convex optimization.
problem Adversarial bandit convex optimization with varying curvature.
method Developed an adaptive algorithm that learns curvature on the fly.
result Achieves optimal regret bounds even with heterogeneous curvature.
New method improves optimization and DP in FL.
problem Combining strong DP and optimization in FL.
method Combining clipping, momentum, and error feedback.
result Optimal convergence rate and near optimal DP guarantees.
The thesis clarifies when local updates outperform centralized methods in heterogeneous data environments.
problem Understanding when local updates are more effective than centralized or mini-batch methods in distributed optimization.
method Fine-grained consensus-error-based analysis framework, focusing on bounded second-order heterogeneity and third-order smoothness.
result Local updates outperform centralized or mini-batch methods under realistic models of data heterogeneity.
A new method for online prediction uncertainty quantification in non-exchangeable panel data.
problem Challenges in quantifying predictive uncertainty for non-exchangeable panel data.
method Online conformal prediction framework for non-exchangeable panel data, using similarity weights and adaptive miscoverage levels.
result Improves coverage on worst-covered target units through adaptive interval-width allocation.
The paper tackles personalized policy learning from diverse data sources in a federated setting.
problem Learning personalized decision policies from observational bandit feedback across multiple heterogeneous data sources.
method Introduces a novel regret analysis for distinguishing global and local regret, and presents a federated policy learning algorithm using local policies trained with doubly robust offline policy evaluation strategies.
result Establishes finite-sample upper bounds on global and local regret, characterizing them by source heterogeneity and distribution shift.
RLHF uses human feedback to train AI models, posing statistical challenges.
problem Aligning AI models with human preferences using noisy, subjective feedback.
method Supervised fine-tuning, reward modeling, policy optimization, statistical ideas.
result Statistical methods for reward function learning and policy optimization.
Algorithm aggregates rewards from multiple players to learn related tasks in online bandit learning.
problem Learning related but slightly different tasks in an online setting with heterogeneous feedback.
method RobustAgg(ε) algorithm that aggregates rewards from different players. result Achieves instance-dependent regret guarantees and nearly matching lower bounds.
Bayesian model identifies three types of travelers adapting to feedback.
problem Capturing adaptive, feedback-driven travel behavior in heterogeneous individuals.
method Latent Class Reinforcement Learning (LCRL) model with Variational Bayes estimation.
result Three distinct traveler classes identified: context-dependent, persistent exploitative, and exploratory.
New methods improve distributed optimization on non-iid data.
problem Communication bottleneck in distributed machine learning models.
method Two types of distributed gradient compression methods (D-QSGD and D-EF-SGD) analyzed for non-iid data.
result D-EF-SGD performs better than D-QSGD on non-iid data but can still slow down with high data skewness.
Develops methods to correct bias in AI feedback for more accurate alignment.
problem Systematic bias in AI feedback compared to human labels.
method Two debiased alignment methods: DDPO and DIPO.
result Methods improve alignment efficiency and performance close to human-labeled data.
Improves search performance by transferring knowledge from recommender system.
problem Cold start and feedback loop problems in search retrieval.
method Zero-Shot Heterogeneous Transfer Learning framework.
result Significant improvements in relevance and user interactions over production system.
We introduce a stochastic heterogeneous interacting-agent model for the short-time non-equilibrium evolution of excess demand and price in a stylized asset market. We consider a combination of social interaction within peer groups and individually heterogeneous fundamentalist trading decisions which take into account t…
Study shows how sentiment shocks affect equity markets, revealing asymmetries and state-dependent effects.
problem Understanding how sentiment shocks propagate through equity markets and their impact on different investor groups.
method Used four independent proxies with sign-aligned kappa-rho parameters, calibrated a structural model to link sentiment to returns.
result A one standard deviation sentiment shock has a 1.06 basis point impact, with effects amplified over 11.2 months and concentrated in retail-tilted stocks.
We consider a financial market model which consists of a financial asset and a large number of interacting agents classified into many types. Different types of agents are heterogeneous in their price expectations. Each agent can change its type based on the current empirical distribution of the types and the equilibri…
In recent years, Variational Autoencoders (VAEs) have been shown to be highly effective in both standard collaborative filtering applications and extensions such as incorporation of implicit feedback. We extend VAEs to collaborative filtering with side information, for instance when ratings are combined with explicit t…
Many recommendation algorithms rely on user data to generate recommendations. However, these recommendations also affect the data obtained from future users. This work aims to understand the effects of this dynamic interaction. We propose a simple model where users with heterogeneous preferences arrive over time. Based…
This paper analyzes error feedback in compressed federated learning for non-convex optimization problems.
problem Reducing communication cost in federated learning with biased gradient compression.
method Proposes Fed-EF, a compressed federated learning scheme with error feedback, and analyzes its convergence rate and performance under partial client participation.
result Fed-EF can match the convergence rate of full-precision FL under data heterogeneity with a linear speedup and no extra slow-down factor due to stale error compensation.
ATLAS uses LLMs to adaptively trade by optimizing prompts and coordinating agents.
problem Adapting LLMs for real-time financial decision-making in noisy markets.
method ATLAS integrates structured market data, uses Adaptive-OPRO for prompt optimization, and employs multi-agent coordination.
result Adaptive-OPRO consistently outperforms fixed prompts in financial trading.
We propose two rational expectation models of transient financial bubbles with heterogeneous arbitrageurs and positive feedbacks leading to self-reinforcing transient stochastic faster-than-exponential price dynamics. As a result of the nonlinear feedbacks, the termination of a bubble is found to be characterized by a …
The paper tackles robust policy learning from multiple data sources.
problem Learning a policy that generalizes across diverse settings from multiple heterogeneous data sources.
method Proposes a minimax regret optimization objective and a policy learning algorithm combining doubly robust offline policy evaluation and no-regret learning.
result Achieves minimal worst-case mixture regret up to a moderated vanishing rate of the total data across all sources.
A general theory of innovation and progress in human society is outlined, based on the combat between two opposite forces (conservatism/inertia and speculative herding "bubble" behavior). We contend that human affairs are characterized by ubiquitous ``bubbles'', which involve huge risks which would not otherwise be tak…
Improved error feedback method reduces communication complexity in distributed training.
problem Improving convergence in distributed training methods with compression techniques.
method Introduced and improved a modern form of error feedback (EF21) by reducing its theoretical communication complexity.
result Communication complexity depends on the arithmetic mean of smoothness parameters, leading to significant improvements.
Develops a framework for analyzing multi-agent and many-body systems with feedback loops.
problem Optimal order of multi-agent and general many-body systems
method Derive macroscopic properties and optimal degree of order
result Optimal degree of order balances productivity, stability, and adaptability
Response time improves alignment with diverse human preferences.
problem Standard aggregation of feedback ignores heterogeneity and anonymity.
method Augmenting feedback with response time data and modeling decisions with DDM.
result Estimator of heterogeneous preferences converges to true average preference.
Learning weights in a spiking neural network with hidden neurons, using local, stable and online rules, to control non-linear body dynamics is an open problem. Here, we employ a supervised scheme, Feedback-based Online Local Learning Of Weights (FOLLOW), to train a network of heterogeneous spiking neurons with hidden l…
Paper introduces MoTEF for faster decentralized optimization with compressed communication.
problem Efficiency bottleneck in decentralized machine learning applications.
method Integrates communication compression with Momentum Tracking and Error Feedback.
result Significantly outperforms existing methods under arbitrary data heterogeneity.
Feedback alignment methods need to be evaluated for accuracy and gradient cosine similarity.
problem Evaluating feedback alignment methods
method Proposed diagnostic evaluation protocol
result Identified silent failures in standard reporting pair
Adaptive reward models capture individual preferences from human feedback.
problem Learning a reward model that can be specialised to a user.
method Empirical risk minimisation and PAC bound analysis.
result Adaptive reward models benefit from the heterogeneity of user preferences.
Recently, matrix factorization-based recommendation methods have been criticized for the problem raised by the triangle inequality violation. Although several metric learning-based approaches have been proposed to overcome this issue, existing approaches typically project each user to a single point in the metric space…
EControl improves fast distributed optimization with compression and error control.
problem Stable convergence issues in distributed training with compression.
method Proposes EControl to regulate error compensation and prove fast convergence.
result Proves fast convergence for EControl in various convex settings without additional assumptions.
Consider a platform that wants to learn a personalized policy for each user, but the platform faces the risk of a user abandoning the platform if she is dissatisfied with the actions of the platform. For example, a platform is interested in personalizing the number of newsletters it sends, but faces the risk that the u…
New model for personalized online advertising with multi-user interaction.
problem Realistic online advertising scenarios with multiple users interacting simultaneously.
method Introduces Multi-User Contextual Cascading Bandit (MCCB) model and proposes UCBBP and AUCBBP algorithms.
result Proves UCBBP and AUCBBP achieve optimal regret bounds for multi-user context.
FedConPE improves conversational recommender systems efficiency and privacy.
problem Efficiently eliciting user preferences in interactive systems with heterogeneous clients.
method Phase elimination-based federated conversational bandit algorithm with adaptive key term construction.
result Minimizes uncertainty across all dimensions in feature space and offers improved efficiency and privacy.
Retraining stabilizes model influence on data.
problem Performativity in predictive models leads to feedback loops.
method Developed the stable signal principle to address retraining dynamics.
result Repeated risk minimization converges geometrically to stable signal direction.
In this paper we seek to demonstrate the predictability of stock market returns and explain the nature of this return predictability. To this end, we introduce investors with different investment horizons into the news-driven, analytic, agent-based market model developed in Gusev et al. (2015). This heterogeneous frame…
We analyze the stability properties of equilibrium solutions and periodicity of orbits in a two-dimensional dynamical system whose orbits mimic the evolution of the price of an asset and the excess demand for that asset. The construction of the system is grounded upon a heterogeneous interacting agent model for a singl…
The challenge in controlling stochastic systems in which low-probability events can set the system on catastrophic trajectories is to develop a robust ability to respond to such events without significantly compromising the optimality of the baseline control policy. This paper presents CelluDose, a stochastic simulatio…
CausalLongPFN predicts counterfactual outcomes from time-series data.
problem Predicting future outcomes under varying treatments in time-series data with confounding and heterogeneity.
method Prior-fitted network pretrained on synthetic episodes of temporal structural causal models.
result CausalLongPFN outperforms domain-trained models on factual and counterfactual prediction tasks.
New algorithms reduce matching market regret to log(T) with improved stability.
problem Minimizing regret in two-sided matching markets with bandit feedback.
method Phase-based algorithm with local arm deletion to improve stability.
result Achieves Θ(log(T)) regret for markets with uniqueness consistency.
End-to-end framework learns LLM routing from observational data.
problem Compounding errors in decoupled approaches and reliance on full-feedback data.
method Causal end-to-end framework minimizing decision-making regret from observational data.
result Method outperforms existing baselines across different embedding models.
EF21 improves convergence in distributed machine learning models.
problem Improving convergence in distributed machine learning models.
method Proposes EF21, a new error feedback mechanism.
result EF21 achieves a fast O(1/T) convergence rate for smooth nonconvex problems. New insights into cascade feedback linearization of control systems.
problem Obtaining a cascade feedback linearization for invariant control systems.
method Introducing truncated versions of operators from the calculus of variations to prove new theorems.
result Established new geometry and foundational theorems for future work.
New method learns from either positive or negative feedback alone.
problem Limited applicability of existing preference optimization methods in scenarios with only unpaired feedback.
method Decouples learning from positive and negative feedback, using expectation-maximization (EM) to optimize probability of positive outcomes and explicitly incorporate negative examples.
result Stable learning from negative feedback alone demonstrated.