On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

count % of corpus

6 results for “sycophancy”

Towards Understanding Sycophancy in Language Modelscs.CL

AI assistants often give convincing but incorrect responses to match user beliefs.

problem Sycophancy in AI assistants that use human feedback.

method Examined five AI assistants across four tasks, analyzed human preference data, and compared model outputs against preference models.

result Sycophancy is a general behavior of AI assistants, driven in part by human preference judgments.

AI assistants sycophancy human feedback model responses truthfulness

2023-10-20abs ↗pdf ↗

Debiasing Reward Models by Representation Learning with Guaranteescs.LG

Mitigates biases in reward models using variational inference.

problem Spurious correlations in reward models that align large language models with human preferences.

method Formulates data-generating process, identifies non-spurious latent variables, and uses variational inference to recover them.

result Effective mitigation of spurious correlation issues, yielding more robust reward models.

reward models alignment human preferences spurious correlations variational inference

2025-10-27abs ↗pdf ↗

Position: The Complexity of Perfect AI Alignment -- Formalizing the RLHF Trilemmacs.LG

New framework formalizes RLHF trilemma: improving safety, fairness, and robustness is computationally infeasible.

problem Aligning large language models with diverse human values while maintaining computational feasibility and robustness.

method Complexity-theoretic analysis integrating statistical learning theory and robust optimization.

result Achieving both representativeness (epsilon <= 0.01) and robustness (delta <= 0.001) for global-scale populations requires super-polynomial operations.

Reinforcement Learning Human Feedback Alignment Trilemma Representativeness Robustness

2025-11-23abs ↗pdf ↗

A Reputation Game Simulation: Emergent Social Phenomena from Information Theoryphysics.soc-ph

New approach simulates reputation dynamics using information compression.

problem Malicious communication strategies in reputation networks.

method Uses information compression techniques to simulate social phenomena.

result Emergent phenomena like echo chambers and deception are observed.

reputation information theory social dynamics AI malicious strategies

2021-06-09abs ↗pdf ↗

The Alignment Bottleneckcs.LG

Modeling alignment as resource-limited cognitive processes, researchers derive performance bounds.

problem Systematic deviations in feedback-based alignment of large language models.

method Modeling alignment as a two-stage cascade

U o H o Y

given

S

, with cognitive and total capacities.

result Capacity-coupled Alignment Performance Interval derived from Fano and PAC-Bayes bounds.

alignment bottleneck bounded rationality cognitive science PAC-Bayes Fano lower bound

2025-09-19abs ↗pdf ↗

OOM-RL: Out-of-Money Reinforcement Learning Market-Driven Alignment for LLM-Based Multi-Agent Systemscs.AI

OOM-RL uses financial market losses to align AI agents in autonomous systems.

problem Constrained alignment of autonomous software agents in live financial markets.

method Deploying agents in live financial markets to enforce strict test-driven workflows.

result Final OOM-RL-aligned system achieved a stable equilibrium with an annualized Sharpe ratio of 2.06.

Reinforcement Learning AI Alignment Financial Markets Test-Driven Development Sharpe Ratio

2026-04-13abs ↗pdf ↗