Language-based methods improve human similarity approximations without requiring many human judgments.
problem Approximating human similarity judgments using pre-trained deep neural networks (DNNs) is challenging and expensive.
method Developed language-based methods to approximate human similarity judgments, validated with adaptive tag collection pipeline.
result Language-based methods significantly improve performance over DNN-based methods with fewer human judgments.
Develops a statistical framework to measure uncertainty in model rankings based on human preferences.
problem Uncertainty in model rankings based on human preferences due to mismatch between human and model preferences.
method Statistical framework using pairwise comparisons by humans and models to provide rank-sets for each model.
result Rank-sets constructed using only pairwise comparisons by strong models often do not cover the true ranking of human preferences.
New metrics predict human sentence comprehension across languages.
problem Predicting human sentence comprehension using computational models.
method Developed sentence-level metrics using multilingual large language models.
result Achieved high accuracy in predicting human sentence reading speeds.
Reward learning enables the application of reinforcement learning (RL) to tasks where reward is defined by human judgment, building a model of reward by asking humans questions. Most work on reward learning has used simulated environments, but complex information about values is often expressed in natural language, and…
Linking human whole-body motion and natural language is of great interest for the generation of semantic representations of observed human behaviors as well as for the generation of robot behaviors based on natural language input. While there has been a large body of research in this area, most approaches that exist to…
Proposes a robust algorithm for aligning large language models with human preferences.
problem Misspecification in preference models, reference policies, and reward functions.
method Doubly robust preference optimization algorithm.
result Superior and more robust performance compared to state-of-the-art algorithms.
Stable and consistent model alignment for language models without assuming human preference models.
problem Lack of statistical consistency in existing alignment methods.
method Relative density ratio optimization between preferred and mixture of preferred and non-preferred data distributions.
result Our approach achieves statistical consistency and stability, providing tighter convergence guarantees.
The recognition of sign language is a challenging task with an important role in society to facilitate the communication of deaf persons. We propose a new approach of Spatial-Temporal Graph Convolutional Network to sign language recognition based on the human skeletal movements. The method uses graphs to capture the si…
Large language models predict human sensory judgments across multiple modalities.
problem Determining the extent of perceptual information in language.
method State-of-the-art large language models were used to predict sensory judgments across six psychophysical datasets.
result Large language models can predict human sensory judgments across multiple modalities with significant correlation to human data.
Paper proposes a new RLHF framework for human preference learning.
problem Handling dependent online human preference outcomes with dynamic contexts.
method Two-stage algorithm with ε-greedy followed by exploitation; anti-concentration inequalities and matrix martingale concentration techniques. result Our method achieves optimal regret bound and asymptotic normality of estimators.
We conduct a large-scale, systematic study to evaluate the existing evaluation methods for natural language generation in the context of generating online product reviews. We compare human-based evaluators with a variety of automated evaluation procedures, including discriminative evaluators that measure how well machi…
New technique reduces language biases in large language models.
problem Gap between language and thought in large language models.
method Proposes Language-of-Thoughts (LoT) technique to adjust order and tokens.
result Significantly reduces language biases and improves reasoning tasks.
Efficiently identifies good policies by choosing contexts for human feedback.
problem Efficiently acquiring human feedback for preference alignment in large language models.
method Formalizes active exploration as a dueling bandit problem and proposes an active exploration algorithm with a polynomial worst-case regret bound.
result Proposed method outperforms baselines with limited human preferences on various language models and datasets.
Direct Density Ratio Optimization aligns LLMs with human preferences without assuming specific models.
problem Statistical inconsistency in aligning LLMs with human preferences.
method Direct Density Ratio Optimization (DDRO) estimates density ratio directly.
result DDRO is statistically consistent, converging to true human preferences as data grows.
Framework ensures alignment between humans and machines in LLMs.
problem Human-machine misalignment in LLMs scoring mechanisms.
method Lightweight calibration framework for blackbox models.
result Provably guarantees alignment between humans and machines.
Many challenges in natural language processing require generating text, including language translation, dialogue generation, and speech recognition. For all of these problems, text generation becomes more difficult as the text becomes longer. Current language models often struggle to keep track of coherence for long pi…
New benchmark for causal reasoning from human video descriptions.
problem Lack of diversity in event types and natural language descriptions, and differences from human judgments.
method Iterative event cloze task and data augmentation techniques.
result Improved data collection efficiency and diverse causal judgments.
Paper proposes a robust RLHF algorithm for LLMs, improving response preference over baselines.
problem Reward model misspecifications in RLHF for LLMs.
method Proposes a robust algorithm to reduce reward and policy estimator variance, theoretically and empirically validated.
result Consistently outperforms existing methods on LLM benchmark datasets, favoring 77-81% of responses over baselines.
Fine-tunes language models with captions for differential equation solving.
problem Overreliance on function data in operator learning.
method Integrates human knowledge through captions and fine-tunes language models.
result Significantly enhanced performance and reduced function data requirements.
Bayesian EnKF improves sentence comprehension uncertainty modeling.
problem Uncertainty in human language comprehension, especially with ambiguous inputs.
method Bayesian framework using ensemble Kalman filter (EnKF) for uncertainty quantification.
result Enhanced model's ability to approximate human cognitive processing with linguistic ambiguities.
Modeling human language learning with multi-checkpoint machine translation.
problem Improving machine translation quality for language education.
method Ensemble of multi-checkpoints from a single model, sampling n-best sequences.
result Achieved 37.57 macro F1 score, outperforming baseline.
LLMs struggle with financial reasoning but can outperform the market with human oversight.
problem Financial reasoning failures in LLM-generated stock market predictions.
method Evaluated four LLMs using three prompting strategies and compared to human oversight.
result LLMs require human oversight to fully realize their potential in financial markets.
Alpha-GPT 2.0 integrates human insights into AI-driven investment research.
problem Efficiency and precision in quantitative investment research.
method Iterative Human-AI interaction using large language models.
result Enhanced efficiency and precision in quantitative investment research.
A novel memory mechanism for reinforcement learning agents that stores past events in human-readable language.
problem Lack of interpretability in reinforcement learning agent's memory mechanisms.
method Uses CLIP to associate visual inputs with language tokens, then feeds these tokens to a pretrained language model.
result Significantly faster convergence on challenging continuous recognition tasks.
FinHEAR combines LLMs with human expertise for better financial decision-making.
problem Challenges in financial decision-making for language models.
method Multi-agent framework with specialized LLMs for historical analysis, event interpretation, and expert retrieval.
result FinHEAR outperforms baselines in financial tasks with higher accuracy and risk-adjusted returns.
Embodied cognition states that semantics is encoded in the brain as firing patterns of neural circuits, which are learned according to the statistical structure of human multimodal experience. However, each human brain is idiosyncratically biased, according to its subjective experience history, making this biological s…
ATD measures language distance using neural models, recovering linguistic groupings.
problem Lack of a unified quantitative measure for cross-linguistic distance.
method Pretrained multilingual language models, attention mechanisms, optimal transport.
result ATD quantifies representational distance between languages, recovering linguistic groupings.
As modern deep networks become more complex, and get closer to human-like capabilities in certain domains, the question arises of how the representations and decision rules they learn compare to the ones in humans. In this work, we study representations of sentences in one such artificial system for natural language pr…
Decoding strategies often exclude human-like tokens, creating a detectable gap in generated text.
problem Decoding strategies exclude contextually appropriate but statistically rare tokens, creating a detectable gap in generated text.
method Analysis of 1.8 million texts across 8 language models, 5 decoding strategies, and 53 hyperparameter configurations.
result 8-18% of human-selected tokens fall outside typical truncation boundaries, indicating a detectable gap.
EBRM improves robustness and generalization of language model rewards.
problem Challenges in capturing complex human preferences and generalizing to unseen data in reward models.
method Energy-Based Reward Model (EBRM) that models reward distribution explicitly, using conflict-aware data filtering, label-noise-aware contrastive training, and hybrid initialization.
result Significant improvements in robustness and generalization, up to 5.97% improvement in safety-critical alignment tasks.
Method tackles uncertainty in reward models for LLMs from heterogeneous human feedback.
problem Uncertainty in reward models for LLMs from heterogeneous human feedback.
method Heterogeneous preference framework and alternating gradient descent algorithm.
result Established theoretical guarantees for estimator convergence and asymptotic distribution.
Efficient exploration improves large language model performance with fewer queries.
problem Improving large language model performance with fewer human feedback queries.
method Sequentially generates queries, fits a reward model to feedback, uses double Thompson sampling with epistemic neural network uncertainty.
result Efficient exploration enables high performance with far fewer queries.
Transformer models improve financial sentiment measurement.
problem Capturing nuanced sentiment from financial news articles.
method Transformer-based language models for sentiment classification and aggregation.
result Transformer models outperform traditional dictionary-based methods in sentiment classification.
A new method reduces the computational burden of safety alignment for large language models.
problem Safety concerns in large language models and the need to align them with human preferences.
method Optimal dualization approach to reduce constrained alignment to an unconstrained problem.
result Our algorithms MoCAN and PeCAN significantly reduce computational burden and improve training stability.
XPO enhances RLHF by encouraging diverse responses, offering improved sample efficiency.
problem Limited exploration in RLHF leads to suboptimal models and bottlenecks.
method XPO is a simple one-line change to DPO, introducing a novel exploration bonus.
result XPO achieves strong theoretical guarantees and promising empirical performance.
Dynamic meta-learning improves multi-agent communication with natural language.
problem Static population-based approaches struggle with adapting to human partners using natural language.
method Dynamic population-based meta-learning approach.
result Agents outperform all prior work in communicating with seen partners and humans.
New method detects watermarks in LLM-generated text with human edits.
problem Dilution of watermark signals by human edits on LLM-generated text.
method Truncated goodness-of-fit test (Tr-GoF) for robust detection.
result Tr-GoF achieves optimality in robust detection of Gumbel-max watermark.
In this paper, we present Russian language datasets in the digital humanities domain for the evaluation of word embedding techniques or similar language modeling and feature learning algorithms. The datasets are split into two task types, word intrusion and word analogy, and contain 31362 task units in total. The chara…
A promising approach for teaching artificial agents to use natural language involves using human-in-the-loop training. However, recent work suggests that current machine learning methods are too data inefficient to be trained in this way from scratch. In this paper, we investigate the relationship between two categorie…
Computational modeling of human multimodal language is an emerging research area in natural language processing spanning the language, visual and acoustic modalities. Comprehending multimodal language requires modeling not only the interactions within each modality (intra-modal interactions) but more importantly the in…
How can we measure whether a natural language generation system produces both high quality and diverse outputs? Human evaluation captures quality but not diversity, as it does not catch models that simply plagiarize from the training set. On the other hand, statistical evaluation (i.e., perplexity) captures diversity b…
New RLHF approach mitigates bias in aligning LLMs with human preferences.
problem Algorithmic bias in RLHF leading to preference collapse.
method Preference Matching (PM) RLHF, using PM regularizer and conditional variant.
result 29% to 41% improvement in alignment with human preferences.
Introduces MLM dataset for multitask learning across multiple languages and modalities.
problem Training and evaluating multitask systems on diverse data.
method Develops a new benchmark dataset with semantic data and modalities.
result Demonstrates challenges in generalizing on diverse data.
Study compares DSPy teleprompter algorithms for aligning LLM evaluations with human annotations.
problem Aligning LLM evaluation metrics with human annotations.
method Comparative analysis of five teleprompter algorithms within the DSPy framework.
result Certain teleprompters outperform others in detecting hallucinations.
Robots learn actions and language through curiosity-driven self-exploration.
problem Efficient development of actions and language in infants and robots.
method Curiosity-driven self-exploration using Q-learning to amortize active inference.
result Curiosity-driven exploration enables faster learning and compositional generalization.
TraderTalk uses LLMs to simulate human trading interactions in financial markets.
problem Simulating realistic human trading interactions in financial markets.
method Hybrid ABM with LLM-generated behaviors for detailed conversations.
result Successfully replicates trade-to-order volume ratios in financial markets.
SPIN converts weak LLMs to strong ones using self-play.
problem Growing strong LLMs without human-annotated data.
method Self-play fine-tuning (SPIN) starting from a supervised fine-tuned model.
result SPIN significantly improves LLM performance across benchmarks.
UCFE benchmarks LLMs in financial tasks with human feedback.
problem Evaluating LLMs' financial task performance and user satisfaction.
method Hybrid approach combining human expert evaluations and dynamic interactions.
result Significant alignment between benchmark scores and human preferences (Pearson correlation coefficient of 0.78).