arXiv research

L3Ms fine-tune LLMs with constraints for tailored applications.

Optimization with inequality constraints using embedded gradient vector field method

GOPO optimizes large models in Hilbert space, avoiding Kullback-Leibler's curvature.

Paper studies critical points of curvature energies in 4D.

New technique reduces language biases in large language models.

Researchers analyze the structure of token spaces in large language models.

Large language models predict human sensory judgments across multiple modalities.

AudioPaLM combines text and speech models to improve speech processing and translation.

Recently, pre-trained language representation flourishes as the mainstay of the natural language understanding community, e.g., BERT. These pre-trained language representations can create state-of-the-art results on a wide range of downstream tasks. Along with continuous significant performance improvement, the size an…

Large language models have recently achieved state of the art performance across a wide variety of natural language tasks. Meanwhile, the size of these models and their latency have significantly increased, which makes their usage costly, and raises an interesting question: do language models need to be large? We study…

Chronologically consistent models maintain accuracy with time-restricted data.

DS-CP improves reliability of uncertainty quantification for large language models under domain shift.

LLMs translate natural language trading intents into correct option strategies using a domain-specific language.

This work uses conformal prediction to quantify uncertainty in large language models for multiple-choice questions.

Schwarzian derivative connects to Euler-Lagrange equations in variational calculus.

Optimus pre-trains sentences in a latent space for various NLP tasks.

The goal of this paper is to encode equivalently the fractional Lagrange dynamics as a nonholonomic almost Kahler geometry. We use the fractional Caputo derivative generalized for nontrivial nonlinear connections (N-connections) originally introduced in Finsler geometry, with further developments in Lagrange and Hamilt…

The approach to nonholonomic Ricci flows and geometric evolution of regular Lagrange systems [S. Vacaru: J. Math. Phys. \textbf{49} (2008) 043504 \& Rep. Math. Phys. \textbf{63} (2009) 95] is extended to include geometric mechanics and gravity models on Lie algebroids. We prove that such evolution scenarios of geometri…

Large language models can't efficiently reason conditionally in a distribution-free setting.

Infinitesimal variation of Action functional in classical (non-quantum) field theory with higher derivatives is presented in terms of well-defined intrinsic geometric objects independent of the particular field which varies. 'Integration by parts' procedure for this variation is then described in purely formal language…

New bounds show large language models can generalize beyond training data.

Study constructs a Japanese financial LLM benchmark.

LLMs are compared to Markov chains for natural language processing.

This work explains how linear representations in large language models arise from training objectives and gradient descent.

Geometric arbitrage theory reformulates a generic asset model possibly allowing for arbitrage by packaging all asset and their forward dynamics into a stochastic principal fibre bundle, with a connection whose parallel transport encodes discounting and portfolio rebalancing, and whose curvature measures, in this geomet…

POUF fine-tunes large models without labeled data.

Graph signal processing detects hallucinations in large language models.

Develops geometry for Lotka-Volterra model of species competition.

Shai is a 10B model for asset management tasks, outperforming baselines.

Study Federated RL with diverse constraints, proposing new optimization methods.

For a system of second order differential equations we determine a nonlinear connection that is compatible with a given generalized Lagrange metric. Using this nonlinear connection, we can find the whole family of metric nonlinear connections that can be associated with a system of SODE and a generalized Lagrange struc…

Large language models improve futures market factor models in China.

This paper benchmarks FinGPT for financial datasets using open-source large language models.

A large number of objectives have been proposed to train latent variable generative models. We show that many of them are Lagrangian dual functions of the same primal optimization problem. The primal problem optimizes the mutual information between latent and visible variables, subject to the constraints of accurately …

Proposes a robust algorithm for aligning large language models with human preferences.

In natural language processing, it has been observed recently that generalization could be greatly improved by finetuning a large-scale language model pretrained on a large unlabeled corpus. Despite its recent success and wide adoption, finetuning a large pretrained language model on a downstream task is prone to degen…

We introduce a new sampling method for large language models that balances diversity and parallelism.

In this paper are studied the harmonic maps between two generalized Lagrange spaces. At the same time, it is proved that the solutions of $C^2$ class of certain ODEs or PDEs are harmonic maps between certain convenient generalized Lagrange spaces.

This work investigates uncertainty quantification for black-box large language models in natural language generation.

Discrete Lagrange problems solved with Lie group constraints.

Study compares LLMs vs classical models for financial sentiment analysis.

The paper construct a suitable generalized metrical multi-time Lagrange geometrical model for both gravitational and electromagnetic fields, in a general setting. In this construction, the gravitational potentials are described by a distinguished vertical metrical tensor of the form $h^{/alpha/beta}e^{2σ}/phi_{ij}$.

Develops a statistical framework to measure uncertainty in model rankings based on human preferences.

Variational calculus on a vector bundle E equipped with a structure of a general algebroid is developed, together with the corresponding analogs of Euler-Lagrange equations. Constrained systems are introduced in the variational and in the geometrical setting. The constrained Euler-Lagrange equations are derived for ana…

Develops Lagrange-Hamilton geometry for COVID-19 disease dynamics.

ScheduleFree+ improves large language model training without schedules or learning rates.

Efficient exploration improves large language model performance with fewer queries.

Lagrange's map construction ideas influenced later mathematicians.