A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Optimizes contrastive learning with individualized temperatures for better performance on imbalanced datasets.
problem The common practice of using a global temperature parameter ignores the varying semantic similarity across different anchor data.
method Proposes a new robust contrastive loss inspired by distributionally robust optimization (DRO) and an efficient stochastic algorithm for automatic temperature individualization.
result Our method automatically learns a suitable temperature for each sample, improving performance on imbalanced datasets.
Hyperfitting improves LLM generation quality by enhancing diversity, contrary to simple temperature scaling.
problem Improving open-ended generation quality of LLMs with minimal fine-tuning effort.
method Demonstrates that hyperfitting, a phenomenon where LLMs are fine-tuned to near-zero training loss, enhances generation quality and mitigates repetition.
result Hyperfitting is distinct from temperature scaling and involves a dynamic, context-dependent rank reordering mechanism in the final transformer block.
We define a random-matrix ensemble given by the infinite-time covariance matrices of Ornstein-Uhlenbeck processes at different temperatures coupled by a Gaussian symmetric matrix. The spectral properties of this ensemble are shown to be in qualitative agreement with some stylized facts of financial markets. Through the…
Recently, Deep Neural Networks (DNNs) have been achieving impressive results on wide range of tasks. However, they suffer from being well-calibrated. In decision-making applications, such as autonomous driving or medical diagnosing, the confidence of deep networks plays an important role to bring the trust and reliabil…
We introduce a temperature into the exponential function and replace the softmax output layer of neural nets by a high temperature generalization. Similarly, the logarithm in the log loss we use for training is replaced by a low temperature logarithm. By tuning the two temperatures we create loss functions that are non…
We present a novel Metropolis-Hastings method for large datasets that uses small expected-size minibatches of data. Previous work on reducing the cost of Metropolis-Hastings tests yield variable data consumed per sample, with only constant factor reductions versus using the full dataset for each sample. Here we present…
This paper measures temperature in agent systems using volatility.
problem How to measure temperature in agent systems.
method Examined an agent system with two decision options in a news environment, established the measurement equation, and outlined the concept of temperature measurement.
result Illustrated a strategy for influencing average opinion in competing subsystems.
Attention temperature improves robustness of ICL in high-dimensional settings.
problem ICL robustness failure under distribution shift in high dimensions.
method Analyzed a Transformer with approximate softmax attention, derived a closed-form error expression, and showed optimal temperature minimizes error.
result Optimal attention temperature minimizes ICL generalization error under distribution shift.
Trends in terrestrial temperature variability are perhaps more relevant for species viability than trends in mean temperature. In this paper, we develop methodology for estimating such trends using multi-resolution climate data from polar orbiting weather satellites. We derive two novel algorithms for computation that …
We present here the Temporal Clustering Algorithm (TCA), an incremental learning algorithm applicable to problems of anticipatory computing in the context of the Internet of Things. This algorithm was tested in a specific prediction scenario of consumption of an electric water dispenser typically used in tropical count…
We show that the dynamical equations describing the collective behavior of the model introduced by Cavagna et al. i) are not their Eqs. (5,6) but rather ii) are the same as those of the minority game (MG). As a consequence the analytic solution of the MG presented in [PRL, 84, 1824 (2000)] holds also for this model. Fi…
The minute fluctuations of of S&P 500 and NASDAQ 100 indices display Boltzmann statistics over a wide range of positive as well as negative returns, thus allowing us to define a {\em market temperature} for either sign. With increasing time the sharp Boltzmann peak broadens into a Gaussian whose volatility σ measure…
In the spirit of behavioral finance, we study the process of opinion formation among investors using a variant of the 2D Voter Model with a tunable social temperature. Further, a feedback acting on the temperature is introduced, such that social temperature reacts to market imbalances and thus becomes time dependent. I…
A spin model relating physical to financial variables is presented. This work is the first to introduce the concept of negative absolute temperature into stock market dynamics by establishing a rigorous formal analogy between physical and financial variables. Based on this model, an algorithm evaluating negative temper…
Machine learning and deep learning infer surface/groundwater exchange from temperature data.
problem Inferring surface/groundwater exchange from temperature data with high temporal resolution.
method Application of machine learning and deep learning algorithms to infer surface/groundwater exchange flux from subsurface temperature observations.
result DL methods outperform ML methods in interpreting noisy temperature data, especially with a smoothing filter.
We describe an adaptation of the simulated annealing algorithm to nonparametric clustering and related probabilistic models. This new algorithm learns nonparametric latent structure over a growing and constantly churning subsample of training data, where the portion of data subsampled can be interpreted as the inverse …
Labour productivity distribution (dispersion) is studied both theoretically and empirically. Superstatistics is presented as a natural theoretical framework for productivity. The demand index κ is proposed within this framework as a new business index. Japanese productivity data covering small-to-medium to large firm…
Understanding local currents in the North Atlantic region of the ocean is a key part of modelling heat transfer and global climate patterns. Satellites provide a surface signature of the temperature of the ocean with a high horizontal resolution while in situ autonomous probes supply high vertical resolution, but horiz…
We present a model of financial markets originally proposed for a turbulent flow, as a dynamic basis of its intermittent behavior. Time evolution of the price change is assumed to be described by Brownian motion in a power-law potential, where the `temperature' fluctuates slowly. The model generally yields a fat-tailed…
Model uncertainty obtained by variational Bayesian inference with Monte Carlo dropout is prone to miscalibration. The uncertainty does not represent the model error well. In this paper, temperature scaling is extended to dropout variational inference to calibrate model uncertainty. Expected uncertainty calibration erro…