This study investigates abrupt learning dynamics in Transformers, revealing plateau formation and internal representation collapse.
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Trend · papers per month
Ghost mechanism explains abrupt learning in RNNs, revealing constraints on optimization landscapes.
BRPC online Bayesian calibration handles gradual and abrupt system changes.
TradeR uses RL to execute trades in real markets, minimizing surprise and catastrophe.
Transformers exhibit abrupt learning in matrix completion tasks.
Meta-reinforcement learning improves fault-adaptive control efficiency.
Many real-world time series, such as in health, have changepoints where the system's structure or parameters change. Since changepoints can indicate critical events such as onset of illness, it is highly important to detect them. However, existing methods for changepoint detection (CPD) often require user-specified mod…
Optimizes sensor usage for detecting abrupt changes in sensor data.
We propose a multiresolution Gaussian process to capture long-range, non-Markovian dependencies while allowing for abrupt changes. The multiresolution GP hierarchically couples a collection of smooth GPs, each defined over an element of a random nested partition. Long-range dependencies are captured by the top-level GP…
Extracting the underlying trend signal is a crucial step to facilitate time series analysis like forecasting and anomaly detection. Besides noise signal, time series can contain not only outliers but also abrupt trend changes in real-world scenarios. To deal with these challenges, we propose a robust trend filtering al…
Robots can rapidly acquire new skills from demonstrations. However, during generalisation of skills or transitioning across fundamentally different skills, it is unclear whether the robot has the necessary knowledge to perform the task. Failing to detect missing information often leads to abrupt movements or to collisi…
Bayesian theory explains abrupt emergence of copy subcircuit in attention.
A nonparametric family of conditional distributions is introduced, which generalizes conditional exponential families using functional parameters in a suitable RKHS. An algorithm is provided for learning the generalized natural parameter, and consistency of the estimator is established in the well specified case. In ex…
Machine learning detects tipping points in complex systems.
Autoencoder detects subtle changes in time series data.
We show that reinforcement learning agents that learn by surprise (surprisal) get stuck at abrupt environmental transition boundaries because these transitions are difficult to learn. We propose a counter-intuitive solution that we call Mutual Information Minimising Exploration (MIME) where an agent learns a latent rep…
Adversarial training improves linear regression solutions, revealing sparsity and abrupt interpolation.
Study tackles infinitely many-armed bandits with rotting rewards, achieving tight regret bounds.
The volatility of financial instruments is rarely constant, and usually varies over time. This creates a phenomenon called volatility clustering, where large price movements on one day are followed by similarly large movements on successive days, creating temporal clusters. The GARCH model, which treats volatility as a…
New algorithm for nonstationary multi-armed bandits with optimal performance.
A method identifies abrupt changes in functions with fixed confidence under noisy feedback.
The objective of the change-point detection is to discover the abrupt property changes lying behind the time-series data. In this paper, we firstly summarize the definition and in-depth implication of the changepoint detection. The next stage is to elaborate traditional and some alternative model-based changepoint dete…
Enhash detects concept drift in data streams quickly and efficiently.
DeRegiME forecasts with regime structure, improving probabilistic predictions across various time series.
We consider the classical stochastic multi-armed bandit problem with a constraint that limits the total cost incurred by switching between actions to be no larger than a given switching budget. For this problem, we prove matching upper and lower bounds on the optimal (i.e., minimax) regret, and provide efficient rate-o…
High-dimensional curved diffusions show abrupt convergence at a critical time.
Data-driven predictive analytics are in use today across a number of industrial applications, but further integration is hindered by the requirement of similarity among model training and test data distributions. This paper addresses the need of learning from possibly nonstationary data streams, or under concept drift,…
DS-TS adapts to abrupt and smooth changes in bandit problems.
Symmetry unifies AI learning dynamics, complexity, and representation.
Change-point detection (CPD) aims to locate abrupt transitions in the generative model of a sequence of observations. When Bayesian methods are considered, the standard practice is to infer the posterior distribution of the change-point locations. However, for complex models (high-dimensional or heterogeneous), it is n…
Algorithm detects concept drift and adapts models in streaming data.
Study reveals how illiquidity network signals Chinese stock market crashes.
The objective of change-point detection is to discover abrupt property changes lying behind time-series data. In this paper, we present a novel statistical change-point detection algorithm based on non-parametric divergence estimation between time-series samples from two retrospective segments. Our method uses the rela…
We consider a scenario where an artificial agent is reading a stream of text composed of a set of narrations, and it is informed about the identity of some of the individuals that are mentioned in the text portion that is currently being read. The agent is expected to learn to follow the narrations, thus disambiguating…
Transformers learn rich in-context dependencies efficiently.
Bayesian optimisation for dynamically adjusting learning rates in machine learning models.
Research in deep reinforcement learning (RL) has coalesced around improving performance on benchmarks like the Arcade Learning Environment. However, these benchmarks conspicuously miss important characteristics like abrupt context-dependent shifts in strategy and temporal sensitivity that are often present in real-worl…
KCUSUM detects abrupt changes in real-time data streams efficiently.
Continual learning aims to improve the ability of modern learning systems to deal with non-stationary distributions, typically by attempting to learn a series of tasks sequentially. Prior art in the field has largely considered supervised or reinforcement learning tasks, and often assumes full knowledge of task labels …
Study analyzes European energy markets' reactions to 2022 events using Bayesian methods.
A neural network approach to learn Cusp Catastrophe dynamics.
Detecting the emergence of abrupt property changes in time series is a challenging problem. Kernel two-sample test has been studied for this task which makes fewer assumptions on the distributions than traditional parametric approaches. However, selecting kernels is non-trivial in practice. Although kernel selection fo…
The dichotomous coordinate descent (DCD) algorithm has been successfully used for significant reduction in the complexity of recursive least squares (RLS) algorithms. In this work, we generalize the application of the DCD algorithm to RLS adaptive filtering in impulsive noise scenarios and derive a unified update formu…
A new approach for functional data description is proposed in this paper. It consists of a regression model with a discrete hidden logistic process which is adapted for modeling curves with abrupt or smooth regime changes. The model parameters are estimated in a maximum likelihood framework through a dedicated Expectat…
A policy is said to be robust if it maximizes the reward while considering a bad, or even adversarial, model. In this work we formalize two new criteria of robustness to action uncertainty. Specifically, we consider two scenarios in which the agent attempts to perform an action , and (i) with probability , an alt…
Delta hedging, which plays a crucial rôle in modern financial engineering, is a tracking control design for a "risk-free" management. We utilize the existence of trends in financial time series (Fliess M., Join C.: A mathematical proof of the existence of trends in financial time series, Proc. Int. Conf. Systems Theory…
Enhances DGP surrogates for efficient active learning.
This paper addresses parameter estimation for wave equations with Markovian switching.