Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

14.7%29.5%44.2%59.0% · Jun 202019922001200920172026
48 results for Abrupt Learning

This study investigates abrupt learning dynamics in Transformers, revealing plateau formation and internal representation collapse.

problem Abrupt learning in Transformers, particularly during the loss plateau.
method Investigates mechanisms of abrupt learning in shallow Transformers, focusing on attention maps and hidden states.
result Reveals plateau formation, internal representation collapse, and strong repetition bias in outputs.

Ghost mechanism explains abrupt learning in RNNs, revealing constraints on optimization landscapes.

problem Understanding abrupt learning in recurrent neural networks (RNNs) trained on working memory tasks.
method Introducing the ghost mechanism, a process driven by saddle-node bifurcations, to analyze and model abrupt learning.
result A critical learning rate scales as an inverse power law with the timescale of computation, leading to vanishing and oscillatory gradients.

BRPC online Bayesian calibration handles gradual and abrupt system changes.

problem Aligning model outputs with field observations in evolving systems.
method Bayesian Recursive Projected Calibration (BRPC) for streaming data under simulator mismatch and nonstationarity.
result Improves calibration accuracy under gradual changes and robustness under abrupt regime shifts.

TradeR uses RL to execute trades in real markets, minimizing surprise and catastrophe.

problem Minimizing surprise and catastrophe in high-frequency trading.
method Hierarchical RL with energy-based surprise value function.
result TradeR outperforms in abrupt price changes and maintains profitability.

We propose a multiresolution Gaussian process to capture long-range, non-Markovian dependencies while allowing for abrupt changes. The multiresolution GP hierarchically couples a collection of smooth GPs, each defined over an element of a random nested partition. Long-range dependencies are captured by the top-level GP…

2012-09-05abs ↗pdf ↗

Robots can rapidly acquire new skills from demonstrations. However, during generalisation of skills or transitioning across fundamentally different skills, it is unclear whether the robot has the necessary knowledge to perform the task. Failing to detect missing information often leads to abrupt movements or to collisi…

2018-08-06abs ↗pdf ↗

Bayesian theory explains abrupt emergence of copy subcircuit in attention.

problem Understanding the abrupt emergence of the copy subcircuit in attention during training.
method Deriving a closed-form posterior over the attention matrix and reducing it to a low-dimensional order parameter space.
result Derive a phase transition in the amount of training data.

A nonparametric family of conditional distributions is introduced, which generalizes conditional exponential families using functional parameters in a suitable RKHS. An algorithm is provided for learning the generalized natural parameter, and consistency of the estimator is established in the well specified case. In ex…

2017-11-15abs ↗pdf ↗

Machine learning detects tipping points in complex systems.

problem Detecting abrupt shifts in complex dynamical systems.
method Equilibrium-informed neural networks (EINNs) trained on candidate equilibrium states.
result EINNs can identify critical thresholds in nonlinear systems.

We show that reinforcement learning agents that learn by surprise (surprisal) get stuck at abrupt environmental transition boundaries because these transitions are difficult to learn. We propose a counter-intuitive solution that we call Mutual Information Minimising Exploration (MIME) where an agent learns a latent rep…

2020-01-16abs ↗pdf ↗

Adversarial training improves linear regression solutions, revealing sparsity and abrupt interpolation.

problem Adversarial attacks on linear regression models.
method Formulated as a convex problem, adversarial training is used to find robust solutions that are sparse and interpolate data.
result Adversarial training with small disturbances gives the solution with the minimum-norm that interpolates the training data, revealing abrupt transition into interpolation.

Study tackles infinitely many-armed bandits with rotting rewards, achieving tight regret bounds.

problem Infinitely many-armed bandits with rotting rewards.
method Adaptive sliding window UCB algorithm for slow and abrupt rotting scenarios.
result Achieves tight regret bounds for both slow and abrupt rotting scenarios.

The volatility of financial instruments is rarely constant, and usually varies over time. This creates a phenomenon called volatility clustering, where large price movements on one day are followed by similarly large movements on successive days, creating temporal clusters. The GARCH model, which treats volatility as a…

2012-12-25abs ↗pdf ↗

New algorithm for nonstationary multi-armed bandits with optimal performance.

problem Nonstationary multi-armed bandits with changing model parameters over time.
method Adaptive Resetting Bandit (ADR-bandit) algorithm using adaptive windowing techniques.
result ADR-bandit achieves nearly optimal performance in both abrupt and gradual changes.

A method identifies abrupt changes in functions with fixed confidence under noisy feedback.

problem Identifying abrupt changes in piecewise constant functions quickly and with certainty.
method Fixed-confidence piecewise constant bandit problem, focusing sampling efforts near change points.
result Asymptotically optimal method proven computationally efficient and effective in experiments.

The objective of the change-point detection is to discover the abrupt property changes lying behind the time-series data. In this paper, we firstly summarize the definition and in-depth implication of the changepoint detection. The next stage is to elaborate traditional and some alternative model-based changepoint dete…

2019-08-20abs ↗pdf ↗

Enhash detects concept drift in data streams quickly and efficiently.

problem Detecting abrupt, gradual, virtual, or recurring events in data streams.
method Uses projection hash to insert incoming samples and detects concept drift.
result Enhash has competitive performance and moderate resource requirements compared to existing ensemble learners.

DeRegiME forecasts with regime structure, improving probabilistic predictions across various time series.

problem Probabilistic forecasting discards residual uncertainty, and distribution shifts are hard to capture.
method DeRegiME uses a sparse variational Gaussian process with a nonstationary regime-mixing kernel to separate latent uncertainty regimes.
result DeRegiME improves NLPD by 20.3% on average across benchmarks, with gains on CRPS and MSE.

We consider the classical stochastic multi-armed bandit problem with a constraint that limits the total cost incurred by switching between actions to be no larger than a given switching budget. For this problem, we prove matching upper and lower bounds on the optimal (i.e., minimax) regret, and provide efficient rate-o…

2019-05-26abs ↗pdf ↗

Data-driven predictive analytics are in use today across a number of industrial applications, but further integration is hindered by the requirement of similarity among model training and test data distributions. This paper addresses the need of learning from possibly nonstationary data streams, or under concept drift,…

2017-10-18abs ↗pdf ↗

Change-point detection (CPD) aims to locate abrupt transitions in the generative model of a sequence of observations. When Bayesian methods are considered, the standard practice is to infer the posterior distribution of the change-point locations. However, for complex models (high-dimensional or heterogeneous), it is n…

2019-10-22abs ↗pdf ↗

Bayesian optimisation for dynamically adjusting learning rates in machine learning models.

problem Dynamic adjustment of learning rates schedules in machine learning models.
method Probabilistic model based on latent Gaussian processes and auto-/regressive formulation.
result Flexibly adjusts learning rates schedules to abrupt changes of behaviours.

Continual learning aims to improve the ability of modern learning systems to deal with non-stationary distributions, typically by attempting to learn a series of tasks sequentially. Prior art in the field has largely considered supervised or reinforcement learning tasks, and often assumes full knowledge of task labels …

2019-10-31abs ↗pdf ↗

Study analyzes European energy markets' reactions to 2022 events using Bayesian methods.

problem Detecting structural breakpoints in energy and financial markets during turbulent times.
method Combines Hurst exponent for market efficiency, BEAST for abrupt changes, and seasonal/trend analysis.
result Markets exhibit varied reactions to critical events, affecting their trends and breakpoints.

Detecting the emergence of abrupt property changes in time series is a challenging problem. Kernel two-sample test has been studied for this task which makes fewer assumptions on the distributions than traditional parametric approaches. However, selecting kernels is non-trivial in practice. Although kernel selection fo…

2019-01-18abs ↗pdf ↗

The dichotomous coordinate descent (DCD) algorithm has been successfully used for significant reduction in the complexity of recursive least squares (RLS) algorithms. In this work, we generalize the application of the DCD algorithm to RLS adaptive filtering in impulsive noise scenarios and derive a unified update formu…

2019-08-18abs ↗pdf ↗

Delta hedging, which plays a crucial rôle in modern financial engineering, is a tracking control design for a "risk-free" management. We utilize the existence of trends in financial time series (Fliess M., Join C.: A mathematical proof of the existence of trends in financial time series, Proc. Int. Conf. Systems Theory…

2010-05-03abs ↗pdf ↗

This paper addresses parameter estimation for wave equations with Markovian switching.

problem Parameter estimation for wave equations with abrupt changes.
method Bayesian statistical framework using discrete sparse Bayesian learning.
result Strong performance in parameter estimation for variable coefficient PDEs.