A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New model stabilizes asynchronous LTI systems, independent of synchronous stability.
New stability conditions for ZO methods reveal unique regularization effects.
This paper presents a stochastic behavior analysis of a kernel-based stochastic restricted-gradient descent method. The restricted gradient gives a steepest ascent direction within the so-called dictionary subspace. The analysis provides the transient and steady state performance in the mean squared error criterion. It…
New method reveals insights about stochastic optimization methods using modified equations.
Constrained adaptive filtering algorithms inculding constrained least mean square (CLMS), constrained affine projection (CAP) and constrained recursive least squares (CRLS) have been extensively studied in many applications. Most existing constrained adaptive filtering algorithms are developed under mean square error (…
Artificial neural network training with stochastic gradient descent can be destabilized by "bad batches" with high losses. This is often problematic for training with small batch sizes, high order loss functions or unstably high learning rates. To stabilize learning, we have developed adaptive learning rate clipping (A…
We study the stability vis a vis adversarial noise of matrix factorization algorithm for matrix completion. In particular, our results include: (I) we bound the gap between the solution matrix of the factorization method and the ground truth in terms of root mean square error; (II) we treat the matrix factorization as …
Paper analyzes robustness of data-selective Volterra NLMS algorithm.
WAVE improves stability in reinforcement learning by adaptively weighting critic's loss.
Layer normalization (LayerNorm) has been successfully applied to various deep neural networks to help stabilize training and boost model convergence because of its capability in handling re-centering and re-scaling of both inputs and weight matrix. However, the computational overhead introduced by LayerNorm makes these…
Paper uses Time Series Transformer for bank stability prediction.
New results show flat minima in neural networks suffer from high dimensionality.
Paper introduces stability in model averaging and proposes a L2-penalty method.
New adaptive importance samplers improve stability and accuracy.
This study analyzes AdaGrad's stability and convergence in non-convex optimization.
Gradient filters track moving parameters under noisy data and misspecification.
DEQGAN uses GANs to solve differential equations without supervision.
Adaptive filtering algorithms operating in reproducing kernel Hilbert spaces have demonstrated superiority over their linear counterpart for nonlinear system identification. Unfortunately, an undesirable characteristic of these methods is that the order of the filters grows linearly with the number of input data. This …
Identifies bilinear systems from a single trajectory with optimal sample complexity.
Double Q-learning has the same mean-squared error as Q-learning under certain conditions.
The kernel least mean squares (KLMS) algorithm is a computationally efficient nonlinear adaptive filtering method that "kernelizes" the celebrated (linear) least mean squares algorithm. We demonstrate that the least mean squares algorithm is closely related to the Kalman filtering, and thus, the KLMS can be interpreted…
New activation function BrownianReLU improves LSTM network performance on financial time series.
A new method for estimating adversarial strategies in nonlinear systems.
Paper shows ERM's suboptimality due to bias, not variance.
In this work we propose an adversarial learning approach to generate high resolution MRI scans from low resolution images. The architecture, based on the SRGAN model, adopts 3D convolutions to exploit volumetric information. For the discriminator, the adversarial loss uses least squares in order to stabilize the traini…
This paper analyzes sampling from heavy-tailed distributions using discretized Itô diffusions.
Neural network training is usually accomplished by solving a non-convex optimization problem using stochastic gradient descent. Although one optimizes over the networks parameters, the main loss function generally only depends on the realization of the neural network, i.e. the function it computes. Studying the optimiz…
Machine learning improves American option pricing accuracy.
This study calculates the maximum error of a famous estimation method.
Paper proposes a DNN-driven AF framework for improved generalization.
The paper evaluates company investment value using machine learning models.
We consider adaptive system identification problems with convex constraints and propose a family of regularized Least-Mean-Square (LMS) algorithms. We show that with a properly selected regularization parameter the regularized LMS provably dominates its conventional counterpart in terms of mean square deviations. We es…
Generative adversarial networks (GANs) are highly effective unsupervised learning frameworks that can generate very sharp data, even for data such as images with complex, highly multimodal distributions. However GANs are known to be very hard to train, suffering from problems such as mode collapse and disturbing visual…
New method optimizes tail dependence coefficient estimation.
This paper explains CART random forests using stochastic control theory.
Nonparametric modeling approaches show very promising results in the area of system identification and control. A naturally provided model confidence is highly relevant for system-theoretical considerations to provide guarantees for application scenarios. Gaussian process regression represents one approach which provid…
The Matérn covariance function is a popular choice for prediction in spatial statistics and uncertainty quantification literature. A key benefit of the Matérn class is that it is possible to get precise control over the degree of mean-square differentiability of the random process. However, the Matérn class possesses e…
Improved Least-Squares Monte Carlo with finite-difference ansatz.
Cryptocurrency prices predicted using LSTM, SVM, and polynomial regression.
Despite the simplicity and intuitive interpretation of Minimum Mean Squared Error (MMSE) estimators, their effectiveness in certain scenarios is questionable. Indeed, minimizing squared errors on average does not provide any form of stability, as the volatility of the estimation error is left unconstrained. When this v…
We formulate the problem of neural network optimization as Bayesian filtering, where the observations are the backpropagated gradients. While neural network optimization has previously been studied using natural gradient methods which are closely related to Bayesian inference, they were unable to recover standard optim…
Proposes adversarial method to estimate Riesz representer.
This paper concerns error bounds for recursive equations subject to Markovian disturbances. Motivating examples abound within the fields of Markov chain Monte Carlo (MCMC) and Reinforcement Learning (RL), and many of these algorithms can be interpreted as special cases of stochastic approximation (SA). It is argued tha…
Study on LMMSE estimation with model mismatch, quantifying MSE trade-offs.
A fast method for LOOCV in k-NN regression reduces computation time.
Kernel adaptive filters (KAF) are a class of powerful nonlinear filters developed in Reproducing Kernel Hilbert Space (RKHS). The Gaussian kernel is usually the default kernel in KAF algorithms, but selecting the proper kernel size (bandwidth) is still an open important issue especially for learning with small sample s…
New theory shows large learning rates prevent overfitting in neural networks.
New methods for estimating complex causal effects in econometrics.