A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
21cmEMU speeds up EoR simulations by 10^4x, predicting key observables with sub-percent accuracy.
New method predicts quasar continuum near Lyman-α with high precision and accuracy.
ResUNet-CMB neural network reconstructs CMB effects from noisy data.
Paper uses ResUNet-CMB to reconstruct cosmic polarization rotation from CMB data.
Few-shot learning aims to train efficient predictive models with a few examples. The lack of training data leads to poor models that perform high-variance or low-confidence predictions. In this paper, we propose to meta-learn the ensemble of epoch-wise empirical Bayes models (E3BM) to achieve robust predictions. "Epoch…
A deep learning model, named IITNet, is proposed to learn intra- and inter-epoch temporal contexts from raw single-channel EEG for automatic sleep scoring. To classify the sleep stage from half-minute EEG, called an epoch, sleep experts investigate sleep-related events and consider the transition rules between the foun…
Epoch-GDA achieves optimal convergence rate for SCSC min-max problems.
This paper tackles overfitting in CTR models by introducing Multi-Epoch learning with Data Augmentation.
In unsupervised learning, collecting more data is not always a costly process unlike the training. For example, it is not hard to enlarge the 40GB WebText used for training GPT-2 by modifying its sampling methodology considering how many webpages there are in the Internet. On the other hand, given that training on this…
This paper explains double descent in linear neural networks, identifying new factors.
The paper analyzes how repeating epochs affects data scaling in linear regression.
One epoch training yields certifiably robust models.
Improves early stopping in deep networks by adjusting stepsizes.
Adaptive regularization prevents overfitting in large-scale sparse feature models.
Automatic sleep staging has been often treated as a simple classification problem that aims at determining the label of individual target polysomnography (PSG) epochs one at a time. In this work, we tackle the task as a sequence-to-sequence classification problem that receives a sequence of multiple epochs as input and…
Historical economic growth in Asia (excluding Japan) is analysed. It is shown that Unified Growth Theory is contradicted by the data, which were used (but not analysed) during the formulation of this theory. Unified Growth Theory does not explain the mechanism of economic growth. It explains the mechanism of Malthusian…
We consider stochastic strongly convex optimization with a complex inequality constraint. This complex inequality constraint may lead to computationally expensive projections in algorithmic iterations of the stochastic gradient descent~(SGD) methods. To reduce the computation costs pertaining to the projections, we pro…
Topic models have proven to be a useful tool for discovering latent structures in document collections. However, most document collections often come as temporal streams and thus several aspects of the latent structure such as the number of topics, the topics' distribution and popularity are time-evolving. Several mode…
Unbalanced GANs stabilize GAN training by pre-training the generator with VAE.
A new learning scheme improves model efficiency and performance.
A method for faster neural architecture search using low-fidelity training.
Oil price data have a complicated multi-scale structure that may vary with time. We use time-frequency analysis to identify the main features of these variations and, in particular, the regime shifts. The analysis is based on a wavelet-based decomposition and analysis of the associated scale spectrum. The joint estimat…
This paper identifies and analyzes the Epochal Sawtooth Phenomenon in training loss curves.
We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging…
Correctly identifying sleep stages is important in diagnosing and treating sleep disorders. This work proposes a joint classification-and-prediction framework based on CNNs for automatic sleep staging, and, subsequently, introduces a simple yet efficient CNN architecture to power the framework. Given a single input epo…
We present a brief overview of random matrix theory (RMT) with the objectives of highlighting the computational results and applications in financial markets as complex systems. An oft-encountered problem in computational finance is the choice of an appropriate epoch over which the empirical cross-correlation return ma…
Poly-view contrastive learning improves image representation learning.
Optimal SGD rates achieved with shuffling, covering non-convex and convex cases.
RAE improves image representation learning with simplified design choices.
New active learning framework for multiclass classification beyond realizability assumption.
Within a statistical learning setting, we propose and study an iterative regularization algorithm for least squares defined by an incremental gradient method. In particular, we show that, if all other parameters are fixed a priori, the number of passes over the data (epochs) acts as a regularization parameter, and prov…
It is hypothesized that price charts can be empirically decomposed into two components as random and non random. The non random component, which can be treated as approximately regular behavior of the prices (trend) in an epoch, is a geometric line. Thus, the random component fluctuates around the non random component …
High fidelity behavior prediction of intelligent agents is critical in many applications. However, the prediction model trained on the training set may not generalize to the testing set due to domain shift and time variance. The challenge motivates the adoption of online adaptation algorithms to update prediction model…
Deep learning models can overfit noisy data without losing generalization.
InstanceFlow visualizes classifier confusion over training epochs.
Deep learning is rapidly becoming a go-to tool for many artificial intelligence problems due to its ability to outperform other approaches and even humans at many problems. Despite its popularity we are still unable to accurately predict the time it will take to train a deep learning network to solve a given problem. T…
While existing work on neural architecture search (NAS) tunes hyperparameters in a separate post-processing step, we demonstrate that architectural choices and other hyperparameter settings interact in a way that can render this separation suboptimal. Likewise, we demonstrate that the common practice of using very few …
Stochastic gradient decent~(SGD) and its variants, including some accelerated variants, have become popular for training in machine learning. However, in all existing SGD and its variants, the sample size in each iteration~(epoch) of training is the same as the size of the full training set. In this paper, we propose a…
Adam optimizer's bias is influenced by mini-batch size and momentum hyperparameters.
While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that there exists a convex loss function for which the stability gap fo…
Determining the appropriate batch size for mini-batch gradient descent is always time consuming as it often relies on grid search. This paper considers a resizable mini-batch gradient descent (RMGD) algorithm based on a multi-armed bandit for achieving best performance in grid search by selecting an appropriate batch s…
In this technical report, we consider an approach that combines the PPO objective and K-FAC natural gradient optimization, for which we call PPOKFAC. We perform a range of empirical analysis on various aspects of the algorithm, such as sample complexity, training speed, and sensitivity to batch size and training epochs…
Study finds common poetic themes across languages over time.
A long-standing problem in the theory of stochastic gradient descent (SGD) is to prove that its without-replacement version RandomShuffle converges faster than the usual with-replacement version. We present the first (to our knowledge) non-asymptotic solution to this problem, which shows that after a "reasonable" numbe…
New algorithm reduces regret from sqrt(T) to polylog(T) in stochastic contextual linear bandits.
The paper uses LSMC to price capped American options with time-dependent caps.
In this paper we study the problem of minimizing the average of a large number () of smooth convex loss functions. We propose a new method, S2GD (Semi-Stochastic Gradient Descent), which runs for one or several epochs in each of which a single full gradient and a random number of stochastic gradients is computed, fo…
Averaging recent model checkpoints speeds up training time.