A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
LoRA fine-tuning creates intruder dimensions that can cause forgetting, and a new law predicts when this happens.
problem Predicting when LoRA fine-tuning creates intruder dimensions that can cause catastrophic forgetting.
method Derived a per-layer critical update strength s∗ and an exact secular-equation characterization of the updated spectrum.
result The law localizes the empirical threshold within a factor of two on 82% of layers and separates intruder-bearing from intruder-free layers at deployment.
Unified learning-rate scale for CNNs and ResNets, avoiding depth imbalance.
problem Challenges in choosing an appropriate learning rate for deep networks, especially as depth increases.
method Introduces Arithmetic-Mean μP (AM-μP), constraining network-wide average pre-activation second moment to a constant scale, combined with residual-aware He fan-in initialization.
result Demonstrates a −3/2 scaling law for learning rates across depths, enabling zero-shot learning-rate transfer.
Training convolutional neural network models is memory intensive since back-propagation requires storing activations of all intermediate layers. This presents a practical concern when seeking to deploy very deep architectures in production, especially when models need to be frequently re-trained on updated datasets. In…
The advance of modern sensor technologies enables collection of multi-stream longitudinal data where multiple signals from different units are collected in real-time. In this article, we present a non-parametric approach to predict the evolution of multi-stream longitudinal data for an in-service unit through borrowing…
One of the main difficulties in analyzing neural networks is the non-convexity of the loss function which may have many bad local minima. In this paper, we study the landscape of neural networks for binary classification tasks. Under mild assumptions, we prove that after adding one special neuron with a skip connection…
A new learning strategy using two GP layers for inhomogeneous data.
problem Addressing inhomogeneous empirical correlation structures in data.
method Modeling the function as a sample function of a non-stationary Gaussian Process (GP) nested within multiple stationary GPs, with hyperparameters dependent on the outer GP.
result The approach is sufficient with two GP layers, and the model can be implemented using MCMC.
We propose a voted dual averaging method for online classification problems with explicit regularization. This method employs the update rule of the regularized dual averaging (RDA) method, but only on the subsequence of training examples where a classification error is made. We derive a bound on the number of mistakes…
We characterize convolutional neural networks with respect to the relative amount of features per layer. Using a skew normal distribution as a parametrized framework, we investigate the common assumption of monotonously increasing feature-counts with higher layers of architecture designs. Our evaluation on models with …
We propose an algorithm, semismooth Newton coordinate descent (SNCD), for the elastic-net penalized Huber loss regression and quantile regression in high dimensional settings. Unlike existing coordinate descent type algorithms, the SNCD updates each regression coefficient and its corresponding subgradient simultaneousl…
We present an overview of techniques for quantizing convolutional neural networks for inference with integer weights and activations. Per-channel quantization of weights and per-layer quantization of activations to 8-bits of precision post-training produces classification accuracies within 2% of floating point networks…
We apply supervised deep neural networks (DNNs) for pricing and calibration of both vanilla and exotic options under both diffusion and pure jump processes with and without stochastic volatility. We train our neural network models under different number of layers, neurons per layer, and various different activation fun…
Understanding tie strength in social networks, and the factors that influence it, have received much attention in a myriad of disciplines for decades. Several models incorporating indicators of tie strength have been proposed and used to quantify relationships in social networks, and a standard set of structural networ…
Unified Latent Dynamics unifies model-free and model-based reinforcement learning.
problem Combining the efficiency of model-free methods with the representational strengths of model-based approaches.
method Embedding state-action pairs into a latent space where the true value function is approximately linear, using synchronized updates of encoder, value, and policy networks.
result ULD achieves cross-domain competence with minimal tuning and a fraction of the parameter footprint.
For any positive integer k, there exist neural networks with Θ(k3) layers, Θ(1) nodes per layer, and Θ(1) distinct parameters which can not be approximated by networks with O(k) layers unless they are exponentially large --- they must possess Ω(2k) nodes. This result is proved here for a class o…
Our goal is to learn a semantic parser that maps natural language utterances into executable programs when only indirect supervision is available: examples are labeled with the correct execution result, but not the program itself. Consequently, we must search the space of programs for those that output the correct resu…