A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
LLMs struggle with arithmetic tasks unless they use high numerical precision.
SINGD improves KFAC for memory-efficiency and stability in low-precision training.
The paper addresses numerical integration issues in SV models, proposing a fast regime switching algorithm.
With ever-increasing computational demand for deep learning, it is critical to investigate the implications of the numeric representation and precision of DNN model weights and activations on computational efficiency. In this work, we explore unconventional narrow-precision floating-point representations as it relates …
GNMR controls runtime stability in low-precision language model training.
The study analyzes numerical stability in large language models using mixed-precision arithmetic.
Two-layer networks struggle with high frequencies due to numerical and computational limitations.
This work studies scaling laws for low-precision training in high-dimensional linear regression.
Low-precision DNNs have been extensively explored in order to reduce the size of DNN models for edge devices. Recently, the posit numerical format has shown promise for DNN data representation and compute with ultra-low precision in [5..8]-bits. However, previous studies were limited to studying posit for DNN inference…
Study on numerical reliability of AD for MaxPool in neural nets.
This paper numerically computes the topological and smooth invariants of Eschenburg spaces with small fourth cohomology group, following Kruggel's determination of the Kreck-Stolz invariants of Eschenburg spaces that satisfy condition C. The GNU GMP arbitrary-precision library is utilised.
Reduced numerical precision is a common technique to reduce computational cost in many Deep Neural Networks (DNNs). While it has been observed that DNNs are resilient to small errors and noise, no general result exists that is capable of predicting a given DNN system architecture's sensitivity to reduced precision. In …
Training of large-scale deep neural networks is often constrained by the available computational resources. We study the effect of limited precision data representation and computation on neural network training. Within the context of low-precision fixed-point computations, we observe the rounding scheme to play a cruc…
Diagonal transformations preserve independence structures in non-Gaussian distributions.
Deep neural networks have enabled progress in a wide variety of applications. Growing the size of the neural network typically results in improved accuracy. As model sizes grow, the memory and compute requirements for training these models also increases. We introduce a technique to train deep neural networks using hal…
Recently, the posit numerical format has shown promise for DNN data representation and compute with ultra-low precision ([5..8]-bit). However, majority of studies focus only on DNN inference. In this work, we propose DNN training using posits and compare with the floating point training. We evaluate on both MNIST and F…
Efforts to reduce the numerical precision of computations in deep learning training have yielded systems that aggressively quantize weights and activations, yet employ wide high-precision accumulators for partial sums in inner-product operations to preserve the quality of convergence. The absence of any framework to an…
Paper derives a simplified formula for Expected Improvement using log-transformed data.
A new method for estimating large-scale linear models with improved precision.
Paper proves conditions for estimating precision matrices with Laplacian constraints.
Kähler-Ricci flow singularity type is independent of initial metric.
Study finds numerical moduli in special 2-flags of length 5.
We present a new method for estimating multivariate, second-order stationary Gaussian Random Field (GRF) models based on the Sparse Precision matrix Selection (SPS) algorithm, proposed by Davanloo et al. (2015) for estimating scalar GRF models. Theoretical convergence rates for the estimated between-response covariance…
Currently, deep neural networks are deployed on low-power portable devices by first training a full-precision model using powerful hardware, and then deriving a corresponding low-precision model for efficient inference on such systems. However, training models directly with coarsely quantized weights is a key step towa…
This paper explains how low-precision arithmetic causes loss spikes in deep learning models.
Study provides bounds for estimating intrinsic dimension using Gaussian kernels.
EMPIR combines low and full precision DNNs to enhance robustness against adversarial attacks.
The paper tackles sparse graph learning under Laplacian-related constraints, improving upon existing methods.
New method estimates portfolio turnover using covariance matrix of returns.
Deep neural networks (DNNs) can be made hardware-efficient by reducing the numerical precision of the weights and activations of the network and by improving the network's resilience to noise. However, this gain in efficiency often comes at the cost of significantly reduced accuracy. In this paper, we present a novel a…
LightOn OPUs accelerate randomized numerical linear algebra, reducing computational costs.
We present any-precision deep neural networks (DNNs), which are trained with a new method that allows the learned DNNs to be flexible in numerical precision during inference. The same model in runtime can be flexibly and directly set to different bit-widths, by truncating the least significant bits, to support dynamic …
BatchNorm helps train quantized networks by avoiding gradient explosion.
Adaptive batch sizes improve active learning efficiency and flexibility.
SCOPE estimator improves covariance and precision matrix estimation.
New insights on Shapley value precision for tabular data predictions.
We implement methods from computational homology to obtain a topological signal of singularity formation in a selection of geometries evolved numerically by Ricci flow. Our approach, based on persistent homology, produces precise, quantitative measures describing the behavior of an entire collection of data across a di…
The study analyzes convergence of adaptive optimizers under low-precision training.
The paper analyzes data augmentation for precision matrix estimation in high dimensions.
We introduce a general framework for estimation of inverse covariance, or precision, matrices from heterogeneous populations. The proposed framework uses a Laplacian shrinkage penalty to encourage similarity among estimates from disparate, but related, subpopulations, while allowing for differences among matrices. We p…
We study the accuracy of estimating the covariance and the precision matrix of a -variate sub-Gaussian distribution along a prescribed subspace or direction using the finite sample covariance. Our results show that the estimation accuracy depends almost exclusively on the components of the distribution that correspo…
The paper analyzes how deep models memorize spurious features.
We deliver a call to arms for probabilistic numerical methods: algorithms for numerical tasks, including linear algebra, integration, optimization and solving differential equations, that return uncertainties in their calculations. Such uncertainties, arising from the loss of precision induced by numerical calculation …
New methods improve insurance data quality for catastrophic events.
We introduce a new strategy designed to help physicists discover hidden laws governing dynamical systems. We propose to use machine learning automatic differentiation libraries to develop hybrid numerical models that combine components based on prior physical knowledge with components based on neural networks. In these…
The Sampled Gaussian Mechanism (SGM)---a composition of subsampling and the additive Gaussian noise---has been successfully used in a number of machine learning applications. The mechanism's unexpected power is derived from privacy amplification by sampling where the privacy cost of a single evaluation diminishes quadr…
Recent exploration of optimal individualized decision rules (IDRs) for patients in precision medicine has attracted a lot of attention due to the heterogeneous responses of patients to different treatments. In the existing literature of precision medicine, an optimal IDR is defined as a decision function mapping from t…
New method analyzes volatility models for option prices, especially in rough volatility.