A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Paper explores reducing precision in SVM for faster text classification.
Reduced precision computation for deep neural networks is one of the key areas addressing the widening compute gap driven by an exponential growth in model size. In recent years, deep learning training has largely migrated to 16-bit precision, with significant gains in performance and energy efficiency. However, attemp…
New framework reduces cost of financial option pricing simulations on FPGAs.
Efforts to reduce the numerical precision of computations in deep learning training have yielded systems that aggressively quantize weights and activations, yet employ wide high-precision accumulators for partial sums in inner-product operations to preserve the quality of convergence. The absence of any framework to an…
Low-precision training reduces computational cost and produces efficient models. Recent research in developing new low-precision training algorithms often relies on simulation to empirically evaluate the statistical effects of quantization while avoiding the substantial overhead of building specific hardware. To suppor…
Paper explores low-precision SGLD for neural networks, reducing costs without sacrificing performance.
Low-precision computation is often used to lower the time and energy cost of machine learning, and recently hardware accelerators have been developed to support it. Still, it has been used primarily for inference - not training. Previous low-precision training algorithms suffered from a fundamental tradeoff: as the num…
The state-of-the-art hardware platforms for training Deep Neural Networks (DNNs) are moving from traditional single precision (32-bit) computations towards 16 bits of precision -- in large part due to the high energy efficiency and smaller bit storage associated with using reduced-precision representations. However, un…
Reduced numerical precision is a common technique to reduce computational cost in many Deep Neural Networks (DNNs). While it has been observed that DNNs are resilient to small errors and noise, no general result exists that is capable of predicting a given DNN system architecture's sensitivity to reduced precision. In …
Neural networks with low-precision weights and activations offer compelling efficiency advantages over their full-precision equivalents. The two most frequently discussed benefits of quantization are reduced memory consumption, and a faster forward pass when implemented with efficient bitwise operations. We propose a t…
Similar to convolution neural networks, recurrent neural networks (RNNs) typically suffer from over-parameterization. Quantizing bit-widths of weights and activations results in runtime efficiency on hardware, yet it often comes at the cost of reduced accuracy. This paper proposes a quantization approach that increases…
Recently there has been significant interest in training machine-learning models at low precision: by reducing precision, one can reduce computation and communication by one order of magnitude. We examine training at reduced precision, both from a theoretical and practical perspective, and ask: is it possible to train …
High-precision machine learning reduces particle physics simulations by orders of magnitude.
Deep neural networks (DNNs) can be made hardware-efficient by reducing the numerical precision of the weights and activations of the network and by improving the network's resilience to noise. However, this gain in efficiency often comes at the cost of significantly reduced accuracy. In this paper, we present a novel a…
Precision farming uses data analysis to optimize crop management.
Study projective derivative cocycles for circle diffeomorphisms.
New methods use RL and DA to improve dosing precision and reduce side effects.
With ever-increasing computational demand for deep learning, it is critical to investigate the implications of the numeric representation and precision of DNN model weights and activations on computational efficiency. In this work, we explore unconventional narrow-precision floating-point representations as it relates …
Efficient Bitwidth Search optimizes neural network quantization for better performance.
Deep neural networks have enabled progress in a wide variety of applications. Growing the size of the neural network typically results in improved accuracy. As model sizes grow, the memory and compute requirements for training these models also increases. We introduce a technique to train deep neural networks using hal…
Mixed-precision CA-SGD for generalized linear models on GPUs
Paper proves existence of a universal codebook for low-precision quantization.
We present an overview of techniques for quantizing convolutional neural networks for inference with integer weights and activations. Per-channel quantization of weights and per-layer quantization of activations to 8-bits of precision post-training produces classification accuracies within 2% of floating point networks…
It is well-known that the precision of data, hyperparameters, and internal representations employed in learning systems directly impacts its energy, throughput, and latency. The precision requirements for the training algorithm are also important for systems that learn on-the-fly. Prior work has shown that the data and…
StatQAT optimizes quantization for deep networks, reducing computational cost and memory usage.
A new optimization method reduces memory and compute requirements for deep learning.
The paper improves support recovery in high-dimensional precision matrix estimation using meta learning.
A new deep learning method using Boolean logic reduces training and inference energy.
In recent years, dynamically growing data and incrementally growing number of classes pose new challenges to large-scale data classification research. Most traditional methods struggle to balance the precision and computational burden when data and its number of classes increased. However, some methods are with weak pr…
Model explains how stablecoin runs are influenced by large sales and reserve quality.
Vaisman's theorem extended to locally reducible Kähler spaces.
This work studies scaling laws for low-precision training in high-dimensional linear regression.
In this paper a novel modification of the multilevel Monte Carlo approach, allowing for further significant complexity reduction, is proposed. The idea of the modification is to use the method of control variates to reduce variance at level zero. We show that, under a proper choice of control variates, one can reduce t…
SPEQ improves quantized neural networks by stochastic precision sharing and cosine similarity loss.
Distributed Lion optimizes large model training by reducing communication costs.
Predicting drug-target interactions (DTI) is an essential part of the drug discovery process, which is an expensive process in terms of time and cost. Therefore, reducing DTI cost could lead to reduced healthcare costs for a patient. In addition, a precisely learned molecule representation in a DTI model could contribu…
We show that reducible braids which are, in a Garside-theoretical sense, as simple as possible within their conjugacy class, are also as simple as possible in a geometric sense. More precisely, if a braid belongs to a certain subset of its conjugacy class which we call the stabilized set of sliding circuits, and if it …
Low-precision DNNs have been extensively explored in order to reduce the size of DNN models for edge devices. Recently, the posit numerical format has shown promise for DNN data representation and compute with ultra-low precision in [5..8]-bits. However, previous studies were limited to studying posit for DNN inference…
Graphical models have gained a lot of attention recently as a tool for learning and representing dependencies among variables in multivariate data. Often, domain scientists are looking specifically for differences among the dependency networks of different conditions or populations (e.g. differences between regulatory …
Knowledge bases (KBs) are the backbone of many ubiquitous applications and are thus required to exhibit high precision. However, for KBs that store subjective attributes of entities, e.g., whether a movie is "kid friendly", simply estimating precision is complicated by the inherent ambiguity in measuring subjective phe…
SCOPE estimator improves covariance and precision matrix estimation.
Introduces robust and decomposable AP for image retrieval.
The study analyzes convergence of adaptive optimizers under low-precision training.
We develop Hodge theory for a Riemannian manifold with a background closed 3-form, H. Precisely, we prove that if the metric connections with torsion have holonomy groups , then the -Laplacian preserves the irreducible representations of the Lie algebras of the holonomy groups on the space o…
A new method selects inducing points to optimize high-throughput Bayesian optimisation.
In this paper, for a variety of nonholonomic (reducible) Hamiltonian systems, we first give to various distributional Hamiltonian systems, by analyzing carefully the dynamics and structures of the nonholonomic Hamiltonian systems. Secondly, we derive precisely the geometric constraint conditions of the induced distribu…
State of the art deep learning models have made steady progress in the fields of computer vision and natural language processing, at the expense of growing model sizes and computational complexity. Deploying these models on low power and mobile devices poses a challenge due to their limited compute capabilities and str…
Proposes a new method for selecting regularization parameters in sparse precision matrix estimation.