arXiv research

Finite-precision learning of $anh$ networks is limited by the Monte Carlo rate.

Paper approximates solutions for complex decision processes with limited precision.

Low-precision streaming PCA estimates the leading eigenvector with limited precision.

Training of large-scale deep neural networks is often constrained by the available computational resources. We study the effect of limited precision data representation and computation on neural network training. Within the context of low-precision fixed-point computations, we observe the rounding scheme to play a cruc…

It is well-known that the precision of data, hyperparameters, and internal representations employed in learning systems directly impacts its energy, throughput, and latency. The precision requirements for the training algorithm are also important for systems that learn on-the-fly. Prior work has shown that the data and…

Given a compact closed subset $M$ of a line segment in $\mathbb{R}^3$, we construct a sequence of minimal surfaces $Σ_k$ embedded in a neighborhood $C$ of the line segment that converge smoothly to a limit lamination of $C$ away from $M$. Moreover, the curvature of this sequence blows up precisely on $M$, and the limit…

Study best arm identification with limited precision sampling in bandits.

Constructs hyperbolic reflection groups with 3D limit sets.

Link signature limit depends on linking matrix under specific polynomial condition.

Low precision networks in the reinforcement learning (RL) setting are relatively unexplored because of the limitations of binary activations for function approximation. Here, in the discrete action ATARI domain, we demonstrate, for the first time, that low precision policy distillation from a high precision network pro…

Low-precision computation is often used to lower the time and energy cost of machine learning, and recently hardware accelerators have been developed to support it. Still, it has been used primarily for inference - not training. Previous low-precision training algorithms suffered from a fundamental tradeoff: as the num…

Maximizing the speed and precision of communication while minimizing power dissipation is a fundamental engineering design goal. Also, biological systems achieve remarkable speed, precision and power efficiency using poorly understood physical design principles. Powerful theories like information theory and thermodynam…

SINGD improves KFAC for memory-efficiency and stability in low-precision training.

The paper improves precision matrix estimation by SLOPE, especially in high-dimensional settings.

Proposes rounding method for precise treatment effect estimation under budget constraints.

This paper examines the precision of estimators of Quantile-Based Risk Measures (Value at Risk, Expected Shortfall, Spectral Risk Measures). It first addresses the question of how to estimate the precision of these estimators, and proposes a Monte Carlo method that is free of some of the limitations of existing approac…

Mixed-precision CA-SGD for generalized linear models on GPUs

Deep neural networks have enabled progress in a wide variety of applications. Growing the size of the neural network typically results in improved accuracy. As model sizes grow, the memory and compute requirements for training these models also increases. We introduce a technique to train deep neural networks using hal…

The paper analyzes the excess risk of PCA and provides a precise characterization.

Study G-H limits of surfaces with boundary, focusing on same Euler characteristic.

Active learning selects both observations and annotation precision for Gaussian Processes.

This paper improves low-precision sampling using SGHMC for deep learning models.

The paper improves Bayesian precision matrix estimation for high-dimensional sparse data.

The paper examines how deep linear neural networks behave as they become infinitely wide.

AI-enabled precision medicine promises a transformational improvement in healthcare outcomes by enabling data-driven personalized diagnosis, prognosis, and treatment. However, the well-known "curse of dimensionality" and the clustered structure of biomedical data together interact to present a joint challenge in the hi…

We study local ($n+1$)-webs of codimension 1 on a manifold of dimension $n.$ We give a complete description of their possible Lie algebras of infinitesimal diffeomorphisms. More precisely we show that these Lie algebras are direct products of sub-algebras which are isomorphic to $\frak{sl}(2),$ to the non-commutative 2…

Recent machine learning methods use increasingly large deep neural networks to achieve state of the art results in various tasks. The gains in performance come at the cost of a substantial increase in computation and storage requirements. This makes real-time implementations on limited resources hardware a challenging …

This paper focuses on stochastic orders and its applications : policy limits and deductibles. Further, many applications and some examples are given : comparison of two families of copulas, individual and collective risk model, reinsurance contracts and dependent portfolios increase risk. More precisely, we propose a n…

UCB algorithms improve on bandit problems with precise regret analysis and adaptive inference.

Derives scaling limits and fluctuations for SGD in high dimensions.

While Recurrent Neural Networks (RNNs) are famously known to be Turing complete, this relies on infinite precision in the states and unbounded computation time. We consider the case of RNNs with finite precision whose computation time is linear in the input length. Under these limitations, we show that different RNN va…

EMPIR combines low and full precision DNNs to enhance robustness against adversarial attacks.

Study identifies three quantization regimes for ReLU networks.

Currently, deep neural networks are deployed on low-power portable devices by first training a full-precision model using powerful hardware, and then deriving a corresponding low-precision model for efficient inference on such systems. However, training models directly with coarsely quantized weights is a key step towa…

We improve Gaussian copula models for imputing mixed data types with precise approximations.

This paper reviews random forest methods for analyzing longitudinal data in precision medicine.

Let $(M,g)$ be a smooth compact Riemannian surface with no boundary. Given a smooth vector field $V$ with finitely many zeroes on $M$, we study the distribution of the number of tangencies to $V$ of the nodal components of random band-limited functions. It is determined that in the high-energy limit, these obey a unive…

Machine learning model predicts DFT total energy to complete basis set limit.

Low-precision DNNs have been extensively explored in order to reduce the size of DNN models for edge devices. Recently, the posit numerical format has shown promise for DNN data representation and compute with ultra-low precision in [5..8]-bits. However, previous studies were limited to studying posit for DNN inference…

Study shows flow convergence to smooth K-Ricci outside a divisor with cusp singularity.

The paper sets fundamental limits for ERM in high dimensions.

Trans-Glasso uses transfer learning to estimate precision matrices from related studies.

We use a Lagrangian perspective to show the limiting absorption principle on Riemannian scattering, i.e. asymptotically conic, spaces, and their generalizations. More precisely we show that, for non-zero spectral parameter, the `on spectrum', as well as the `off-spectrum', spectral family is Fredholm in function spaces…

The two-sphere valued wave map flow on a Lorentzian domain R x Sigma, where Sigma is any flat two-torus, is studied. The Cauchy problem with initial data tangent to the moduli space of holomorphic maps Sigma -> S^2 is considered, in the limit of small initial velocity. It is proved that wave maps, in this limit, conver…

The study characterizes lamination limits and homeomorphisms in 3D handlebodies.

Improved deep learning model deployment on tiny MCUs with mixed-precision quantization.

We review recent quantitative results on the approximation of mean field diffusion equations by large systems of interacting particles, obtained by optimal coupling methods. These results concern a larger range of models, more precise senses of convergence and links with the long time behaviour of the systems to be con…

Investigates fast prediction rates with limited expert advice.