arXiv research

We accelerate the power method for strong low-rank approximation using fast sketching.

SEFR is a fast, energy-efficient classifier for ultra-low power devices.

Deep learning detects pneumonia with 36x compression on low-power devices.

This paper tackles energy-efficient machine learning on low-power devices.

Improved cardiac arrhythmia detection in wearable devices with neural networks.

Low precision networks in the reinforcement learning (RL) setting are relatively unexplored because of the limitations of binary activations for function approximation. Here, in the discrete action ATARI domain, we demonstrate, for the first time, that low precision policy distillation from a high precision network pro…

Review of efficient neural networks for TinyML on resource-constrained devices.

A new method improves convergence in low-rank approximation.

New method detects if data points were used in training models with low cost and high power.

Low-rank structure have been profoundly studied in data mining and machine learning. In this paper, we show a dense matrix $X$'s low-rank approximation can be rapidly built from its left and right random projections $Y_1=XA_1$ and $Y_2=X^TA_2$, or bilateral random projection (BRP). We then show power scheme can further…

New KWS neural networks improve accuracy and power efficiency.

CoNNTrA trains DNNs with low-power, low-memory constraints.

Based on empirical financial time-series, we show that the "silence-breaking" probability follows a super-universal power law: the probability of observing a large movement is inversely proportional to the length of the on-going low-variability period. Such a scaling law has been previously predicted theoretically [R. …

We present power low rank ensembles (PLRE), a flexible framework for n-gram language modeling where ensembles of low rank matrices and tensors are used to obtain smoothed probability estimates of words in context. Our method can be understood as a generalization of n-gram modeling to non-integer n, and includes standar…

LoRA enhances model adaptability without increasing parameters.

Language models exhibit low-rank structure, which can be used for generation.

MoEs can efficiently model complex tasks with low-dimensionality and sparsity.

Currently, deep neural networks are deployed on low-power portable devices by first training a full-precision model using powerful hardware, and then deriving a corresponding low-precision model for efficient inference on such systems. However, training models directly with coarsely quantized weights is a key step towa…

We consider first-order differential operators with locally bounded measurable coefficients on vector bundles with measurable coefficient metrics. Under a mild set of assumptions, we demonstrate the equivalence between the essential self-adjointness of such operators to a negligible boundary property. When the operator…

Survey on GKM theory in low dimensions, highlighting combinatorics-geometry interplay.

Power-SMC reduces inference latency for training-free LLM reasoning.

We consider the problem of learning a high-dimensional but low-rank matrix from a large-scale dataset distributed over several machines, where low-rankness is enforced by a convex trace norm constraint. We propose DFW-Trace, a distributed Frank-Wolfe algorithm which leverages the low-rank structure of its updates to ac…

Tensor regression networks achieve high compression rate of neural networks while having slight impact on performances. They do so by imposing low tensor rank structure on the weight matrices of fully connected layers. In recent years, tensor regression networks have been investigated from the perspective of their comp…

Power quandles improve group invariants and allow group presentations.

This work connects diffusion models to power iteration, revealing how low frequencies emerge earlier.

Enhances power of covariance matrix tests for high-dimensional data.

Statistical query algorithms and low-degree tests are nearly equivalent in high-dimensional hypothesis testing.

Innovative neural networks reduce memory usage for efficient, accurate segmentation.

PowerGossip compresses model differences for decentralized deep learning with low-rank linear compressors.

We study the use of power weighted shortest path distance functions for clustering high dimensional Euclidean data, under the assumption that the data is drawn from a collection of disjoint low dimensional manifolds. We argue, theoretically and experimentally, that this leads to higher clustering accuracy. We also pres…

For a symplectic manifold with quantizing line bundle, a choice of almost complex structure determines a Laplacian acting on tensor powers of the bundle. For high tensor powers Guillemin-Uribe showed that there is a well-defined cluster of low-lying eigenvalues, whose distribution is described by a spectral density fun…

FANN-on-MCU enables efficient neural network inference on IoT devices.

Paper applies FloatSD8 to LSTM networks, reducing complexity and power.

Random projections are able to perform dimension reduction efficiently for datasets with nonlinear low-dimensional structures. One well-known example is that random matrices embed sparse vectors into a low-dimensional subspace nearly isometrically, known as the restricted isometric property in compressed sensing. In th…

State of the art deep learning models have made steady progress in the fields of computer vision and natural language processing, at the expense of growing model sizes and computational complexity. Deploying these models on low power and mobile devices poses a challenge due to their limited compute capabilities and str…

Improved convergence for overparameterized low-rank matrix sensing.

Testing independence is of significant interest in many important areas of large-scale inference. Using extreme-value form statistics to test against sparse alternatives and using quadratic form statistics to test against dense alternatives are two important testing procedures for high-dimensional independence. However…

We develop latent variable models for Bayesian learning based low-rank matrix completion and reconstruction from linear measurements. For under-determined systems, the developed methods are shown to reconstruct low-rank matrices when neither the rank nor the noise power is known a-priori. We derive relations between th…

We consider a compact Riemannian manifold with a Hermitian line bundle whose curvature is non-degenerate. The Laplacian acting on high tensor powers (the semiclassical regime) of the bundle exhibits a cluster of low-energy states. We demonstrate that the orthogonal projectors onto these states are the Fourier component…

A new method for anomaly detection adapts to local non-stationarity in low-data regimes.

We constructed an analog electrical circuit which generates fluctuations in which probability density function has power law tails. In the circuit fluctuations with an arbitrary exponent of the power law can be obtained by adjusting the resistance. With this low cost circuit the random fluctuations which have the simil…

SMPI recovers tensor spikes from noisy data with improved performance.

In this paper, a novel joint transmit power and resource allocation approach for enabling ultra-reliable low-latency communication (URLLC) in vehicular networks is proposed. The objective is to minimize the network-wide power consumption of vehicular users (VUEs) while ensuring high reliability in terms of probabilisti…

Efficient FPGA dropout algorithm reduces memory usage.

There is growing interest in being able to run neural networks on sensors, wearables and internet-of-things (IoT) devices. However, the computational demands of neural networks make them difficult to deploy on resource-constrained edge devices. To meet this need, our work introduces a new recurrent unit architecture th…

We simplify neural networks to 3D to study their topological changes.

The study explains transformer scaling laws using statistical and approximation theories.

Paper develops fast low-rank approximation for smoothing splines.