arXiv research

Study on function sensitivity in random DNNs using large deviation theory.

This paper justifies the use of straight-through estimator in training quantized neural nets.

Channel pruning and weight binarization improve keyword spotting accuracy.

Progressive stochastic binarization improves deep network inference efficiency.

QubitHD improves HD computing ML efficiency without sacrificing accuracy.

TentacleNet improves binarized CNNs, reducing accuracy loss and memory usage.

New framework reduces LLM complexity by directly finetuning in Boolean domain.

Develops a binarized GNN for more efficient graph embeddings.

Binarization of digital documents is the task of classifying each pixel in an image of the document as belonging to the background (parchment/paper) or foreground (text/ink). Historical documents are often subjected to degradations, that make the task challenging. In the current work a deep neural network architecture …

A new adaptive binarization technique using fuzzy integrals improves image quality.

This work improves DNN weight quantization with ADMM, achieving lossless binarization and reduced search space.

Binary representation is desirable for its memory efficiency, computation speed and robustness. In this paper, we propose adjustable bounded rectifiers to learn binary representations for deep neural networks. While hard constraining representations across layers to be binary makes training unreasonably difficult, we s…

This work tackles large action spaces in RL by binarizing actions.

FPGAs accelerate DNNs with binarized weights, improving power and speed.

New insights into BNN optimization redefine latent weights as inertia.

New method for ordinal data improves recommendation systems.

Convolutional neural networks have recently achieved significant breakthroughs in various image classification tasks. However, they are computationally expensive,which can make their feasible mplementation on embedded and low-power devices difficult. In this paper convolutional neural network binarization is implemente…

Understanding properties of deep neural networks is an important challenge in deep learning. In this paper, we take a step in this direction by proposing a rigorous way of verifying properties of a popular class of neural networks, Binarized Neural Networks, using the well-developed means of Boolean satisfiability. Our…

User and item features of side information are crucial for accurate recommendation. However, the large number of feature dimensions, e.g., usually larger than 10^7, results in expensive storage and computational cost. This prohibits fast recommendation especially on mobile applications where the computational resource …

A method for efficient approximate inference on discrete distributions.

Our goal is to design architectures that retain the groundbreaking performance of CNNs for landmark localization and at the same time are lightweight, compact and suitable for applications with limited computational resources. To this end, we make the following contributions: (a) we are the first to study the effect of…

New model CRS combines transparency and high performance for classification.

This paper uses cPF to build recommender systems from raw count data.

This work tackles catastrophic forgetting in neural networks by mimicking brain's metaplasticity.

01 loss neural network improves adversarial robustness on CIFAR10.

This paper introduces GLT for better input data representation in BNN and proposes a compact topology with block pruning.

We consider the problem of deep neural net compression by quantization: given a large, reference net, we want to quantize its real-valued weights using a codebook with $K$ entries so that the training loss of the quantized net is minimal. The codebook can be optimally learned jointly with the net, or fixed, as for bina…

A method to reduce knowledge graph embedding models by binarizing parameters.

Binary feedback outperforms ordinal comparisons in ranking recovery.

Pruned neural networks learn digital circuits with 99% weight reduction.

High error rates improve neural network performance and reduce power consumption.

This study connects ReLU neural networks to toric geometry to analyze function realization.

A new method clusters mixed-type data tables effectively.

Prior distributions of binarized natural images are learned by using a Boltzmann machine. According the results of this study, there emerges a structure with two sublattices in the interactions, and the nearest-neighbor and next-nearest-neighbor interactions correspondingly take two discriminative values, which reflect…

The paper proves deep ReLU networks can die and proposes a new initialization method to prevent it.

B-CP reduces knowledge graph model size by replacing real-valued embeddings with binary values.

The paper investigates how target normalization and momentum affect dying ReLUs in neural networks.

Single ReLU neuron's gradient dynamics reveal support vectors as key to generalization.

Large deviation principle for deep neural networks with ReLU activation.

Study shows efficient neural network approach for stochastic bandits.

Binarized Neural Networks (BNNs) have recently attracted significant interest due to their computational efficiency. Concurrently, it has been shown that neural networks may be overly sensitive to "attacks" - tiny adversarial changes in the input - which may be detrimental to their use in safety-critical domains. Desig…

Hard to learn ReLU with Gaussian data, but can approximate efficiently.

The paper analyzes deep ReLU CNNs' approximation properties in 2D space.

Proves existence of optimal shallow neural networks with ReLU activation.

Gradient descent learns ReLU functions with non-zero bias efficiently.

New radar-based method improves multiclass classification of road users, especially in challenging conditions.

Deep ReLU networks can be simplified to a three-layer model.

This article concerns the expressive power of depth in neural nets with ReLU activations and bounded width. We are particularly interested in the following questions: what is the minimal width $w_{\text{min}}(d)$ so that ReLU nets of width $w_{\text{min}}(d)$ (and arbitrary depth) can approximate any continuous functio…