arXiv research

We introduce a novel scheme to train binary convolutional neural networks (CNNs) -- CNNs with weights and activations constrained to {-1,+1} at run-time. It has been known that using binary weights and activations drastically reduce memory size and accesses, and can replace arithmetic operations with more efficient bit…

Reintroduces straight-through estimators for binary neural networks.

A new weighted MCC measure improves classifier performance evaluation.

Based on the misleading expectation that weighted network properties always offer a more complete description than purely topological ones, current economic models of the International Trade Network (ITN) generally aim at explaining local weighted properties, not local binary ones. Here we complement our analysis of th…

Recurrent neural networks (RNNs) have shown excellent performance in processing sequence data. However, they are both complex and memory intensive due to their recursive nature. These limitations make RNNs difficult to embed on mobile devices requiring real-time processes with limited hardware resources. To address the…

DAMVI algorithm improves imbalanced binary classification by adjusting weights of examples and classifiers.

We present an alternative layer to convolution layers in convolutional neural networks (CNNs). Our approach reduces the complexity of convolutions by replacing it with binary decisions. Those binary decisions are used as indexes to conditional distributions where each weight represents a leaf in a decision tree. This m…

The training of stochastic neural network models with binary ($\pm1$) weights and activations via continuous surrogate networks is investigated. We derive new surrogates using a novel derivation based on writing the stochastic neural network as a Markov chain. This derivation also encompasses existing variants of the s…

For the efficient execution of deep convolutional neural networks (CNN) on edge devices, various approaches have been presented which reduce the bit width of the network parameters down to 1 bit. Binarization of the first layer was always excluded, as it leads to a significant error increase. Here, we present the novel…

A new test method improves goodness-of-fit tests for copulas.

New algorithm detects communities in weighted networks, improving on binary ones.

Study links K-stability of certain surfaces to binary forms, proving stability and non-stability conditions.

Proposes a new binary classification model inspired by fluid phase separation.

We consider the $\mathfrak{aff}(n|1)-$module structure on the spaces of differential bilinear operators acting on the superspaces of weighted densities. We classify $\mathfrak{aff}(n|1)-$invariant binary differential operators acting on the spaces of weighted densities. This result allows us to compute the first $\math…

Recent events such as the global financial crisis have renewed the interest in the topic of economic networks. One of the main channels of shock propagation among countries is the International Trade Network (ITN). Two important models for the ITN structure, the classical gravity model of trade (more popular among econ…

New method improves accuracy of quantized neural networks.

This paper optimizes binary linear classifiers by tuning their weight vectors.

We propose a new algorithm for training neural networks with binary activations and multi-level weights, which enables efficient processing-in-memory circuits with embedded nonvolatile memories (eNVM). Binary activations obviate costly DACs and ADCs. Multi-level weights leverage multi-level eNVM cells. Compared to exis…

Low bit-width weights and activations are an effective way of combating the increasing need for both memory and compute power of Deep Neural Networks. In this work, we present a probabilistic training method for Neural Network with both binary weights and activations, called BLRNet. By embracing stochasticity during tr…

Study calibrates high-dimensional binary classifiers using angle between estimator and true weights.

Enhanced Random Forests outperform XGBoost across binary classification datasets.

Over the $(1,n)$-dimensional real superspace, $n>1$, we classify $\mathcal{K}(n)$-invariant binary differential operators acting on the superspaces of weighted densities, where $\mathcal{K}(n)$ is the Lie superalgebra of contact vector fields. This result allows us to compute the first differential cohomology of %the L…

Binary autoencoder with sparse hidden layer preserves information and zero reconstruction error.

Proposes volumization for neural networks to control bias-variance tradeoff.

This paper reformulates $F_β$ for better model performance and interpretation.

New algorithm improves online binary classification with constant time complexity.

Neural networks with binary weights are computation-efficient and hardware-friendly, but their training is challenging because it involves a discrete optimization problem. Surprisingly, ignoring the discrete nature of the problem and using gradient-based methods, such as the Straight-Through Estimator, still works well…

This study uncovers how neural architectures and weights interact in classification tasks.

Recent breakthroughs in computer vision make use of large deep neural networks, utilizing the substantial speedup offered by GPUs. For applications running on limited hardware, however, high precision real-time processing can still be a challenge. One approach to solving this problem is training networks with binary or…

Stochasticity and limited precision of synaptic weights in neural network models are key aspects of both biological and hardware modeling of learning processes. Here we show that a neural network model with stochastic binary weights naturally gives prominence to exponentially rare dense regions of solutions with a numb…

Significant success has been reported recently using deep neural networks for classification. Such large networks can be computationally intensive, even after training is over. Implementing these trained networks in hardware chips with a limited precision of synaptic weights may improve their speed and energy efficienc…

This paper investigates whether the gravity model (GM) can explain the statistical properties of the International Trade Network (ITN). We fit data on international-trade flows with a GM specification using alternative fitting techniques and we employ GM estimates to build a weighted predicted ITN, whose topological pr…

Binary Neural Networks (BNNs) are neural networks which use binary weights and activations instead of the typical 32-bit floating point values. They have reduced model sizes and allow for efficient inference on mobile or embedded devices with limited power and computational resources. However, the binarization of weigh…

Defines diversification as a binary relationship between financial portfolios.

Many applications require recovering a ground truth low-rank matrix from noisy observations of the entries, which in practice is typically formulated as a weighted low-rank approximation problem and solved by non-convex optimization heuristics such as alternating minimization. In this paper, we provide provable recover…

We prove overfitting in minimal and random NNs, tempering the effect.

Paper studies multiclass classifiers from binary classifiers, proving methods and demonstrating advantages.

The mesoscopic organization of complex systems, from financial markets to the brain, is an intermediate between the microscopic dynamics of individual units (stocks or neurons, in the mentioned cases), and the macroscopic dynamics of the system as a whole. The organization is determined by "communities" of units whose …

A new term weighting scheme TF-IDFC-RF outperforms others in sentiment analysis.

We improve deep threshold networks' memorization capacity exponentially.

Quantum computing improves training of binary neural networks.

Sequences of correlated binary patterns can represent many time-series data including text, movies, and biological signals. These patterns may be described by weighted combinations of a few dominant structures that underpin specific interactions among the binary elements. To extract the dominant correlation structures …

Sparse random networks reduce communication in federated learning.

We formulate learning of a binary autoencoder as a biconvex optimization problem which learns from the pairwise correlations between encoded and decoded bits. Among all possible algorithms that use this information, ours finds the autoencoder that reconstructs its inputs with worst-case optimal loss. The optimal decode…

It is common wisdom that no nation is an isolated economic island. All nations participate in the global economy and are linked together through trade and finance. Here we analyze international trade network (ITN), being the network of import-export relationships between countries. We show that in each year over the an…

We introduce a minorization-maximization approach to optimizing common measures of discovery significance in high energy physics. The approach alternates between solving a weighted binary classification problem and updating class weights in a simple, closed-form manner. Moreover, an argument based on convex duality sho…

Optimizes binary regression models with gradient ascent-descent methods.

Paper discusses extending Gini score for tied rankings and case weights.