QubitHD improves HD computing ML efficiency without sacrificing accuracy.
problem Trade-off between energy efficiency and classification accuracy in HD computing-based ML.
method Stochastically binarizes HD-based algorithms while maintaining comparable classification accuracies.
result 65% improvement in energy efficiency and 95% improvement in training time on FPGA.
Binarization of digital documents is the task of classifying each pixel in an image of the document as belonging to the background (parchment/paper) or foreground (text/ink). Historical documents are often subjected to degradations, that make the task challenging. In the current work a deep neural network architecture …
New radar-based method improves multiclass classification of road users, especially in challenging conditions.
problem Accurate classification of multiple road users in challenging scenarios.
method 50 features extracted from radar data, subset chosen, tested on random forest and LSTM classifiers, addressed data imbalance issues.
result Substantial improvements in multiclass classification compared to ordinary methods.
Channel pruning and weight binarization improve keyword spotting accuracy.
problem Improving accuracy of keyword spotting in neural networks.
method Group-wise splitting method using group Lasso penalty for channel sparsity, combined with 1-bit weight precision.
result Achieved over 50% channel sparsity with minimal accuracy loss.
Progressive stochastic binarization improves deep network inference efficiency.
problem Efficient inference of deep networks with reduced memory and computational resources.
method A progressive stochastic binarization scheme for deep networks that uses small integers and fixed shifts.
result Matches the accuracy of previous binarized approaches and reduces inference costs by up to 33%.
TentacleNet improves binarized CNNs, reducing accuracy loss and memory usage.
problem Excessive accuracy loss in binarized CNNs.
method Parallelization inspired by ensemble learning theory, end-to-end trainable compact topology.
result Significant memory savings compared to state-of-the-art binary ensemble methods.
New framework reduces LLM complexity by directly finetuning in Boolean domain.
problem Reducing the complexity of large language models (LLMs) while maintaining performance.
method Proposes a novel framework using multi-kernel Boolean parameters for direct finetuning in the Boolean domain.
result Significantly reduces complexity during both finetuning and inference, outperforming recent techniques.
Develops a binarized GNN for more efficient graph embeddings.
problem Real-valued GNN parameters limit efficiency and scalability.
method Integrates binarization into GNN-based graph embedding approaches.
result Binarized graph neural network (BGN) achieves state-of-the-art performance with significant efficiency gains.
A new adaptive binarization technique using fuzzy integrals improves image quality.
problem Improving image thresholding quality.
method FLAT (Fuzzy Local Adaptive Thresholding) based on fuzzy integrals.
result The proposed FLAT method produces better image quality than traditional algorithms and neural networks.
This work improves DNN weight quantization with ADMM, achieving lossless binarization and reduced search space.
problem Improving DNN model compression and accuracy with low bit quantization.
method Extending ADMM framework for DNN weight quantization with progressive multi-step approach.
result Achieved lossless and fully binarized DNNs with reduced accuracy loss.
Binary representation is desirable for its memory efficiency, computation speed and robustness. In this paper, we propose adjustable bounded rectifiers to learn binary representations for deep neural networks. While hard constraining representations across layers to be binary makes training unreasonably difficult, we s…
This work tackles large action spaces in RL by binarizing actions.
problem Large action spaces in reinforcement learning cause significant challenges.
method Sequentializing actions and binarizing the action space.
result Binarizing the action space can significantly improve RL algorithms and reduce state space size.
FPGAs accelerate DNNs with binarized weights, improving power and speed.
problem Power and speed limitations of GPU-accelerated DNNs.
method Binarized neural networks on FPGAs using OpenCL.
result Near state-of-the-art performance with >16x power savings.
New framework for evaluating multiclass classifier calibration.
problem Ensuring classifiers are well-calibrated for trustworthy predictions.
method Utility Calibration framework that measures calibration error relative to a utility function.
result Unified and robust interpretation of existing calibration metrics.
Radar-based classification improves accuracy of autonomous driving by identifying new classes.
problem Challenges in classifying sparse radar data for autonomous driving.
method Ensemble of classifiers using one-vs-one and one-vs-all strategies, with feature selection.
result Improved classification performance and identification of novel classes.
New insights into BNN optimization redefine latent weights as inertia.
problem Optimizing Binarized Neural Networks (BNNs) with latent weights.
method Interpreted latent weights as inertia and introduced Binary Optimizer (Bop).
result Demonstrated improved performance of Bop on CIFAR-10 and ImageNet.
New method for ordinal data improves recommendation systems.
problem Improving recommendation systems with ordinal data.
method Ordinal Non-negative Matrix Factorization (OrdNMF) for ordinal data.
result OrdNMF outperforms existing methods in recommendation experiments.
Convolutional neural networks have recently achieved significant breakthroughs in various image classification tasks. However, they are computationally expensive,which can make their feasible mplementation on embedded and low-power devices difficult. In this paper convolutional neural network binarization is implemente…
Understanding properties of deep neural networks is an important challenge in deep learning. In this paper, we take a step in this direction by proposing a rigorous way of verifying properties of a popular class of neural networks, Binarized Neural Networks, using the well-developed means of Boolean satisfiability. Our…
Paper introduces impact curves for evaluating binarized regression models with varying costs.
problem Evaluating binarized regression models with varying costs and instance-specific utility.
method Proposes impact curves to optimize binary decisions across different utilities.
result Impact curves identify conditions where one model is favored over another and quantify model improvement.
User and item features of side information are crucial for accurate recommendation. However, the large number of feature dimensions, e.g., usually larger than 10^7, results in expensive storage and computational cost. This prohibits fast recommendation especially on mobile applications where the computational resource …
TDA classifies MNIST digits with reduced feature set.
problem Classifying MNIST digits using machine learning.
method Persistent homology for feature generation and classification.
result 5x reduction in feature set size with similar accuracy.
A method for efficient approximate inference on discrete distributions.
problem Applying SVGD to discrete distributions.
method Transforming discrete distributions to piecewise continuous distributions for SVGD application.
result Outperforms traditional algorithms and ensemble methods on discrete graphical models.
Our goal is to design architectures that retain the groundbreaking performance of CNNs for landmark localization and at the same time are lightweight, compact and suitable for applications with limited computational resources. To this end, we make the following contributions: (a) we are the first to study the effect of…
New model CRS combines transparency and high performance for classification.
problem Need models with transparent structure and high classification performance.
method Concept Rule Sets (CRS) with Multilayer Logical Perceptron (MLLP) and Random Binarization (RB).
result CRS outperforms state-of-the-art approaches and has low complexity.
This paper uses cPF to build recommender systems from raw count data.
problem Sparse, over-dispersed and bursty count data make direct use in recommender systems challenging.
method Compound Poisson Factorization (cPF) with a unified framework (dcPF) and adaptive algorithm.
result dcPF achieves better recommendation scores than Poisson Factorization on raw or binarized data.
This work tackles catastrophic forgetting in neural networks by mimicking brain's metaplasticity.
problem Catastrophic forgetting in neural networks, where new tasks erase previously learned ones.
method Interpreting binarized neural networks as metaplastic systems, adjusting their training technique.
result Training technique reduces catastrophic forgetting without needing previously presented data.
01 loss neural network improves adversarial robustness on CIFAR10.
problem Improving machine learning models' resistance to adversarial attacks.
method Proposes a single hidden layer 01 loss neural network trained with stochastic coordinate descent.
result 01 loss network has the largest adversarial distortion compared to other models.
This paper introduces GLT for better input data representation in BNN and proposes a compact topology with block pruning.
problem Improving input data representation for Binary Neural Networks (BNN).
method Generic Learned Thermometer (GLT) for encoding, block pruning and Knowledge Distillation for compact topology.
result Significant accuracy gains and lightweight fully-binarized models with limited accuracy degradation.
We consider the problem of deep neural net compression by quantization: given a large, reference net, we want to quantize its real-valued weights using a codebook with K entries so that the training loss of the quantized net is minimal. The codebook can be optimally learned jointly with the net, or fixed, as for bina…
A method to reduce knowledge graph embedding models by binarizing parameters.
problem Large memory requirements for tensor factorization models in knowledge graph completion.
method Introducing a quantization function to binarize parameters of CP tensor decomposition.
result Successfully reduced model size by more than an order of magnitude while maintaining task performance.
Binary feedback outperforms ordinal comparisons in ranking recovery.
problem Challenges the conventional wisdom that ordinal comparisons offer richer information.
method Proposes a parametric framework for modeling ordinal paired comparisons, binarizing ordinal data, and proving faster convergence rates for binary comparisons.
result Binarizing ordinal data significantly improves ranking recovery accuracy and exhibits a substantial performance gap.
Study on function sensitivity in random DNNs using large deviation theory.
problem Understanding function sensitivity in finite-size deep neural networks.
method Large deviation theory and path integral analysis applied to random DNNs with ReLU and sign activations.
result Random DNNs with ReLU activations are more robust to parameter perturbations.
Pruned neural networks learn digital circuits with 99% weight reduction.
problem Efficiently train deep neural networks with minimal weights.
method Constrained binarized networks to zero or one weights.
result Pruned networks achieve similar performance to standard networks with 99% weight reduction.
High error rates improve neural network performance and reduce power consumption.
problem Training neural networks with high Bit Error Rates (BERs).
method Trained three Binarized Convolutional Neural Network architectures on various datasets with high BERs.
result High BERs do not significantly degrade test accuracy, enabling more efficient hardware.
A new method clusters mixed-type data tables effectively.
problem Clustering data with mixed types (numerical and categorical).
method Two-step approach: binarize mixed data, then co-cluster.
result Shows improved clustering of mixed-type data compared to MCA.
Prior distributions of binarized natural images are learned by using a Boltzmann machine. According the results of this study, there emerges a structure with two sublattices in the interactions, and the nearest-neighbor and next-nearest-neighbor interactions correspondingly take two discriminative values, which reflect…
B-CP reduces knowledge graph model size by replacing real-valued embeddings with binary values.
problem Storage inefficiency in vector embeddings for large knowledge graphs.
method Binarized CANDECOMP/PARAFAC (B-CP) decomposition algorithm.
result B-CP reduces model size by more than an order of magnitude while maintaining task performance.
Binarized Neural Networks (BNNs) have recently attracted significant interest due to their computational efficiency. Concurrently, it has been shown that neural networks may be overly sensitive to "attacks" - tiny adversarial changes in the input - which may be detrimental to their use in safety-critical domains. Desig…
A new CI test avoids information loss in discretized data.
problem Incorrect CI conclusions from discretized data.
method Proposes a sample-efficient CI test using GMM and nodewise regression.
result Derives an accurate test statistic and establishes its asymptotic distribution.
In this work, we propose an end-to-end block-based auto-encoder system for image compression. We introduce novel contributions to neural-network based image compression, mainly in achieving binarization simulation, variable bit rates with multiple networks, entropy-friendly representations, inference-stage code optimiz…
This work enhances deep neural networks with robust features to improve their robustness against adversarial attacks.
problem Vulnerability of deep neural networks to adversarial images.
method Augmenting classification pipelines with robust features like binarization and group extraction.
result Improved robustness and training speed on adversarial inputs, with significant improvements over state-of-the-art methods.
New memory allocation scheme improves image generation performance.
problem Improving episodic and semantic memory representation in neural networks.
method Developed a hierarchical latent variable model with differentiable, locally block allocated latent memory.
result Improved conditional likelihood values on various datasets.
This master's thesis discusses an important issue regarding how algorithmic decision making (ADM) is used in crime forecasting. In America forecasting tools are widely used by judiciary systems for making decisions about risk offenders based on criminal justice for risk offenders. By making use of such tools, the judic…
LUTNet optimizes FPGA for neural network inference, achieving high efficiency.
problem Redundancy in neural networks leads to inefficient hardware implementations.
method Exploits LUTs' flexibility for efficient neural network inference on FPGAs.
result Significant area savings with comparable accuracy compared to state-of-the-art binarized neural networks.
Gaudy images help train deep neural networks with less data.
problem Training deep neural networks with limited real data from visual cortex neurons.
method Used high-contrast binarized natural images (gaudy images) to train DNNs.
result Reduced training data needed for accurate DNN predictions of visual cortex neuron responses.
CNN detects phase transitions in Potts models without prior knowledge.
problem Detecting phase transitions in q-state Potts models using deep learning. method Trained a deep CNN on Ising model spin configurations and temperatures, then tested on Potts model images.
result Deep CNN accurately detects phase transitions in Potts models, including high- and low-temperature regions.
Binarized Neural Network (BNN) removes bitwidth redundancy in classical CNN by using a single bit (-1/+1) for network parameters and intermediate representations, which has greatly reduced the off-chip data transfer and storage overhead. However, a large amount of computation redundancy still exists in BNN inference. B…