Study binary activated deep neural networks using PAC-Bayesian theory.
problem Generalization bounds for binary activated deep neural networks.
method Developed an end-to-end framework and provided PAC-Bayesian generalization bounds.
result Nonvacuous PAC-Bayesian generalization bounds for binary activated deep neural networks.
BinaryDuo improves BNNs by coupling binary activations, outperforming state-of-the-art models.
problem Gradient mismatch in BNNs due to binarizing activations.
method Using gradient of smoothed loss function to estimate gradient mismatch, proposing BinaryDuo scheme with coupled ternary activations.
result BinaryDuo outperforms state-of-the-art BNNs on various benchmarks.
Active learning improves EDFA model accuracy with binary features.
problem Lack of labeled training data for EDFA devices.
method Active learning strategy for binary features using sparse linear models.
result Improved prediction and accelerated query generation.
Paper introduces ABC-Net, a binary CNN that maintains high accuracy with reduced memory and power.
problem Accuracy loss in binary CNNs during inference.
method Approximating full-precision weights with binary bases and using multiple binary activations.
result ABC-Net achieves comparable prediction accuracy to full-precision CNNs, even on challenging datasets.
New method predicts activity coefficients for binary mixtures without using physical descriptors.
problem Predicting activity coefficients for unexplored binary mixtures.
method Probabilistic matrix factorization model.
result Method outperforms state-of-the-art models requiring less training effort.
New algorithm trains binary-activation, multi-level RNNs for noise-resilient, ADC-/DAC-free PIM inference.
problem Training noise-resilient, ADC-/DAC-free neural networks.
method Binary activations and multi-level weights for eNVM-based processing-in-memory circuits.
result Higher accuracy and noise resilience for recurrent networks compared to existing methods.
RBMs model binary interactions with hidden node activation effects.
problem Understanding how RBM hidden node activation affects binary variable distributions.
method Investigated RBM marginal distributions with different hidden node activation functions.
result Found exact expressions for RBM marginals as interacting binary variables.
Paper proposes HTAF for stable training of binary neural networks.
problem Challenges in training binary neural networks with gradient-based optimization.
method HTAF is a smooth approximation to the Heaviside function that enables stable training.
result HTAF enables stable training of various binary neural networks with gradient-based optimization.
Reintroduces straight-through estimators for binary neural networks.
problem Training neural networks with binary weights and activations is challenging due to gradient issues and discrete weight optimization.
method Derives ST methods as estimators in the SBN model, analyzes properties and estimation accuracy, explains latent weights and mirror descent method.
result Reintroduces ST methods as sound approximations and provides clearer application and improvements.
Probabilistic BLRNet uses binary weights and activations for efficient neural networks.
problem Efficiently training and deploying deep neural networks with limited memory and compute.
method Probabilistic training method for binary weights and activations, introducing stochastic operations.
result BLRNet achieves performance comparable to full-precision networks while using fewer bits.
Binary autoencoder with sparse hidden layer preserves information and zero reconstruction error.
problem Preserving information and zero reconstruction error in binary neural networks.
method Binary autoencoder with random binary weights, sparse hidden layer, and varying neuron thresholds.
result Zero reconstruction error for any input with a large hidden layer and varying neuron thresholds.
New algorithms for binary classification with abstention, achieving near-optimal performance.
problem Binary classification with abstention in various sampling models.
method Proposed active learning algorithms for three abstention settings, analyzed in a non-parametric framework.
result Upper-bounds and matching lower-bounds on the excess risk of the algorithms, demonstrating near-optimality.
New approach for active learning in overparameterized models.
problem Efficiently labeling datasets in machine learning.
method MaxiMin Active Learning for nonparametric or overparameterized models.
result Automatically identifies decision boundaries and data clusters.
Model detects patterns in noisy binary data, explaining neuron activity in terms of cell assemblies.
problem Detecting structure in noisy or approximate repeats of patterns in sparse binary data.
method Probabilistic binary latent variable model based on Noisy-OR model, inferring sparse activity in latent variables.
result Model successfully extracts and explains latent structure in spiking neural data.
We consider active maximum a posteriori (MAP) inference problem for Hidden Markov Models (HMM), where, given an initial MAP estimate of the hidden sequence, we select to label certain states in the sequence to improve the estimation accuracy of the remaining states. We develop an analytical approach to this problem for…
New method for estimating gradients in stochastic binary networks.
problem Challenges in training neural networks with binary activations and weights.
method Combines sampling and analytic approximation steps to estimate gradients accurately.
result Significantly reduced variance at the cost of small bias, leading to practical tradeoffs.
Active learning method for binary classification with variable selection.
problem Efficiently label subjects in large datasets for binary classification.
method Model-based active learning with sequential variable selection.
result Proposed procedure reduces training cost/time and improves classification model.
DAL improves active learning for neural networks with large batch sizes.
problem Efficiently choosing examples to label for neural networks with large batch sizes.
method DAL treats active learning as a binary classification task to make labeled and unlabeled sets indistinguishable.
result DAL performs on par with state-of-the-art methods in medium and large query batch sizes.
This paper analyzes the training dynamics of binary neural networks using information bottleneck.
problem Training binary neural networks is challenging due to discontinuity in activation functions.
method The approach uses the Information Bottleneck principle to analyze BNN training dynamics.
result Training dynamics of BNNs are different from DNNs, with both phases occurring simultaneously.
Paper tackles binary feedbacks in contextual search learning.
problem Learning underlying mean value function in context with binary feedbacks.
method Tri-section search combined with margin-based active learning.
result Algorithm achieves O(1/ε2) queries for ε-estimation accuracy. Model neural plasticity as binary optimization to dynamically activate or deactivate network units.
problem Dynamic learning and adaptability of neural networks.
method Model neural plasticity as an L0-norm regularized binary optimization problem, where units can be activated or deactivated based on a cost-benefit tradeoff. result Demonstrates that a single parameter k can modulate learning dynamics, unifying network sparsification and expansion. Modeling and estimating dynamic graphs from binary pattern sequences.
problem Extracting dominant correlation structures from time-dependent binary patterns.
method State-space model of an Ising-type network composed of multiple undirected graphs, sequential Bayes algorithm.
result The method outperforms traditional methods in uncovering overlapping graphs and estimating dynamics of weights.
Improves GAN training by guiding the discriminator to have more diverse binary activation patterns.
problem Stability and convergence issues in GAN training.
method Binarized Representation Entropy (BRE) regularization to guide the discriminator's model capacity allocation.
result Improves GAN training stability and convergence speed, higher sample quality, and higher classification accuracy.
MeliusNet improves binary neural networks to match MobileNet-v1 accuracy.
problem Achieving high accuracy with binary neural networks on mobile devices.
method Alternating DenseBlocks and ImprovementBlocks to increase feature capacity and quality.
result MeliusNet matches MobileNet-v1 accuracy on ImageNet, improving binary network performance.
Dual neural network architecture improves accuracy and interpretability.
problem Improving neural network interpretability and accuracy.
method Stacked recurrent and feedforward layers, binary activation function.
result Binary activation leads to simpler, more interpretable models with higher accuracy.
The problem of active diagnosis arises in several applications such as disease diagnosis, and fault diagnosis in computer networks, where the goal is to rapidly identify the binary states of a set of objects (e.g., faulty or working) by sequentially selecting, and observing, (noisy) responses to binary valued queries. …
Quantized neural networks can represent all fixed-point functions under certain conditions.
problem Expressive power of quantized neural networks under fixed-point arithmetic.
method Analyzing necessary and sufficient conditions for quantized networks to represent all fixed-point functions.
result Various popular activation functions satisfy the sufficient condition for representing all fixed-point functions.
A new active learning method for one-class classification using two classifiers.
problem Reducing manual labeling efforts in one-class classification.
method Uses two one-class classifiers for active learning, proposing new query strategies.
result Improved results compared to existing methods on various datasets.
Deep learning has become a powerful and popular tool for a variety of machine learning tasks. However, it is challenging to understand the mechanism of deep learning from a theoretical perspective. In this work, we propose a random active path model to study collective properties of deep neural networks with binary syn…
Binary encoding enables neural networks to extrapolate periodic functions.
problem Extrapolating periodic functions without prior knowledge of their form.
method Normalized Base-2 Encoding (NB2E) for continuous numerical values.
result MLPs using NB2E can successfully extrapolate diverse periodic signals.
A greedy active learning algorithm for logistic regression reduces model size and training size.
problem Binary classification with reduced model size and training size.
method Modified batch subject selection strategy with greedy variable selection.
result Competitive performance with smaller training size and model size.
Paper tackles privacy in active learning for sensitive data.
problem Privacy leak in active learning with sensitive data.
method Privacy-preserving active learning approach with quantifiable guarantees.
result Tradeoff between privacy, utility, and annotation budget demonstrated.
New binary AA methods improve on existing techniques.
problem Binary data limitations in AA methods.
method Proposed two optimization frameworks for binary AA.
result Superior performance on synthetic and real binary data.
Efficiently classifies binary labels with XOR queries, even under noisy conditions.
problem Binary classification with unknown labels using XOR queries.
method Effective query type and an efficient inference algorithm for noisy conditions.
result Achieves information-theoretic limit on optimal number of queries.
A new activation function improves credit scoring accuracy for imbalanced datasets.
problem Imbalanced datasets in credit scoring lead to underestimation of misclassification costs.
method Introduces ASIG, an asymmetric adjusted Sigmoid function.
result ASIG-embedded classifier outperforms traditional classifiers across various imbalance ratios.
A new multi-class active learning method combining informativeness and representativeness.
problem Efficiently labeling large datasets with limited resources.
method A hybrid informative and representative criterion approach for multi-class active learning.
result The proposed method outperforms state-of-the-art methods on multiple UCI datasets.
Paper explores activity recognition and prediction in real homes using sensor data and video.
problem Improving accuracy of activity recognition and prediction in real home environments.
method Binary sensor data, depth video data, field trial, probabilistic methods, LSTM networks, transfer learning, IIR filter.
result Achieved good accuracy in predicting next sensor event and its mean time of occurrence using LSTM model.
This paper investigates the problem of active learning for binary label prediction on a graph. We introduce a simple and label-efficient algorithm called S2 for this task. At each step, S2 selects the vertex to be labeled based on the structure of the graph and all previously gathered labels. Specifically, S2 queries f…
Safe RL with binary feedback using SABRE algorithm.
problem Safe reinforcement learning with binary safety feedback.
method SABRE algorithm, combining active learning and reinforcement learning.
result Provable safe policy with high probability, no unsafe actions during training.
There is a pressing need to build an architecture that could subsume these networks under a unified framework that achieves both higher performance and less overhead. To this end, two fundamental issues are yet to be addressed. The first one is how to implement the back propagation when neuronal activations are discret…
Paper proposes algorithms for active learning of reject option classifiers.
problem Active learning of reject option classifiers is unaddressed in machine learning.
method Developed novel algorithms using double ramp and double sigmoid loss functions.
result Proposed algorithms efficiently reduce the number of labeled examples required.
This paper addresses the problem of learning a task from demonstration. We adopt the framework of inverse reinforcement learning, where tasks are represented in the form of a reward function. Our contribution is a novel active learning algorithm that enables the learning agent to query the expert for more informative d…
Binary representation is desirable for its memory efficiency, computation speed and robustness. In this paper, we propose adjustable bounded rectifiers to learn binary representations for deep neural networks. While hard constraining representations across layers to be binary makes training unreasonably difficult, we s…
This paper develops convex surrogates for optimizing the multi-label F-measure.
problem Optimizing the F-measure for multi-label classification is computationally hard.
method Designing convex surrogate losses calibrated for the F-measure.
result The F-measure for multi-label problems has a rank of at most s2+1. The paper analyzes features learned by deep neural networks from malware binaries.
problem Costly feature engineering in malware classification.
method Examined byte-level activations and their connection to original features through parsing and disassembly.
result Identified interesting features learned by deep neural networks and their relation to traditional features.
An active learner is given a class of models, a large set of unlabeled examples, and the ability to interactively query labels of a subset of these examples; the goal of the learner is to learn a model in the class that fits the data well. Previous theoretical work has rigorously characterized label complexity of activ…
A new method uses SVMs and active learning for efficient fragility curve estimation.
problem Estimating fragility curves for structures under seismic and other excitations.
method Support Vector Machines (SVMs) coupled with active learning algorithm.
result Efficient estimation of fragility curves with reduced numerical calculations.
Quantized neural networks can improve robustness against adversarial attacks.
problem Adversarial attacks on neural networks with low-precision weights and activations.
method Proposed a third benefit of very low-precision neural networks: improved robustness against some adversarial attacks. Focused on weights and activations quantized to ±1, and conducted black-box and white-box experiments.
result Non-scaled binary neural networks can reduce the impact of iterative attacks, but do not artificially mask gradients.