Majority voting neural networks improve binary compressed sensing for sparse signal recovery.
problem Sparse signal recovery in binary compressed sensing.
method Majority voting neural networks with a cross entropy-like term and L1 regularization.
result The majority voting neural network achieves excellent recovery performance, approaching optimal performance as the number of component nets grows.
A new method speeds up learning sparse binary networks.
problem Learning sparse binary pairwise Markov networks efficiently.
method Formulated as sparse multiple logistic regression, uses coordinate descent with strong screening rules.
result Substantial speedup with no loss of accuracy, more stable on unbalanced data.
Binary autoencoder with sparse hidden layer preserves information and zero reconstruction error.
problem Preserving information and zero reconstruction error in binary neural networks.
method Binary autoencoder with random binary weights, sparse hidden layer, and varying neuron thresholds.
result Zero reconstruction error for any input with a large hidden layer and varying neuron thresholds.
New pruning technique reduces index size for DNNs.
problem Irregular index form in fine-grained pruning limits parallelism and memory usage.
method Proposes a low-rank binary index matrix for efficient compression and decompression.
result Fine-grained pruning with binary matrices achieves lower memory footprint and higher parallelism.
This study uses neural networks to solve interpolation problems with sparse, infinitely wide layers.
problem Exact data interpolation using sparse, infinitely wide neural networks.
method Atomic norm framework to derive convex hulls and equivalent convex formulations.
result Simple characterizations of convex hulls for different constraints on network weights and biases.
Sparse random networks reduce communication in federated learning.
problem Large communication cost in federated learning.
method Freeze random weights, train stochastic binary mask to sparsify.
result Improves accuracy, reduces communication, speeds convergence.
Bayesian sparsification reduces memory usage in RNNs without significant loss of accuracy.
problem High memory usage in RNNs for text analysis tasks.
method Apply Sparse Variational Dropout and Binary Variational Dropout to sparsify RNNs.
result Achieve 99.5% sparsity on sentiment analysis with no quality drop and up to 87% on language modeling with slight loss of accuracy.
Sparse-penalized deep neural networks improve performance in weakly dependent processes.
problem Nonparametric regression and classification under weak dependence.
method Sparse-penalized deep neural networks with oracle inequalities and convergence rates established.
result The proposed estimators outperform non-penalized ones in simulations.
Active learning improves EDFA model accuracy with binary features.
problem Lack of labeled training data for EDFA devices.
method Active learning strategy for binary features using sparse linear models.
result Improved prediction and accelerated query generation.
The study develops a supervised and unsupervised WTA model for sparse binary projections.
problem Sparse binary projections in high-dimensional spaces.
method Supervised and unsupervised WTA models with efficient algorithms.
result Significantly improved results in similarity search tasks.
Single neurons can perform as well as dense networks in binary and multi-class recognition tasks.
problem Designing efficient neural networks for recognition tasks.
method Investigated the use of single or multiple neurons in neural networks for binary and multi-class recognition tasks.
result Sparse networks can be as efficient as dense networks in both binary and multi-class tasks.
Sparse binary compression reduces communication costs in distributed deep learning.
problem Limited communication bandwidth in distributed deep learning.
method Combines gradient sparsification, binarization, and optimal weight update encoding.
result Reduces upstream communication by more than four orders of magnitude.
BEGIN network models binary data without parametric assumptions.
problem Conditional independence in non-parametric families of binary data.
method BEGIN network models binary data using sparse linear representations and block factorizations.
result BEGIN network captures conditional independence for arbitrary binary and multinomial variables.
SparseMix clusters sparse high dimensional binary data efficiently.
problem Clustering sparse high dimensional binary data.
method SparseMix is a mixture model designed for sparse data, using an on-line Hartigan optimization algorithm.
result SparseMix builds partitions with higher compatibility with reference grouping than related methods.
A new hashing method improves accuracy by learning an orthogonal transform.
problem Improving hashing accuracy for similarity search.
method Procrustean Orthogonal Sparse Hashing (POSH) method.
result POSH outperforms state-of-the-art hashing methods.
Fruit fly brain network learns word embeddings using sparse binary codes.
problem Learning semantic word representations from text.
method Inspired by mushroom body neural network, sparse binary hash codes.
result Fruit fly network achieves comparable NLP performance with reduced resources.
We present a scalable Bayesian model for low-rank factorization of massive tensors with binary observations. The proposed model has the following key properties: (1) in contrast to the models based on the logistic or probit likelihood, using a zero-truncated Poisson likelihood for binary data allows our model to scale …
MCLNN improves sound classification with fewer parameters.
problem Improving sound classification accuracy with fewer parameters.
method MCLNN uses a binary mask to induce sparseness in frequency bands, automating feature exploration.
result MCLNN achieves competitive results on Urbansound8k with 12% fewer parameters.
Efficiently learns quantizable embeddings for fast search.
problem Learning binary hamming code representations for search efficiency.
method Directly learns a quantizable embedding representation and sparse binary hash code end-to-end.
result Achieves state-of-the-art search accuracy and significant speedup.
Quantum algorithm improves sparse vector recovery from noisy measurements.
problem Accurately recover sparse vectors from noisy linear measurements.
method Formulated as a QUBO task, solved using quantum technology.
result Quantum approach outperforms classical methods in sparse coding.
Unified approach tackles logical constraints in mixed-integer optimization.
problem Logical constraints in mixed-integer optimization problems.
method Express logical constraints non-linearly, reformulate as convex binary optimization, solve using outer-approximation.
result Solves problems faster and at larger scale than existing methods.
A new 2.5D U-net for 3D segmentation reduces memory constraints.
problem Large storage requirements for 3D convolutions in neural networks.
method Transform volumetric data into sequences of 2D images, apply 2D convolutions, and reconstruct.
result Outperforms existing methods in volumetric segmentation tasks.
New algorithm learns sparse GLMs for binary outcomes efficiently.
problem Sparse modeling of binary outcomes in high-dimensional data.
method Iterative hard thresholding algorithm (BIHT) for sparse GLMs.
result BIHT achieves statistical optimality for logistic regression.
A new deep learning model for tabular data improves accuracy over GBDT.
problem Improving accuracy in tabular data classification.
method Differentiable forest with sparse attention mechanism.
result The differentiable forest achieves higher accuracy than GBDT on tabular datasets.
Deep networks learn sparse hierarchical features without CoD.
problem Overparameterized deep networks struggle with the curse of dimensionality.
method Norm-constrained neural networks for sparse compositional functions.
result Deep networks can learn sparse hierarchical features efficiently.
Sparse neural networks trained on commodity hardware achieve high accuracy for high-dimensional data.
problem Training and deploying modern ANNs on commodity hardware due to model size and data volume challenges.
method Introducing a technique to train truly sparse neural networks with fixed parameter count throughout training.
result Achieved higher accuracy than traditional methods and trained models with over 1 million neurons on a typical laptop.
New spectral clustering method improves community detection in sparse networks.
problem Community detection in sparse networks using spectral clustering.
method Data-driven regularization and novel spectral truncation for adjacency matrix.
result Consistency results for community detection in general SBM and beyond.
Graph matching with feature vectors is solved using a two-layer graph neural network.
problem Graph matching in the presence of sparse binary features.
method Two-layer graph neural network with graph structure.
result Graph neural network can recover correct mapping with high probability under certain conditions.
EC2T creates sparse and ternary neural networks for resource-constrained devices.
problem Deploying deep neural networks on resource-constrained devices.
method Entropy-Constrained Trained Ternarization (EC2T) framework.
result EC2T creates sparse and ternary neural networks that are efficient in terms of storage and computation.
New binary matrices improve compressed sensing with faster and less storage requirements.
problem Achieving robust sparse recovery with binary measurement matrices.
method Derived bounds and conditions for binary matrices to satisfy the robust null space property (RNSP).
result Binary matrices with girth six are nearly optimal for compressed sensing.
Proposes a method to learn sparse and low-rank interactions in Ising models with latent variables.
problem Learning sparse interactions in Ising models with latent variables.
method Sparse + low-rank decomposition of Ising model parameters using convex regularized likelihood problem.
result Consistency properties in high-dimensional settings with growing number of variables and samples.
Theory explains why DARTS favors deep architectures over shallow ones.
problem DARTS selects architectures with dominated skip connections, leading to performance degradation.
method Theoretical analysis of operations' effects on network optimization; introduces sparse binary gates and path-depth-wise regularization.
result Theoretical proof that architectures with more skip connections converge faster.
A new framework for sparse regression models with slow variations.
problem Parameter estimation for sparse regression models with slow variations.
method Formulated as a mixed-integer optimization problem, then reformulated as a binary convex optimization problem with a novel relaxation technique.
result Efficiently solves the problem to provable optimality using a cutting plane-type algorithm.
Unified framework for ternary neural networks reduces memory and computation.
problem Training deep neural networks with limited precision and memory.
method Discretization of activations and weights, derivative approximation, and state transition constraints.
result Ternary networks can be reduced to sparse binary networks, termed GXNOR-Nets.
Efficient methods for sparse random projections improve classification accuracy in very high-dimensional data.
problem Handling very high-dimensional sparse data efficiently.
method Non-iterative and iterative classification methods using sparse random projections and Jaccard kernel.
result Non-iterative methods yield larger, more accurate models than iterative methods.
Estimates binary labels from dependent data using Markov Random Fields.
problem Statistical estimation from dependent data across spatial, temporal, and social domains.
method Modeling dependencies as Markov Random Fields and providing efficient estimation algorithms.
result Statistically efficient estimation rates for Ising models from a single sample.
Efficiently poisons offline RLHF models by flipping preference labels.
problem Vulnerability of offline RLHF models to preference label flipping attacks.
method Developed two attack methods: BAL-A and BMP-A, solving a structured binary sparse approximation problem.
result Demonstrated that flipping one preference label induces a parameter-independent shift in the DPO gradient, enabling structured binary sparse approximation.
A fast binary embedding method preserves Euclidean distances in high-dimensional data.
problem Preserving Euclidean distances in high-dimensional datasets.
method Stable noise-shaping quantization of Ax with A a sparse Gaussian random matrix, followed by a linear transformation. result Euclidean distances are approximated by the ℓ1 norm on binary sequences, leading to accurate binary codes. New exact tests detect changepoints in binary and count data, especially when normal approximations fail.
problem Detecting changepoints in multichannel binary and count data.
method Exact tests combining two-sample conditional tests with multiplicity correction.
result Exact tests are much more powerful than asymptotic tests in various settings.
New method for robustly recovering sparse signals from noisy data.
problem Recovering sparse signals from corrupted measurements with outliers.
method Sparse Bayesian learning with binary indicator hyperparameters and hierarchical priors.
result The method achieves better performance than existing techniques.
A method using competitive experience replay enhances learning from sparse rewards.
problem Learning from sparse rewards in reinforcement learning.
method Competitive experience replay method that augments sparse rewards through an exploration competition between agents.
result The method leads to faster convergence and improved task performance.
Study 1-bit compressive sensing with generative models, improving recovery accuracy.
problem Accurately recover sparse vectors from binary measurements with generative models.
method Analyzes noiseless and noisy 1-bit measurements with i.i.d.~Gaussian and Lipschitz continuous generative priors, proving sample complexity bounds and stability properties.
result Proves sample complexity bounds and stability properties for 1-bit compressive sensing with generative models.
Proposes a model selection procedure for high-dimensional binary classification using sparse logistic regression.
problem High-dimensional binary classification with sparse logistic regression.
method Penalized maximum likelihood with complexity penalty on model size, Slope estimator for logistic regression.
result Proposed complexity penalty is rate-optimal in the minimax sense.
RGAM builds more accurate models by preferring linear features over non-linear ones.
problem Building accurate models when linearity assumption is poor.
method Multi-stage algorithm guided by the principle of preferring linear features.
result RGAM can fit sparse generalized additive models at scale for various data types.
Novel approximation hierarchy for sparse quadratic programs.
problem Sparse Quadratic Programs with Cardinality Constraints.
method Exploits rank-dominating eigenvectors for min-max optimization over binary variables.
result Efficient screening of nonzero elements with scalable optimization algorithms.
New method learns binary decision trees efficiently.
problem Learning binary decision trees for data partitioning.
method Argmin differentiation for discrete and continuous parameters.
result Produces competitive binary trees with fast training.
Study proves consistency of spectral clustering on hierarchical networks.
problem Consistency of spectral clustering on hierarchical stochastic block models.
method Recursive bi-partitioning algorithm based on Fiedler vector of graph Laplacian.
result Strong consistency of the method under various model parameters.
SAPSAM trains CNNs on lung CTs with binary labels, improving CPA detection and localization.
problem Chronic Pulmonary Aspergillosis (CPA) detection and localization on CT scans using binary labels.
method Binary labels, average intensity projections, 2D RGB-like images, hierarchical CNN architectures.
result High classification accuracy, precise localization, predictive power of 2-year survival.