A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New flaw found in SAP defense, reducing its effectiveness to 0.1%.
Neural networks are known to be vulnerable to adversarial examples. Carefully chosen perturbations to real images, while imperceptible to humans, induce misclassification and threaten the reliability of deep learning systems in the wild. To guard against adversarial examples, we take inspiration from game theory and ca…
A new method prunes neural networks efficiently without losing effectiveness.
Deep neural networks have dramatically achieved great success on a variety of challenging tasks. However, most successful DNNs have an extremely complex structure, leading to extensive research on model compression.As a significant area of progress in model compression, traditional gradual pruning approaches involve an…
Pruning neural networks adds differential privacy noise, preserving data utility.
Optimizes pruning masks for neural networks using probabilistic fine-tuning and PAC-Bayes bounds.
Real-time pruning during training reduces network size and training time.
With the rapid scaling up of deep neural networks (DNNs), extensive research studies on network model compression such as weight pruning have been performed for improving deployment efficiency. This work aims to advance the compression beyond the weights to neuron activations. We propose the joint regularization techni…
Gibbs pruning optimizes neural networks by combining physics and regularization.
Artificial neural network has achieved unprecedented success in the medical domain. This success depends on the availability of massive and representative datasets. However, data collection is often prevented by privacy concerns and people want to take control over their sensitive information during both training and u…
Sparsification is an efficient approach to accelerate CNN inference, but it is challenging to take advantage of sparsity in training procedure because the involved gradients are dynamically changed. Actually, an important observation shows that most of the activation gradients in back-propagation are very close to zero…
To improve the execution speed and efficiency of neural networks in embedded systems, it is crucial to decrease the model size and computational complexity. In addition to conventional compression techniques, e.g., weight pruning and quantization, removing unimportant activations can reduce the amount of data communica…
Sparse Transformers degrade semantic information first, with early layers encoding more.
A new pruning method reduces neural network computation without retraining.
TENP prunes experts and neurons in Mixture-of-Experts models for efficient deployment.
Recent advances in the sparse neural network literature have made it possible to prune many large feed forward and convolutional networks with only a small quantity of data. Yet, these same techniques often falter when applied to the problem of recovering sparse recurrent networks. These failures are quantitative: when…
This work presents deep asymmetric networks with a set of node-wise variant activation functions. The nodes' sensitivities are affected by activation function selections such that the nodes with smaller indices become increasingly more sensitive. As a result, features learned by the nodes are sorted by the node indices…
We propose a new formulation for pruning convolutional kernels in neural networks to enable efficient inference. We interleave greedy criteria-based pruning with fine-tuning by backpropagation - a computationally efficient procedure that maintains good generalization in the pruned network. We propose a new criterion ba…
Dynamic pruning during training reduces deep network complexity without significant accuracy loss.
Proposes a method to improve surrogate models by incorporating sensitivity information.
i-SpaSP prunes neural networks by identifying important groups of parameters, improving pruning efficiency.
Pruning filters in neural networks improves performance and reduces costs.
Holistic Filter Pruning reduces DNN complexity efficiently.
A new method prunes neural networks faster and more efficiently.
A new pruning method finds sparse minimizers in flat regions of deep neural networks.
This research investigates selectively pruning hyper and hypo neurons to improve neural network generalization.
This work extends neural networks to automatically select features by stochastically penalizing feature involvement.
We show implicit filter level sparsity manifests in convolutional neural networks (CNNs) which employ Batch Normalization and ReLU activation, and are trained with adaptive gradient descent techniques and L2 regularization or weight decay. Through an extensive empirical study (Mehta et al., 2019) we hypothesize the mec…
To address the limitations of existing magnitude-based pruning algorithms in cases where model weights or activations are of large and similar magnitude, we propose a novel perspective to discover parameter redundancy among channels and accelerate deep CNNs via channel pruning. Precisely, we argue that channels reveali…
Expanding neural networks improves their learning from noisy data.
The study examines if ReLU activation function is optimal for modularity in neural networks.
This paper introduces GLT for better input data representation in BNN and proposes a compact topology with block pruning.
Network pruning is aimed at imposing sparsity in a neural network architecture by increasing the portion of zero-valued weights for reducing its size regarding energy-efficiency consideration and increasing evaluation speed. In most of the conducted research efforts, the sparsity is enforced for network pruning without…
EVGAE improves VGAE's latent representation learning by mitigating over-pruning.
Binary Stochastic Filtering (BSF), the algorithm for feature selection and neuron pruning is proposed in this work. The method defines filtering layer which penalizes amount of the information involved in the training process. This information could be the input data or output of the previous layer, which directly lead…
Neural networks are easier to optimise when they have many more weights than are required for modelling the mapping from inputs to outputs. This suggests a two-stage learning procedure that first learns a large net and then prunes away connections or hidden units. But standard training does not necessarily encourage ne…
Bayesian Bits unifies quantization and pruning through gradient optimization.
Pruning CNNs by removing less important filters based on empirical loss changes.
SeReNe prunes neurons with low sensitivity to reduce network size.
New SDP method certifies neural network robustness across all classes efficiently.
Bayesian neural networks improve macroeconomic forecasting and model nonlinearities.
FS&P uses birth-death process to ensure global convergence of stochastic conic particle gradient descent.
We present novel understandings of the Gamma-Poisson (GaP) model, a probabilistic matrix factorization model for count data. We show that GaP can be rewritten free of the score/activation matrix. This gives us new insights about the estimation of the topic/dictionary matrix by maximum marginal likelihood estimation. In…
Neural plasticity is an important functionality of human brain, in which number of neurons and synapses can shrink or expand in response to stimuli throughout the span of life. We model this dynamic learning process as an -norm regularized binary optimization problem, in which each unit of a neural network (e.g., …
Lookahead pruning extends single-layer optimization to multi-layer, outperforming magnitude-based pruning.
Enhanced network threat detection using KG, LLM, and imbalanced learning.
Outlier detection (also known as anomaly detection or deviation detection) is a process of detecting data points in which their patterns deviate significantly from others. It is common to have outliers in industry applications, which could be generated by different causes such as human error, fraudulent activities, or …
Paper tackles active labeling for partial supervision.