A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Greedy pruning reduces neural networks by a logarithmic number of tickets, improving accuracy.
Greedy selection finds smaller, more accurate subnetworks.
Pruning + fine-tuning reduces model complexity and generalizes well for matrix sensing.
The study finds a theoretical bound for pre-training iterations needed for pruning to yield good subnetwork performance.
A new method quantifies feature-map discriminativeness for efficient pruning of deep neural networks.
We propose a new random pruning method (called "submodular sparsification (SS)") to reduce the cost of submodular maximization. The pruning is applied via a "submodularity graph" over the ground elements, where each directed edge is associated with a pairwise dependency defined by the submodular function. In each s…
We propose a new formulation for pruning convolutional kernels in neural networks to enable efficient inference. We interleave greedy criteria-based pruning with fine-tuning by backpropagation - a computationally efficient procedure that maintains good generalization in the pruned network. We propose a new criterion ba…
A new energy-efficient pruning method for federated learning.
Predicting human fixations from images has recently seen large improvements by leveraging deep representations which were pretrained for object recognition. However, as we show in this paper, these networks are highly overparameterized for the task of fixation prediction. We first present a simple yet principled greedy…
This work presents an approach to automatically induction for non-greedy decision trees constructed from neural network architecture. This construction can be used to transfer weights when growing or pruning a decision tree, allowing non-greedy decision tree algorithms to automatically learn and adapt to the ideal arch…
New method prunes large causal bounds LPs for scalable inference.
Convolutional neural networks (CNNs) achieve state-of-the-art performance in a wide variety of tasks in computer vision. However, interpreting CNNs still remains a challenge. This is mainly due to the large number of parameters in these networks. Here, we investigate the role of compression and particularly pruning fil…
Architecture design has become a crucial component of successful deep learning. Recent progress in automatic neural architecture search (NAS) shows a lot of promise. However, discovered architectures often fail to generalize in the final evaluation. Architectures with a higher validation accuracy during the search phas…
Submodular function maximization finds application in a variety of real-world decision-making problems. However, most existing methods, based on greedy maximization, assume it is computationally feasible to evaluate F, the function being maximized. Unfortunately, in many realistic settings F is too expensive to evaluat…
We propose a greedy mixture reduction algorithm which is capable of pruning mixture components as well as merging them based on the Kullback-Leibler divergence (KLD). The algorithm is distinct from the well-known Runnalls' KLD based method since it is not restricted to merging operations. The capability of pruning (in …
In the context of variable selection, ensemble learning has gained increasing interest due to its great potential to improve selection accuracy and to reduce false discovery rate. A novel ordering-based selective ensemble learning strategy is designed in this paper to obtain smaller but more accurate ensembles. In part…
Bayesian Decision Trees are known for their probabilistic interpretability. However, their construction can sometimes be costly. In this article we present a general Bayesian Decision Tree algorithm applicable to both regression and classification problems. The algorithm does not apply Markov Chain Monte Carlo and does…
PILOT is a fast algorithm for linear model trees that outperforms existing methods.
Markov networks are widely used in many Machine Learning applications including natural language processing, computer vision, and bioinformatics . Learning Markov networks have many complications ranging from intractable computations involved to the possibility of learning a model with a huge number of parameters. In t…
Deep neural networks have been proven powerful at processing perceptual data, such as images and audio. However for tabular data, tree-based models are more popular. A nice property of tree-based models is their natural interpretability. In this work, we present Deep Neural Decision Trees (DNDT) -- tree models realised…
Lookahead pruning extends single-layer optimization to multi-layer, outperforming magnitude-based pruning.
Recent pruning methods at initialization fall short of random pruning's accuracy.
ICE-Pruning accelerates deep neural network pruning by 9.61x.
DSA efficiently allocates sparsity across layers for budgeted pruning.
Boosted decision trees enjoy popularity in a variety of applications; however, for large-scale datasets, the cost of training a decision tree in each round can be prohibitively expensive. Inspired by ideas from the multi-arm bandit literature, we develop a highly efficient algorithm for computing exact greedy-optimal d…
Pruning neural network parameters is often viewed as a means to compress models, but pruning has also been motivated by the desire to prevent overfitting. This motivation is particularly relevant given the perhaps surprising observation that a wide variety of pruning approaches increase test accuracy despite sometimes …
Dynamic pruning during training reduces deep network complexity without significant accuracy loss.
The study reveals flaws in pruning criteria and proposes a new assumption for better filter selection.
Deep neural networks have dramatically achieved great success on a variety of challenging tasks. However, most successful DNNs have an extremely complex structure, leading to extensive research on model compression.As a significant area of progress in model compression, traditional gradual pruning approaches involve an…
Gibbs pruning optimizes neural networks by combining physics and regularization.
We examine how recently documented, fundamental phenomena in deep learning models subject to pruning are affected by changes in the pruning procedure. Specifically, we analyze differences in the connectivity structure and learning dynamics of pruned models found through a set of common iterative pruning techniques, to …
New statistical mechanics analysis shows edge pruning outperforms node pruning in neural networks.
This paper introduces blind adversarial pruning to balance accuracy, efficiency, and robustness in neural networks.
Speeds up training and inference by pruning entire channels before training.
This paper analyzes privacy risks in neural network pruning and proposes a defense mechanism.
Neural network pruning lacks standardized benchmarks and metrics.
Bayesian inference improves neural network pruning efficiency.
AlphaPruning optimizes LLM pruning using HT-SR theory for better performance.
This work characterizes the fundamental limit of network pruning using statistical dimension and convex geometry.
A new framework explains why early pruning works well.
CupNet prunes neural nets for cup-shaped data.
This work explores the importance of model weights and Hessian bias in pruning.
A method to combine saliency metrics for better CNN pruning decisions.
Pruning improves model generalization in over-parameterized models, contradicting traditional theories.
Hyperflux models pruning as a system to reveal weight importance.
This paper studies a theoretical pruning method for RNNs to reduce computational costs.
Neural network for subgraph similarity computation with pruning.
Network pruning is a promising avenue for compressing deep neural networks. A typical approach to pruning starts by training a model and then removing redundant parameters while minimizing the impact on what is learned. Alternatively, a recent approach shows that pruning can be done at initialization prior to training,…