CupNet prunes neural nets for cup-shaped data.
problem Pruning neural networks for cup-shaped data.
method Used simulated cup drawing data to prune a neural network.
result Pruning effectively reduces network size for cup-shaped data.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “data pruning”
Data pruning algorithms struggle in high compression regimes, as shown by theoretical and empirical studies.
problem Limitations of score-based data pruning algorithms in high compression regimes.
method Theoretical and empirical analysis of score-based data pruning algorithms.
result Score-based data pruning algorithms fail in high compression regimes due to 'No Free Lunch' theorems.
Pruning neural networks adds differential privacy noise, preserving data utility.
problem Achieving differential privacy in neural networks without sacrificing data utility.
method Proving equivalence between pruning and adding differential privacy noise to hidden-layer activations.
result Pruning can be a more effective alternative to adding differential privacy noise for neural networks.
Randomly initialized networks can perform as well as pruned networks.
problem Understanding and improving network pruning methods.
method Sanity checks on recent pruning methods, proposing random tickets.
result Randomly initialized networks can perform as well as pruned networks.
New algorithm finds important synapses without training data.
problem Finding important synapses in neural networks without data.
method Iterative Synaptic Flow Pruning (SynFlow) based on conservation law.
result Algorithm consistently outperforms existing pruning methods.
Recent advances in the sparse neural network literature have made it possible to prune many large feed forward and convolutional networks with only a small quantity of data. Yet, these same techniques often falter when applied to the problem of recovering sparse recurrent networks. These failures are quantitative: when…
Speeds up training and inference by pruning entire channels before training.
problem Training and inference speed in deep neural networks.
method Structured pruning applied before training, focusing on removing entire channels and hidden units.
result 2x speedup in training and 3x speedup in inference.
Optimizes pruning masks for neural networks using probabilistic fine-tuning and PAC-Bayes bounds.
problem Improving neural network performance through adaptive pruning of weights.
method Optimizes stochastic pruning masks by minimizing expected loss, considering data-adaptive regularization and feature alignment.
result Probabilistic fine-tuning leads to improved test error over baseline methods in neural networks.
A framework for privacy-preserving DNN pruning and acceleration.
problem Privacy concerns in DNN weight pruning for mobile devices.
method ADMM-based iterative pruning with synthetic data, compiler optimizations.
result 4.2X, 2.5X, and 2.0X speedup with almost no accuracy loss.
Pruning method removes less important features in linear models.
problem Removing less important features in linear models trained by gradient flow.
method Iterative Magnitude Pruning (IMP) applied to linear models trained by gradient flow.
result IMP prunes features with smallest projection onto the data.
Pruning is an efficient model compression technique to remove redundancy in the connectivity of deep neural networks (DNNs). Computations using sparse matrices obtained by pruning parameters, however, exhibit vastly different parallelism depending on the index representation scheme. As a result, fine-grained pruning ha…
Improved pruning method finds winning neural network subnetworks.
problem Finding a small subnetwork that performs as well as a full neural network.
method Data-dependent pruning criterion using gradient of training loss.
result Data-dependent pruning improves existing pruning algorithms.
New pruning method breaks power law scaling, potentially reducing error to exponential.
problem Improving neural network performance through scaling alone is costly.
method Developed a new data pruning metric to break power law scaling.
result Pruned datasets show better than power law scaling on various image datasets.
Network pruning is a promising avenue for compressing deep neural networks. A typical approach to pruning starts by training a model and then removing redundant parameters while minimizing the impact on what is learned. Alternatively, a recent approach shows that pruning can be done at initialization prior to training,…
Pruned neural networks' error scales predictably with architecture and task.
problem Understanding the predictability of pruning across different scales and architectures.
method Functionally approximated the error of pruned networks, showing it is predictable in terms of invariant tying width, depth, and pruning level.
result The error of pruned networks follows a scaling law with interpretable coefficients that depend on architecture and task.
This paper introduces blind adversarial pruning to balance accuracy, efficiency, and robustness in neural networks.
problem Balancing accuracy, efficiency, and robustness in neural networks with limited resources.
method Adversarial pruning with a cutoff-scale strategy to dynamically adjust the strength of adversarial examples.
result Blind adversarial pruning improves the overall AER of pruned models compared to adversarial pruning.
A method to combine saliency metrics for better CNN pruning decisions.
problem Improving CNN pruning decisions by combining multiple saliency metrics.
method Proposes a method to compose different saliency metrics for better CNN pruning decisions.
result The composition of saliencies avoids many poor pruning choices identified by individual saliencies.
Artificial neural networks (ANNs) may not be worth their computational/memory costs when used in mobile phones or embedded devices. Parameter-pruning algorithms combat these costs, with some algorithms capable of removing over 90% of an ANN's weights without harming the ANN's performance. Removing weights from an ANN i…
This paper analyzes privacy risks in neural network pruning and proposes a defense mechanism.
problem Privacy risks in neural network pruning due to membership inference attacks.
method Investigates the impact of pruning on prediction divergence and proposes a self-attention membership inference attack.
result Proposed defense mechanism mitigates privacy risks while maintaining sparsity and accuracy.
PruneNet efficiently prunes channels in deep networks, improving accuracy and performance.
problem Improving deep neural network performance and efficiency through channel pruning.
method PruneNet uses a computationally light-weight optimization step to identify and prune channels based on layer redundancy.
result Pruned ResNet models achieve higher accuracy and better performance than non-pruned models.
Gradient-based training and pruning for radial basis function networks in materials physics.
problem Interpretable and robust machine learning for materials physics problems.
method Gradient-based training and pruning of radial basis function networks with closed-form optimization criteria.
result Pruned models provide compact and interpretable versions of larger models, offering insights into atom-level migration processes.
Alpha-trimming prunes trees in random forests to improve predictive performance.
problem Improving predictive performance of random forests by locally adaptive tree pruning.
method Alpha-trimming is a fast pruning algorithm that prunes trees in a random forest based on signal-to-noise ratio, controlled by a tuning parameter.
result Alpha-trimming often lowers mean squared prediction error compared to fully grown random forests.
The most common method for DNN pruning is hard thresholding of network weights, followed by retraining to recover any lost accuracy. Recently developed smart pruning algorithms use the DNN response over the training set for a variety of cost functions to determine redundant network weights, leading to less accuracy deg…
New method prunes classification trees for biased data.
problem Pruning classification trees in imbalanced training data.
method Optimal pruning procedure for inhomogeneous data.
result First efficient procedure for optimal pruning under covariate shift.
Safe Pattern Pruning reduces pattern explosion in predictive pattern mining.
problem Exponential growth of patterns in structured data.
method Safe Pattern Pruning (SPP) method.
result Effective model building in practical data analysis.
Recent pruning methods at initialization fall short of random pruning's accuracy.
problem Improving neural network accuracy through pruning at initialization.
method Various pruning methods (SNIP, GraSP, SynFlow, magnitude pruning) are evaluated; per-layer pruning decisions are proposed.
result Randomly shuffling or sampling initial weights preserves or improves accuracy, suggesting challenges with pruning heuristics.
Pruning at initialization fails to find sparse subnetworks, revealing information-theoretic barriers.
problem Difficulty in finding sparse subnetworks without training the full model.
method Analysis of effective parameter count and mutual information between sparsity mask and data.
result Pruning at initialization cannot find sparse subnetworks due to high mutual information.
Dataset pruning is the process of removing sub-optimal tuples from a dataset to improve the learning of a machine learning model. In this paper, we compared the performance of different algorithms, first on an unpruned dataset and then on an iteratively pruned dataset. The goal was to understand whether an algorithm (s…
ICE-Pruning accelerates deep neural network pruning by 9.61x.
problem Efficiently pruning deep neural networks while maintaining accuracy.
method Iterative pruning with automatic fine-tuning steps, freezing strategy, and custom learning rate scheduler.
result Significantly reduces pruning time by up to 9.61x.
Proposes clustering and pruning to simplify causal data fusion models.
problem Combining observational and experimental data to identify causal effects.
method Generalizes pruning and clustering operations for multiple data sources.
result Derives conditions for inferring causal effects from simplified models.
DSA efficiently allocates sparsity across layers for budgeted pruning.
problem Efficiently distributing resources (sparsity) across layers in pruning under resource constraints.
method DSA uses differentiable pruning to find continuous layer-wise pruning ratios via gradient-based optimization.
result DSA achieves superior performance and significantly reduces the time cost of pruning.
Gradient-based method prunes large models to create transferable architectures.
problem Creating transferable architectures from large models with limited fine-tuning data.
method Gradient-based algorithm for architecture pruning and subset selection.
result Successfully retrain architectures on new tasks with few fine-tuning data.
A new method prunes neural network channels based on operation characteristics.
problem Compressing deep neural networks efficiently and maintaining accuracy.
method Differentiable masks for channel pruning considering BN and ReLU.
result Outstanding performance in accuracy with less resources compared to state-of-the-art methods.
Geometric pruning rules improve change point detection in multiple time series.
problem Detecting multiple changes in multiple independent time series.
method Dynamic programming algorithms with inequality-based and geometric pruning rules.
result Geometric pruning rules offer close-to-linear time complexity for multiple independent time series.
Magnitude-based pruning is one of the simplest methods for pruning neural networks. Despite its simplicity, magnitude-based pruning and its variants demonstrated remarkable performances for pruning modern architectures. Based on the observation that magnitude-based pruning indeed minimizes the Frobenius distortion of a…
New compression theory justifies model pruning for neural networks.
problem Improving neural network performance with reduced model size.
method Information-theoretic rate-distortion theory applied to NN compression.
result Pruning improves model performance on CIFAR-10 and ImageNet datasets.
This paper finds a new way to compress CNN weights, improving on pruning and quantization.
problem Improving performance and storage efficiency of CNNs.
method Identifying and exploiting repeated patterns in CNN weight tensors, using Huffman coding and block sparse matrix formats.
result Achieved compaction ratios of 1.4x to 3.1x in addition to pruning and quantization.
Pruning neural network parameters is often viewed as a means to compress models, but pruning has also been motivated by the desire to prevent overfitting. This motivation is particularly relevant given the perhaps surprising observation that a wide variety of pruning approaches increase test accuracy despite sometimes …
New pruning method for sparse additive models speeds up causal structure learning.
problem Efficiently prune spurious edges from fully-connected DAG induced by estimated topological order.
method Sparse additive models combined with randomized tree embedding and group-wise sparse regression.
result Significantly faster than existing pruning methods while maintaining comparable accuracy.
Gradual pruning reduces inference cost by pruning least important channels during training.
problem Reduction of deep neural network inference cost.
method Gradual channel pruning using feature relevance scores during training.
result Achieved significant model compression with minimal accuracy loss.
Dynamic pruning during training reduces deep network complexity without significant accuracy loss.
problem High memory and computational requirements of deep networks during training and inference.
method Dynamic pruning of convolutional filters during training, using L1 normalization for optimization.
result L1 normalization-based pruning yields up to 50% reduction in filters with minimal accuracy loss.
The study reveals flaws in pruning criteria and proposes a new assumption for better filter selection.
problem Flaws in existing pruning criteria for CNNs.
method Empirical experiments and Convolutional Weight Distribution Assumption.
result The Convolutional Weight Distribution Assumption improves filter selection in pruning.
Optimizes neural networks for solving problems with pruning and ensembles of minimal structures.
problem Improving neural network performance and interpretability.
method Pruning neural networks based on the principle of controlling training and pruning, using sensitivity indicators and logically transparent NN.
result Ensemble of minimal neural networks provides diverse forecasting algorithms and identifies areas for further data collection.
Dynamic sample pruning speeds up spatio-temporal forecasting models.
problem Training deep learning models on large, redundant datasets is computationally expensive.
method Dynamic sample pruning based on real-time learning state.
result Significant acceleration of training speed with improved performance.
Deep neural networks have dramatically achieved great success on a variety of challenging tasks. However, most successful DNNs have an extremely complex structure, leading to extensive research on model compression.As a significant area of progress in model compression, traditional gradual pruning approaches involve an…
Gibbs pruning optimizes neural networks by combining physics and regularization.
problem Large neural networks are impractical for many applications.
method Combines statistical physics and stochastic regularization to train and prune networks simultaneously.
result Gibbs pruning achieves state-of-the-art performance on ResNet-56.
We examine how recently documented, fundamental phenomena in deep learning models subject to pruning are affected by changes in the pruning procedure. Specifically, we analyze differences in the connectivity structure and learning dynamics of pruned models found through a set of common iterative pruning techniques, to …
Federated learning (FL) allows model training from local data collected by edge/mobile devices while preserving data privacy, which has wide applicability to image and vision applications. A challenge is that client devices in FL usually have much more limited computation and communication resources compared to servers…