The sophisticated structure of Convolutional Neural Network (CNN) allows for outstanding performance, but at the cost of intensive computation. As significant redundancies inevitably present in such a structure, many works have been proposed to prune the convolutional filters for computation cost reduction. Although ex…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “redundancy filtering”
Compressing convolutional neural networks (CNNs) is essential for transferring the success of CNNs to a wide variety of applications to mobile devices. In contrast to directly recognizing subtle weights or filters as redundant in a given CNN, this paper presents an evolutionary method to automatically eliminate redunda…
The redundancy is widely recognized in Convolutional Neural Networks (CNNs), which enables to remove unimportant filters from convolutional layers so as to slim the network with acceptable performance drop. Inspired by the linear and combinational properties of convolution, we seek to make some filters increasingly clo…
The study reveals flaws in pruning criteria and proposes a new assumption for better filter selection.
problem Flaws in existing pruning criteria for CNNs.
method Empirical experiments and Convolutional Weight Distribution Assumption.
result The Convolutional Weight Distribution Assumption improves filter selection in pruning.
CPOT prunes deep networks by identifying redundant filters using optimal transport.
problem Redundant filters in deep neural networks make models hard to deploy on resource-limited platforms.
method CPOT uses optimal transport to find the mean of channel distributions, pruning redundant information.
result CPOT outperforms state-of-the-art methods in pruning ResNet models and image-to-image translation tasks.
Novel approach reduces DNN redundancy and enhances computational efficiency.
problem Redundant parameters in large-scale DNNs pose hardware deployment challenges.
method WGSEF regularization technique for structured sparsity.
result Reduces redundancy and enhances computational efficiency.
We present a provable, sampling-based approach for generating compact Convolutional Neural Networks (CNNs) by identifying and removing redundant filters from an over-parameterized network. Our algorithm uses a small batch of input data points to assign a saliency score to each filter and constructs an importance sampli…
We compare in this paper several feature selection methods for the Naive Bayes Classifier (NBC) when the data under study are described by a large number of redundant binary indicators. Wrapper approaches guided by the NBC estimation of the classification error probability out-perform filter approaches while retaining …
Convolutional neural networks (CNNs) achieve state-of-the-art performance in a wide variety of tasks in computer vision. However, interpreting CNNs still remains a challenge. This is mainly due to the large number of parameters in these networks. Here, we investigate the role of compression and particularly pruning fil…
Data is said to follow the transform (or analysis) sparsity model if it becomes sparse when acted on by a linear operator called a sparsifying transform. Several algorithms have been designed to learn such a transform directly from data, and data-adaptive sparsifying transforms have demonstrated excellent performance i…
With the advent of Big Data era, data reduction methods are highly demanded given its ability to simplify huge data, and ease complex learning processes. Concretely, algorithms that are able to filter relevant dimensions from a set of millions are of huge importance. Although effective, these techniques suffer from the…
To improve the resilience of distributed training to worst-case, or Byzantine node failures, several recent approaches have replaced gradient averaging with robust aggregation methods. Such techniques can have high computational costs, often quadratic in the number of compute nodes, and only have limited robustness gua…
Enhances financial time series forecasting with a multi-period learning framework.
problem Accurate financial time series forecasting requires considering both short-term and long-term trends.
method Proposes a Multi-period Learning Framework (MLF) with three modules: Inter-period Redundancy Filtering, Learnable Weighted-average Integration, and Multi-period self-Adaptive Patching.
result Improves financial time series forecasting accuracy and efficiency.
Proposes a new feature selection method integrating feature relationships.
problem Feature selection in machine learning models.
method Integrates feature-feature and feature-target relationships via penalized mRMR.
result Correctly identifies inactive features, reducing false discoveries.
Following recent advances in morphological neural networks, we propose to study in more depth how Max-plus operators can be exploited to define morphological units and how they behave when incorporated in layers of conventional neural networks. Besides showing that they can be easily implemented with modern machine lea…
Convolution is a central operation in Convolutional Neural Networks (CNNs), which applies a kernel to overlapping regions shifted across the image. However, because of the strong correlations in real-world image data, convolutional kernels are in effect re-learning redundant data. In this work, we show that this redund…
Online Maximum Likelihood Estimation of the Parameters of Partially Observed Diffusion Processesmath.OC
We revisit the problem of estimating the parameters of a partially observed diffusion process, consisting of a hidden state process and an observed process, with a continuous time parameter. The estimation is to be done online, i.e. the parameter estimate should be updated recursively based on the observation filtratio…
Inf-FS selects features by graph paths, ranking them for infinite feature sets.
problem Feature selection in large datasets with relevance and redundancy.
method Graph-based feature selection with infinite paths, evaluating feature subsets using matrix power series and Markov chains.
result Inf-FS outperforms other methods in various feature selection scenarios.
Funnel-Transformer reduces computation by compressing sequence data.
problem Redundant token-level representations in language processing.
method Gradually compresses sequence of hidden states to a shorter one.
result Funnel-Transformer outperforms standard Transformer with fewer FLOPs.
This work compares data reduction criteria for online Gaussian Processes.
problem The computational complexity of Gaussian Processes limits their applicability to small datasets and streaming scenarios.
method Unified comparison of several data reduction criteria, analyzing computational complexity and reduction behavior.
result Practical guidelines for choosing a suitable data reduction criterion for online Gaussian Processes.
Feature Selection for Regression Problems Based on the Morisita Estimator of Intrinsic Dimensionstat.ML
Data acquisition, storage and management have been improved, while the key factors of many phenomena are not well known. Consequently, irrelevant and redundant features artificially increase the size of datasets, which complicates learning tasks, such as regression. To address this problem, feature selection methods ha…
This paper explores how balancing and filtering techniques affect predictive multiplicity in machine learning models.
problem Predictive multiplicity due to Rashomon effect in high-stakes environments.
method Investigates the impact of balancing and filtering techniques on predictive multiplicity using 21 real-world datasets.
result Data-centric AI strategies can mitigate predictive multiplicity, but preprocessing methods may introduce it.
The paper deals with the adaptation of a new measure for the unsupervised feature selection problems. The proposed measure is based on space filling concept and is called the coverage measure. This measure was used for judging the quality of an experimental space filling design. In the present work, the coverage measur…
Holistic Filter Pruning reduces DNN complexity efficiently.
problem Redundant parameters in deep neural networks.
method Holistic Filter Pruning (HFP) for efficient DNN training.
result Achieves state-of-the-art performance with 60% reduction in multiplications on ImageNet.
Review and benchmark 58 feature selection methods for ML applications.
problem Selecting appropriate features for predictive modeling in ML applications.
method Synthesizes literature on feature selection benchmarking and evaluates 58 methods.
result Well-performing candidate methods identified: random forest, DISR, JIM.
It is well known that Convolutional Neural Networks (CNNs) have significant redundancy in their filter weights. Various methods have been proposed in the literature to compress trained CNNs. These include techniques like pruning weights, filter quantization and representing filters in terms of a basis functions. Our ap…
Improved change point detection using matched filters for non-parametric tests.
problem False positives and localization ambiguity in non-parametric two-sample tests.
method Derived and applied matched filters for various two-sample tests.
result Matched filters reduce false positives and improve test precision.
The introduction of automated flight control and management systems have made possible aircraft designs that sacrifice arodynamic stability in order to incorporate stealth technology intro their shape, operate more efficiently, and are highly maneuverable. Therefore, modern flight management systems are reliant on mult…
This paper deals with a new filter algorithm for selecting the smallest subset of features carrying all the information content of a data set (i.e. for removing redundant features). It is an advanced version of the fractal dimension reduction technique, and it relies on the recently introduced Morisita estimator of Int…
RGCF improves collaborative filtering by refining graph convolution embeddings.
problem GCN-based recommendation models introduce noise and redundancy, limiting high-order connectivity capture.
method Developed RGCF, a new GCN-based Collaborative Filtering model with redesigned embeddings.
result RGCF significantly outperforms state-of-the-art models on public datasets.
SAE-FiRE: Enhancing Earnings Surprise Predictions Through Sparse Autoencoder Feature Selectionq-fin.CP
SAE-FiRE extracts key financial info from long documents, improving earnings surprise predictions.
problem Predicting earnings surprises from long, redundant financial documents.
method Sparse Autoencoder feature selection to filter out noise and identify key dimensions.
result SAE-FiRE significantly outperforms baseline approaches in financial datasets.
Novel framework improves graph learning for out-of-distribution generalization.
problem Graph out-of-distribution generalization challenges in neural networks.
method Invariant Graph Learning based on Information bottleneck theory (InfoIGL).
result Achieves state-of-the-art performance in graph classification tasks under OOD generalization.
Quantifying multivariate redundancy with maximum entropy decompositions of mutual informationphysics.data-an
Williams and Beer (2010) proposed a nonnegative mutual information decomposition, based on the construction of redundancy lattices, which allows separating the information that a set of variables contains about a target variable into nonnegative components interpretable as the unique information of some variables not p…
This work enhances collaborative inference privacy by minimizing conditional entropy and boosting robustness against model inversion attacks.
problem Privacy leakage in collaborative inference systems via model inversion attacks.
method Theoretical proof and derivation of a differentiable measure for bounding conditional entropy, followed by a CEM algorithm to maximize it.
result Theoretical proof and experimental validation show that CEM consistently boosts inversion robustness without compromising feature utility or efficiency.
New method quantifies redundant information using information bottleneck.
problem Quantifying redundant information among multiple sources.
method Formulated as an information bottleneck problem, termed redundancy bottleneck.
result Extracts information that best predicts the target without revealing source identity.
Redundancy improves learning stability and generalization in structured systems.
problem Understanding redundancy in structured systems for learning and generalization.
method Developed a theoretical framework that redefines redundancy as a geometric principle unifying various measures.
result Redundancy balances structure and coupling, leading to optimal stability and generalization.
Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations. GCNs derive inspiration primarily from recent deep learning approaches, and as a result, may inherit unnecessary complexity and redundant computation. In…
Transformers reduce redundancy by focusing on invariant relational quantities.
problem Substantial internal redundancy in Transformer models due to coordinate-dependent representations and continuous symmetries.
method Reformulate representations, attention mechanisms, and optimization dynamics in terms of invariant relational quantities, eliminating redundant degrees of freedom by construction.
result Architectures that operate directly on relational structures, providing a principled geometric framework for reducing parameter redundancy and analyzing optimization.
Convolutional neural networks (CNN) are generally designed with a heuristic initialization of network architecture and trained for a certain task. This often leads to overparametrization after learning and induces redundancy in the information flow paths within the network. This robustness and reliability is at the inc…
Most existing feature selection methods are insufficient for analytic purposes as soon as high dimensional data or redundant sensor signals are dealt with since features can be selected due to spurious effects or correlations rather than causal effects. To support the finding of causal features in biomedical experiment…
Mutual information has been successfully adopted in filter feature-selection methods to assess both the relevancy of a subset of features in predicting the target variable and the redundancy with respect to other variables. However, existing algorithms are mostly heuristic and do not offer any guarantee on the proposed…
Paper proposes redundancy-free features for zero-shot object recognition.
problem Redundant visual features degrade zero-shot object recognition.
method Project original features into a new, statistically independent space.
result RFF-GZSL achieves competitive results on benchmark datasets.
This work explains scaling laws as redundancy laws in deep learning.
problem The mathematical origins of scaling laws in deep learning models remain unclear.
method Kernel regression and analysis of data covariance spectra.
result Scaling laws can be explained as redundancy laws, revealing the learning curve's slope depends on data redundancy.
Study on neural networks to identify redundancy issues in safe machine learning.
problem Identifying redundancy in neural network architectures for safe machine learning.
method Experiments with MNIST database using neural network classifiers.
result Underlines difficulties in using neural network classifiers for safe systems.
Feature selection and reducing the dimensionality of data is an essential step in data analysis. In this work, we propose a new criterion for feature selection that is formulated as conditional information between features given the labeled variable. Instead of using the standard mutual information measure based on Kul…
We simplify SSL by approximating redundant structural components with low-rank factorization.
problem Improving self-supervised learning performance with limited labeled data.
method Low-rank approximation of structural redundancy, introducing ε_s to measure approximation quality.
result The proposed method enhances SSL performance, as shown by theoretical and experimental validations.
Redundancy in deep neural network (DNN) models has always been one of their most intriguing and important properties. DNNs have been shown to overparameterize, or extract a lot of redundant features. In this work, we explore the impact of size (both width and depth), activation function, and weight initialization on th…
This paper introduces a new measure to identify model redundancy in compressed CNNs.
problem Identifying remaining model redundancy in compressed CNNs.
method Developed a statistical formulation of CNNs and compressed CNNs via tensor decomposition, revealing discrepancies in sample complexity and model redundancy.
result Introduced a new model redundancy measure, the ratio, for compressed CNNs.