Paper compresses deep neural networks by eliminating redundant neurons.
problem Challenges in deploying deep learning models due to high parameter count.
method Exploits non-linear redundancy to compress neural networks without loss.
result Reduces network size by up to 99% with minimal performance loss.
This paper enhances ML algorithms by improving data locality and reducing redundancy.
problem Improving performance of machine learning algorithms with complex data.
method Exploiting data locality and reuse in memory hierarchies of modern processors.
result Efficient implementation of machine learning algorithms can be achieved by reusing computation results.
Study finds 10% redundant images in image classification datasets.
problem Redundancy in large image classification datasets.
method Analysis of CIFAR-10 and ImageNet datasets to identify redundant images.
result 10% of images are redundant and can be removed without significant loss of performance.
This paper explores how optimizing data access and reducing redundancy can improve machine learning algorithm performance.
problem Performance issues in machine learning algorithms due to data locality and redundancy.
method Analysis of data access patterns and computational redundancy in machine learning algorithms, identifying opportunities for reuse and experimentation.
result Initial indicative results show potential for improving performance through data access optimization and reuse of computation results.
Paper proposes efficient BNN inference techniques on FPGA.
problem Redundancy in BNN inference leading to high computation and data access costs.
method Analyzed image similarity and BNN kernel weights to exploit redundancy. Proposed two types of fast and energy-efficient architectures.
result 80% reduction in computation and 40% in buffer access, achieving 17% power reduction.
Accurate prediction of suicide risk in mental health patients remains an open problem. Existing methods including clinician judgments have acceptable sensitivity, but yield many false positives. Exploiting administrative data has a great potential, but the data has high dimensionality and redundancies in the recording …
PoWER-BERT speeds up BERT inference by eliminating redundant word-vectors.
problem Improving BERT inference speed without sacrificing accuracy.
method Eliminating redundant word-vectors using a self-attention-based significance measure and learning the number of vectors to eliminate.
result Up to 4.5x reduction in inference time with <1% loss in accuracy on GLUE benchmark.
Two new Hie-TAN and Hie-TAN-Lite algorithms improve TAN for hierarchical feature spaces.
problem Learning dependencies in hierarchical feature spaces.
method Exploits hierarchical parent-child relationships as constraints to learn a dependency tree.
result Hie-TAN-Lite outperforms Hie-TAN and other methods in predictive accuracy.
Novel channel pruning method accelerates deep CNNs by removing redundant, similar features.
problem Limitations of existing magnitude-based pruning algorithms in similar magnitude cases.
method Hierarchical clustering of channels based on probabilistic similarity metrics, without sparsity training.
result 30% reduction in FLOPs for pruned ResNet-50 on ImageNet, outperforming baseline.
This paper finds a new way to compress CNN weights, improving on pruning and quantization.
problem Improving performance and storage efficiency of CNNs.
method Identifying and exploiting repeated patterns in CNN weight tensors, using Huffman coding and block sparse matrix formats.
result Achieved compaction ratios of 1.4x to 3.1x in addition to pruning and quantization.
Coded Federated Learning speeds up training in edge computing networks.
problem Slow convergence in Federated Learning due to heterogeneity and stochastic fluctuations.
method Exploiting statistical properties of compute and communication delays, distributed kernel embedding, and random Fourier features.
result Significant performance gains for CodedFedL in distributed non-linear regression and classification problems.
Gradient boosts monomial-order-free basis construction algorithms.
problem Lack of theoretical properties in monomial-order-free basis construction algorithms.
method Exploits gradient to sidestep spurious vanishing, achieve consistent output, and remove redundant bases.
result Proposes methods that equip monomial-order-free algorithms with theoretical properties.
Funnel-Transformer reduces computation by compressing sequence data.
problem Redundant token-level representations in language processing.
method Gradually compresses sequence of hidden states to a shorter one.
result Funnel-Transformer outperforms standard Transformer with fewer FLOPs.
Scientific discovery is limited by hypothesis redundancy, and hybrid methods can exploit non-local exploration.
problem Limitation of scientific discovery due to hypothesis redundancy.
method Hybrid discovery systems combining structured local search with LLM-generated non-local proposals.
result Hybrid methods can exploit non-local exploration when three geometric conditions co-occur.
Williams and Beer (2010) proposed a nonnegative mutual information decomposition, based on the construction of redundancy lattices, which allows separating the information that a set of variables contains about a target variable into nonnegative components interpretable as the unique information of some variables not p…
The paper introduces a method to explain redundancy in deep CNNs using unit impulse response.
problem Redundancy in deep CNNs leads to unnecessary computations and increased cost.
method Empirical demonstration and unit impulse response analysis to identify and quantify redundancy across layers and depth.
result Identifies and quantifies redundancy in deep CNNs, providing better insights into their internal dynamics.
Redundancy improves learning stability and generalization in structured systems.
problem Understanding redundancy in structured systems for learning and generalization.
method Developed a theoretical framework that redefines redundancy as a geometric principle unifying various measures.
result Redundancy balances structure and coupling, leading to optimal stability and generalization.
New method quantifies redundant information using information bottleneck.
problem Quantifying redundant information among multiple sources.
method Formulated as an information bottleneck problem, termed redundancy bottleneck.
result Extracts information that best predicts the target without revealing source identity.
Max-plus operators improve neural network filter selection and pruning.
problem Improving neural network efficiency and reducing redundancy.
method Exploiting Max-plus operators in neural network layers for filter selection and model pruning.
result Max-plus layers enhance filter selection and reduce redundancy without performance loss.
New BO methods exploit parallel experiments, reducing search time and improving solution quality.
problem Limitation of Bayesian optimization in exploiting parallel experiments.
method Propose new parallel BO paradigms that exploit the structure of the system to partition the design space.
result Significantly reduce search time and increase probability of finding global solutions.
NAT optimizes neural architectures to improve performance without extra cost.
problem Redundant operations in neural architectures consume memory and degrade performance.
method Transformed Markov Decision Process (MDP) and reinforcement learning to replace redundant operations with more efficient ones.
result Transformed architectures outperform original and existing methods on CIFAR-10 and ImageNet datasets.
Transformers reduce redundancy by focusing on invariant relational quantities.
problem Substantial internal redundancy in Transformer models due to coordinate-dependent representations and continuous symmetries.
method Reformulate representations, attention mechanisms, and optimization dynamics in terms of invariant relational quantities, eliminating redundant degrees of freedom by construction.
result Architectures that operate directly on relational structures, providing a principled geometric framework for reducing parameter redundancy and analyzing optimization.
Study shows DNNs often extract redundant features, influenced by network size and activation function.
problem Redundancy in deep neural network features.
method Hierarchical clustering of features based on cosine distances, varying network sizes and activation functions.
result Network size and activation function are key factors in DNN redundancy.
Paper proposes redundancy-free features for zero-shot object recognition.
problem Redundant visual features degrade zero-shot object recognition.
method Project original features into a new, statistically independent space.
result RFF-GZSL achieves competitive results on benchmark datasets.
This work explains scaling laws as redundancy laws in deep learning.
problem The mathematical origins of scaling laws in deep learning models remain unclear.
method Kernel regression and analysis of data covariance spectra.
result Scaling laws can be explained as redundancy laws, revealing the learning curve's slope depends on data redundancy.
Framework for generating multiple clusterings from multi-view data.
problem Challenges in finding optimal clustering criteria and handling incomplete multi-view data.
method DiMVMC framework that optimizes multiple decoder deep networks to complete data views and generate shared representations.
result DiMVMC outperforms state-of-the-art competitors in generating multiple clusterings with high diversity and quality.
Disagreement-based approaches generate multiple classifiers and exploit the disagreement among them with unlabeled data to improve learning performance. Co-training is a representative paradigm of them, which trains two classifiers separately on two sufficient and redundant views; while for the applications where there…
New framework improves multivariate time series forecasting by minimizing redundant information.
problem Improving multivariate time series forecasting with deep learning techniques.
method Cross-variable Decorrelation Aware feature Modeling (CDAM) and Temporal correlation Aware Modeling (TAM) to refine Channel-mixing and exploit temporal correlations.
result Significantly surpasses existing models in comprehensive tests.
I2C enables agents to learn efficient communication without redundancy.
problem Redundant broadcast communication in multi-agent cooperation.
method I2C learns a prior for agent-agent communication via causal inference and reinforcement learning.
result I2C reduces communication overhead and improves multi-agent cooperative performance.
Study on neural networks to identify redundancy issues in safe machine learning.
problem Identifying redundancy in neural network architectures for safe machine learning.
method Experiments with MNIST database using neural network classifiers.
result Underlines difficulties in using neural network classifiers for safe systems.
In this paper we construct a learning architecture for high dimensional time series sampled by sensor arrangements. Using a redundant wavelet decomposition on a graph constructed over the sensor locations, our algorithm is able to construct discriminative features that exploit the mutual information between the sensors…
This paper proposes a new method to compress CNNs for medical image analysis, improving efficiency and accuracy.
problem Large memory and computational requirements of CNNs in resource-constrained environments.
method Hierarchical spatio-channel low-rank compression framework that partitions feature maps into spatial regions and groups channels according to co-activation patterns within each region.
result The proposed method achieves significant FLOP reduction, inference speed-up, and improved classification accuracy compared to existing methods.
This work proposes a collaborative multi-head attention layer to reduce model size without sacrificing accuracy.
problem Over-parameterization in transformer models trained with large datasets.
method Proposes a collaborative multi-head attention layer that shares key/query projections.
result Reduction in model size by 4 for same accuracy and speed.
CodedFedL speeds up federated learning in MEC networks by 15x.
problem Slow convergence in federated learning due to heterogeneity and stochastic fluctuations.
method Injects structured coding redundancy into federated learning to mitigate stragglers and speed up training.
result CodedFedL speeds up the training procedure by up to 15x compared to benchmark schemes.
We simplify SSL by approximating redundant structural components with low-rank factorization.
problem Improving self-supervised learning performance with limited labeled data.
method Low-rank approximation of structural redundancy, introducing ε_s to measure approximation quality.
result The proposed method enhances SSL performance, as shown by theoretical and experimental validations.
This paper introduces a new measure to identify model redundancy in compressed CNNs.
problem Identifying remaining model redundancy in compressed CNNs.
method Developed a statistical formulation of CNNs and compressed CNNs via tensor decomposition, revealing discrepancies in sample complexity and model redundancy.
result Introduced a new model redundancy measure, the K/R ratio, for compressed CNNs. Optimized GPRNN reduces model complexity and overfitting, improving performance.
problem Overfitting in neural networks and high model complexity.
method Gaussian Process Regression - Neural Network hybrid with optimized redundant coordinates.
result Optimized GPRNN achieves lower test set error with fewer terms/neurons.
One of Powell's generators is not necessary.
problem Unresolved conjecture about generating Goeritz group.
method Short argument showing redundancy of one generator.
result One of Powell's generators is a consequence of others.
The study analyzes optimization trajectories in neural networks to reveal redundancy and redundancy-reducing strategies.
problem Understanding the directional structure and redundancy in neural network optimization.
method Introducing natural notions of complexity for optimization trajectories and analyzing their directional nature.
result Training only scalar batchnorm parameters can match the performance of training the entire network, indicating potential for hybrid optimization schemes.
This work introduces RISE to explain LLMs more reliably by distinguishing essential context.
problem Identifying which context elements influence LLM outputs reliably.
method RISE (Redundancy-Insensitive Scoring of Explanation) method.
result RISE provides more robust explanations than traditional methods.
This work identifies redundant tests in conditional-independence-based discovery that can improve graphical model accuracy.
problem Reliability and sensitivity of conditional-independence-based discovery algorithms.
method Analysis of redundant tests and their impact on error detection and correction.
result Redundant tests can improve graphical model accuracy but not all are beneficial.
Contrastive learning works well with redundant data views.
problem Improving unsupervised learning with redundant data views.
method Theoretical analysis of contrastive learning in a multi-view setting.
result Linear functions of learned representations are nearly optimal on prediction tasks when views are redundant.
Inf-FS selects features by graph paths, ranking them for infinite feature sets.
problem Feature selection in large datasets with relevance and redundancy.
method Graph-based feature selection with infinite paths, evaluating feature subsets using matrix power series and Markov chains.
result Inf-FS outperforms other methods in various feature selection scenarios.
DETOX improves distributed training resilience with redundancy and robust aggregation.
problem Byzantine node failures in distributed training.
method Combines redundancy and robust aggregation methods.
result Substantial increase in robustness with nearly linear runtime.
A new model improves clustering by reducing redundancy in mixture of local EPCAs.
problem Redundancy in traditional mixture models causes ambiguity in clustering.
method Introduced a repulsiveness-encouraging prior and a DPP for diversity in DEPCAM model.
result The method reduces model redundancy and improves clustering performance.
This paper explains adversarial examples as feature redundancy abuse.
problem Understanding and mitigating adversarial examples in machine learning.
method Information-theoretic model to explain adversarial attacks.
result Feature redundancy is necessary for adversarial examples.
New method distinguishes feature relevance in non-linear contexts.
problem Finding relevant features with preserved redundancies.
method Random forest models and statistical methods.
result Distinguishes strong from weak feature relevance in non-linear problems.
In the field of exploratory data mining, local structure in data can be described by patterns and discovered by mining algorithms. Although many solutions have been proposed to address the redundancy problems in pattern mining, most of them either provide succinct pattern sets or take the interests of the user into acc…