New algorithm for 2D SWDFT reduces redundant calculations.
problem Redundant calculations in overlapping windows for 2D SWDFT.
method Tree data structure based on Cooley-Tukey FFT for efficient computation.
result Our algorithm reduces operations to O(N0N1n0n1). This paper explores how optimizing data access and reducing redundancy can improve machine learning algorithm performance.
problem Performance issues in machine learning algorithms due to data locality and redundancy.
method Analysis of data access patterns and computational redundancy in machine learning algorithms, identifying opportunities for reuse and experimentation.
result Initial indicative results show potential for improving performance through data access optimization and reuse of computation results.
This paper enhances ML algorithms by improving data locality and reducing redundancy.
problem Improving performance of machine learning algorithms with complex data.
method Exploiting data locality and reuse in memory hierarchies of modern processors.
result Efficient implementation of machine learning algorithms can be achieved by reusing computation results.
Hierarchical learning models, such as mixture models and Bayesian networks, are widely employed for unsupervised learning tasks, such as clustering analysis. They consist of observable and hidden variables, which represent the given data and their hidden generation process, respectively. It has been pointed out that co…
New method disentangles high-order effects in feature importance.
problem Quantifying cooperative effects in feature importance.
method Adaptive Leave One Covariate Out (LOCO) method to decompose LOCO into two-body and higher-order components.
result Decomposes LOCO into two-body and higher-order components, highlighting synergistic and redundant effects.
New method quantifies multivariate redundancy using maximum entropy decompositions.
problem Elusive multivariate measures of redundancy that comply with nonnegativity and axioms.
method Maximum entropy framework, rooted tree-based decompositions of mutual information.
result Quantifies different multivariate redundancy contributions.
Network deconvolution removes redundant data to improve neural network performance.
problem Redundant data in neural networks makes training challenging.
method Network deconvolution optimally removes pixel-wise and channel-wise correlations before each layer.
result Network deconvolution improves performance across various datasets.
Proposes a novel network for CTR prediction by learning modality-specific and modality-invariant representations.
problem Learning good representation of items from multimodal features in E-commerce is challenging due to redundant information across modalities.
method Introduces a Multimodal Adversarial Representation Network (MARN) that calculates modality-specific weights and learns modality-invariant representations.
result Consistently achieves remarkable improvements over state-of-the-art methods in CTR prediction.
A federated method for feature selection in multi-label data.
problem Feature selection in multi-label data for distributed and federated environments.
method Semi-Supervised Federated Multi-Label Feature Selection (SSFMLFS) using fuzzy information measures.
result SSFMLFS outperforms other methods in feature selection for multi-label data in federated settings.
The paper introduces a method to explain redundancy in deep CNNs using unit impulse response.
problem Redundancy in deep CNNs leads to unnecessary computations and increased cost.
method Empirical demonstration and unit impulse response analysis to identify and quantify redundancy across layers and depth.
result Identifies and quantifies redundancy in deep CNNs, providing better insights into their internal dynamics.
Redundancy improves learning stability and generalization in structured systems.
problem Understanding redundancy in structured systems for learning and generalization.
method Developed a theoretical framework that redefines redundancy as a geometric principle unifying various measures.
result Redundancy balances structure and coupling, leading to optimal stability and generalization.
New method quantifies redundant information using information bottleneck.
problem Quantifying redundant information among multiple sources.
method Formulated as an information bottleneck problem, termed redundancy bottleneck.
result Extracts information that best predicts the target without revealing source identity.
Transformers reduce redundancy by focusing on invariant relational quantities.
problem Substantial internal redundancy in Transformer models due to coordinate-dependent representations and continuous symmetries.
method Reformulate representations, attention mechanisms, and optimization dynamics in terms of invariant relational quantities, eliminating redundant degrees of freedom by construction.
result Architectures that operate directly on relational structures, providing a principled geometric framework for reducing parameter redundancy and analyzing optimization.
Study shows DNNs often extract redundant features, influenced by network size and activation function.
problem Redundancy in deep neural network features.
method Hierarchical clustering of features based on cosine distances, varying network sizes and activation functions.
result Network size and activation function are key factors in DNN redundancy.
Paper proposes redundancy-free features for zero-shot object recognition.
problem Redundant visual features degrade zero-shot object recognition.
method Project original features into a new, statistically independent space.
result RFF-GZSL achieves competitive results on benchmark datasets.
This work explains scaling laws as redundancy laws in deep learning.
problem The mathematical origins of scaling laws in deep learning models remain unclear.
method Kernel regression and analysis of data covariance spectra.
result Scaling laws can be explained as redundancy laws, revealing the learning curve's slope depends on data redundancy.
RMCSE improves voltage estimation in low-observability distribution systems.
problem Insufficient measurements in distribution system state estimation.
method Combines matrix completion and power system model, minimizes rank and residual with different weights.
result Robust voltage estimation in low-observability systems without bad data detection.
Study on neural networks to identify redundancy issues in safe machine learning.
problem Identifying redundancy in neural network architectures for safe machine learning.
method Experiments with MNIST database using neural network classifiers.
result Underlines difficulties in using neural network classifiers for safe systems.
Study finds 10% redundant images in image classification datasets.
problem Redundancy in large image classification datasets.
method Analysis of CIFAR-10 and ImageNet datasets to identify redundant images.
result 10% of images are redundant and can be removed without significant loss of performance.
Clustering helps reduce spatial data size by identifying redundant points.
problem Too much spatial data makes it hard to answer research questions.
method Density-based clustering to compress data into representative features.
result Reduces spatial data set size by identifying and removing redundant points.
We simplify SSL by approximating redundant structural components with low-rank factorization.
problem Improving self-supervised learning performance with limited labeled data.
method Low-rank approximation of structural redundancy, introducing ε_s to measure approximation quality.
result The proposed method enhances SSL performance, as shown by theoretical and experimental validations.
This paper introduces a new measure to identify model redundancy in compressed CNNs.
problem Identifying remaining model redundancy in compressed CNNs.
method Developed a statistical formulation of CNNs and compressed CNNs via tensor decomposition, revealing discrepancies in sample complexity and model redundancy.
result Introduced a new model redundancy measure, the K/R ratio, for compressed CNNs. Optimized GPRNN reduces model complexity and overfitting, improving performance.
problem Overfitting in neural networks and high model complexity.
method Gaussian Process Regression - Neural Network hybrid with optimized redundant coordinates.
result Optimized GPRNN achieves lower test set error with fewer terms/neurons.
One of Powell's generators is not necessary.
problem Unresolved conjecture about generating Goeritz group.
method Short argument showing redundancy of one generator.
result One of Powell's generators is a consequence of others.
The study analyzes optimization trajectories in neural networks to reveal redundancy and redundancy-reducing strategies.
problem Understanding the directional structure and redundancy in neural network optimization.
method Introducing natural notions of complexity for optimization trajectories and analyzing their directional nature.
result Training only scalar batchnorm parameters can match the performance of training the entire network, indicating potential for hybrid optimization schemes.
A genetic algorithm with community detection improves feature selection accuracy.
problem Feature selection in high-dimensional datasets.
method Community detection in genetic algorithm for feature selection.
result The new approach yields improved classification accuracy.
This work introduces RISE to explain LLMs more reliably by distinguishing essential context.
problem Identifying which context elements influence LLM outputs reliably.
method RISE (Redundancy-Insensitive Scoring of Explanation) method.
result RISE provides more robust explanations than traditional methods.
This work identifies redundant tests in conditional-independence-based discovery that can improve graphical model accuracy.
problem Reliability and sensitivity of conditional-independence-based discovery algorithms.
method Analysis of redundant tests and their impact on error detection and correction.
result Redundant tests can improve graphical model accuracy but not all are beneficial.
Contrastive learning works well with redundant data views.
problem Improving unsupervised learning with redundant data views.
method Theoretical analysis of contrastive learning in a multi-view setting.
result Linear functions of learned representations are nearly optimal on prediction tasks when views are redundant.
DETOX improves distributed training resilience with redundancy and robust aggregation.
problem Byzantine node failures in distributed training.
method Combines redundancy and robust aggregation methods.
result Substantial increase in robustness with nearly linear runtime.
iSTFTNet speeds up mel-spectrogram vocoders without sacrificing quality.
problem Efficiently converting mel-spectrograms to speech with minimal computation.
method Replaces convolutional layers with iSTFT after frequency dimension reduction.
result Significant reduction in computational cost with comparable quality.
This paper explains adversarial examples as feature redundancy abuse.
problem Understanding and mitigating adversarial examples in machine learning.
method Information-theoretic model to explain adversarial attacks.
result Feature redundancy is necessary for adversarial examples.
Study on how deterministic dependencies affect information synergy and redundancy.
problem Understanding how deterministic dependencies impact information synergy and redundancy.
method Systematic analysis of deterministic dependencies on information decomposition.
result Identifies how negative terms can originate from deterministic dependencies and discusses implications for neural coding.
New method distinguishes feature relevance in non-linear contexts.
problem Finding relevant features with preserved redundancies.
method Random forest models and statistical methods.
result Distinguishes strong from weak feature relevance in non-linear problems.
Paper compresses deep neural networks by eliminating redundant neurons.
problem Challenges in deploying deep learning models due to high parameter count.
method Exploits non-linear redundancy to compress neural networks without loss.
result Reduces network size by up to 99% with minimal performance loss.
FactorMiner discovers financial alpha factors with low redundancy.
problem Finding novel financial alpha factors in a vast search space.
method Modular Skill Architecture and Experience Memory to distill and guide exploration.
result FactorMiner constructs a diverse library of high-quality factors with competitive performance.
Wide neural networks' last hidden layers split into groups of redundant neurons.
problem Understanding why wide neural networks generalize well despite overfitting.
method Analyzed the last hidden layer representations of various convolutional neural networks.
result Wide hidden layers split into groups of redundant neurons, which help generalize.
BNs extract sparse, non-redundant features from climate data networks.
problem Redundant information in correlation networks limits physical feature extraction.
method Construct data-driven complex networks using Bayesian Networks.
result Sparse, non-redundant features reveal generalizable physical features.
Spectral dimensionality reduction algorithms are widely used in numerous domains, including for recognition, segmentation, tracking and visualization. However, despite their popularity, these algorithms suffer from a major limitation known as the "repeated Eigen-directions" phenomenon. That is, many of the embedding co…
mRMR method selects relevant features for marketing models without redundancy.
problem Selecting relevant features from large feature spaces in machine learning.
method Extends mRMR framework with non-linear redundancy and model-based relevance measures.
result Implemented mRMR method in production for Uber's marketing machine learning platform.
HAGs eliminate redundant computations in GNNs, improving training efficiency.
problem Redundant computations in GNNs leading to inefficiencies.
method Hierarchically Aggregated computation Graphs (HAGs) to manage and eliminate redundant computations.
result Significant improvement in training efficiency (up to 2.8x) with HAGs.
Collage inference uses redundancy to reduce cloud image classification latency variance.
problem Reducing latency variance in cloud image classification.
method Integrates collage-cnn for low-cost redundancy in multi-image classification.
result Significant reduction in 99th percentile tail latency and inference latency variation.
Collage-CNN reduces cloud inference latency by 1.47X with 9X reduced latency variation.
problem Reducing latency variance in cloud machine learning inference.
method Proposes a novel Collage-CNN model that combines multiple images for classification, providing redundancy and cost efficiency.
result Significant reduction in 99th percentile tail latency and variation in inference latency.
Redundancy helps speed up slow nodes in distributed learning.
problem Slow nodes (stragglers) bottleneck distributed optimization and learning performance.
method Encode data with redundancy, dynamically exclude stragglers, and compensate losses.
result Optimization algorithms converge to solutions even with straggling nodes.
This work prunes CNN filters based on their functionality, not just size.
problem Redundant filters in CNNs waste computation resources.
method Functionality-oriented filter pruning method.
result Pruning based on functionality optimizes computation and interprets filter importance.
Cell-based NAS search spaces are redundant and lack novelty.
problem Redundancy and lack of novelty in cell-based neural architecture search.
method Empirical analysis of architectures from popular cell-based search spaces.
result Randomly sampled architectures can match or outperform state-of-the-art results.
Redundancy in AI perception systems doesn't guarantee independent error occurrences.
problem Lack of direct statistical evidence of super-human automated driving performance.
method Investigated the effectiveness of redundancy in neural networks for independent error occurrences.
result Errors in neural networks for computer vision tasks are correlated, not independent.
Estimates interactions between modalities for multimodal data.
problem Accurately quantifying interactions between different data types.
method Developed Lightweight Sample-wise Multimodal Interaction (LSMI) estimator using pointwise information theory.
result LSMI reveals fine-grained dynamics in multimodal data.