Despite the great success achieved in machine learning (ML), adversarial examples have caused concerns with regards to its trustworthiness: A small perturbation of an input results in an arbitrary failure of an otherwise seemingly well-trained ML model. While studies are being conducted to discover the intrinsic proper…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Redundant Examples”
Convolutional neural networks (CNN) are generally designed with a heuristic initialization of network architecture and trained for a certain task. This often leads to overparametrization after learning and induces redundancy in the information flow paths within the network. This robustness and reliability is at the inc…
BNs extract sparse, non-redundant features from climate data networks.
problem Redundant information in correlation networks limits physical feature extraction.
method Construct data-driven complex networks using Bayesian Networks.
result Sparse, non-redundant features reveal generalizable physical features.
Optimized GPRNN reduces model complexity and overfitting, improving performance.
problem Overfitting in neural networks and high model complexity.
method Gaussian Process Regression - Neural Network hybrid with optimized redundant coordinates.
result Optimized GPRNN achieves lower test set error with fewer terms/neurons.
PAC-Bayesian bounds improve understanding of K-NN classifier performance.
problem Improving the understanding of K-NN classifier's generalization error.
method PAC-Bayesian analysis applied to K-NN classifier in kernel space.
result PAC-Bayesian bounds provide a function of the number of redundant training examples.
Quantifying multivariate redundancy with maximum entropy decompositions of mutual informationphysics.data-an
Williams and Beer (2010) proposed a nonnegative mutual information decomposition, based on the construction of redundancy lattices, which allows separating the information that a set of variables contains about a target variable into nonnegative components interpretable as the unique information of some variables not p…
The use of variable selection methods is particularly appealing in statistical problems with functional data. The obvious general criterion for variable selection is to choose the `most representative' or `most relevant' variables. However, it is also clear that a purely relevance-oriented criterion could lead to selec…
New method quantifies redundant information using information bottleneck.
problem Quantifying redundant information among multiple sources.
method Formulated as an information bottleneck problem, termed redundancy bottleneck.
result Extracts information that best predicts the target without revealing source identity.
Redundancy improves learning stability and generalization in structured systems.
problem Understanding redundancy in structured systems for learning and generalization.
method Developed a theoretical framework that redefines redundancy as a geometric principle unifying various measures.
result Redundancy balances structure and coupling, leading to optimal stability and generalization.
Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic pieces of information about sensory or behavioral variables. Williams and Beer (…
A data proxy reduces NAS computation cost by preserving key examples.
problem High computation cost in Neural Architecture Search (NAS).
method Measuring example importance based on relative accuracy ranking and creating a data proxy.
result Preserves almost same relative accuracy ranking with 10-20x smaller data proxy.
Transformers reduce redundancy by focusing on invariant relational quantities.
problem Substantial internal redundancy in Transformer models due to coordinate-dependent representations and continuous symmetries.
method Reformulate representations, attention mechanisms, and optimization dynamics in terms of invariant relational quantities, eliminating redundant degrees of freedom by construction.
result Architectures that operate directly on relational structures, providing a principled geometric framework for reducing parameter redundancy and analyzing optimization.
While deep learning is a powerful tool for natural language processing (NLP) problems, successful solutions to these problems rely heavily on large amounts of annotated samples. However, manually annotating data is expensive and time-consuming. Active Learning (AL) strategies reduce the need for huge volumes of labeled…
Paper proposes redundancy-free features for zero-shot object recognition.
problem Redundant visual features degrade zero-shot object recognition.
method Project original features into a new, statistically independent space.
result RFF-GZSL achieves competitive results on benchmark datasets.
This work explains scaling laws as redundancy laws in deep learning.
problem The mathematical origins of scaling laws in deep learning models remain unclear.
method Kernel regression and analysis of data covariance spectra.
result Scaling laws can be explained as redundancy laws, revealing the learning curve's slope depends on data redundancy.
Study on neural networks to identify redundancy issues in safe machine learning.
problem Identifying redundancy in neural network architectures for safe machine learning.
method Experiments with MNIST database using neural network classifiers.
result Underlines difficulties in using neural network classifiers for safe systems.
We simplify SSL by approximating redundant structural components with low-rank factorization.
problem Improving self-supervised learning performance with limited labeled data.
method Low-rank approximation of structural redundancy, introducing ε_s to measure approximation quality.
result The proposed method enhances SSL performance, as shown by theoretical and experimental validations.
Redundancy in deep neural network (DNN) models has always been one of their most intriguing and important properties. DNNs have been shown to overparameterize, or extract a lot of redundant features. In this work, we explore the impact of size (both width and depth), activation function, and weight initialization on th…
This paper introduces a new measure to identify model redundancy in compressed CNNs.
problem Identifying remaining model redundancy in compressed CNNs.
method Developed a statistical formulation of CNNs and compressed CNNs via tensor decomposition, revealing discrepancies in sample complexity and model redundancy.
result Introduced a new model redundancy measure, the ratio, for compressed CNNs.
The paper tackles feature selection for ordinal regression, considering feature redundancies and privileged information.
problem Discovering relevant factors in ranked data with potentially redundant features and privileged information.
method Develops feature relevance bounds for linear ordinal regression, considering feature redundancies and privileged information.
result Identifies all strongly and weakly relevant features and their type of relevance.
Hallmarks of Optimization Trajectories in Neural Networks: Directional Exploration and Redundancycs.LG
The study analyzes optimization trajectories in neural networks to reveal redundancy and redundancy-reducing strategies.
problem Understanding the directional structure and redundancy in neural network optimization.
method Introducing natural notions of complexity for optimization trajectories and analyzing their directional nature.
result Training only scalar batchnorm parameters can match the performance of training the entire network, indicating potential for hybrid optimization schemes.
This work introduces RISE to explain LLMs more reliably by distinguishing essential context.
problem Identifying which context elements influence LLM outputs reliably.
method RISE (Redundancy-Insensitive Scoring of Explanation) method.
result RISE provides more robust explanations than traditional methods.
On Different Notions of Redundancy in Conditional-Independence-Based Discovery of Graphical Modelscs.LG
This work identifies redundant tests in conditional-independence-based discovery that can improve graphical model accuracy.
problem Reliability and sensitivity of conditional-independence-based discovery algorithms.
method Analysis of redundant tests and their impact on error detection and correction.
result Redundant tests can improve graphical model accuracy but not all are beneficial.
Contrastive learning works well with redundant data views.
problem Improving unsupervised learning with redundant data views.
method Theoretical analysis of contrastive learning in a multi-view setting.
result Linear functions of learned representations are nearly optimal on prediction tasks when views are redundant.
Many objective Bayesian optimization tackles redundant objectives in expensive black-box functions.
problem Efficiently optimizing multiple expensive and noisy black-box functions with redundant objectives.
method Proposes a metric to identify redundant objectives and a Bayesian optimization algorithm to stop evaluating them.
result Reduces computational cost by stopping evaluation of redundant objectives, improving efficiency.
Generating labeled training datasets has become a major bottleneck in Machine Learning (ML) pipelines. Active ML aims to address this issue by designing learning algorithms that automatically and adaptively select the most informative examples for labeling so that human time is not wasted labeling irrelevant, redundant…
New method distinguishes feature relevance in non-linear contexts.
problem Finding relevant features with preserved redundancies.
method Random forest models and statistical methods.
result Distinguishes strong from weak feature relevance in non-linear problems.
Paper compresses deep neural networks by eliminating redundant neurons.
problem Challenges in deploying deep learning models due to high parameter count.
method Exploits non-linear redundancy to compress neural networks without loss.
result Reduces network size by up to 99% with minimal performance loss.
Large datasets have been crucial to the success of deep learning models in the recent years, which keep performing better as they are trained with more labelled data. While there have been sustained efforts to make these models more data-efficient, the potential benefit of understanding the data itself, is largely unta…
FactorMiner: A Self-Evolving Agent with Skills and Experience Memory for Financial Alpha Discoveryq-fin.TR
FactorMiner discovers financial alpha factors with low redundancy.
problem Finding novel financial alpha factors in a vast search space.
method Modular Skill Architecture and Experience Memory to distill and guide exploration.
result FactorMiner constructs a diverse library of high-quality factors with competitive performance.
Wide neural networks' last hidden layers split into groups of redundant neurons.
problem Understanding why wide neural networks generalize well despite overfitting.
method Analyzed the last hidden layer representations of various convolutional neural networks.
result Wide hidden layers split into groups of redundant neurons, which help generalize.
Spectral dimensionality reduction algorithms are widely used in numerous domains, including for recognition, segmentation, tracking and visualization. However, despite their popularity, these algorithms suffer from a major limitation known as the "repeated Eigen-directions" phenomenon. That is, many of the embedding co…
This paper enhances ML algorithms by improving data locality and reducing redundancy.
problem Improving performance of machine learning algorithms with complex data.
method Exploiting data locality and reuse in memory hierarchies of modern processors.
result Efficient implementation of machine learning algorithms can be achieved by reusing computation results.
Machine learning (ML) is probably the first and foremost used technique to deal with the size and complexity of the new generation of data. In this paper, we analyze one of the means to increase the performances of ML algorithms which is exploiting data locality. Data locality and access patterns are often at the heart…
In 1980 J. Powell proposed that five specific elements sufficed to generate the Goeritz group of any Heegaard splitting of . This conjecture remains unresolved for genus . Here a short argument shows that one of his proposed generators is redundant, in fact a consequence of three of the other four.
Cell-based NAS search spaces are redundant and lack novelty.
problem Redundancy and lack of novelty in cell-based neural architecture search.
method Empirical analysis of architectures from popular cell-based search spaces.
result Randomly sampled architectures can match or outperform state-of-the-art results.
Does Redundancy in AI Perception Systems Help to Test for Super-Human Automated Driving Performance?cs.LG
Redundancy in AI perception systems doesn't guarantee independent error occurrences.
problem Lack of direct statistical evidence of super-human automated driving performance.
method Investigated the effectiveness of redundancy in neural networks for independent error occurrences.
result Errors in neural networks for computer vision tasks are correlated, not independent.
Estimates interactions between modalities for multimodal data.
problem Accurately quantifying interactions between different data types.
method Developed Lightweight Sample-wise Multimodal Interaction (LSMI) estimator using pointwise information theory.
result LSMI reveals fine-grained dynamics in multimodal data.
Signed pairwise interactions conflate uniqueness, redundancy, and synergy
problem Signed pairwise interactions conflate uniqueness, redundancy, and synergy
method Stochastic Hi-Fi
result Stochastic Hi-Fi recovers structure missed by scalar baselines
Efficient algorithm removes redundant nodes and obsolete samples in machine learning.
problem Pruning redundant nodes and removing obsolete training samples in machine learning.
method Deduced decremented learning algorithms from incremental learning algorithms, using inverse Cholesterol factor and unitary transformation.
result Proposed decremented learning algorithms efficiently prune redundant nodes and remove obsolete training samples.
End-to-end framework learns from imperfect annotations directly.
problem Training machine learning models on imperfect human annotations.
method End-to-end framework merging aggregation with model training and modeling annotator competencies.
result Accuracy gains of up to 25% over state-of-the-art annotation aggregation methods.
New CR representations are found and shown to be redundant.
problem Identifying and classifying CR representations of 3-manifolds.
method Experimental computation of limit sets and exact computations of triangle groups.
result Many CR representations are redundant and conjugate.
Label-Based Diversity Measure Among Hidden Units of Deep Neural Networks: A Regularization Methodcs.LG
New method reduces overfitting in deep neural networks by measuring and regulating hidden unit diversity.
problem Overfitting in deep neural networks.
method Introduces a new redundancy measure based on mutual information to improve generalization.
result Reduction of redundancy improves generalization capacity, reducing overfitting.
PoWER-BERT speeds up BERT inference by eliminating redundant word-vectors.
problem Improving BERT inference speed without sacrificing accuracy.
method Eliminating redundant word-vectors using a self-attention-based significance measure and learning the number of vectors to eliminate.
result Up to 4.5x reduction in inference time with <1% loss in accuracy on GLUE benchmark.
Optimal algorithms for mixable losses in dynamic environments with reduced redundancy.
problem Online optimization of mixable loss functions in a dynamic environment.
method Introduce online mixture schemes with polynomial and logarithmic time complexities.
result Achieves optimal redundancy up to a constant multiplicity gap.
Reducing the latency variance in machine learning inference is a key requirement in many applications. Variance is harder to control in a cloud deployment in the presence of stragglers. In spite of this challenge, inference is increasingly being done in the cloud, due to the advent of affordable machine learning as a s…
Feature selection has attracted significant attention in data mining and machine learning in the past decades. Many existing feature selection methods eliminate redundancy by measuring pairwise inter-correlation of features, whereas the complementariness of features and higher inter-correlation among more than two feat…
To improve the resilience of distributed training to worst-case, or Byzantine node failures, several recent approaches have replaced gradient averaging with robust aggregation methods. Such techniques can have high computational costs, often quadratic in the number of compute nodes, and only have limited robustness gua…