When learning a new concept, not all training examples may prove equally useful for training: some may have higher or lower training value than others. The goal of this paper is to bring to the attention of the vision community the following considerations: (1) some examples are better than others for training detector…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,181 papers · 148 categories
Trend · papers per month
48 results for “training examples”
Adversarial examples improve speech recognition robustness.
problem Improving robustness of speech recognition systems.
method Adversarial examples generated dynamically during training to augment data.
result 23% relative word error rate reduction on Aurora-4.
Enhances single-step adversarial training to defend against iterative adversarial examples.
problem Defending against iterative adversarial examples in neural networks.
method Identified and leveraged empirical properties of Iter-Adv to improve Single-Adv.
result Enhanced Single-Adv to defend against iterative adversarial examples with improved accuracy and reduced training cost.
New method defends against both single-step and iterative adversarial examples.
problem Defending against adversarial examples, especially iterative ones, is computationally expensive.
method Single-Step Adversarial Training (SST) with modifications.
result Our method outperforms state-of-the-art methods in both accuracy and training time.
Loss-guided training accelerates node embedding methods on graphs.
problem Training efficiency in graph learning methods with implicit positive examples.
method Dynamic adjustment of training distribution based on loss values.
result Significant acceleration in training and computation over static methods.
Transformers trained on random classification tasks generalize well and can overfit without error.
problem Understanding how transformers generalize and overfit in-context.
method Analysis of implicit regularization during gradient descent training.
result Transformers can overfit without error and still generalize well.
RelatIF selects more intuitive training examples for explaining model predictions.
problem Influence functions identify outliers as explanatory examples, leading to poor explanations.
method RelatIF separates global and local influence, optimizing for local relative to global effects.
result Examples selected by RelatIF are more intuitive than those from influence functions.
GanDef uses GANs to defend against adversarial examples in neural networks.
problem Defending against adversarial examples in neural networks.
method GAN-based adversarial training defense using a competition game to regulate feature selection.
result GanDef trains a classifier to defend against adversarial examples with high accuracy.
Simple regularization methods mimic adversarial training's robustness.
problem Expensive adversarial training for robustness.
method Label smoothing and logit squeezing.
result Achieves strong adversarial robustness without adversarial examples.
New attack reveals adversarial training's inability to handle both and norms.
problem Adversarial training's inability to ensure robustness across and norms.
method Proposed a new attack to expose the weakness of adversarial training.
result Adversarial training fails to achieve robustness in both and norms.
Study shows removing outliers from training sets improves model robustness.
problem Vulnerability of deep neural networks to adversarial examples.
method Proposed a framework to detect and remove outliers from the training set to improve model robustness.
result Demonstrated that removing outliers from the training set can enhance model robustness.
This study explores how examples influence ICL in LLMs.
problem Understanding how examples impact in-context learning in large language models.
method Theoretical study with a probabilistic model extending from Gaussian mixture model.
result The impact of pre-training knowledge and examples on ICL prediction accuracy.
This paper introduces 'General Cyclical Training' for neural networks.
problem Improving training efficiency and performance of neural networks.
method Cyclical training phases with varying hyperparameters, batch sizes, loss functions, and data augmentation.
result Cyclical weight decay, softmax temperature, and gradient clipping enhance model accuracy.
CCAT improves model robustness to various adversarial attacks.
problem Robustness to adversarial attacks does not generalize to unseen threat models.
method CCAT biases models towards low confidence predictions on adversarial examples.
result CCAT increases robustness against multiple adversarial attack norms and types.
New training scheme reduces adversarial examples by increasing decision boundary margin.
problem Vulnerability of neural networks to adversarial examples.
method Differential training using a loss function on feature differences.
result Differential training significantly reduces adversarial examples.
Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficultycs.LG
Deep networks prioritize easier examples over harder ones, leading to faster training.
problem Understanding how deep networks prioritize examples of varying difficulty.
method Investigated the effect of linear vs non-linear learning modes on example difficulty.
result Non-linear dynamics tend to sequentialize the learning of examples of increasing difficulty.
Efficiently computes per-example gradients in CNNs for differential privacy.
problem Computing per-example gradients in CNNs for differential privacy.
method Comparison of existing strategies and introduction of a new per-example gradient calculation method.
result The new method is advantageous depending on model architecture and training.
Free adversarial training improves robustness without generating adversarial examples.
problem Training robust models against adversarial attacks is costly and impractical for large-scale datasets.
method Recycles gradient information from parameter updates to generate adversarial examples.
result Free adversarial training achieves comparable robustness to PGD training at negligible cost.
ZK-GanDef uses GANs to defend against adversarial examples without using them during training.
problem Adversarial examples challenge the effectiveness of neural network classifiers.
method Generative adversarial net (GAN) based zero knowledge adversarial training defense.
result Enhances test accuracy on adversarial examples by up to 49.17% compared to zero knowledge approaches.
SPAT improves adversarial robustness by preserving semantics in adversarial training.
problem Adversarial examples often have different semantics than original data, introducing unintended biases.
method Semantics-preserving adversarial training (SPAT) that encourages pixel perturbation shared among all classes.
result SPAT improves adversarial robustness and achieves state-of-the-art results in CIFAR-10 and CIFAR-100.
Method trains deep models to explain predictions with fewer examples.
problem Difficulty in humans understanding deep model predictions.
method Simultaneously trains prediction and explanation models with sparse regularization.
result Improves faithfulness of explanations with fewer examples while maintaining predictive performance.
Predicts optimal training dataset sizes per class for machine learning models.
problem Optimizing training dataset sizes for class-specific machine learning models.
method Algorithm based on space-filling design of experiments, models like powerlaw curves and generalized linear models.
result The algorithm predicts optimal training dataset sizes per class for improved model performance.
D3M debiases models by selectively removing problematic examples.
problem Model failures on underrepresented subgroups.
method Isolates and removes specific training examples that cause failures.
result Efficiently trains debiased classifiers with minimal example removal.
New algorithms estimate unknown training examples to improve ML model generalization.
problem Distribution shift between training and testing data leads to poor generalization.
method Combining species-estimation techniques with data-driven methods.
result Correcting the training set with unknown examples improves model robustness.
A two-step defense method generates strong adversarial examples at low cost.
problem Vulnerability of deep neural networks to adversarial attacks.
method Develops a two-step defense approach that generates strong adversarial examples using FGSM at a lower computational cost compared to traditional multi-step adversarial training.
result Demonstrates effectiveness of the two-step defense approach against various attack methods with comparable robustness to traditional multi-step adversarial training.
Paper tackles adversarial attacks on A3C path finding, proposing Gradient Band-based Adversarial Training.
problem Adversarial attacks on A3C path finding.
method Gradient Band-based Adversarial Training with CDG method.
result Gradient Band-based Adversarial Training achieves high attack immunity.
A novel method to train networks from each other's adversarial examples to resist black-box attacks.
problem Machine learning models can be fooled by adversarial examples, especially in black-box attacks.
method Simultaneous adversarial training combining two networks to learn from each other's adversarial examples.
result The method improves the networks' resilience to black-box attacks.
Adversarial examples are malicious inputs designed to fool machine learning models. They often transfer from one model to another, allowing attackers to mount black box attacks without knowledge of the target model's parameters. Adversarial training is the process of explicitly training a model on adversarial examples,…
A new method reduces adversarial training time without overfitting.
problem Catastrophic overfitting in single-step adversarial training.
method FGSMPR: FGSM with PGD Regularization.
result Reduces the gap to multi-step adversarial training.
BPN defends against adversarial attacks by generating beneficial perturbations.
problem Adversarial attacks cause deep neural networks to misclassify clean inputs.
method BPN generates beneficial perturbations during training to neutralize future adversarial attacks.
result BPN is robust to adversarial examples and more efficient than classical adversarial training.
Exact distributed algorithm trains Random Forest models on very large datasets.
problem Training Random Forest models on extremely large datasets (billions of examples).
method Exact distributed algorithm without approximating best split search.
result Trains Random Forest models on up to 18 billion examples, significantly faster than existing methods.
Many machine learning algorithms are based on the assumption that training examples are drawn independently. However, this assumption does not hold anymore when learning from a networked sample where two or more training examples may share common features. We propose an efficient weighting method for learning from netw…
AdvGAN generates high-quality adversarial examples using GANs.
problem Vulnerability of deep neural networks to adversarial examples.
method Generative adversarial networks (GANs) to learn and approximate input distributions.
result AdvGAN achieves high attack success rates on various models.
Algorithm predicts with optimal loss by abstaining from uncertain test examples.
problem Predicting with training data not matching test data.
method Transductive abstention algorithm using labeled and unlabeled test examples.
result Optimal prediction loss guarantees with additional term for abstaining cost.
Two-layer networks trained on low-dimensional subspaces are vulnerable to adversarial examples.
problem Vulnerability of two-layer neural networks to adversarial examples on low-dimensional subspaces.
method Analysis of gradient behavior and effect of initialization scale and regularization.
result Decreasing initialization scale or adding L2 regularization can improve robustness to adversarial perturbations orthogonal to the data.
Overparameterized autoencoders can memorize training examples.
problem Understanding generalization in overparameterized neural networks.
method Analyzing autoencoders of varying depths and types.
result Autoencoders concentrate learned functions around training examples.
Ordering examples in modular arithmetic training affects model performance.
problem Training efficiency and model performance in modular arithmetic.
method Varying example ordering while keeping other conditions constant.
result Adversarially structured ordering suppresses learning, while fixed-ordering strategies achieve high accuracy.
Study finds some examples are forgotten during neural network learning.
problem Understanding forgetting during neural network training.
method Defined forgetting event and analyzed across multiple data sets.
result High frequency of certain examples being forgotten, some not.
Selective-Backprop accelerates deep learning by focusing on high-loss examples.
problem Training deep neural networks is computationally expensive.
method Prioritizes examples with high loss for gradient computation, skipping others.
result Accelerates training up to 3.5x faster than standard SGD.
One Man's Trash is Another Man's Treasure: Resisting Adversarial Examples by Adversarial Examplescs.LG
Transform images with adversarial examples for robust classification.
problem Adversarial examples mislead deep neural networks in image classification.
method Transform input images with adversarial examples from a pre-trained model.
result Significantly more robust against a wide range of attacks compared to state-of-the-art methods.
A new method for robust training under label noise using weighted gradient descent.
problem Overfitting to noisy examples in machine learning.
method Exponentiated gradient reweighting for flexible handling of noisy data.
result Improved generalization in noisy classification and PCA problems.
Adversarial training adds dynamic perturbations to neural networks for robustness.
problem Accuracy trade-off and lack of diversity in adversarial examples.
method Dynamic adversarial perturbations in the parameter space of neural networks, updating perturbation biases during training.
result Adversarial training with negligible cost and reduced accuracy trade-off.
New method selects diverse mini-batches for active learning.
problem Reduce labeled data for deep learning models.
method Sequential selection of diverse mini-batches using K-means clustering.
result Achieves comparable or better performance than previous methods.
Reduces transferability of adversarial examples between models.
problem Transferability of adversarial examples between different models trained on different datasets.
method Investigates the angle between input-output Jacobians of different models to demonstrate relevance and explains asymmetry in terms of gradient norms.
result Simple modification reduces adversarial example transferability between models.
Adambs adapts Adam to prioritize important training examples.
problem Training deep learning models efficiently with varying example importance.
method Adapt Adam using a bandit sampling approach to prioritize examples based on their importance.
result Adambs improves convergence rate compared to Adam in some cases.
New framework explains adversarial examples in neural nets.
problem Adversarial examples in deep neural networks are counterintuitive and hard to explain.
method Introduces Dimpled Manifold Model to explain training phases and adversarial examples.
result Shows how adversarial examples arise from training dynamics and dimpling phase.
GraphAT improves graph neural networks by dynamically considering connected examples in adversarial training.
problem Graph neural networks are vulnerable to adversarial perturbations due to connections between examples.
method GraphAT dynamically regularizes based on graph structure to resist adversarial perturbations.
result GraphAT outperforms normal training on GCN by 4.51% in node classification accuracy.
Adversarial training improves model robustness with Voronoi constraints.
problem Adversarial examples mislead machine learning models, leading to incorrect classifications.
method Geometric framework using Voronoi cells to constrain adversarial training.
result Adversarial training with Voronoi constraints produces robust models.