Simple regularization methods mimic adversarial training's robustness.
problem Expensive adversarial training for robustness.
method Label smoothing and logit squeezing.
result Achieves strong adversarial robustness without adversarial examples.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “simple training”
Simple method disentangles content and style from pre-trained vision models.
problem Learning interpretable features in visual representations.
method Probabilistic linear entanglement model and simple disentanglement algorithm.
result Method provably disentangles content and style features.
A simple regularization technique speeds up training of Neural ODEs.
problem Training Neural ODEs is computationally expensive.
method Randomly sampling the end time of the ODE during training.
result Significantly decreases training time and improves performance.
Two regularization methods improve deep neural network performance on noisy data.
problem Improving deep neural network performance on noisy labeled data.
method Two simple regularization methods: distance regularization and auxiliary variable.
result Gradient descent with either method achieves generalization guarantee on clean data despite noisy labels.
New implicit regularization drives deep networks towards simple models.
problem Training deep neural networks with noise.
method Stochastic gradient descent with perturbed labels, analyzing dynamics near zero-error parameters.
result The training dynamics are governed by an implicit regularization term, leading to simpler models.
Simple noise training improves image recognition robustness.
problem Modern neural networks struggle with unseen image corruptions.
method Additive Gaussian and Speckle noise training, adversarial training against worst-case noise.
result Significant improvement in robustness on ImageNet-C and MNIST-C benchmarks.
Neural networks have recently had a lot of success for many tasks. However, neural network architectures that perform well are still typically designed manually by experts in a cumbersome trial-and-error process. We propose a new method to automatically search for well-performing CNN architectures based on a simple hil…
A new method for effective VAE training using calibrated decoders.
problem Training VAEs requires hyperparameter tuning, leading to inefficiency.
method Calibrated decoders that learn uncertainty and automatically determine information retention.
result Calibrated decoders can simplify VAE training without heuristic modifications.
SMT trains generative models by estimating mixture scores, outperforming existing methods.
problem Training one-step generative models efficiently and effectively.
method Score-of-Mixture Training (SMT) estimates the score of mixture distributions between real and fake samples.
result SMT/SMD outperform existing methods on CIFAR-10 and ImageNet 64x64 datasets.
New algorithm trains neural nets on simple skills to learn complex tasks faster.
problem Learning complex tasks through simple imitation.
method Train neural networks on simple, easy-to-learn skills to accelerate learning of complex, hard-to-learn tasks.
result Consistently outperforms state-of-the-art baseline in training speed and performance.
ABNN converts pre-trained DNNs into BNNs for reliable uncertainty quantification.
problem Uncertainty quantification in deep neural networks (DNNs) is challenging and critical for real-world applications.
method Adaptable Bayesian Neural Network (ABNN) that transforms pre-trained DNNs into BNNs with minimal overhead.
result ABNN achieves state-of-the-art performance in image classification and semantic segmentation tasks.
Improved neural language models using adversarial training.
problem Overfitting in large-scale neural language models.
method Adversarial training mechanism to introduce adversarial noise to output embedding layers.
result Improved perplexity scores and BLEU scores on various language tasks.
SRNNs achieve LSTMs' online regression performance in less time.
problem Training LSTMs is time-consuming.
method First-order training algorithm with linear time complexity.
result SRNNs provide similar regression performance as LSTMs in shorter training time.
A simple method compresses neural network weights using entropy penalties.
problem Neural network weight compression for scalability and accuracy.
method Reparameterization with learned entropy penalty for compression.
result Maximized classification accuracy and model compressibility.
The paper analyzes the dynamics of a simple neural network using a mean-field approach.
problem Understanding the training dynamics of neural networks, especially in classification tasks.
method Developed an analytic theory using a mean-field limit for a simple neural network.
result Explicitly solved the dynamics of a linearly separable dataset with a linear hinge loss.
A simple method for learning activation functions in neural networks.
problem Determining the best activation function for neural networks is challenging.
method Adding local subnetworks with a small amount of neurons to the neural network.
result The proposed method leads to better results compared to using a pre-defined activation function.
Training large-scale image recognition models is computationally expensive. This raises the question of whether there might be simple ways to improve the test performance of an already trained model without having to re-train or fine-tune it with new data. Here, we show that, surprisingly, this is indeed possible. The …
The paper proposes a new algorithm to select subsets of training data for better accuracy and explainability.
problem Tackles the challenge of balancing accuracy and explainability in pattern recognition.
method Identifies multiple subsets with simple local patterns by clustering similar instances.
result The sub-setting algorithm outperformed traditional decision trees by 15% on the international stroke dataset.
TuneUp improves GNN training by focusing on hard-to-learn nodes.
problem Sub-optimal training of GNNs on all nodes equally.
method Two-stage training: base GNN + tail node improvement.
result Significant improvement in tail node prediction performance.
Adversarial samples are perturbed inputs crafted to mislead the machine learning systems. A training mechanism, called adversarial training, which presents adversarial samples along with clean samples has been introduced to learn robust models. In order to scale adversarial training for large datasets, these perturbati…
Ensemble models improve prediction calibration for mismatched distributions.
problem Calibration issues in deep neural networks with mismatched train and test distributions.
method Simple data augmentation and mixing techniques for ensemble models.
result Improves calibration and accuracy on CIFAR10 and CIFAR100 benchmarks.
Simple k-NN filtering improves model accuracy on noisy labels.
problem Training models with noisy labels reduces performance and is hard to identify.
method A simple -nearest neighbor-based filtering approach on the logit layer.
result Improves model accuracy compared to recent methods.
Simple fine-tuning of GANs with frozen discriminator outperforms existing methods.
problem Fine-tuning GANs requires numerous data and heavy computation.
method Freeze lower layers of the discriminator for fine-tuning.
result FreezeD significantly outperforms previous techniques on various datasets.
We learn recurrent neural network optimizers trained on simple synthetic functions by gradient descent. We show that these learned optimizers exhibit a remarkable degree of transfer in that they can be used to efficiently optimize a broad range of derivative-free black-box functions, including Gaussian process bandits,…
Without access to large compute clusters, building random forests on large datasets is still a challenging problem. This is, in particular, the case if fully-grown trees are desired. We propose a simple yet effective framework that allows to efficiently construct ensembles of huge trees for hundreds of millions or even…
Large deep neural networks are powerful, but exhibit undesirable behaviors such as memorization and sensitivity to adversarial examples. In this work, we propose mixup, a simple learning principle to alleviate these issues. In essence, mixup trains a neural network on convex combinations of pairs of examples and their …
IBP-R improves verified adversarial robustness with simple, effective interval bound propagation.
problem Improving verifiability of adversarially trained networks.
method Coupling adversarial attacks with interval bound propagation for minimized verification gap.
result State-of-the-art verified robustness-accuracy trade-offs for small perturbations on CIFAR-10.
Simple method improves deep learning with noisy labels.
problem Deep learning's overfitting to noisy labels.
method Adds a variance regularization term to penalize neural network's Jacobian norm.
result Achieves state-of-the-art performance with high noise tolerance.
AFR simplifies reducing reliance on spurious features, improving model performance.
problem Reducing reliance on spurious features for out-of-distribution generalization.
method Automatic Feature Reweighting (AFR) updates the model with a weighted loss.
result AFR improves model performance on benchmarks with minimal compute.
AutoClip automatically adjusts gradient clipping for better audio separation.
problem Improving generalization in audio source separation networks.
method Adaptive gradient clipping based on historical gradient norms.
result Improves generalization performance in audio source separation networks.
RFM simplifies generative modeling on complex geometries without simulation.
problem Training generative models on non-Euclidean geometries is challenging.
method Riemannian Flow Matching (RFM) constructs a premetric for efficient vector field computation.
result RFM achieves state-of-the-art performance on various non-Euclidean datasets.
Improved WBP decoding with simple scaling and SNR adaptation.
problem Efficiently decoding weighted Tanner graphs with reduced complexity.
method Simple-scaling models with machine learning for edge weights, and parameter adapter networks.
result Simple scaling with few parameters can achieve near-maximum-likelihood performance.
A simple GI loss improves temporal generalization without complex methods.
problem Temporal drift between train and test distributions in evolving data.
method Gradient Interpolation (GI) loss to regularize temporal complexity.
result GI loss outperforms complex methods on real-world datasets.
New method protects sensitive data in deep learning training.
problem Protecting sensitive data in deep learning training.
method Distributed layer-partitioned training with step-wise activation functions.
result Experimental results show the method is simple and effective.
This paper analyzes training and generalization of simple 2-layer neural nets.
problem Why overparameterized neural nets generalize well despite fitting any data.
method Tighter characterization of training speed, data-dependent complexity measure, and tracking training and generalization dynamics via a related kernel.
result Generalization bound independent of network size, with completely independent sample complexity.
Neural networks and linear systems linked, revealing training loss and kernel limitations.
problem Exploring the training loss and limitations of neural networks and their kernels.
method Drawing connections between neural networks and under-determined linear systems, providing lower bounds, and analyzing gradient descent.
result Zero training loss achievable for neural networks under certain conditions, but not for ReLU kernels.
A simple method flags images as out-of-distribution based on their distance to nearest neighbors.
problem Detecting images not aligned with a trained model's in-distribution data.
method Flag images as OOD if their average distance to K nearest neighbors is large in the classifier's representation space.
result Simple methods can outperform more complex ones when considering learned representations.
Proposes a simple framework to balance task difficulty in multi-task learning.
problem Varying difficulty levels among different tasks in multi-task learning.
method Introduces a Balanced Multi-Task Learning (BMTL) framework that transforms training losses to balance task difficulty.
result Empirical studies show state-of-the-art performance of the proposed BMTL framework.
Estimates neural architecture performance speedily.
problem Accurately evaluating neural architectures' generalization performance.
method Estimates final test performance based on training speed.
result Consistently outperforms other alternatives in correlation with true test performance.
A simple fix helps saliency methods pass sanity checks.
problem Saliency methods fail simple sanity checks.
method Compute saliency maps for all labels, use competition to identify and remove less relevant pixels.
result CGI method is efficient, requires no additional training, and uses only the input and gradient.
Neural nets learn simple distributions first, then more complex ones.
problem Understanding how neural networks generalize from simple to complex functions.
method Stochastic gradient descent training, synthetic data, CIFAR10, ImageNet pre-training.
result Neural networks initially use lower-order statistics, then higher-order ones.
MMCGAN uses explicit manifold learning to improve GAN performance.
problem GAN mode collapse and unstable training.
method Introduces Minimum Manifold Coding (MMC) as a prior to guide GAN training.
result MMCGAN effectively alleviates mode collapse and stabilizes GAN training.
Unlabeled data improves adversarial robustness in machine learning.
problem Improving robustness of machine learning models against adversarial attacks.
method Semisupervised learning with self-training using unlabeled data.
result Semisupervised learning with unlabeled data significantly improves adversarial robustness.
Last layer retraining improves robustness to spurious correlations without high computational costs.
problem Neural networks can rely on spurious features like backgrounds for predictions.
method Simple last layer retraining on large models.
result Last layer retraining matches or outperforms state-of-the-art approaches on spurious correlation benchmarks.
Automates neural network design without training, speeding up search by seconds.
problem Time and effort in hand-designing deep neural networks.
method Predict trained accuracy from untrained network state using activation overlap.
result Search for powerful networks in seconds, verified on various benchmarks.
Simplified image clustering achieves competitive results without text-based embeddings.
problem Complexity and resource requirements of state-of-the-art clustering methods.
method SCP: trains a small cluster head using pre-trained vision model features and positive data pairs.
result SCP achieves highly competitive performance on various benchmark datasets.
Simple linear model outperforms deep models on sparse data.
problem Sparse data in recommender systems.
method Linear autoencoder with closed-form solution.
result Simple model achieves better ranking accuracy than deep models.
Simple ML models generate art completions.
problem Creating art through machine learning.
method Single-output classification and regression models trained on various image datasets.
result Generated images complete missing parts of input images.