Pre-training improves model robustness and uncertainty.
problem Improving model robustness and uncertainty in machine learning.
method Adversarial pre-training and task-specific methods.
result Approximately a 10% absolute improvement in adversarial robustness.
Paper improves adversarial training using a learned optimizer.
problem Improving robustness of deep learning models against adversarial attacks.
method Empirically identified PGD attack's limitations and used a learning-to-learn framework to train an adaptive inner optimizer.
result The proposed framework consistently improves model robustness over traditional adversarial training methods.
CAT improves robustness of neural networks by customizing perturbation levels.
problem Poor generalization of adversarial training on clean and perturbed data.
method CAT adapts perturbation level and label for each sample.
result CAT achieves better clean and robust accuracy than previous methods.
TuneUp improves GNN training by focusing on hard-to-learn nodes.
problem Sub-optimal training of GNNs on all nodes equally.
method Two-stage training: base GNN + tail node improvement.
result Significant improvement in tail node prediction performance.
MLPerf benchmarks ML training to drive performance improvements.
problem Unique challenges in ML training benchmarks.
method Developed MLPerf to overcome ML training's specific challenges.
result Quantitatively evaluated MLPerf's effectiveness.
New sampling method improves model training speed and robustness.
problem Improving model training speed and robustness.
method Periodic sampling of model weights for gradient optimization methods.
result Better, faster, and more robust convergence with minimal computation time.
Self-training outperforms pre-training on COCO object detection and segmentation datasets.
problem The effectiveness of pre-training in improving object detection and segmentation models is limited.
method Investigated self-training as an alternative method to utilize additional data.
result Self-training consistently improves model performance across various dataset sizes and data augmentation levels.
Improved GAN training with defined weights speeds up convergence and accuracy.
problem Challenges in training Generative Adversarial Networks (GANs).
method Proposed a training variation with defined weights to assist GAN training.
result The new algorithm improves GAN performance by 5% to 50% in synthetic and image datasets.
FIRE PBT improves neural network training by focusing on long-term performance.
problem Greedy decision mechanisms in PBT lead to poor long-term performance.
method FIRE PBT uses a fitness metric to encourage long-term performance over short-term improvements.
result FIRE PBT outperforms PBT on ImageNet and matches hand-tuned learning rates.
Adaptive networks improve model robustness through conditional normalization.
problem Limited robustness of adversarial-trained networks due to network capacity and training samples.
method Proposes a conditional normalization module to adapt networks during adversarial training.
result Adaptive networks outperform both clean validation accuracy and robustness compared to non-adaptive counterparts.
We improve private training accuracy with learning rate schedules and matrix factorizations.
problem Private training with learning rate schedules and correlated noise.
method General upper and lower bounds for learning rate schedules, memory-efficient constructions, and schedule-aware factorizations.
result Schedule-aware factorizations improve accuracy in private training.
Improved GAN training stability through clipping and reweighting.
problem Inconsistent GAN training leading to inferior performance.
method Proposes a variational GAN framework with probability ratio clipping and sample reweighting.
result Significantly improved performance across various GAN tasks.
Federated learning improves keyboard suggestions without user data.
problem Improving search suggestion quality without direct user data access.
method Applied federated learning for global-scale model training and deployment.
result Significant quality increases in search suggestions.
Improves neural network performance by enriching training dataset.
problem Achieving worst-case performance guarantees in neural networks.
method Adapting training dataset during training to reduce worst-case violations.
result Improved worst-case performance guarantees in neural networks.
Curriculum GANs improve GAN training by making tasks harder over time.
problem Improving training of Generative Adversarial Networks (GANs).
method A curriculum learning strategy that increases discriminator strength progressively.
result State-of-the-art image generation results achieved.
Training on mixed distributions improves test performance even when components are unrelated.
problem Improving test performance with mismatched training and test distributions.
method Analyzing mixture distributions with different training and test proportions.
result Distribution shift can be beneficial, improving test performance even when components are unrelated.
Improved neural network robustness with instance-specific perturbation margins.
problem Adversarial training fails to generalize well to unperturbed test set.
method Instance adaptive adversarial training with sample-specific perturbation margins.
result Test accuracy improves with a marginal drop in robustness.
A new method to speed up training and improve model performance.
problem Training models in computer vision can be slow and resource-intensive.
method Multiply loss by a random number to reduce backpropagation gradient.
result The method can improve training speed and model performance.
Pruning improves model generalization in over-parameterized models, contradicting traditional theories.
problem Pruning's effect on generalization in over-parameterized models.
method Empirical study on standard pruning algorithms and additional regularization effects.
result Pruning leads to better training and regularization, improving generalization.
Retraining with predicted labels improves model accuracy in noisy settings.
problem Improving model accuracy with noisy or corrupted labels.
method Retraining with predicted hard labels in a linearly separable binary classification setting.
result Retraining with predicted labels can increase model accuracy, as proven theoretically.
Improves adversarial training generalization with domain adaptation.
problem Weak generalization of adversarial training due to lack of representative adversarial samples.
method Adversarial Training with Domain Adaptation (ATDA) method.
result ATDA greatly improves adversarial training generalization and model smoothness.
Generative models improve adversarial robustness by adding synthetic data.
problem Improving robustness in machine learning models trained on limited data.
method Using synthetic data generated from a large dataset to augment the original training set.
result Generative models can significantly reduce the robust-accuracy gap compared to models trained with additional real data.
New method improves adversarial robustness without extra training steps.
problem Improving robustness of deep learning models against adversarial attacks.
method Guided Complement Entropy (GCE) training paradigm.
result GCE achieves better adversarial robustness with improved performance.
SPAT improves adversarial robustness by preserving semantics in adversarial training.
problem Adversarial examples often have different semantics than original data, introducing unintended biases.
method Semantics-preserving adversarial training (SPAT) that encourages pixel perturbation shared among all classes.
result SPAT improves adversarial robustness and achieves state-of-the-art results in CIFAR-10 and CIFAR-100.
Improves information cascade models using contrastive training and DSTs.
problem Improving models of information cascades using limited labeled data.
method Proposes a contrastive training procedure for models of information cascades as directed spanning trees (DSTs).
result Unsupervised training with additional content features achieves significantly better results, reaching half the accuracy of a fully supervised model.
Fine-tuning with pre-training data improves performance.
problem Limited training data for tasks.
method Theoretical analysis of excess risk bound and selection of pre-training data subset.
result Improvement in generalization performance with pre-training data.
New method improves adversarial training efficiency and robustness.
problem High computational costs and lack of stability in adversarial training.
method Backward smoothing for randomized smoothing of random initialization.
result Our method achieves similar model robustness as state-of-the-art methods but with significantly less training time.
Improved GAN training speed and quality with FastGAN.
problem Slower convergence and less expressive models in GAN training.
method Adversarial training with FastGAN algorithm.
result Better generation quality with less training time.
New strategy improves GNN pre-training for graph datasets.
problem Effective pre-training for graph neural networks on graph datasets.
method Expressive pre-training of GNNs at both node and graph levels.
result Significant improvement in generalization across downstream tasks.
Post-training corrections boost time-series forecasting accuracy.
problem Improving forecasting accuracy of large models after training.
method Sequential application of carefully selected corrections to predictions.
result Up to 30% improvement in forecasting accuracy with minimal overhead.
Improved pre-trained embeddings through effective entropy maximization.
problem Developing high-quality pre-trained embeddings for future tasks.
method E2MC criterion defined in terms of low-dimensional constraints.
result Significant improvement in downstream performance.
Rob-GAN combines generator, discriminator, and adversarial attack for improved robustness and quality.
problem Improving robustness and quality of GAN-generated images under adversarial attacks.
method Rob-GAN framework that jointly optimizes generator and discriminator in the presence of adversarial attacks.
result Rob-GAN improves convergence speed, image quality, and robustness of discriminators under strong adversarial attacks.
AGMMNs improve learning of copula models by adaptively selecting kernels.
problem Learning dependence structures in copula models.
method Adaptive bandwidth selection for MMD in GMMNs, increasing kernels based on validation loss.
result AGMMNs significantly improve training performance over GMMNs and parametric models.
Meta-learning improves DNN generalization on standard supervised learning.
problem Improving deep neural networks' generalization without adding more parameters.
method MLTP simulates meta-training by considering a batch of samples as a task, optimizing for both current and new tasks.
result MLTP consistently improves DNN generalization across various sizes and datasets.
Improves sample quality of generative models using energy-based methods.
problem Low sample quality in generative models.
method Constructs an energy function on latent space, trains an energy-based model, and generates improved samples.
result Significant improvement in sample quality with minimal computational overhead.
Improved image classifier performance with faster training using adaptive learning rates.
problem Lower accuracy and training time in small datasets.
method Dynamic learning rate tuning for different model architectures.
result Two-fold to ten-fold speedup in accuracy over state-of-the-art methods.
We simplify diffusion models by defining a design space and improving FID scores.
problem Complexity in diffusion-based generative models.
method Defined a design space, improved sampling and training processes, and pre-conditioned score networks.
result Improved FID scores of 1.79 for CIFAR-10 and 1.97 for unconditional settings.
Cost-effective method improves and re-purposes pre-trained GANs by fine-tuning class-embeddings.
problem Fine-tuning BigGANs from scratch is impractical due to instability and high computational cost.
method Fine-tuning only the class-embedding layer of pre-trained GANs.
result Significantly improved realism and diversity of samples, re-purposed for new tasks, and de-biased or improved diversity.
Adversarial examples improve speech recognition robustness.
problem Improving robustness of speech recognition systems.
method Adversarial examples generated dynamically during training to augment data.
result 23% relative word error rate reduction on Aurora-4.
Adversarial training improves robustness against common corruptions.
problem Improving robustness of models against common corruptions.
method Adversarial training with ℓp perturbation and learned perceptual similarity. result Our approach leads to state-of-the-art performance on common corruptions.
New training method improves model performance across various tasks.
problem Training deep neural networks using only the ground-truth class information.
method Maximizes likelihood of ground-truth classes while neutralizing complement classes.
result Training with both primary and complement objectives improves model performance across multiple tasks.
Improves model performance with unlabeled test data.
problem Improving model robustness to distribution shifts.
method Self-supervised learning using a single unlabeled test sample.
result Improves performance on diverse image classification benchmarks.
Ensemble models improve prediction calibration for mismatched distributions.
problem Calibration issues in deep neural networks with mismatched train and test distributions.
method Simple data augmentation and mixing techniques for ensemble models.
result Improves calibration and accuracy on CIFAR10 and CIFAR100 benchmarks.
LOGAN optimizes GAN training by improving adversarial dynamics.
problem Training GANs is challenging due to delicate adversarial dynamics and potential divergence.
method Integrates natural gradient-based latent optimisation into CS-GAN.
result Significant improvement in GAN training performance, achieving state-of-the-art results.
Joint training improves model accuracy by selectively using privileged information.
problem Two-stage training can lead to model failure with noisy privileged information.
method Joint training of two models to use privileged information selectively.
result Joint training outperforms two-stage baselines on synthetic and real-world tasks.
WCAT improves adversarial robustness on CIFAR-10.
problem Improving robustness against adversarial examples.
method Lipschitz regularization of the loss function.
result 11% improvement in adversarial robustness on CIFAR-10.
Improved disentanglement of data factors using recursive training.
problem Current unsupervised disentanglement methods are inconsistent and fail to achieve levels of disentanglement seen in supervised approaches.
method Introduced PBT for VAEs, used UDR for heuristic scoring, and developed recursive rPU-VAE approach.
result Recursive training leads to robust disentanglement of data factors across multiple datasets.
Noise improves model quality in non-linear neural networks during decentralized training.
problem Improving generalization of locally trained neural networks.
method Injecting noise into the weights of neural networks during decentralized training.
result Noise injection improves model quality for non-linear neural networks, but not for linear models.