Paper shows adversarial training can be fooled by new type of noise.
problem Adversarial training can be fooled by new types of noise.
method Designing ADVIN, a new type of inducing noise.
result ADVIN can degrade adversarial training robustness by 99.9%.
Adversarial training can degrade standard accuracy even when optimal for robust accuracy.
problem Tradeoff between standard and robust accuracy in adversarial training.
method Analyzes adversarial training's impact on standard accuracy, even when optimal for robust accuracy.
result Even with optimal predictors, adversarial training can still degrade standard accuracy.
New techniques improve 16-bit training accuracy without 32-bit units.
problem Training deep learning models with only 16-bit floating-point units.
method Studied BFloat16 units and applied stochastic rounding and Kahan summation techniques.
result Up to 7% absolute validation accuracy gain in 16-bit-FPU training.
The paper studies adversarial training for linear regression models.
problem Understanding the tradeoffs between robust and standard accuracy in adversarial training.
method Characterizes the fundamental tradeoff and specific adversarial training approach for linear regression with Gaussian features.
result Precise characterization of the standard and robust accuracy tradeoff in high-dimensional settings.
JTT improves model worst-group accuracy without group annotations.
problem Low worst-group accuracy in standard ERM models with spurious correlations.
method Two-stage approach: first ERM, then upweight misclassified examples.
result JTT closes 75% of the gap in worst-group accuracy compared to group DRO.
Study reveals how initialization scale affects training accuracy in linear networks.
problem Understanding implicit bias in linear classification models.
method Asymptotic analysis of gradient flow trajectories and training loss minimization.
result Implicit bias is more complex at reasonable initialization scales and training accuracies.
Mixed-size training improves CNN accuracy and speed.
problem Training CNNs on fixed image sizes limits their adaptability to various image sizes.
method Mixed-size training: training on multiple image sizes at once.
result Models trained with mixed-size images achieve higher accuracy and faster inference.
This work reduces computation cost for on-device CNN training.
problem High computation cost during on-device CNN training.
method Self-supervised instance filtering and error map pruning.
result Substantial computation saving without significant accuracy loss.
Neural network accuracy improves with denser training samples.
problem Improving neural network accuracy on unseen test samples.
method Bounding empirical training error smoothed across activation regions and using it to discard high-risk test samples.
result Discarding high-risk test samples based on error bounds improves prediction accuracy by up to 20%.
The paper analyzes adversarial training effects on classification accuracy.
problem Understanding adversarial training's impact on standard and robust accuracy.
method Derived precise statistical analysis for binary classification problems with Gaussian data.
result Theoretical explanation of standard and robust accuracy trends for adversarial training.
Self-training with noisy student-teacher boosts keyword spotting accuracy.
problem Robust keyword spotting in challenging conditions.
method Aggressive data augmentation and self-training with noisy student-teacher approach.
result Significant accuracy improvement in difficult conditions, up to 60%.
MotherNets speed up deep ensemble learning with shared costs.
problem Training deep neural network ensembles is computationally expensive and time-consuming.
method MotherNets train a single or small set of MotherNets to share data movement and computation costs across ensembles, generating target ensembles by transferring function from MotherNets.
result MotherNets provide a new Pareto frontier for accuracy vs. training cost, scaling improvements as ensemble size increases.
Adversarial training can lead to unfair accuracy disparities between different groups.
problem Adversarial training algorithms introduce unfair accuracy disparities between different groups of data.
method Propose a Fair-Robust-Learning (FRL) framework to mitigate unfairness in adversarial defenses.
result Empirical and theoretical validation of FRL's effectiveness in mitigating unfairness.
Learning ReLU networks to high uniform accuracy requires exponentially many samples.
problem Achieving high uniform accuracy on ReLU networks for security-critical applications.
method Quantified the number of training samples needed for any algorithm to guarantee uniform accuracy.
result The minimal number of training samples scales exponentially with network depth and input dimension.
Our system trains ImageNet in 6.6 minutes using 2048 Tesla P40 GPUs.
problem Training large-scale deep neural networks efficiently and accurately.
method Mixed-precision training, extremely large mini-batch size optimization, and optimized all-reduce algorithms.
result Trains ResNet-50 to 75.8% top-1 test accuracy in 6.6 minutes.
Differential privacy reduces model accuracy more for underrepresented groups.
problem Differential privacy impacts model accuracy differently across groups.
method Training neural networks with differential privacy (DP-SGD).
result DP-SGD reduces accuracy more for underrepresented groups.
This paper proposes BAT to balance accuracy and robustness in adversarial training.
problem Balancing accuracy and robustness in adversarial training models.
method Blind adversarial training (BAT) uses a cutoff-scale strategy to adaptively estimate a nonuniform budget for AEs.
result BAT improves the overall robustness of adversarial training models.
FasTrCaps reduces CapsNet training time by 58.6% while maintaining accuracy.
problem The high training time of Capsule Networks (CapsNets).
method Integrating multiple lightweight optimizations and a novel learning rate policy (WarmAdaBatch) into FasTrCaps framework.
result FasTrCaps framework can reduce CapsNet training time by 58.6% while preserving accuracy.
A new method improves SSVEP BCI to recognize responses in sub-second time with high accuracy.
problem Achieving high accuracy in SSVEP BCI with short response times.
method CSTA (Combining Spatial-Filtering and Temporal Alignment) method.
result CSTA achieves maximum mean accuracy of 97.43% in sub-second response time.
Joint training improves model accuracy by selectively using privileged information.
problem Two-stage training can lead to model failure with noisy privileged information.
method Joint training of two models to use privileged information selectively.
result Joint training outperforms two-stage baselines on synthetic and real-world tasks.
Approximate multipliers boost CNN training speed, power, and area at slight accuracy cost.
problem Improving CNN training performance in terms of speed, power, and area.
method Simulation of approximate multipliers' impact on CNN training, hybrid training method combining approximate and exact multipliers.
result Using approximate multipliers for most of training enhances speed, power, and area with minimal accuracy loss.
Study shows accuracy of neural networks depends more on error location than percentage of error.
problem Effect of noise on accuracy in incremental learning neural networks.
method Empirical study using Perceptron, Feed Forward Neural Network, and Radial Basis Function Neural Network.
result Accuracy of neural networks is more dependent on error location than the percentage of error.
Paper proposes using synthetic data to improve face recognition accuracy.
problem Improving face recognition accuracy using real data alone.
method Proposes a GAN that disentangles identity attributes and generates photo-realistic synthetic images.
result Synthetic images generated by the model are photo-realistic and can increase face recognition accuracy.
SAT improves adversarial training by smoothing the loss landscape through curriculum learning.
problem Adversarial training sacrifices clean accuracy for robustness and suffers from large generalization error.
method SAT uses curriculum learning to smooth the adversarial loss landscape, improving both clean and robust accuracy.
result SAT models improve clean and robust accuracy significantly compared to adversarial training and other baselines.
SemiNAS reduces NAS cost by predicting accuracy of unlabeled architectures.
problem Costly evaluation of architectures limits NAS efficiency.
method SemiNAS uses unlabeled architectures to train an accuracy predictor.
result SemiNAS achieves comparable accuracy with less data.
Improves deep neural network training and accuracy with adaptive basis approach.
problem Gap between theoretical and practical performance of deep neural networks.
method Adaptive basis viewpoint, novel initializations, hybrid optimizer.
result Dramatic increases in accuracy and convergence rate for various DNN applications.
Adversarial training can hurt robust accuracy in small sample size scenarios.
problem Adversarial training improves test accuracy but may degrade robustness in limited data settings.
method Analyzes high-dimensional linear classification with noiseless observations, and observes perceptible attacks on image datasets.
result Adversarial training can negatively impact robust generalization in small sample size regimes.
New method preserves both benign and robust accuracy in deep neural networks.
problem Large neural networks have high computational and storage costs.
method Pruning with fine-tuning to preserve both benign and robust accuracy.
result Average 93% benign accuracy, 92.5% empirical robust accuracy, and 85.0% verifiable robust accuracy preserved while compressing by 10x.
Adversarial robustness varies with input data distribution.
problem Vulnerability of adversarial training to input data distribution.
method Study of adversarial training and Bayes classifier behavior.
result Adversarial robustness is sensitive to input data distribution.
Improved CNN with general image processing kernels reduces training time and achieves high accuracy.
problem Training time and accuracy of CNNs.
method Used 41 general-purpose kernels for the first layer of CNNs.
result GFNN reduces training time by 30% and achieves 99.56% accuracy.
Proposes an accuracy-preserving calibration method for DNNs.
problem Calibration of deep neural networks (DNNs) to measure prediction reliability.
method Uses Concrete distribution on the probability simplex to calibrate DNNs without accuracy loss.
result The proposed method outperforms previous methods in accuracy-preserving calibration tasks.
Master thesis applies deep learning to sEMG hand gesture recognition, improving accuracy.
problem Reliability issues in sEMG-based hand gesture recognition due to motion artefacts and variability.
method Used deep learning on Unibo-INAIL dataset, collecting data over 8 sessions of 7 subjects.
result Deep learning architecture yields 81.2% inter-posture test accuracy and 75.9% inter-day test accuracy.
SliceOut speeds up deep learning training without sacrificing accuracy.
problem Frequent model re-training and large model training workloads in deep learning.
method SliceOut uses dropout-inspired scheme to drop contiguous sets of units at random, leveraging GPU memory layout.
result 10-40% speedups and memory reduction with minimal accuracy loss.
TAPAS predicts unseen dataset accuracy without training.
problem Training new models for unseen datasets is time-consuming.
method Deep neural network accuracy predictor that estimates unseen dataset performance.
result Achieves 100 networks per second on a single GPU.
Retraining with predicted labels improves model accuracy in noisy settings.
problem Improving model accuracy with noisy or corrupted labels.
method Retraining with predicted hard labels in a linearly separable binary classification setting.
result Retraining with predicted labels can increase model accuracy, as proven theoretically.
Predicting neural network accuracy from weights without testing.
problem Predicting neural network performance based on weights alone.
method Used simple statistics of weights to rank neural networks' performance.
result Simple predictors can rank networks' performance with high accuracy (R2 score > 0.98).
A new metric estimates classifier accuracy using only training data.
problem Assessing classifier accuracy without cross-validation.
method Bayesian Area Under the ROC Curve (CBAUC) metric for linear classifiers.
result The CBAUC is faster and more accurate than conventional AUC estimators.
This work benchmarks and theorizes robust NAS under adversarial training.
problem Lack of benchmark evaluations and theoretical guarantees for robust NAS architectures under adversarial training.
method Released a comprehensive data set and established a generalization theory using the neural tangent kernel.
result Established a generalization theory for robust NAS architectures under adversarial training.
Improved classifier accuracy by using more of the class-specific structure in trained models.
problem Softmax ignores valuable information encoded in the full array of class response distributions.
method Developed a hybrid classifier (Softmax-Pooling Hybrid, SPH) that uses Softmax on high-scoring samples and a log-likelihood method on low-scoring samples. result Reduces test set error by 6% to 23% using the exact same trained model.
We improve private training accuracy with learning rate schedules and matrix factorizations.
problem Private training with learning rate schedules and correlated noise.
method General upper and lower bounds for learning rate schedules, memory-efficient constructions, and schedule-aware factorizations.
result Schedule-aware factorizations improve accuracy in private training.
Paper proposes an ensemble SVM method for efficient VAD.
problem Efficient and accurate VAD for speech processing.
method Supervised learning with ensemble SVM on large datasets.
result Ensemble SVM outperforms stand-alone SVM in VAD accuracy.
Enhances machine learning accuracy with multi-level training data.
problem Improving accuracy of machine learning algorithms for differential equations.
method Combining coarse and fine resolution training data.
result Significant gains in accuracy over single-level algorithms.
Neural nets trained with linear discriminant initialization converge faster and more accurately.
problem Training feed-forward neural networks efficiently and accurately.
method Initialize first layer weights with linear discriminants.
result Asymptotic higher accuracy and faster convergence.
Predicts neural network accuracy with minimal training, speeding up hyper-parameter tuning.
problem Expensive full training of neural networks during hyper-parameter tuning.
method Train network multiple times, store initial and final accuracies, use curve fitting and SVM for prediction.
result Predicted accuracies match known optimal settings with low computational cost.
This paper improves deep learning models' accuracy with differential privacy using gradient encoding and denoising.
problem Deep learning models leak sensitive information about their training datasets.
method Gradient encoding to map gradients to a smaller vector space, and denoising for post-processing.
result Our technique achieves better model accuracy with differential privacy guarantees compared to state-of-the-art methods.
Band-limited training reduces resource usage without sacrificing accuracy.
problem Resource constraints in training Convolutional Neural Networks (CNNs).
method Artificially constraining the frequency spectra of convolutional filters during training.
result CNNs can leverage lower-frequency components effectively, reducing resource usage.
Paper explores tradeoff between standard and robust accuracy for latent models.
problem Tradeoff between standard accuracy and robust accuracy in adversarial training.
method Revisits adversarial training for latent models, considering Gaussian mixture and generalized linear models.
result Low-dimensional manifold structure mitigates the tradeoff between standard and robust accuracy.
TSSM splits neural networks for parallel training with minimal accuracy loss.
problem Accuracy degradation in parallel training of deep neural networks.
method TSSM reformulates alternating minimization to achieve parallelism with minimal accuracy loss.
result TSSM achieves significant speedup without accuracy loss on multiple datasets.