CNN layers with large norms are still robust to adversarial attacks.
problem Understanding the relationship between layer norms and adversarial robustness in CNNs.
method Theoretical analysis of ℓ1 and ℓ∞ norms, norm decay method, adversarial training frameworks. result Adversarially robust CNNs can have comparable or larger layer norms than non-adversarially robust ones.
Study reveals layers in deep networks can be robust or critical, affecting model performance.
problem Understanding the role of different layers in deep neural networks.
method Empirical study of layer robustness and re-initialization effects.
result Layers in deep neural networks can be categorized as robust or critical, impacting model performance.
New margin measure improves deep learning generalization and robustness.
problem Unclear relationship between output margin and generalization for deep models.
method Introduced 'all-layer margin' for deep neural networks.
result Tighter generalization bounds for neural nets with no exponential depth dependency.
WM layer improves CNN robustness to noise and adversarial attacks.
problem CNNs' susceptibility to noise and adversarial attacks.
method WM layer as a generic architectural addition to CNNs.
result WM variants enhance robustness to noise and adversarial attacks.
Orthogonium offers unified, efficient layers for robust deep learning.
problem Fragmented and computationally demanding implementations of orthogonal and 1-Lipschitz layers.
method Unified, efficient PyTorch library providing orthogonal and 1-Lipschitz layers.
result Reduced overhead and standardized tools for robust experimentation.
LOT improves adversarial robustness by training 1-Lipschitz convolution layers.
problem Improving adversarial robustness of deep neural networks.
method LOT: Layer-wise Orthogonal Training for 1-Lipschitz convolution layers.
result LOT significantly enhances certified robustness of Lipschitz-bounded models.
New method makes neural networks more secure by protecting latent layers from adversarial attacks.
problem Vulnerability of latent layers in adversarially trained models to small perturbations.
method Latent Adversarial Training (LAT) and Latent Attack (LA) algorithms.
result Improves adversarial accuracy by 1-2% on MNIST, CIFAR-10, CIFAR-100 datasets.
Last layer retraining improves robustness to spurious correlations without high computational costs.
problem Neural networks can rely on spurious features like backgrounds for predictions.
method Simple last layer retraining on large models.
result Last layer retraining matches or outperforms state-of-the-art approaches on spurious correlation benchmarks.
LipKernel adds robustness to CNNs by enforcing Lipschitz bounds.
problem Improving robustness of CNNs in real-time applications.
method Dissipative layers parameterized by LMIs and 2-D Roesser model.
result Orders of magnitude faster run-time compared to state-of-the-art methods.
A new neural network model MDRBM improves noise-robustness in classification.
problem Improving noise-robustness in classification problems.
method Stacking a probabilistic-ELM layer on a discriminative restricted Boltzmann machine (DRBM).
result MDRBM outperforms other models, especially in noise-robustness.
New method enforces orthogonality in convolutional layers for improved robustness.
problem Improving adversarial robustness in deep learning models.
method Applying the Cayley transform to skew-symmetric convolutions in the Fourier domain.
result The proposed method preserves orthogonality and enhances adversarial robustness compared to existing techniques.
This work uses Lyapunov theory to improve the robustness of deep neural networks against adversarial attacks.
problem Vulnerability of deep neural networks to subtle adversarial perturbations.
method Treated each layer as a nonlinear dynamical system and used Lyapunov theory for stability and robustness.
result Developed empirically tight bounds on adversarial perturbations and proved stability and robustness globally.
AIBC uses AI and blockchain for efficient resource sharing.
problem Efficient resource sharing in large-scale decentralized systems.
method Two-consensus scheme with DABFT and DPoEV.
result Predictive deep learning for optimal BFT algorithm selection.
Lipschitz regularization improves neural network robustness by coupling weights across layers.
problem Improving neural network robustness under random input uncertainties.
method Regularization of neural networks by their Lipschitz constant, highlighting the coupling effect on weights across layers.
result Lipschitz regularization introduces a tradeoff between robustness and expressiveness, suggesting careful implementation.
AECF improves multimodal inference robustness and calibration.
problem Robustness and calibration issues in multimodal systems with missing inputs.
method Adaptive Entropy-Gated Contrastive Fusion (AECF) layer.
result Improves masked-input mAP by +18 pp at a 50% drop rate.
This study reveals a Min-Max property in LeNet's convolutional layers, enhancing adversarial robustness.
problem Uncertainty in the connection weights of convolutional layers in neural networks.
method Demonstrates the Min-Max property through back propagation-based training and a simplified convolution formulation.
result The Min-Max property improves adversarial robustness, indicating a stronger uncertainty in the model parameters.
BN helps learn fragile features, which can improve adversarial robustness.
problem The role of batch normalization in adversarial training and its impact on robustness.
method Investigated the expressiveness of BN in learning robust features compared to random features.
result Adversarially fine-tuning BN layers can result in non-trivial adversarial robustness.
Two-layer networks trained on low-dimensional subspaces are vulnerable to adversarial examples.
problem Vulnerability of two-layer neural networks to adversarial examples on low-dimensional subspaces.
method Analysis of gradient behavior and effect of initialization scale and regularization.
result Decreasing initialization scale or adding L2 regularization can improve robustness to adversarial perturbations orthogonal to the data.
Semantify-NN verifies neural network robustness against semantic perturbations.
problem Verifying robustness of neural networks against semantic adversarial attacks.
method Inserting semantic perturbation layers (SP-layers) into neural networks to verify robustness.
result Semantify-NN significantly improves robustness verification performance over ℓp-norm-based methods. The paper analyzes different neural network architectures for 3D point cloud processing.
problem Understanding the effects of specific intermediate-layer network architectures on 3D point cloud processing.
method Designing five metrics to diagnose various types of DNNs and conducting comparative studies.
result The hypotheses on the effects of specific intermediate-layer network architectures on the representation capacity of DNNs are verified.
New law establishes robustness for neural networks with bounded weights.
problem Ensuring robustness of neural networks against adversarial attacks.
method Deriving a lower bound on Lipschitz constant for arbitrary model classes with bounded Rademacher complexity.
result Established a law of robustness for weight-bounded neural networks, requiring log(n) layers for robust fitting.
ARFF reduces spectral bias in SGD-trained neural networks.
problem Spectral bias in two-layer neural networks.
method Comparison of SGD and ARFF on spectral bias and robustness.
result ARFF yields a closer to zero spectral bias compared to SGD.
Introduces bi-temperature logistic loss for more robust training.
problem Training neural nets with noise robustness.
method Introduces two temperatures into logistic loss and Bregman divergences.
result Training becomes more robust to noise with bi-temperature loss.
PLLay adds topological layers to deep learning models efficiently.
problem Efficiently incorporating topological features into deep learning models.
method Persistence landscapes for differentiable topological features.
result PLLay improves model learnability and robustness.
Enhances deep learning models to resist adversarial attacks.
problem Protecting deep learning models from adversarial examples.
method Combines two mechanisms: increased robustness at the cost of accuracy and improved accuracy without robustness guarantee.
result Combining mechanisms provides robustness against adversarial examples while maintaining accuracy.
Improves interpretability of neural network intermediate layers by making concept activations more robust and effective.
problem Challenges in interpreting neural network decisions and learning in intermediate layers due to opacity and shared interactions.
method Proposes A-CAV to increase effectiveness and employs Gram-Schmidt process to improve robustness.
result Significant improvement in recall rate of concept images from 18.35% to 76.83% for VGG16, with reduced variance in recall across different random seeds.
Inserts proximal mapping into deep networks for better regularization.
problem Effective regularization of deep learning models to handle adversarial perturbations and correlations between modalities.
method Proposes a new layer that directly produces regularized hidden layer outputs using proximal mapping.
result Outperforms state-of-the-art methods in robust temporal learning and multiview modeling.
A neural network detects anomalies without labels by identifying the underlying subspace.
problem Unsupervised anomaly detection in data.
method Robust Subspace Recovery (RSR) layer within an autoencoder.
result RSR layer effectively distinguishes inliers from outliers in latent space.
RAD improves robustness to domain annotation noise without explicit domain annotations.
problem Robustness to domain annotation noise in training data.
method Regularized Annotation of Domains (RAD) for last layer retraining.
result RAD outperforms state-of-the-art methods even with 5% noise in training data.
Paper proposes continuous residual layers for graph neural networks.
problem Low-pass filtering effect in GCN-based models.
method Integrates Ordinary Differential Equations (ODE) to produce outputs of continuous residual layers.
result Continuous residual layers achieve better results than non-residual modules in multiple layers.
Tensor dropout improves CNN robustness and efficiency.
problem Limited generalization and robustness of deep CNNs.
method Tensor layers with low-rank structures and tensor dropout.
result Tensor dropout outperforms other methods in robustness and efficiency.
Two-layer neural networks need more neurons to be robust.
problem Understanding the robustness of two-layer neural networks and the role of overparametrization.
method Investigation of the tradeoffs between network size and robustness, using Lipschitz constant as a measure.
result A conjecture that robustness requires overparametrization, with precise bounds for different cases.
End-to-end portfolio system accounts for model risk.
problem Model risk in portfolio selection.
method Distributionally robust optimization with convex duality.
result Explicitly accounts for model risk in portfolio selection.
Verifying robustness of neural network classifiers has attracted great interests and attention due to the success of deep neural networks and their unexpected vulnerability to adversarial perturbations. Although finding minimum adversarial distortion of neural networks (with ReLU activations) has been shown to be an NP…
Study improves robustness of deep fusion models against single source noise.
problem Ensuring robustness of deep fusion models against noise added to a single input source.
method Proposed two approaches: a carefully designed loss function and a convolutional fusion layer.
result Deep fusion models become robust against noise applied to a single source, preserving performance on clean data.
New method improves adversarial training robustness.
problem Lack of tight upper bounds for adversarial training.
method Holistic expansion of the network for upper bound minimization.
result RUB and aRUB methods are more robust than state-of-the-art methods.
Improved robustness of 1D CNNs for heart arrhythmia classification.
problem Improving the robustness of 1D CNNs for classification tasks.
method Parameterization using Cayley transform and controllability Gramian for Lipschitz-bounded CNNs.
result Improved robustness of trained Lipschitz-bounded 1D CNNs for heart arrhythmia classification.
We identify and approximate weights of two-layer neural networks from few samples.
problem Identifying and approximating weights of two-layer neural networks from limited data.
method Active sampling of finite difference approximations to Hessians, solving robust nonlinear programs, and gradient descent.
result Stable recovery of network weights under verifiable conditions.
LOTOS improves ensemble robustness by promoting orthogonal transformations.
problem Transferability of adversarial examples threatens robustness of classification models.
method LOTOS promotes orthogonality among sub-spaces of transformations in ensemble models.
result LOTOS increases robust accuracy of ensembles by 6 percentage points against black-box attacks.
New Muon and Momo variants improve neural network optimization robustness.
problem Improving neural network optimization methods.
method Systematic exploration of non-Euclidean gradient descent variants.
result Momo variants of Muon are more robust to hyperparameter tuning.
This work evaluates machine learning-based hotspot detectors on synthesized layout patterns.
problem Evaluating model robustness and generality of machine learning-based hotspot detectors.
method Developed an automatic layout generation tool to synthesize various layout patterns and tested machine learning-based detectors on these synthesized layouts.
result Machine learning-based detectors need continuous study for robustness and generality in DFM flows.
The paper proposes a method to create robust neural networks for automated driving.
problem Creating neural networks that can accurately predict road conditions and distances.
method The method introduces a non-standard loss function with tolerance to account for label variability and allows for deviations from labels.
result The proposed method results in a neural network that can robustly predict road conditions and distances, even with small label variations.
This research analyzes how input and output layers affect deep neural networks' resistance to adversarial attacks.
problem The vulnerability of deep neural networks to adversarial inputs, especially non-gradient based attacks.
method Analysis of three different fully connected dense network classes with manipulated input and output layers.
result Manipulating input and output layers can significantly enhance a deep neural network's robustness against adversarial attacks.
New method approximates neural network training for robustness.
problem Training robust neural networks with adversarial input perturbations.
method Stochastic convex optimization approach to adversarial training.
result Method achieves better adversarial robustness and performance.
Paper presents a defense framework against adversarial examples.
problem Vulnerability of deep neural networks to adversarial examples.
method Cross-layer strategic ensemble defense with input and output transformations.
result Strategic ensemble defense achieves high defense success rates and robustness.
The paper introduces REQNNs for robust 3D point cloud processing.
problem 3D point cloud processing robustness to rotations.
method Revised neural networks using quaternion features for rotation-equivariance.
result REQNNs exhibit higher rotation robustness compared to original networks.
Gradient descent biases neural networks to use an average of features, leading to non-robustness.
problem Non-robustness in neural networks due to feature averaging.
method Theoretical analysis and experiments on binary classification tasks.
result Gradient descent trains networks to rely on an average of features, making them vulnerable to adversarial attacks.
Efficiently detects anomalies in videos with reduced computation.
problem Anomaly detection in videos for robust and efficient systems.
method Powers-of-two weights and denoising for CNN simplification and robustness.
result 10% faster detection with comparable accuracy and robustness.