arXiv research

AWP improves robustness by flattening weight loss landscape.

Adversarial weight perturbations can inject backdoors into trained neural models.

In this paper, we propose a perturbation framework to measure the robustness of graph properties. Although there are already perturbation methods proposed to tackle this problem, they are limited by the fact that the strength of the perturbation cannot be well controlled. We firstly provide a perturbation framework on …

Stochastic neural net weights are used in a variety of contexts, including regularization, Bayesian neural nets, exploration in reinforcement learning, and evolution strategies. Unfortunately, due to the large number of weights, all the examples in a mini-batch typically share the same weight perturbation, thereby limi…

Most random ReLU networks are vulnerable to small, Euclidean adversarial perturbations.

LoRA fine-tuning explained with gradient dynamics for low-rank perturbations.

Uncertainty computation in deep learning is essential to design robust and reliable systems. Variational inference (VI) is a promising approach for such computation, but requires more effort to implement and execute compared to maximum-likelihood methods. In this paper, we propose new natural-gradient algorithms to red…

Let $Ω$ be an open half-space or slab in $\mathbb{R}^{n+1}$ endowed with a perturbation of the Gaussian measure of the form $f(p):=\exp(ω(p)-c|p|^2)$, where $c>0$ and $ω$ is a smooth concave function depending only on the signed distance from the linear hyperplane parallel to $\partialΩ$. In this work we follow a varia…

Gradient-based methods introduce noise that penalizes models sensitive to weight perturbations.

Graph convolutional networks (GCNs) are vulnerable to perturbations of the graph structure that are either random, or, adversarially designed. The perturbed links modify the graph neighborhoods, which critically affects the performance of GCNs in semi-supervised learning (SSL) tasks. Aiming at robustifying GCNs conditi…

New metrics improve scRNA-seq perturbation modeling by reducing mode collapse.

Neural networks trained with backpropagation, the standard algorithm of deep learning which uses weight transport, are easily fooled by existing gradient-based adversarial attacks. This class of attacks are based on certain small perturbations of the inputs to make networks misclassify them. We show that less biologica…

We consider the structured-output prediction problem through probabilistic approaches and generalize the "perturb-and-MAP" framework to more challenging weighted Hamming losses, which are crucial in applications. While in principle our approach is a straightforward marginalization, it requires solving many related MAP …

Adversarial training purifies hidden weights to remove small perturbations.

We present an algorithm for computing class-specific universal adversarial perturbations for deep neural networks. Such perturbations can induce misclassification in a large fraction of images of a specific class. Unlike previous methods that use iterative optimization for computing a universal perturbation, the propos…

Differentiable clustering method using perturbed spanning forests.

There is a growing body of literature showing that deep neural networks are vulnerable to adversarial input modification. Recently this work has been extended from image classification to malware classification over boolean features. In this paper we present several new methods for training restricted networks in this …

RSO uses random weight perturbations to train deep networks without gradients.

Performance-critical machine learning models should be robust to input perturbations not seen during training. Adversarial training is a method for improving a model's robustness to some perturbations by including them in the training process, but this tends to exacerbate other vulnerabilities of the model. The adversa…

Adversarial Training (AT) and Virtual Adversarial Training (VAT) are the regularization techniques that train Deep Neural Networks (DNNs) with adversarial examples generated by adding small but worst-case perturbations to input examples. In this paper, we propose xAT and xVAT, new adversarial training algorithms, that …

Improved neural network convergence with causal Bayesian modeling in retail performance.

We investigate the generalizability of deep learning based on the sensitivity to input perturbation. We hypothesize that the high sensitivity to the perturbation of data degrades the performance on it. To reduce the sensitivity to perturbation, we propose a simple and effective regularization method, referred to as spe…

Differentially private method for estimating individualized treatment rules.

In this paper we introduce a family of stochastic gradient estimation techniques based of the perturbative expansion around the mean of the sampling distribution. We characterize the bias and variance of the resulting Taylor-corrected estimators using the Lagrange error formula. Furthermore, we introduce a family of va…

We construct new examples of Einstein metrics by perturbing the conformal infinity of geometrically finite hyperbolic metrics and by applying the inverse function theorem in suitable weighted Hölder spaces.

Distributed machine learning is an approach allowing different parties to learn a model over all data sets without disclosing their own data. In this paper, we propose a weighted distributed differential privacy (WD-DP) empirical risk minimization (ERM) method to train a model in distributed setting, considering differ…

The paper generalizes K-stability results to singular and weighted settings.

The paper learns perturbation sets from data to improve robustness in machine learning.

TULiP estimates uncertainty for deep learning models safely.

Study analyzes perturbations in singular subspaces under random noise.

Construct minimal Lagrangian surfaces in complex projective plane via loop group method.

We propose a new online algorithm for cumulative regret minimization in a stochastic linear bandit. The algorithm pulls the arm with the highest estimated reward in a linear model trained on its perturbed history. Therefore, we call it perturbed-history exploration in a linear bandit (LinPHE). The perturbed history is …

Convolutional Neural Networks have been a subject of great importance over the past decade and great strides have been made in their utility for producing state of the art performance in many computer vision problems. However, the behavior of deep networks is yet to be fully understood and is still an active area of re…

A new PLL method uses class activation values to improve robustness.

We consider a self-adjoint non-negative operator $H$ in a Hilbert space $\mathsf{L}^2(X,{\rm d}μ)$. We assume that the semigroup $(\mathrm{e}^{-t H})_{t>0}$ is defined by an integral kernel, $p$, which allows an estimate of the form $p(t,x,x)\le F_1(x)F_2(t)$ for all $(x,t)\in X\times\mathbb{R_+}$; we refer to $F_1$ as…

Generic smooth minimal hypersurfaces exist in 8D manifolds.

Establishes convexity and coercivity of K-energy functional for complex tori.

The study analyzes prediction errors in systems with memory kernels, providing bounds and stability results.

Deep learning has become a powerful and popular tool for a variety of machine learning tasks. However, it is challenging to understand the mechanism of deep learning from a theoretical perspective. In this work, we propose a random active path model to study collective properties of deep neural networks with binary syn…

Study optimal strategies for insurer's dividends, investments, and liabilities.

Unified PAC-Bayesian framework for deep learning generalization.

Recent advances show that deep neural networks are not robust to deliberately crafted adversarial examples which many are generated by adding human imperceptible perturbation to clear input. Consider $l_2$ norms attacks, Project Gradient Descent (PGD) and the Carlini and Wagner (C\&W) attacks are the two main methods, …

One of the common tasks in unsupervised learning is dimensionality reduction, where the goal is to find meaningful low-dimensional structures hidden in high-dimensional data. Sometimes referred to as manifold learning, this problem is closely related to the problem of localization, which aims at embedding a weighted gr…

Self-similar solutions to geometric flows are stable under small perturbations.

This work presents a novel approach to train invertible linear layers by adding rank-one perturbations.

The paper introduces a method to improve adversarial robustness in neural networks using randomized perturbations.

We construct a topological Chern-Simons sigma model on a Riemannian three-manifold M with gauge group G whose hyperkahler target space X is equipped with a G-action. Via a perturbative computation of its partition function, we obtain new topological invariants of M that define new weight systems which are characterized…

There has been a recent surge of interest in modeling neural networks (NNs) as Gaussian processes. In the limit of a NN of infinite width the NN becomes equivalent to a Gaussian process. Here we demonstrate that for an ensemble of large, finite, fully connected networks with a single hidden layer the distribution of ou…