arXiv research

Framework combines adversarial training and provable robustness for neural networks.

Recent works have shown the effectiveness of randomized smoothing as a scalable technique for building neural network-based classifiers that are provably robust to $\ell_2$-norm adversarial perturbations. In this paper, we employ adversarial training to improve the performance of randomized smoothing. We design an adap…

Topic models provide a useful method for dimensionality reduction and exploratory data analysis in large text corpora. Most approaches to topic model inference have been based on a maximum likelihood objective. Efficient algorithms exist that approximate this objective, but they have no provable guarantees. Recently, a…

In recent years several adversarial attacks and defenses have been proposed. Often seemingly robust models turn out to be non-robust when more sophisticated attacks are used. One way out of this dilemma are provable robustness guarantees. While provably robust models for specific $l_p$-perturbation models have been dev…

New method uses deep learning to solve inverse problems with provable guarantees.

PARADE generates provably robust adversarial examples for deep networks.

Framework for fair classification with noisy protected attributes and provable guarantees.

Denoised smoothing defends pretrained classifiers against adversarial attacks.

Proposes efficient stochastic algorithms for optimizing NDCG with provable convergence guarantees.

New neural methods for stable control with provable guarantees.

Deep Partition Aggregation defends against poisoning attacks with provable certificates.

Paper develops robust neural network sensors for fuel injection quantities.

In this paper, we propose a provably correct algorithm for convolutive nonnegative matrix factorization (CNMF) under separability assumptions. CNMF is a convolutive variant of nonnegative matrix factorization (NMF), which functions as an NMF with additional sequential structure. This model is useful in a number of appl…

POLICE enforces linear constraints on deep neural networks efficiently.

Lipschitz constraints under L2 norm on deep neural networks are useful for provable adversarial robustness bounds, stable training, and Wasserstein distance estimation. While heuristic approaches such as the gradient penalty have seen much practical success, it is challenging to achieve similar practical performance wh…

Recently, there has been considerable progress on designing algorithms with provable guarantees -- typically using linear algebraic methods -- for parameter learning in latent variable models. But designing provable algorithms for inference has proven to be more challenging. Here we take a first step towards provable i…

Novel algorithm accelerates PnP methods for image deblurring and super-resolution.

Efficient poisoning attack converges to any target classifier with provable convergence.

Paper proposes robust generative models using VAEs.

PatchGuard defends against localized adversarial patches with provable robustness.

This paper concerns dictionary learning, i.e., sparse coding, a fundamental representation learning problem. We show that a subgradient descent algorithm, with random initialization, can provably recover orthogonal dictionaries on a natural nonsmooth, nonconvex $\ell_1$ minimization formulation of the problem, under mi…

PAC-Wrap provides provable guarantees for semi-supervised anomaly detection.

Paper proposes methods to estimate minimal adversarial perturbations for deep neural networks.

Study improves understanding of why agentic theorem provers succeed.

New algorithm finds near-optimal policies efficiently in zero-sum games.

Proposes a method for obtaining interval bounds in off-policy evaluation.

New algorithm provably converges to second-order stationary points in NMF.

We explain theoretically a curious empirical phenomenon: "Approximating a matrix by deterministically selecting a subset of its columns with the corresponding largest leverage scores results in a good low-rank matrix surrogate". To obtain provable guarantees, previous work requires randomized sampling of the columns wi…

Efficient RL algorithm for multinomial logistic MDPs with provable guarantees.

New algorithm for online tensor factorization with provable guarantees.

We introduce a pruning algorithm that provably sparsifies the parameters of a trained model in a way that approximately preserves the model's predictive accuracy. Our algorithm uses a small batch of input points to construct a data-informed importance sampling distribution over the network's parameters, and adaptively …

We analyze a simple prefiltered variation of the least squares estimator for the problem of estimation with biased, semi-parametric noise, an error model studied more broadly in causal statistics and active learning. We prove an oracle inequality which demonstrates that this procedure provably mitigates the variance in…

Machine-learning models for security-critical applications such as bot, malware, or spam detection, operate in constrained discrete domains. These applications would benefit from having provable guarantees against adversarial examples. The existing literature on provable adversarial robustness of models, however, exclu…

While policy-based reinforcement learning (RL) achieves tremendous successes in practice, it is significantly less understood in theory, especially compared with value-based RL. In particular, it remains elusive how to design a provably efficient policy optimization algorithm that incorporates exploration. To bridge su…

Although deep learning has shown its powerful performance in many applications, the mathematical principles behind neural networks are still mysterious. In this paper, we consider the problem of learning a one-hidden-layer neural network with quadratic activations. We focus on the under-parameterized regime where the n…

Develops PLL methods that are provably consistent and compatible with any deep network.

Ridge leverage scores provide a balance between low-rank approximation and regularization, and are ubiquitous in randomized linear algebra and machine learning. Deterministic algorithms are also of interest in the moderately big data regime, because deterministic algorithms provide interpretability to the practitioner …

Graph-based framework for provably robust adversarial training.

New algorithm trains ReLU gates provably in linear time.

We present a training system, which can provably defend significantly larger neural networks than previously possible, including ResNet-34 and DenseNet-100. Our approach is based on differentiable abstract interpretation and introduces two novel concepts: (i) abstract layers for fine-tuning the precision and scalabilit…

We give two provably accurate feature-selection techniques for the linear SVM. The algorithms run in deterministic and randomized time respectively. Our algorithms can be used in an unsupervised or supervised setting. The supervised approach is based on sampling features from support vectors. We prove that the margin i…

This paper introduces a new scalarization method for multi-objective optimization.

In dictionary learning, also known as sparse coding, the algorithm is given samples of the form $y = Ax$ where $x\in \mathbb{R}^m$ is an unknown random sparse vector and $A$ is an unknown dictionary matrix in $\mathbb{R}^{n\times m}$ (usually $m > n$, which is the overcomplete case). The goal is to learn $A$ and $x$. T…

Safe RL with binary feedback using SABRE algorithm.

Topic models, such as Latent Dirichlet Allocation (LDA), posit that documents are drawn from admixtures of distributions over words, known as topics. The inference problem of recovering topics from admixtures, is NP-hard. Assuming separability, a strong assumption, [4] gave the first provable algorithm for inference. F…

Study calibrates high-dimensional binary classifiers using angle between estimator and true weights.

We develop efficient algorithms to estimate the stability of Ordinary Least Squares regression results.

OKRidge solves sparse ridge regression problems for nonlinear systems.