Paper proposes robust generative models using VAEs.
problem Lack of robustness in generative models.
method Formally defined robust lower bound, optimized during training.
result Generative models become more robust to adversarial attacks.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,694 papers · 148 categories
Trend · papers per month
48 results for “provably bounded”
Proposes a method for obtaining interval bounds in off-policy evaluation.
problem Provides provably correct upper and lower bounds for off-policy evaluation.
method Searches for the maximum and minimum values of the expected reward among Lipschitz Q-functions.
result Introduces a Lipschitz value iteration method to monotonically tighten interval bounds.
Efficient RL algorithm for multinomial logistic MDPs with provable guarantees.
problem Model-based RL for episodic MDPs with unknown transition probabilities.
method Upper confidence bound-based algorithm for exploration-exploitation balance.
result Achieves regret bound for multinomial logistic models.
Graph-based framework for provably robust adversarial training.
problem Adversarial robustness of machine learning models.
method Formulates adversarial robustness as loss minimization with a Lipschitz constraint, using graph-based discretization and primal-dual algorithms.
result Establishes a connection between elliptic operators and adversarial learning, and proves fundamental lower bounds on adversarial sensitivity.
Deep Partition Aggregation defends against poisoning attacks with provable certificates.
problem Adversarial poisoning attacks corrupt classifier test-time behavior.
method Deep Partition Aggregation (DPA) is an ensemble method using hash partitions and base models trained on these partitions.
result DPA can certify >= 50% of test images against over 500 poison image insertions on MNIST, and nine insertions on CIFAR-10.
Topic models provide a useful method for dimensionality reduction and exploratory data analysis in large text corpora. Most approaches to topic model inference have been based on a maximum likelihood objective. Efficient algorithms exist that approximate this objective, but they have no provable guarantees. Recently, a…
Lipschitz constraints under L2 norm on deep neural networks are useful for provable adversarial robustness bounds, stable training, and Wasserstein distance estimation. While heuristic approaches such as the gradient penalty have seen much practical success, it is challenging to achieve similar practical performance wh…
Improves neural network verification by merging abstract domains and Lagrangian methods.
problem Prove provable bounds for neural network outputs given input ranges.
method Uses zonotopes within a Lagrangian decomposition to verify deep neural networks.
result Yields bounds that improve upon existing techniques in both time and tightness.
Proves efficient learning of hierarchical structure in meta-reinforcement learning.
problem Lack of provable guarantees for learning hierarchical structures in reinforcement learning.
method Analyzed HRL in meta-RL setting with tabular transition dynamics, providing diversity conditions and regret bounds.
result Sample-efficient recovery of natural hierarchical structure with provable guarantees.
Paper proposes a cost-sensitive conformal training method with provably controllable learning bounds.
problem Uncertainty quantification and learning bounds in conformal prediction.
method Cost-sensitive conformal training algorithm that minimizes the expected size of prediction sets using rank weighting.
result Theoretical analysis shows tightness between weighted objective and expected size of conformal prediction sets.
PatchGuard defends against localized adversarial patches with provable robustness.
problem Localized adversarial patches induce misclassification in machine learning models.
method PatchGuard uses CNNs with small receptive fields and robust masking to detect and mask corrupted features.
result PatchGuard achieves state-of-the-art provable robust accuracy and clean accuracy.
New algorithm reduces neural net error in contextual bandits.
problem Neural contextual bandits with general activation functions.
method Proposed an efficient algorithm with sublinear regret bound.
result Demonstrated provably sublinear regret bound in finite regime.
PARADE generates provably robust adversarial examples for deep networks.
problem Creating robust adversarial examples for deep neural networks.
method Iterative refinement of adversarial regions using optimization to ensure robustness under various perturbations.
result PARADE finds large, provably robust adversarial regions with high volume under various perturbations.
Recently, there has been considerable progress on designing algorithms with provable guarantees -- typically using linear algebraic methods -- for parameter learning in latent variable models. But designing provable algorithms for inference has proven to be more challenging. Here we take a first step towards provable i…
Ridge leverage scores provide a balance between low-rank approximation and regularization, and are ubiquitous in randomized linear algebra and machine learning. Deterministic algorithms are also of interest in the moderately big data regime, because deterministic algorithms provide interpretability to the practitioner …
PAC-Wrap provides provable guarantees for semi-supervised anomaly detection.
problem Ensuring reliable anomaly detection in safety-critical applications.
method PAC-Wrap wraps around existing anomaly detection methods to provide PAC guarantees.
result PAC-Wrap effectively provides rigorous guarantees for various anomaly detectors.
Unified framework for ensemble sampling in nonlinear contextual bandits with provable regret bounds.
problem Efficient exploration in nonlinear contextual bandits with unknown feature dimensions.
method Developed GLM-ES and Neural-ES for generalized linear and neural contextual bandits, respectively, using maximum likelihood estimation on randomly perturbed data.
result Unified high-probability frequentist regret bounds for GLM-ES and Neural-ES, matching state-of-the-art results.
This paper introduces a new scalarization method for multi-objective optimization.
problem Efficiently optimizing multiple conflicting objectives in black box settings.
method Introduces a novel hypervolume scalarization function and uses it to approximate the hypervolume indicator metric.
result Provable convergence to the entire Pareto frontier using random scalarizations and Bayesian optimization.
Recent work has shown that it is possible to train deep neural networks that are provably robust to norm-bounded adversarial perturbations. Most of these methods are based on minimizing an upper bound on the worst-case loss over all possible adversarial perturbations. While these techniques show promise, they often res…
New algorithm for weighted low rank approximation with provable guarantees.
problem Weighted low rank approximation (WLRA) is computationally hard.
method Reweights the low rank solution using the weight matrix itself.
result Provably optimal approximation guarantees for WLRA.
We develop efficient algorithms to estimate the stability of Ordinary Least Squares regression results.
problem Measuring the stability of regression conclusions in low dimensions.
method Efficient algorithms for estimating the minimum number of samples that need to be removed to change a regression conclusion.
result We can estimate stability up to a factor of 3 better than the greedy heuristic and certify stability even for dropping a majority of samples.
Efficient poisoning attack converges to any target classifier with provable convergence.
problem Inducing a corrupted model that misbehaves in favor of an adversary.
method Online convex optimization to find poisoning points incrementally.
result Provably converges to any attainable target classifier.
We prove a lower bound for feature dimension in linear MDPs and propose a novel dynamics aggregation framework.
problem The limitation of feature dimension in linear MDPs and the need for efficient hierarchical reinforcement learning.
method We propose a novel dynamics aggregation framework based on structural dynamics and design a provably efficient hierarchical reinforcement learning algorithm.
result Our algorithm achieves a regret of and meets the condition in most real-world environments.
The paper tackles safe exploration in RL by a conservative safety critic.
problem Safe exploration in reinforcement learning (RL) when partially trained policies are deployed.
method Learning a conservative safety estimate through a critic, provably bounding catastrophic failures.
result The approach provably converges to competitive task performance with significantly lower catastrophic failure rates.
This work studies learning a multi-head attention layer from random examples.
problem Learning a multi-head attention layer from random examples.
method The work initiates the study of provably learning a multi-head attention layer from random examples, providing upper and lower bounds.
result The first nontrivial upper and lower bounds for learning a multi-head attention layer from random examples are given.
Deployment of deep neural networks (DNNs) in safety- or security-critical systems requires provable guarantees on their correct behaviour. A common requirement is robustness to adversarial perturbations in a neighbourhood around an input. In this paper we focus on the norm and aim to compute, for a trained DNN an…
Topic models, such as Latent Dirichlet Allocation (LDA), posit that documents are drawn from admixtures of distributions over words, known as topics. The inference problem of recovering topics from admixtures, is NP-hard. Assuming separability, a strong assumption, [4] gave the first provable algorithm for inference. F…
In deep neural networks, the spectral norm of the Jacobian of a layer bounds the factor by which the norm of a signal changes during forward/backward propagation. Spectral norm regularizations have been shown to improve generalization, robustness and optimization of deep learning methods. Existing methods to compute th…
Recent work has developed methods for learning deep network classifiers that are provably robust to norm-bounded adversarial perturbation; however, these methods are currently only possible for relatively small feedforward networks. In this paper, in an effort to scale these approaches to substantially larger models, w…
On the Minimal Adversarial Perturbation for Deep Neural Networks with Provable Estimation Errorcs.LG
Paper proposes methods to estimate minimal adversarial perturbations for deep neural networks.
problem Quantifying robustness of deep neural networks against adversarial attacks.
method Proposes two lightweight strategies to find minimal adversarial perturbation.
result Approximates theoretical distance for samples close to classification boundary, providing robustness guarantees.
New method gives provable error bounds for neural nets under distribution shift.
problem Proving reliable error bounds for neural networks under distribution shift.
method Optimizing a classifier to disagree with another, using a new 'disagreement loss'.
result Valid error bounds with comparable accuracy to competitive methods.
Black-box variational inference tries to approximate a complex target distribution though a gradient-based optimization of the parameters of a simpler distribution. Provable convergence guarantees require structural properties of the objective. This paper shows that for location-scale family approximations, if the targ…
Why Agentic Theorem Prover Works: A Statistical Provability Theory of Mathematical Reasoning Modelsstat.ML
Study improves understanding of why agentic theorem provers succeed.
problem Understanding which components of agentic theorem provers improve proof success.
method Statistical provability theory and finite-horizon reachability MDP model.
result Bounds provability gap and explains components' effectiveness.
Paper develops efficient RL algorithm for general value function approximation.
problem Lack of theory for RL with general value function approximation.
method Provable efficient RL algorithm using bounded eluder dimension.
result Achieves a regret bound of .
OKRidge solves sparse ridge regression problems for nonlinear systems.
problem Identifying sparse governing equations for nonlinear dynamical systems.
method OKRidge algorithm using saddle point formulation and ADMM-based approach with efficient proximal operators.
result OKRidge achieves provable optimality with significantly faster run times than Gurobi.
Random Fourier Features for Kernel Ridge Regression: Approximation Bounds and Statistical Guaranteescs.LG
Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifica…
This paper addresses the problem of formally verifying desirable properties of neural networks, i.e., obtaining provable guarantees that neural networks satisfy specifications relating their inputs and outputs (robustness to bounded norm adversarial perturbations, for example). Most previous work on this topic was limi…
We present a provable, sampling-based approach for generating compact Convolutional Neural Networks (CNNs) by identifying and removing redundant filters from an over-parameterized network. Our algorithm uses a small batch of input data points to assign a saliency score to each filter and constructs an importance sampli…
SGMs are robust to practical errors via uncertainty quantification.
problem Robustness of SGMs to practical implementation errors.
method Wasserstein uncertainty propagation (WUP) theorem and Bernstein estimates.
result SGMs are provably robust to multiple sources of error.
Recent variational inference methods use stochastic gradient estimators whose variance is not well understood. Theoretical guarantees for these estimators are important to understand when these methods will or will not work. This paper gives bounds for the common "reparameterization" estimators when the target is smoot…
Training certifiable neural networks enables one to obtain models with robustness guarantees against adversarial attacks. In this work, we introduce a framework to bound the adversary-free region in the neighborhood of the input data by a polyhedral envelope, which yields finer-grained certified robustness. We further …
The problem of maximizing precision at the top of a ranked list, often dubbed Precision@k (prec@k), finds relevance in myriad learning applications such as ranking, multi-label classification, and learning with severe label imbalance. However, despite its popularity, there exist significant gaps in our understanding of…
We consider the mixed regression problem with two components, under adversarial and stochastic noise. We give a convex optimization formulation that provably recovers the true solution, and provide upper bounds on the recovery errors for both arbitrary noise and stochastic noise settings. We also give matching minimax …
New AI error correctors improve classifier performance with provable guarantees.
problem Improving AI classifier performance with scarce training data.
method Weakly supervised AI error correctors with performance guarantees.
result Provable performance guarantees for AI error correction.
PAC-Bayesian theory applied to learning optimization algorithms with generalization guarantees.
problem Learning optimization algorithms with provable generalization guarantees and explicit trade-offs.
method PAC-Bayes theory applied to learning-to-optimize, reformulating the learning procedure into a one-dimensional minimization problem.
result Learned optimization algorithms outperform deterministic worst-case analysis algorithms, even in the limit case of guaranteed convergence.
Momentum SGD fails to track nonstationary optima due to drift amplification.
problem Tracking nonstationary optima in stochastic optimization.
method Theoretical analysis of SGD and momentum variants under strong convexity and smoothness.
result Momentum incurs a drift-amplification penalty that diverges as the momentum parameter approaches 1, leading to systematic lag.
New SVM margin bound improves generalization in machine learning.
problem Improving SVM margin bounds for better generalization.
method Stable sample compression schemes to derive new data-dependent generalization bounds.
result Proves a new optimal SVM margin bound with a log factor improvement.
It has been shown that neural network classifiers are not robust. This raises concerns about their usage in safety-critical systems. We propose in this paper a regularization scheme for ReLU networks which provably improves the robustness of the classifier by maximizing the linear regions of the classifier as well as t…