Novel tensor perturbation bounds for orthogonal iteration methods.
problem Developing robust bounds for tensor reconstruction and subspace estimation.
method Blockwise tensor perturbation bounds for high-order orthogonal iteration (HOOI).
result Upper bounds for singular subspace estimation converge linearly and tensor reconstruction error bound is characterized by a simple quantity.
Unified analysis of perturbation-based strategies in stochastic and adversarial bandit problems.
problem Optimality of perturbation-based strategies in multi-armed bandit problems.
method Unified regret analysis for stochastic and adversarial settings, using perturbations of sub-Weibull and bounded support.
result Unified bounds for perturbations in both stochastic and adversarial settings, with optimal perturbations of Frechet-type.
The paper analyzes how quantum PageRank changes with small perturbations.
problem Estimating sensitivity of quantum PageRank to small changes.
method Finite dimensional perturbation theory to estimate changes and bounds.
result Estimation of lower bound of convergence radius and error bounds.
Paper bounds subspace estimator error from noisy projections.
problem Estimating subspaces from noisy data.
method Derives perturbation bound on optimal subspace estimator.
result Fundamental result with implications in matrix completion and clustering.
New method defends deep nets against large perturbations perceptible to humans.
problem Vulnerability of deep nets to adversarial attacks with perceptible but not changing predictions.
method Oracle-Aligned Adversarial Training (OA-AT) to align network predictions with Oracle's.
result Achieves state-of-the-art performance at large perturbation bounds (L-inf of 16/255 on CIFAR-10).
Paper improves variational inference by tightening bounds using perturbation theory.
problem Improving variational inference's bias and KL divergence approximation.
method Revisits perturbation theory to derive corrections that tighten variational bounds.
result New bounds are tighter and more mass-covering, leading to higher likelihoods.
FTPL with Fréchet perturbation achieves near optimal regret bounds for m-set semi-bandit problems.
problem Optimizing regret bounds for m-set semi-bandit problems in adversarial and stochastic settings.
method Follow-the-Perturbed-Leader (FTPL) with Fréchet perturbation.
result Achieves near optimal regret bounds of O ( n m ( d log ( d ) + m 5 / 6 ) ) \mathcal{O}(\sqrt{nm}(\sqrt{d\log(d)}+m^{5/6})) O ( nm ( d log ( d ) + m 5/6 )) in adversarial setting and logarithmic regret in stochastic setting. The paper provides robustness bounds for manifold learning techniques.
problem Understanding the robustness of manifold learning methods.
method Derives perturbation bounds for Procrustes, Classical Scaling, and Trilateration.
result Performance bounds for Isomap, Landmark Isomap, and Maximum Variance Unfolding are derived.
Classical matrix perturbation results, such as Weyl's theorem for eigenvalues and the Davis-Kahan theorem for eigenvectors, are general purpose. These classical bounds are tight in the worst case, but in many settings sub-optimal in the typical case. In this paper, we present perturbation bounds which consider the natu…
The paper studies heat kernels on modified manifolds and bounds their properties.
problem Bounding heat kernels on modified Riemannian manifolds.
method Derives upper bounds and gradient estimates for the heat kernel of ( M , i l d e g ) (M, ilde{g}) ( M , i l d e g ) . result Establishes upper bounds and gradient estimates for the heat kernel of modified manifolds.
Essential self-adjointness proved for perturbed quadharmonic operators on Riemannian manifolds.
problem Proving essential self-adjointness for perturbed quadharmonic operators.
method Using bounded geometry assumptions and a non-positive potential function.
result Essential self-adjointness condition established for perturbed quadharmonic operators.
This paper tackles incomplete multi-view clustering with spectral perturbation theory.
problem Realistic clustering scenario where data instances are missing in certain views.
method Spectral perturbation theory and matrix completion method for incomplete similarity matrix.
result The minimization of perturbation risk bounds maximizes the final fusion result across all views.
The higher order singular value decomposition (HOSVD) of tensors is a generalization of matrix SVD. The perturbation analysis of HOSVD under random noise is more delicate than its matrix counterpart. Recently, polynomial time algorithms have been proposed where statistically optimal estimates of the singular subspaces …
A key problem in research on adversarial examples is that vulnerability to adversarial examples is usually measured by running attack algorithms. Because the attack algorithms are not optimal, the attack algorithms are prone to overestimating the size of perturbation needed to fool the target model. In other words, the…
New approach to adversarial robustness with non-uniform perturbations.
problem Real-world adversaries craft adversarial examples with non-uniform perturbations.
method Proposes non-uniform perturbations based on feature dependencies and data distribution.
result Shows improved robustness to real-world attacks compared to uniform perturbations.
Study robust learning without knowing perturbation sets, using interactions with attackers.
problem Learning robust predictors against unknown adversarial perturbations.
method Examined different interaction models with adversarial attackers, derived bounds on sample complexity and interactions.
result Upper bounds on sample complexity and lower bounds on interactions in various models.
We improve image perturbation defenses using a better-defined Wasserstein threat model.
problem Real-world image perturbations are not pixel-independent, unlike ℓ p \ell_p ℓ p threat models. method We rectify flaws in the Wasserstein threat model and explore stronger attacks and defenses.
result Current Wasserstein-robust models are ineffective against real-world perturbations.
PAC-Bayesian bounds estimate adversarial robustness.
problem Estimating robustness to imperceptible input perturbations.
method PAC-Bayesian framework for averaging over hypotheses.
result General bounds valid for any type of adversarial attacks.
Improved online Lasso reduces regret in sparse linear contextual bandits.
problem Sparse linear contextual bandit problem with inefficient sampling.
method Perturbed adversary approach to alleviate sampling inefficiency.
result Online Lasso achieves O ( k T log d ) \mathcal{O}(\sqrt{kT\log d}) O ( k T log d ) regret bound. We propose a novel framework for the differentially private ERM, input perturbation. Existing differentially private ERM implicitly assumed that the data contributors submit their private data to a database expecting that the database invokes a differentially private mechanism for publication of the learned model. In i…
New algorithm tackles adversarial contextual bandits using stochastic smoothing.
problem Adversarial contextual bandit problems.
method Stochastic smoothing perspective and random perturbation based algorithms.
result Zero-order bound of O ( T ) O(\sqrt{T}) O ( T ) and first-order bound of O ( L T ∗ 2 / 3 ) O(L^{*2/3}_{T}) O ( L T ∗ 2/3 ) for the proposed algorithm. Deterministic bounds for tensor singular values and vectors, differing from matrix cases.
problem Spectral learning of higher-order orthogonally decomposable tensors.
method Deterministic perturbation bounds for singular values and vectors of orthogonally decomposable tensors.
result Perturbation affects each essential singular value/vector in isolation, independent of multiplicity and distance from other singular values.
Paper establishes lower bounds for Gaussian process bandit optimization under various perturbation models.
problem Lower bounds for Gaussian process bandit optimization in noisy and robust settings.
method Novel proof techniques for standard and robust settings, including deterministic strategies.
result Demonstrates inevitable joint dependence of cumulative regret on corruption level and time horizon in robust settings.
Study analyzes perturbations in singular subspaces under random noise.
problem Understanding singular vector and subspace changes in signal-plus-noise models.
method Generalized Davis-Kahan-Wedin theorem for any unitarily invariant norm, considering ℓ ∞ \ell_\infty ℓ ∞ and ℓ 2 , ∞ \ell_{2,\infty} ℓ 2 , ∞ bounds. result Fine-grained insights into singular vector and subspace perturbations, including ℓ ∞ \ell_\infty ℓ ∞ and ℓ 2 , ∞ \ell_{2,\infty} ℓ 2 , ∞ bounds. The Davis-Kahan-Wedin sin Θ \sin Θ sin Θ theorem describes how the singular subspaces of a matrix change when subjected to a small perturbation. This classic result is sharp in the worst case scenario. In this paper, we prove a stochastic version of the Davis-Kahan-Wedin sin Θ \sin Θ sin Θ theorem when the perturbation is a Gaussian rando…
Paper develops robust estimators and strategies for stochastic MABs with heavy-tailed rewards.
problem Stochastic multi-armed bandits with heavy-tailed rewards.
method Proposes a novel robust estimator and perturbation-based exploration strategy.
result Develops upper and lower regret bounds for various perturbations.
Study robust online learning with adversarial perturbations.
problem Learning robust classifiers in the presence of adversarial perturbations.
method Formulated as an online learning problem, considered both realizable and agnostic learnability, defined new dimension controlling mistake/regret bounds.
result Showed new dimension controls mistake/regret bounds, generalized to multiclass hypothesis classes.
New bounds for nearly-linear networks without training.
problem Generalization of neural networks close to linearity.
method Perturbation of linear networks to derive bounds.
result First non-vacuous bounds for neural nets.
Study shows torsion order bounds band-unlinking number for knot cobordisms.
problem Understanding bounds on knot cobordisms using torsion orders.
method Established an inequality involving local maxima, genus, and torsion orders of Khovanov homology.
result Torsion order gives a lower bound for the band-unlinking number.
The paper strengthens a theorem on crossings under linear perturbations with Hausdorff measure estimates.
problem Understanding multiple-point crossings under linear perturbations.
method Establishes a transversality theorem with Hausdorff measure estimates for exceptional parameter sets.
result Explicit upper bounds on the Hausdorff dimension of the exceptional set.
New algorithm minimizes regret in stochastic linear bandits with perturbed history.
problem Minimizing cumulative regret in stochastic linear bandits.
method Perturbed-history exploration in a linear bandit (LinPHE) algorithm.
result Achieves a O ( d n ) O(d \sqrt{n}) O ( d n ) gap-free bound on cumulative regret. The paper proves conditions for Kähler-Einstein metrics to remain Kähler-Einstein under cscK perturbations.
problem Conditions for Kähler-Einstein metrics to remain Kähler-Einstein under cscK perturbations.
method Study of constant scalar curvature Kähler (cscK) metrics on complete non-compact Kähler--Einstein manifolds.
result Sufficient conditions for a cscK perturbation of a Kähler--Einstein metric to remain Kähler--Einstein.
This paper analyzes privacy-preserving methods for sparse model optimization.
problem Privacy-preserving sparse model optimization with non-differentiable norms.
method Differential privacy techniques applied to Frank-Wolfe and objective perturbation algorithms.
result Excess risk bounds for Frank-Wolfe and objective perturbation algorithms are derived.
Our research tackles robustness to multiple perturbations in adversarial training.
problem Defenses against adversarial examples are tailored to single perturbation types and offer no guarantees for others.
method We analyze and train models robust to multiple ℓ p \ell_p ℓ p -bounded and spatial perturbations. result No model trained against multiple attacks achieves robustness competitive with individual training.
Paper analyzes GCNN sensitivity to probabilistic graph perturbations.
problem Investigating how GCNNs handle probabilistic graph errors.
method Establishes error bounds and linear relationships between GSO perturbations and GCNN outputs.
result GCNNs maintain stability under graph edge perturbations if GSO errors are bounded.
SGD generalization bounds derived from information theory.
problem Understanding generalization of SGD for non-convex functions.
method Combining information-theoretic bounds with perturbation analysis.
result Upper bounds on SGD's generalization error based on gradient variance and function smoothness.
Paper proposes an efficient algorithm to compute minimum adversarial perturbation for NN classifiers.
problem Computing the minimum adversarial perturbation for Nearest Neighbor classifiers.
method Formulated as a list of convex quadratic programming problems, solved using efficient algorithms.
result Shows dual solutions as valid lower bounds for adversarial perturbation, aiding robustness verification.
New method improves solving combinatorial optimization problems with smoothed policies.
problem Solving combinatorial optimization problems repeatedly with varying instances.
method Smoothed policies with controlled random perturbations to linear oracle, leading to differentiable surrogate risk.
result Generalization bound decomposes excess risk into bias, estimation, and optimization components.
On a smooth complete Riemannian spin manifold with smooth compact boundary, we demonstrate that the Atiyah-Singer Dirac operator D B \mathrm{D}_{\mathcal B} D B in L 2 \mathrm{L}^{2} L 2 depends Riesz continuously on L ∞ \mathrm{L}^{\infty} L ∞ perturbations of local boundary conditions B {\mathcal B} B . The Lipschitz bound for the map ${…
We prove that the Atiyah-Singer Dirac operator D g {\mathrm D}_{\mathrm g} D g in L 2 {\mathrm L}^2 L 2 depends Riesz continuously on L ∞ {\mathrm L}^{\infty} L ∞ perturbations of complete metrics g {\mathrm g} g on a smooth manifold. The Lipschitz bound for the map ${\mathrm g} \to {\mathrm D}_{\mathrm g}(1 + {\mathrm D}_{\mathrm g}^2)^{…
Study shows transfer of adversarial robustness between different perturbation types is limited.
problem Understanding adversarial robustness across various perturbation types.
method Evaluated 32 attacks of 5 different types on models trained on a subset of ImageNet.
result Adversarial robustness transfer between perturbation types is limited and depends on the specific type of perturbation.
Despite achieving impressive performance, state-of-the-art classifiers remain highly vulnerable to small, imperceptible, adversarial perturbations. This vulnerability has proven empirically to be very intricate to address. In this paper, we study the phenomenon of adversarial perturbations under the assumption that the…
TULiP estimates uncertainty for deep learning models safely.
problem Reliable uncertainty estimation for deep learning models in the open world.
method TULiP considers a hypothetical perturbation, bounds its effect, and computes uncertainty from sampled predictions.
result TULiP achieves state-of-the-art performance in OOD detection benchmarks.
The paper analyzes how quantization affects the Fisher Information Matrix's dominant eigenvalue.
problem The impact of quantization on the Fisher Information Matrix's dominant eigenvalue.
method The study examines spectral perturbation of the empirical Fisher Information Matrix under in-distribution input and quantized parameter perturbations.
result A bound on the eigenvalue under quantization noise, showing it strictly exceeds the unperturbed value at leading order.
This paper presents a new approach, called perturb-max, for high-dimensional statistical inference that is based on applying random perturbations followed by optimization. This framework injects randomness to maximum a-posteriori (MAP) predictors by randomly perturbing the potential function for the input. A classic re…
Unified algorithm for linear bandits with improved regret bound.
problem Adversarial linear bandits with improved regret.
method Self-concordant perturbations in FTPL framework.
result Regret bound of O ( d n ln n ) \mathcal{O}(d\sqrt{n \ln n}) O ( d n ln n ) for hypercube and ℓ 2 \ell_2 ℓ 2 ball. Defense against adversarial examples using k-NN on neural network activations.
problem Adversarial examples that fool machine learning models.
method k-Nearest Neighbor (kNN) on intermediate activations of neural networks.
result Significantly outperforms state-of-the-art defenses on MNIST and CIFAR-10.
Estimates mass of static vacuum metrics with small Bartnik data.
problem Estimating mass of static vacuum metrics with small perturbations.
method Second-order mass estimation using Bartnik data.
result New upper bound on Bartnik mass to fifth order.