Enhances random forests by smoothing predictions for better performance.
problem Suboptimal performance due to piecewise constant predictions in random forests.
method Kernel-based smoothing mechanism to introduce local regularity.
result Smoothed random forest model consistently improves predictive performance.
This study improves scalability of randomized smoothing for certifying classifier robustness.
problem Certifying machine learning classifiers against adversarial attacks is challenging and scalable solutions are needed.
method The study reviews and explores randomized smoothing and its derivatives, focusing on scalability.
result The study provides theoretical guarantees and discusses scalability challenges of randomized smoothing.
New method enhances neural network robustness against adversarial attacks.
problem Enhancing neural network robustness against adversarial attacks.
method Variational framework with per-sample noise level selector.
result Enhanced empirical robustness and certified robustness.
Hierarchical randomized smoothing improves model robustness for complex data.
problem Certifying robustness on complex data (e.g. images, graphs) is challenging.
method Add random noise to a randomly selected subset of entities in a hierarchical manner.
result Hierarchical randomized smoothing yields stronger robustness guarantees with high accuracy.
Randomized smoothing is the current state-of-the-art defense with provable robustness against ℓ2 adversarial attacks. Many works have devised new randomized smoothing schemes for other metrics, such as ℓ1 or ℓ∞; however, substantial effort was needed to derive such new guarantees. This begs the q…
Extends randomized smoothing to certify robustness against various threat models and adversarial perturbations.
problem Certifying robustness of classifiers against adversarial perturbations.
method Develops a method to certify robustness against any ℓp (p∈N>0) minimized adversarial perturbation. result Randomized smoothing suffers from the curse of dimensionality, reducing effective radius as p increases. Enhances robustness of deep neural networks with randomized smoothing.
problem Improving robustness of deep neural networks against noisy inputs and adversarial attacks.
method Introduces a variance-margin trade-off approach to increase certified robust radius using pre-trained models.
result Significant improvement in certified accuracy compared to state-of-the-art methods.
We show how to turn any classifier that classifies well under Gaussian noise into a new classifier that is certifiably robust to adversarial perturbations under the ℓ2 norm. This "randomized smoothing" technique has been proposed recently in the literature, but existing guarantees are loose. We prove a tight robu…
Recent advances in machine learning (ML) algorithms, especially deep neural networks (DNNs), have demonstrated remarkable success (sometimes exceeding human-level performance) on several tasks, including face and speech recognition. However, ML algorithms are vulnerable to \emph{adversarial attacks}, such test-time, tr…
New method provides tighter robustness guarantees for adversarial attacks.
problem Ensuring robustness against adversarial attacks in machine learning models.
method Developed a Second-order Smoothing (SoS) robustness certificate using Gaussian random smoothing.
result SoS certificates are tighter and provide improved robustness on high-dimensional datasets.
RS-Del provides robustness for sequence classifiers against edit distance attacks.
problem Certifying robustness of discrete sequence classifiers against edit distance attacks.
method Randomized deletion (RS-Del) for discrete sequence classifiers, focusing on edit distance-bounded adversaries.
result Achieved a certified accuracy of 91% at an edit distance radius of 128 bytes on malware detection.
SWEEN improves certified robustness via weighted ensembling of smoothed classifiers.
problem Finding optimal base classifiers for randomized smoothing.
method Smoothed WEighted ENsembling (SWEEN) scheme.
result SWEEN achieves optimal certified robustness under mild assumptions.
Paper optimizes statistical estimation for randomized smoothing to reduce adversarial robustness certification time.
problem Efficiently estimating robustness of points against adversarial attacks.
method Developed estimation procedures using confidence sequences and randomized Clopper-Pearson intervals.
result Achieved optimal sample complexities and stronger certificates with reduced computational burden.
New framework tightens certified robustness gaps in machine learning models.
problem Persistent gap between theoretical certified robustness and empirical accuracy.
method Leverages Lipschitz continuity and novel confidence intervals.
result Improves robust accuracy, compressing the gap between theory and practice.
In spite of several notable efforts, explaining the generalization of deterministic non-smooth deep nets, e.g., ReLU-nets, has remained challenging. Existing approaches for deterministic non-smooth deep nets typically need to bound the Lipschitz constant of such deep nets but such bounds are quite large, may even incre…
Develops accelerated methods for optimization using low-dimensional projected-gradient information.
problem Optimization with low-dimensional projected-gradient information and Nesterov acceleration.
method Randomized-subspace Nesterov accelerated gradient methods for smooth convex and strongly convex optimization.
result Established accelerated oracle-complexity guarantees and unified basis for comparing sketch families.
Paper introduces a new regularization method for kernel gradient descent learning.
problem Preventing overfitting in kernel gradient descent learning.
method Random smoothing regularization as novel convolution-based smoothing kernels.
result Optimal convergence rates achieved in various function spaces.
Bounds on Gaussian approximation for neural networks with novel smoothing techniques.
problem Approximating the distribution of wide random neural networks.
method Stein's method, Gaussian smoothing, Laplacian operators, Cameron-Martin space.
result First bounds on Gaussian approximation of wide random neural networks.
Randomized classifiers have been shown to provide a promising approach for achieving certified robustness against adversarial attacks in deep learning. However, most existing methods only leverage Gaussian smoothing noise and only work for ℓ2 perturbation. We propose a general framework of adversarial certificati…
Unified smoothing for robust classification improves accuracy.
problem Improving robustness of classifiers against adversarial attacks.
method Learned smoothed densities and randomized smoothing.
result Provable robust accuracies higher than state-of-the-art defenses.
Consider a random smooth Gaussian field G(x):F→R, where F is a compact in Rd. We derive a formula for average area of a surface generated by the equation G(x)=0 and give some applications. As an auxiliary result we obtain an integral expression for area of a surface induced by zeros of a \e…
Input-dependent smoothing mitigates classical issues but suffers from the curse of dimensionality.
problem Certifiably robust classifiers with input-dependent smoothing suffer from the curse of dimensionality.
method Proposed a theoretical and practical framework for input-dependent smoothing under strict restrictions.
result Input-dependent smoothing mitigates some classical issues but is limited by the curse of dimensionality.
New method improves certified robustness for classifier confidence.
problem Certifying confidence in classifier predictions.
method Randomized smoothing with modified Neyman-Pearson lemma.
result Certified radii for prediction confidence improved.
Bayesian methods estimate regression functions on submanifolds using graph Laplacian eigenbasis.
problem Estimating regression functions on unknown smooth submanifolds.
method Random geometric graph structure, Bayesian priors based on random basis expansion in graph Laplacian eigenbasis.
result Posterior contraction rates are minimax optimal for any positive smoothness index.
New method improves adversarial training efficiency and robustness.
problem High computational costs and lack of stability in adversarial training.
method Backward smoothing for randomized smoothing of random initialization.
result Our method achieves similar model robustness as state-of-the-art methods but with significantly less training time.
The study of topological properties of random smooth maps, focusing on Kac-Rice formula and Betti numbers.
problem Topological and geometric properties of random smooth maps.
method Developed a general framework for differential geometric and topological issues of smooth Gaussian Random Fields, generalized Kac-Rice formula, applied to Kostlan random polynomials, and proved an original theorem in Differential Topology.
result The Betti numbers of the solution of a system of regular equations cannot decrease under a C0-small perturbation of the equations. Linear statistics of random zero sets are integrals of smooth differential forms over the zero set and as such are smooth analogues of the volume of the random zero set inside a fixed domain. We derive an asymptotic expansion for the variance of linear statistics of the zero divisors of random holomorphic sections of p…
New method improves robustness of large models without sacrificing accuracy.
problem Improving robustness of large pre-trained models without accuracy loss.
method Multi-scale diffusion denoised smoothing, selectively applying smoothing at multiple noise scales.
result Strong certified robustness at high noise levels with accuracy close to non-smoothed classifiers.
Study on Gaussian random fields' singularities on manifolds.
problem Understanding singularities of Gaussian random fields on manifolds.
method Computed expected values of singularities under various conditions.
result Explicit formulae for singularities under different constraints.
New insights into simple kernel smoothing reveal surprising asymptotics.
problem Understanding precise asymptotics of Nadaraya-Watson kernel smoothing.
method Using ideas from the random energy model in statistical physics.
result Sharp asymptotics for the NW predictor on the sphere.
Interesting data often concentrate on low dimensional smooth manifolds inside a high dimensional ambient space. Random projections are a simple, powerful tool for dimensionality reduction of such data. Previous works have studied bounds on how many projections are needed to accurately preserve the geometry of these man…
Random forests are a powerful method for non-parametric regression, but are limited in their ability to fit smooth signals, and can show poor predictive performance in the presence of strong, smooth effects. Taking the perspective of random forests as an adaptive kernel method, we pair the forest kernel with a local li…
The fragility of modern machine learning models has drawn a considerable amount of attention from both academia and the public. While immense interests were in either crafting adversarial attacks as a way to measure the robustness of neural networks or devising worst-case analytical robustness verification with guarant…
Improves safety region certification for smoothed classifiers without changing smoothing scheme.
problem Certified safety regions for smoothed classifiers are often small compared to optimal.
method Generalizes certified radius calculation as nested optimization problem, uses 0th-1st order information, and designs efficient estimators.
result Certified safety regions are significantly larger than current methods, achieving significant improvements on various metrics.
A new method for efficient BNC parameter estimation outperforms HDP smoothing.
problem Efficiently estimating parameters for Bayesian network classifiers to match or exceed random forest performance.
method Uses log-linear regression to approximate hierarchical Dirichlet process (HDP) smoothing, making the approach simpler and faster.
result Our method outperforms HDP smoothing while being orders of magnitude faster and competitive with random forests.
Recent works have shown the effectiveness of randomized smoothing as a scalable technique for building neural network-based classifiers that are provably robust to ℓ2-norm adversarial perturbations. In this paper, we employ adversarial training to improve the performance of randomized smoothing. We design an adap…
Study uniform convergence of random walk Laplacians to diffusion Laplacian on smooth manifolds.
problem Uniform convergence of random walk Laplacians to diffusion Laplacian on smooth manifolds.
method Analysis of random walks on geometric and directed kNN graphs, using concentration tools and differential geometry.
result Uniform convergence of kNN Laplacians to diffusion Laplacian, without continuity of transition kernel. Paper improves neural network robustness certification with tighter radii estimates.
problem Certifying neural networks' robustness against adversarial attacks.
method Advanced algorithms for discrete and continuous domains, optimizing sample size, standard deviation, and temperature.
result Significant improvement in certified test-set accuracy with tighter certified radii bounds.
This thesis predicts the distribution of smoothed zeros of random sections on line bundles.
problem Predicting the distribution of smoothed zeros of random sections on line bundles.
method Developing smoothing operators on discrete surfaces and computing the expected sum of indices on each face.
result Predictions on the distribution of smoothed section's signed zeros with multiplicity.
The paper improves Kaczmarz algorithm with momentum for linear least squares.
problem Improving convergence of the Kaczmarz algorithm for linear least squares.
method Integrates geometrically smoothed momentum into the randomized Kaczmarz algorithm.
result Proves expected error reduction in singular vector directions.
New algorithm learns halfspaces over hypercube with random bit flips.
problem Agnostic learning of Boolean halfspaces over discrete domains is computationally hard.
method Smoothed analysis with random bit flips for discrete inputs.
result First efficient algorithm for smoothed agnostic learning of halfspaces over Boolean hypercube.
We analyze convergence rates of stochastic optimization procedures for non-smooth convex optimization problems. By combining randomized smoothing techniques with accelerated gradient methods, we obtain convergence rates of stochastic optimization procedures, both in expectation and with high probability, that have opti…
New method improves optimization algorithms without Lipschitz smoothness.
problem Improving optimization algorithms in the absence of Lipschitz smoothness.
method Dual kernel conditioning (DKC) to provide dual Lipschitz continuity.
result First complexity bounds and iterate convergence for random reshuffling mirror descent.
This paper assesses Gaussian and Exponential mechanisms for certifying adversarial robustness.
problem Certifying adversarial robustness using randomized smoothing mechanisms.
method Proposes a generic framework to assess the appropriateness of randomized smoothing mechanisms.
result Gaussian mechanism is an appropriate option for certifying both ℓ2-norm and ℓ∞-norm robustness. It is well-known that classifiers are vulnerable to adversarial perturbations. To defend against adversarial perturbations, various certified robustness results have been derived. However, existing certified robustnesses are limited to top-1 predictions. In many real-world applications, top-k predictions are more rel…
Improved analysis and new algorithm for gradient-free optimization of smooth functions.
problem Minimization of highly smooth functions with noisy oracle information.
method Two zero-order projected gradient descent algorithms based on randomization over the ℓ2 and ℓ1 spheres, with improved analysis and theoretical guarantees. result Improved convergence rates and theoretical guarantees for various function classes.
Improved kernel ridge regression for large datasets using weighted random binning.
problem Efficiently approximating kernel matrices for large-scale datasets.
method Introduced weighted random binning features for locality sensitive hashing.
result Weighted random binning features generate Gaussian processes of any desired smoothness.
RESTA defends LLMs against jailbreaking attacks by adding random noise to embeddings.
problem Vulnerability of LLMs to jailbreaking attacks that generate harmful outputs.
method Adds random noise to embedding vectors and aggregates during token generation.
result RESTA achieves superior robustness versus utility tradeoffs compared to baseline defenses.