Improves black-box adversarial attacks with a transfer-based prior.
problem Low attack success rates and poor query efficiency in black-box adversarial attacks.
method P-RGF method that integrates a transfer-based prior and query information.
result Significantly reduces the number of queries needed for successful attacks.
Improved deep learning models using new attribution priors and expected gradients.
problem Improving interpretability and performance of deep learning models.
method Introducing new attribution priors and expected gradients method that satisfies interpretability axioms.
result Improves model performance across various real-world tasks.
This paper analyzes and guarantees convergence of prior-guided ZO algorithms.
problem Understanding convergence properties of prior-guided zeroth-order optimization algorithms.
method Analysis of convergence under a greedy descent framework with various gradient estimators, and development of ARS algorithm.
result Convergence guarantee for prior-guided random gradient-free (PRGF) algorithms and accelerated random search (ARS) algorithm.
The paper deals with learning probability distributions of observed data by artificial neural networks. We suggest a so-called gradient conjugate prior (GCP) update appropriate for neural networks, which is a modification of the classical Bayesian update for conjugate priors. We establish a connection between the gradi…
New methods improve black-box adversarial attacks with fewer queries.
problem Efficiently craft adversarial examples without access to gradients.
method Prior-guided random gradient-free algorithms based on biased sampling and gradient averaging.
result Both methods require fewer queries and achieve higher success rates.
K-priors enable quick adaptation with minimal retraining.
problem Machine learning models struggle to adapt to changes efficiently.
method Combines weight and function-space priors to reconstruct past gradients.
result Adaptation with K-priors achieves similar performance to full retraining with less data.
New method improves black-box adversarial attacks with gradient priors.
problem Generating adversarial examples in a black-box setting.
method Bandit optimization-based algorithm with gradient priors.
result Improves black-box attacks by using gradient priors, reducing queries and failures.
Bayesian approach improves deep image prior for image reconstruction.
problem Improving performance of deep image prior for image reconstruction tasks.
method Derive Bayesian approach using stochastic gradient Langevin, showing asymptotic equivalence to Gaussian process prior.
result Improves denoising and impainting results for image reconstruction tasks.
Paper uses SGLD to recover signals from generative models, proving convergence under mild conditions.
problem Signal recovery from generative priors in compressed sensing.
method Stochastic Gradient Langevin Dynamics (SGLD) for signal recovery.
result SGLD converges to the true signal under mild assumptions on the generative model.
Bayesian priors for neural networks are improved by incorporating weight correlations and tail behavior.
problem Improving Bayesian priors for neural networks to better reflect true beliefs and performance.
method Analyzed summary statistics of neural network weights in different architectures and incorporated these observations into new priors.
result Improved performance on image classification datasets by using new priors that account for weight correlations and tail behavior.
New method uses diffusion models to solve inverse problems.
problem Solving ill-posed inverse problems with powerful priors.
method Formulate posterior sampling as a regularized Wasserstein gradient flow in latent space.
result Demonstrates improved performance on standard benchmarks.
DP-RandP improves privacy-utility tradeoff in DP-SGD by learning priors from random processes.
problem Improving the performance of differentially private stochastic gradient descent (DP-SGD) on private data.
method A three-phase approach that learns priors from images generated by random processes and transfers these priors to private data.
result New state-of-the-art accuracy on CIFAR10, CIFAR100, MedMNIST, and ImageNet for various privacy budgets.
Proposes using equivariant generative models for compressed sensing with unknown orientations.
problem Recovering signals with unknown orientations from underdetermined systems of linear measurements.
method Equivariant variational autoencoder as a generative prior for compressed sensing.
result Signals with unknown orientations can be recovered using iterative gradient descent on the latent space of equivariant models.
New framework uses deep generative priors for robust phase retrieval.
problem Highly ill-posed and non-linear phase retrieval problem.
method Regularization through deep generative priors with gradient descent.
result Effective for random Gaussian and Fourier friendly measurements.
New approach to neural networks by incorporating observation noise and arbitrary prior means.
problem Misspecification on noisy data and limitations of NTK-GP equivalence.
method Introducing a regularizer for observation noise and proposing a shifted network for arbitrary prior means.
result Removes key obstacles to practical Gaussian process modeling in neural networks.
New algorithms adapt to both gradient norms and comparator norms in online learning.
problem Adapting to both gradient norms and comparator norms in online learning.
method Developed parameter-free and scale-free algorithms for unbounded online convex optimization.
result Improved regret bounds for scale-invariant online prediction with linear models.
Study MAP estimation for PnP priors with SGD, proving convergence and demonstrating practical applications.
problem Theoretical analysis and practical implementation of PnP priors for Bayesian imaging problems.
method Maximum-a-posteriori estimation with Plug & Play priors and stochastic gradient descent.
result Convergence proof for MAP computation by PnP-SGD under realistic assumptions on the denoiser.
Develops a simulation-based method to translate expert knowledge into prior distributions for Bayesian models.
problem Effective incorporation of expert knowledge into prior distributions for diverse model structures.
method Simulation-based stochastic gradient descent to learn hyperparameters of parametric priors from expert knowledge.
result Method is adaptable to various elicitation techniques and independent of model structure.
New method uses deep learning to solve linear inverse problems.
problem Solving linear inverse problems with high-dimensional signals.
method Stochastic coarse-to-fine gradient ascent procedure using implicit prior from denoising CNN.
result General algorithm for solving linear inverse problems without additional training.
New algorithm uses untrained neural networks for image recovery, offering better compression.
problem Using untrained neural networks for image recovery and theoretical guarantees.
method Projected gradient descent scheme for solving linear and non-linear inverse problems.
result The method achieves better compression rates for the same image quality compared to hand-crafted priors.
SAP learns efficient task-specific parameter subspaces for few-shot learning.
problem Efficient few-shot learning with limited data.
method Subspace Adaptation Prior (SAP) learns task-specific parameter subspaces for efficient few-shot learning.
result SAP yields superior or competitive performance in few-shot image classification.
Bayesian framework inserts prior knowledge in RL for faster task solving.
problem Faster transfer learning across reinforcement learning tasks.
method Bayesian posterior distribution combining task-specific and prior knowledge.
result Significant speed ups achieved in maze solving.
The paper proposes a method to learn differentially private variational autoencoders with term-wise gradient aggregation.
problem Learning variational autoencoders with differential privacy constraints and multiple divergences.
method Term-wise Differentially Private SGD (DP-SGD) that crafts randomized gradients for each loss term, keeping sensitivity at O(1).
result The method reduces the amount of noise needed for differential privacy, allowing better learning.
Multimodal MAML adapts faster to tasks from a diverse distribution.
problem Limitations of gradient-based meta-learners in learning from multimodal task distributions.
method Augmenting MAML with a capability to identify and adapt to tasks from a multimodal distribution.
result Multimodal MAML adapts faster to tasks from a multimodal distribution.
New priors for deep neural networks converge to Gaussian processes.
problem Improving the performance and stability of deep neural networks.
method Extending prior distributions to include non-zero means and partially exchangeable priors, leading to a new Gaussian process model.
result The new Gaussian process model avoids pathologies and improves performance on regression problems.
Two new estimators improve VAE training for hierarchical and prior parameters.
problem Efficient gradient estimation for VAEs with hierarchical and prior parameters.
method Developed two generalizations of Doubly-Reparameterized Gradient Estimators (DReGs) for VAEs.
result Improved training of conditional and hierarchical VAEs on image modeling tasks.
The vast majority of optimization and online learning algorithms today require some prior information about the data (often in the form of bounds on gradients or on the optimal parameter value). When this information is not available, these algorithms require laborious manual tuning of various hyperparameters, motivati…
This paper distills financial indicators into neural networks to reduce noise and improve accuracy.
problem Reduction of non-stationary noise in financial time series data.
method Co-distillation of smaller networks trained on indicators to transfer prior knowledge and reduce overfitting.
result The proposed method outperforms traditional methods in terms of speed and accuracy on real financial datasets.
This paper solves quadratic systems with sparse or generative priors.
problem Recovering signals from quadratic systems with full-rank matrices.
method Thresholded Wirtinger flow (TWF) and projected gradient descent (PGD) algorithms.
result The proposed methods significantly outperform existing algorithms in signal recovery.
New method makes variational inference robust for Bayesian neural networks.
problem Fragility of variational Bayes in neural networks.
method Deterministic approximation of moments and hierarchical prior selection.
result Good predictive performance in heteroscedastic regression.
Transformer-based method for causal discovery with prior knowledge integration.
problem Complex nonlinear dependencies and spurious correlations in time series data.
method Multi-layer Transformer forecaster with gradient-based causal structure extraction and attention masking for prior knowledge integration.
result Significant improvement in causal discovery and causal lag estimation compared to state-of-the-art methods.
New methods combine MALA and mGRAD for scalable Bayesian inference in high-dimensional state-space models.
problem Bayesian inference in high-dimensional state-space models with limited scalability.
method Combines gradient-based MALA and prior-informed mGRAD for scalable inference.
result Extends classical MCMC methods to handle multiple time steps and particles.
New method corrects deep neural network variance for outlier data.
problem Outliers in deep neural network outputs.
method Gradient Conjugate Priors (GCP) network.
result Corrected variance is in a small neighborhood of ground truth variance.
Adam converges with high probability under unconstrained non-convex smooth stochastic optimizations.
problem Theoretical limitations of Adam's convergence under unconstrained non-convex smooth stochastic optimizations.
method Deep analysis of Adam's convergence rate under affine variance noise, without bounded gradient assumptions.
result Adam converges to the stationary point with a high probability rate of $\mathcal{O}\left({
m poly}(\log T)/\sqrt{T}
ight)$.
New attack recovers user-level information from large batch images.
problem Recovering private information from user-level gradients in distributed learning.
method Proposes a gradient inversion attack using a denoising diffusion model as a prior.
result Demonstrates recovery of realistic facial images and private attributes.
Neural-g models mixtures of densities with flexibility and accuracy.
problem Accurately estimating prior densities in g-models. method Neural network with softmax output for valid probability densities.
result Neural-g captures various prior shapes including flat, heavy-tailed, and discontinuous.
Proposes LARS for better priors in VAEs, improving performance.
problem Underfitting in VAEs with simple priors.
method Learned Accept/Reject Sampling (LARS) with efficient prior estimation.
result LARS priors improve VAE performance on standard datasets.
P-BO reduces black-box adversarial attacks by 10x with Bayesian optimization and function prior.
problem Efficiently generating adversarial examples against black-box models.
method Prior-guided Bayesian Optimization (P-BO) with a function prior initialized from a surrogate model.
result Significantly reduces the number of queries needed for adversarial attacks.
Gradient descent reveals the exact implicit bias via dual optimization for linearly separable data.
problem Characterizing the implicit bias of gradient descent on linearly separable data.
method Primal-dual analysis with smoothed margin for general losses, and exponential loss with specific step sizes.
result Proves faster convergence rates for implicit bias and margin maximization.
Improved scalable machine learning under heavy-tailed data.
problem Machine learning scalability under heavy-tailed data without strong convexity.
method Simple robust validation sub-routine to boost confidence in gradient-based sub-processes.
result Substantial improvement in dimension dependence without strong convexity.
Gradient descent recovers low-rank matrices from corrupted measurements with double over-parameterization.
problem Robust recovery of low-rank matrices from grossly corrupted measurements.
method Gradient descent with discrepant learning rates for double over-parameterized models.
result Gradient descent with discrepant learning rates provably recovers the underlying matrix without prior knowledge on rank or sparsity.
BONG optimizes Bayesian inference online with natural gradient descent.
problem Sequential Bayesian inference in online settings.
method Bayesian online natural gradient (BONG) approach based on variational Bayes.
result BONG outperforms other online VB methods in non-conjugate settings.
Unified framework reduces NFEs for inverse problems.
problem High computational costs and degraded reconstruction quality in existing LDM-based inverse solvers.
method Consistency Regularised Gradient Flows for posterior sampling and prompt optimization.
result Significantly reduced computational cost with state-of-the-art performance.
Paper examines convergence rate of PGD for BP objective in inverse problems.
problem Optimizing ill-posed linear inverse problems using BP vs LS.
method Analysis of PGD convergence rate for BP objective, comparison with proximal gradient method.
result PGD converges faster for BP objective due to inherent properties.
fBNNs use stochastic processes for variational inference in neural networks.
problem Difficulties in specifying priors and posteriors in high-dimensional weight spaces.
method Maximize Evidence Lower Bound (ELBO) on stochastic processes, using spectral Stein gradient estimator.
result fBNNs provide reliable uncertainty estimates and extrapolate well with structured priors.
ErasureHead speeds up distributed GD with approximate gradient coding.
problem Mitigating delays in distributed gradient descent.
method Approximate gradient coding to tolerate delays.
result ErasureHead converges as quickly as GD and has faster runtime under probabilistic delays.
Gradient descent and SGD achieve low test error in specific network weight regimes.
problem Optimizing two-layer ReLU networks with standard initialization.
method Gradient flow and stochastic gradient descent, analyzing margins and weight norms.
result Gradient descent and SGD can achieve globally maximal margins under certain constraints.
New algorithm proves convergence for MAP estimation with denoisers.
problem Proving convergence of MAP estimation methods using pretrained denoisers.
method A simple gradient descent algorithm on smoothed proximal objectives.
result Algorithm provably converges to the proximal operator under log-concavity.