New DP optimization methods for sparse gradients, improving on existing algorithms.
problem Differentially private optimization with sparse gradients in high-dimensional settings.
method Improved bounds for mean estimation, pure- and approximate-DP algorithms for stochastic convex optimization.
result First nearly dimension-independent rates for DP optimization with sparse gradients.
Paper characterizes phase transition of TV minimization for sparse-gradient signal recovery.
problem Characterizing phase transitions of TV minimization for sparse-gradient signal recovery.
method Combines AMP conjectured phase transition curve and high-dimensional convex geometry.
result Fully characterizes the phase transition curve of TV minimization.
Variable selection and dimension reduction are two commonly adopted approaches for high-dimensional data analysis, but have traditionally been treated separately. Here we propose an integrated approach, called sparse gradient learning (SGL), for variable selection and dimension reduction via learning the gradients of t…
New sparsity operator reduces variance reduction methods' computational cost.
problem Reduce computational cost of variance reduction methods.
method Introduce random-top-k operator to estimate gradient sparsity and reduce operations per update.
result Our algorithm consistently outperforms SpiderBoost in various tasks.
signADAM++ uses confidence functions to train deep neural networks faster.
problem Training deep neural networks efficiently and effectively.
method Integrates sign operation into ADAM and introduces a confidence function.
result signADAM++ generates more sparse gradients and improves performance.
Two algorithms optimize high-dimensional convex functions using sparse gradient or function value queries.
problem Optimizing high-dimensional convex functions with sparse gradient or function value queries.
method Two algorithms: successive component/feature selection and noisy mirror descent using Lasso gradient estimates.
result Both algorithms have logarithmically dependent convergence rates on the problem's dimensionality.
Improves deep learning optimization with a new stochastic gradient method.
problem Noisy and sparse gradients in deep learning optimization.
method Proposes a family of double adaptive stochastic gradient methods (DASGrad).
result Analyzes theoretical convergence improvements and empirical validation.
Gradient-enhanced GSA uses Poincaré chaos expansions for accurate sensitivity analysis.
problem Accurately estimating Sobol' indices with limited data.
method Integrates sparse, gradient-enhanced regression with Poincaré chaos expansions for derivative-based sensitivity analysis.
result Accurately estimated Sobol' indices using limited data.
Proposes a gradient-based variable selection method for binary classification in RKHS.
problem Variable selection in high-dimensional data analysis.
method Gradient-based representation of large-margin classifier with group-lasso penalty.
result Selection consistency and risk bound of the estimated classifier.
A new algorithm estimates sparse gradients on graphs with improved risk bounds.
problem Estimating sparse gradients on graph-structured data.
method Tree-Projected Gradient Descent algorithm for gradient-sparse parameters.
result Achieves risk bound of ns∗log(1+s∗p). A hashing technique reduces deep learning costs and energy usage.
problem Expensive matrix operations in deep learning training and testing.
method Randomized hashing for maximum inner product search and adaptive dropouts.
result Reduces overall computational cost by 95%, with 1% accuracy loss.
Paper proposes Sp-GD for sparse max-affine regression with theoretical guarantees.
problem Sparse max-affine regression model selection and estimation.
method Sparse Gradient Descent (Sp-GD) initialization using sparse PCA and covering search.
result Sp-GD provides ε-accurate estimates with optimal number of observations.