New black-box reductions simplify online learning algorithms.
problem Designing adaptive and parameter-free online learning algorithms.
method Introducing black-box reductions to simplify analysis and improve regret guarantees.
result Improved regret bounds for parameter-free learning.
New method defends machine learning models from black-box attacks.
problem Protecting machine learning models from black-box transfer attacks.
method Randomized ensemble technique with provable security guarantee.
result Empirical evidence of security for random binary classifiers.
TVR optimizes black-box simulators by targeting variance reduction over control and noise parameters.
problem Optimizing black-box simulators with uncertain parameters.
method Targeted Variance Reduction (TVR) method that optimizes (x,θ) jointly. result Improved robust optimization performance over state-of-the-art methods.
New analysis improves black-box k-PCA algorithms, reducing parameter loss.
problem Designing efficient k-PCA algorithms with black-box access to a 1-PCA oracle. method Black-box deflation methods, analyzing ePCA and cPCA approximations.
result Deflation methods suffer no asymptotic parameter loss for k-cPCA in feasible regimes. A new optimization method reduces variance in derivative-free updates.
problem Derivative-free optimization for black-box models with high dimensionality.
method Stochastic Zeroth-order method with Variance Reduction (SZVR-G).
result The method achieves sublinear complexity to problem dimensionality, improving over existing techniques.
New method turns optimization algorithms into uniformly stable learning algorithms for non-Euclidean norms.
problem Non-Euclidean norms in binary classification problems.
method Black-box reduction method using uniformly convex regularizers.
result Achieves optimal statistical risk bounds on excess risk for non-Euclidean norms.
Proposes variance reduction for optimizing permutation models.
problem High variance in gradient estimates for discrete latent variables.
method Control variates for the Plackett-Luce distribution.
result Optimization of black-box functions over permutations using SGD.
New approach turns optimal stationary RL into non-stationary RL without prior knowledge.
problem Optimal RL in non-stationary environments without prior knowledge of non-stationarity.
method Black-box reduction of optimal stationary RL algorithms to non-stationary RL.
result Achieves optimal dynamic regret bounds in various RL settings.
A new method reduces variance in black-box variational inference.
problem High variance in black-box variational inference.
method Importance sampling from an overdispersed distribution.
result Effective reduction in variance with negligible overhead.
We generalize stochastic smoothing for gradient estimation of non-differentiable functions.
problem Gradient estimation for non-differentiable functions.
method Developed a general framework for relaxation and gradient estimation of non-differentiable black-box functions using stochastic smoothing with reduced assumptions.
result Empirically validated the effectiveness of variance reduction strategies for various non-differentiable tasks.
Optimal reductions apply machine learning methods to various optimization tasks.
problem Applying machine learning methods to diverse optimization tasks.
method Develops optimal and practical reductions between different optimization objectives.
result New reductions lead to faster training times for linear classifiers.
New method for black-box adversarial attacks using pretrained model embeddings.
problem Efficiently attacking unknown target networks with high-level semantic patterns.
method Learn a low-dimensional embedding using a pretrained model, then search within the embedding space.
result Significant reduction in the number of queries for black-box adversarial attacks.
Efficient method reduces black-box adversarial queries for neural networks.
problem Solving for adversarial examples in black-box settings with limited query access.
method Efficient combinatorial optimization surrogate for gradient estimation.
result State-of-the-art black-box attack performance with reduced query count.
New method wraps black-box classifiers to reduce bias.
problem Reduction of bias in black-box predictions.
method Post-processing with α-trees and boosting algorithms.
result Demonstrated effectiveness in reducing bias across multiple fairness metrics.
Proposes a method to stabilize Black Box Variational Inference using the James-Stein estimator.
problem Stability issues and fine-tuning required in basic Black Box Variational Inference.
method Reframe stochastic gradient ascent as multivariate estimation problem using James-Stein estimator.
result Provides a simpler method with consistent performance in terms of model fit and convergence time.
Paper tackles gradient-free minimax optimization with variance reduction for faster convergence.
problem Gradient-free minimax optimization problems in machine learning.
method Variance reduction technique to design a novel zeroth-order gradient descent ascent algorithm.
result Achieves the best known query complexity of O(κ(d₁ + d₂)ε⁻³), outperforming previous methods.
Private learning can be used to efficiently solve online learning problems.
problem The relationship between differentially private learning and online learning efficiency.
method Derive an efficient black-box reduction from differentially private learning to online learning from expert advice.
result An efficient differentially private learner implies an efficient online learner.
Paper presents ZO-SVRG for faster nonconvex optimization.
problem Gradient-free optimization challenges in nonconvex settings.
method Comprehensive theoretical analysis, novel ZO-SVRG algorithm, accelerated versions.
result ZO-SVRG achieves best rate for ZO stochastic optimization.
This work surveys unsupervised learning methods for high-dimensional uncertainty quantification in complex PDEs.
problem Uncertainty quantification in high-dimensional stochastic inputs of complex PDEs.
method Review and investigation of thirteen dimension reduction methods including linear and nonlinear, spectral, blind source separation, convex and non-convex methods.
result Manifold PCE (m-PCE) provides a cost-effective approach compared to deep neural network-based surrogates.
Deep learning reduces complex data to simpler predictors.
problem High-dimensional data reduction in input-output models.
method Hierarchical layers of latent features for constructing predictors.
result Deep learning is a black-box method for high-dimensional function estimation.
AutoZOOM reduces black-box attack query counts by 93% on MNIST, CIFAR-10, and ImageNet.
problem Efficiently attack black-box neural networks with minimal model queries.
method AutoZOOM uses an autoencoder and adaptive gradient estimation for query-efficient black-box attacks.
result Significant reduction in model queries (93%) without sacrificing attack success rate and visual quality.
Paper presents a reduction-based framework for conservative bandits and RL with improved lower and upper bounds.
problem Conservative bandits and reinforcement learning problems.
method Reduction technique to calculate necessary and sufficient budget from baseline policy.
result Improved lower and upper bounds for various conservative settings.
Paper proposes a neural network for learning better importance sampling.
problem Improving variance reduction in Monte Carlo rendering.
method Uses a neural network to learn desired densities in the primary sample space of a rendering algorithm.
result Effective variance reduction demonstrated in practical scenarios.
Paper presents efficient IS for tail risk estimation with machine learning features.
problem Estimating Value at Risk and Conditional Value at Risk with black-box access.
method Efficient Importance Sampling algorithm with self-structuring transformation.
result Asymptotically optimal variance reduction in logarithmic scale.
New approach tackles non-stationary multi-agent games with black-box methods.
problem Challenges in learning equilibria in non-stationary multi-agent systems.
method Versatile black-box approach applicable to various games, including general-sum, potential, and Markov games.
result Achieves optimal regret bounds for non-stationary games, with or without knowledge of total variation.
Sparse perturbations improve convergence in SZO methods for faster training.
problem Dependency of SZO methods on function dimensionality limits their convergence speed.
method Sparse perturbations reduce the effective dimensionality of the optimization problem.
result Sparse SZO optimization leads to faster convergence in training loss and test accuracy.
Improves scalability and efficiency of mixture models in black-box variational inference.
problem Scaling mixture models in black-box variational inference leads to high parameter and time costs.
method Introduces MISVAE for amortized mixture parameter space and new ELBO estimators.
result Achieves superior estimation performance with fewer parameters and shorter inference time.
Kalman Gradient Descent optimizes machine learning models by reducing variance in stochastic optimization.
problem Reducing variance in stochastic gradient descent to improve optimization performance.
method Uses Kalman filtering to adaptively reduce gradient variance in stochastic gradient descent.
result Improved performance on various machine learning tasks including neural networks and black box variational inference.
ZOO attack bypasses substitute models for black-box DNN attacks.
problem Vulnerability of DNNs to adversarial examples.
method Zeroth order optimization for gradient estimation.
result ZOO attack outperforms existing black-box attacks.
Bayesian optimization finds game equilibria efficiently.
problem Finding game equilibria in derivative-free, expensive contexts.
method Bayesian optimization framework with sequential sampling decisions.
result Equilibria can be found reliably at a lower cost.
Paper proposes hybrid approach for transparent credit scoring models.
problem Lack of transparency in machine learning models limits their use in regulated environments.
method Post-hoc interpretation of black-box models guides feature selection, followed by training glass-box models.
result Reduces feature usage from 106 to 10 while maintaining comparable performance.
Optimizes nonconvex optimization by converting it to static regret minimization.
problem Nonconvex optimization challenges in machine learning.
method Black-box online-to-nonconvex conversion with static regret minimization oracles.
result Achieves optimal convergence rates for nonconvex optimization.
Paper introduces a new IS scheme for estimating distribution tails of complex models.
problem Scalability and feasibility issues in traditional IS schemes for rich models.
method Develops a self-structuring IS approach guided by large deviations principles.
result First to achieve asymptotically optimal variance reduction across various multivariate distributions.
Develops an SSBO algorithm for global optimization of expensive models.
problem Global optimization of expensive black-box models.
method Asynchronous hybrid-criterion with interval reduction.
result Improves global search ability and local search efficiency.
DRLViz interprets deep RL agent memory for better understanding.
problem Understanding complex deep RL agent memory.
method Visual analytics interface to reduce and interpret memory vectors.
result Experts can better understand and investigate agent decisions.
BasisVAE combines VAE and clustering for tabular data analysis.
problem Lack of insights in tabular high-dimensional data analysis.
method Combines VAE with probabilistic clustering prior for joint dimensionality reduction and clustering.
result Learned one-hot basis function representation for translation-invariant features.
New method extends surrogate modeling to high dimensions using dimensionality reduction.
problem Curse of dimensionality limits surrogate models to low dimensions.
method Combines Kriging, polynomial chaos expansions, and kernel PCA for high-dimensional problems.
result High-dimensional problems can be solved using the proposed method.
Ensemble of diverse CNNs detects and mitigates adversarial attacks.
problem Detecting and defending against adversarial attacks.
method An ensemble of specialized CNNs with a voting mechanism.
result Significant reduction in adversarial attack risk rate.
DarkSight visualizes deep classifiers more effectively than t-SNE.
problem Interpreting black box classifiers like deep networks.
method DarkSight embeds data points into a low-dimensional space to compress deep classifiers, using dark knowledge for a new confidence measure.
result DarkSight visualizations are more informative and yield a new confidence measure.
PESMOC optimizes multiple expensive functions with constraints using entropy reduction.
problem Simultaneous optimization of multiple expensive functions with constraints.
method Iterative entropy reduction strategy based on predictive entropy search.
result PESMOC provides better recommendations with fewer evaluations than random search.
A framework reduces bias in sampling from posterior distributions.
problem Reducing bias in sampling from posterior distributions.
method A black-box debiasing scheme generating weighted samples.
result Improves accuracy of posterior sampling without increasing variance.
Efficient distributed SGD method improves training speed and robustness.
problem Scaling up stochastic optimization for large-scale data.
method Combining adaptivity and variance reduction techniques for distributed SGD.
result Achieves linear speedup, constant memory, and logarithmic communication rounds.
New algorithms improve privacy in statistical estimation by making them robust.
problem Improving privacy in statistical estimation methods.
method Black-box reduction from privacy to robustness, using Sum-of-Squares method.
result Design of polynomial-time private estimators with optimal tradeoffs among sample complexity, accuracy, and privacy.
VR-ConfTr reduces noise in CP training, leading to more stable and efficient model performance.
problem Improving the efficiency and stability of conformal prediction during model training.
method Variance-reduced conformal training (VR-ConfTr) that incorporates variance reduction in gradient estimation.
result VR-ConfTr achieves faster convergence and smaller prediction sets compared to existing methods.
New method learns low-dimensional models for systems with non-polynomial terms.
problem Modeling systems with non-polynomial nonlinear terms that are spatially local and given in analytic form.
method Non-intrusive model reduction method that learns operators for linear and polynomially nonlinear dynamics via a least-squares problem incorporating given non-polynomial terms.
result Comparable accuracy to intrusive methods that require full knowledge of governing equations.
A new method for optimizing functions without gradients, improving efficiency and convergence.
problem Optimizing functions without gradient information in machine learning.
method Hybrid Gradient Descent (HGE) using random and coordinate-wise gradient estimates.
result The proposed method achieves optimal convergence rates in convex cases and generalizes to non-convex cases.
Paper tackles LDP bandits learning with improved results and sub-linear regret.
problem Contextual bandits learning with LDP privacy constraints.
method Simple black-box reduction frameworks for context-free bandits, extended to GLB.
result First result for BCO with multi-point feedback under LDP, sub-linear regret for GLB.
A new tree-based model for multivariate responses interprets piecewise linear regimes.
problem Recovering piecewise multivariate linear regimes in complex data.
method Twoblock clustering trees with coskewness-based dimension reduction.
result Recovery of piecewise linear regimes in data.