We propose a new randomized ensemble technique with a provable security guarantee against black-box transfer attacks. Our proof constructs a new security problem for random binary classifiers which is easier to empirically verify and a reduction from the security of this new model to the security of the ensemble classi…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Black-box Reduction”
TVR optimizes black-box simulators by targeting variance reduction over control and noise parameters.
problem Optimizing black-box simulators with uncertain parameters.
method Targeted Variance Reduction (TVR) method that optimizes jointly.
result Improved robust optimization performance over state-of-the-art methods.
We introduce several new black-box reductions that significantly improve the design of adaptive and parameter-free online learning algorithms by simplifying analysis, improving regret guarantees, and sometimes even improving runtime. We reduce parameter-free online learning to online exp-concave optimization, we reduce…
New analysis improves black-box -PCA algorithms, reducing parameter loss.
problem Designing efficient -PCA algorithms with black-box access to a 1-PCA oracle.
method Black-box deflation methods, analyzing ePCA and cPCA approximations.
result Deflation methods suffer no asymptotic parameter loss for -cPCA in feasible regimes.
New method turns optimization algorithms into uniformly stable learning algorithms for non-Euclidean norms.
problem Non-Euclidean norms in binary classification problems.
method Black-box reduction method using uniformly convex regularizers.
result Achieves optimal statistical risk bounds on excess risk for non-Euclidean norms.
New approach turns optimal stationary RL into non-stationary RL without prior knowledge.
problem Optimal RL in non-stationary environments without prior knowledge of non-stationarity.
method Black-box reduction of optimal stationary RL algorithms to non-stationary RL.
result Achieves optimal dynamic regret bounds in various RL settings.
We generalize stochastic smoothing for gradient estimation of non-differentiable functions.
problem Gradient estimation for non-differentiable functions.
method Developed a general framework for relaxation and gradient estimation of non-differentiable black-box functions using stochastic smoothing with reduced assumptions.
result Empirically validated the effectiveness of variance reduction strategies for various non-differentiable tasks.
The diverse world of machine learning applications has given rise to a plethora of algorithms and optimization methods, finely tuned to the specific regression or classification task at hand. We reduce the complexity of algorithm design for machine learning by reductions: we develop reductions that take a method develo…
New method wraps black-box classifiers to reduce bias.
problem Reduction of bias in black-box predictions.
method Post-processing with α-trees and boosting algorithms.
result Demonstrated effectiveness in reducing bias across multiple fairness metrics.
Derivative-free optimization has become an important technique used in machine learning for optimizing black-box models. To conduct updates without explicitly computing gradient, most current approaches iteratively sample a random search direction from Gaussian distribution and compute the estimated gradient along that…
Proposes a method to stabilize Black Box Variational Inference using the James-Stein estimator.
problem Stability issues and fine-tuning required in basic Black Box Variational Inference.
method Reframe stochastic gradient ascent as multivariate estimation problem using James-Stein estimator.
result Provides a simpler method with consistent performance in terms of model fit and convergence time.
Paper tackles gradient-free minimax optimization with variance reduction for faster convergence.
problem Gradient-free minimax optimization problems in machine learning.
method Variance reduction technique to design a novel zeroth-order gradient descent ascent algorithm.
result Achieves the best known query complexity of O(κ(d₁ + d₂)ε⁻³), outperforming previous methods.
This work surveys unsupervised learning methods for high-dimensional uncertainty quantification in complex PDEs.
problem Uncertainty quantification in high-dimensional stochastic inputs of complex PDEs.
method Review and investigation of thirteen dimension reduction methods including linear and nonlinear, spectral, blind source separation, convex and non-convex methods.
result Manifold PCE (m-PCE) provides a cost-effective approach compared to deep neural network-based surrogates.
We study the relationship between the notions of differentially private learning and online learning in games. Several recent works have shown that differentially private learning implies online learning, but an open problem of Neel, Roth, and Wu \cite{NeelAaronRoth2018} asks whether this implication is {\it efficient}…
We present a new method for black-box adversarial attack. Unlike previous methods that combined transfer-based and scored-based methods by using the gradient or initialization of a surrogate white-box model, this new method tries to learn a low-dimensional embedding using a pretrained model, and then performs efficient…
Paper presents a reduction-based framework for conservative bandits and RL with improved lower and upper bounds.
problem Conservative bandits and reinforcement learning problems.
method Reduction technique to calculate necessary and sufficient budget from baseline policy.
result Improved lower and upper bounds for various conservative settings.
Solving for adversarial examples with projected gradient descent has been demonstrated to be highly effective in fooling the neural network based classifiers. However, in the black-box setting, the attacker is limited only to the query access to the network and solving for a successful adversarial example becomes much …
Learning models with discrete latent variables using stochastic gradient descent remains a challenge due to the high variance of gradient estimates. Modern variance reduction techniques mostly consider categorical distributions and have limited applicability when the number of possible outcomes becomes large. In this w…
AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networkscs.CV
Recent studies have shown that adversarial examples in state-of-the-art image classifiers trained by deep neural networks (DNN) can be easily generated when the target model is transparent to an attacker, known as the white-box setting. However, when attacking a deployed machine learning service, one can only acquire t…
Paper presents efficient IS for tail risk estimation with machine learning features.
problem Estimating Value at Risk and Conditional Value at Risk with black-box access.
method Efficient Importance Sampling algorithm with self-structuring transformation.
result Asymptotically optimal variance reduction in logarithmic scale.
New approach tackles non-stationary multi-agent games with black-box methods.
problem Challenges in learning equilibria in non-stationary multi-agent systems.
method Versatile black-box approach applicable to various games, including general-sum, potential, and Markov games.
result Achieves optimal regret bounds for non-stationary games, with or without knowledge of total variation.
Sparse perturbations improve convergence in SZO methods for faster training.
problem Dependency of SZO methods on function dimensionality limits their convergence speed.
method Sparse perturbations reduce the effective dimensionality of the optimization problem.
result Sparse SZO optimization leads to faster convergence in training loss and test accuracy.
Improves scalability and efficiency of mixture models in black-box variational inference.
problem Scaling mixture models in black-box variational inference leads to high parameter and time costs.
method Introduces MISVAE for amortized mixture parameter space and new ELBO estimators.
result Achieves superior estimation performance with fewer parameters and shorter inference time.
Importance sampling is one of the most widely used variance reduction strategies in Monte Carlo rendering. In this paper, we propose a novel importance sampling technique that uses a neural network to learn how to sample from a desired density represented by a set of samples. Our approach considers an existing Monte Ca…
Paper proposes hybrid approach for transparent credit scoring models.
problem Lack of transparency in machine learning models limits their use in regulated environments.
method Post-hoc interpretation of black-box models guides feature selection, followed by training glass-box models.
result Reduces feature usage from 106 to 10 while maintaining comparable performance.
Optimizes nonconvex optimization by converting it to static regret minimization.
problem Nonconvex optimization challenges in machine learning.
method Black-box online-to-nonconvex conversion with static regret minimization oracles.
result Achieves optimal convergence rates for nonconvex optimization.
Paper introduces a new IS scheme for estimating distribution tails of complex models.
problem Scalability and feasibility issues in traditional IS schemes for rich models.
method Develops a self-structuring IS approach guided by large deviations principles.
result First to achieve asymptotically optimal variance reduction across various multivariate distributions.
BasisVAE combines VAE and clustering for tabular data analysis.
problem Lack of insights in tabular high-dimensional data analysis.
method Combines VAE with probabilistic clustering prior for joint dimensionality reduction and clustering.
result Learned one-hot basis function representation for translation-invariant features.
Deep neural networks (DNNs) are one of the most prominent technologies of our time, as they achieve state-of-the-art performance in many machine learning tasks, including but not limited to image classification, text mining, and speech processing. However, recent research on DNNs has indicated ever-increasing concern o…
Ensemble of diverse CNNs detects and mitigates adversarial attacks.
problem Detecting and defending against adversarial attacks.
method An ensemble of specialized CNNs with a voting mechanism.
result Significant reduction in adversarial attack risk rate.
We introduce Kalman Gradient Descent, a stochastic optimization algorithm that uses Kalman filtering to adaptively reduce gradient variance in stochastic gradient descent by filtering the gradient estimates. We present both a theoretical analysis of convergence in a non-convex setting and experimental results which dem…
Deep Learningstat.ML
Deep learning (DL) is a high dimensional data reduction technique for constructing high-dimensional predictors in input-output models. DL is a form of machine learning that uses hierarchical layers of latent features. In this article, we review the state-of-the-art of deep learning from a modeling and algorithmic persp…
Game theory finds nowadays a broad range of applications in engineering and machine learning. However, in a derivative-free, expensive black-box context, very few algorithmic solutions are available to find game equilibria. Here, we propose a novel Gaussian-process based approach for solving games in this context. We f…
A framework reduces bias in sampling from posterior distributions.
problem Reducing bias in sampling from posterior distributions.
method A black-box debiasing scheme generating weighted samples.
result Improves accuracy of posterior sampling without increasing variance.
New algorithms improve privacy in statistical estimation by making them robust.
problem Improving privacy in statistical estimation methods.
method Black-box reduction from privacy to robustness, using Sum-of-Squares method.
result Design of polynomial-time private estimators with optimal tradeoffs among sample complexity, accuracy, and privacy.
VR-ConfTr reduces noise in CP training, leading to more stable and efficient model performance.
problem Improving the efficiency and stability of conformal prediction during model training.
method Variance-reduced conformal training (VR-ConfTr) that incorporates variance reduction in gradient estimation.
result VR-ConfTr achieves faster convergence and smaller prediction sets compared to existing methods.
Operator inference for non-intrusive model reduction of systems with non-polynomial nonlinear termsmath.NA
New method learns low-dimensional models for systems with non-polynomial terms.
problem Modeling systems with non-polynomial nonlinear terms that are spatially local and given in analytic form.
method Non-intrusive model reduction method that learns operators for linear and polynomially nonlinear dynamics via a least-squares problem incorporating given non-polynomial terms.
result Comparable accuracy to intrusive methods that require full knowledge of governing equations.
Thanks to their versatility, ease of deployment and high-performance, surrogate models have become staple tools in the arsenal of uncertainty quantification (UQ). From local interpolants to global spectral decompositions, surrogates are characterised by their ability to efficiently emulate complex computational models …
We present DRLViz, a visual analytics interface to interpret the internal memory of an agent (e.g. a robot) trained using deep reinforcement learning. This memory is composed of large temporal vectors updated when the agent moves in an environment and is not trivial to understand due to the number of dimensions, depend…
Interpreting black box classifiers, such as deep networks, allows an analyst to validate a classifier before it is deployed in a high-stakes setting. A natural idea is to visualize the deep network's representations, so as to "see what the network sees". In this paper, we demonstrate that standard dimension reduction m…
A new method for optimizing functions without gradients, improving efficiency and convergence.
problem Optimizing functions without gradient information in machine learning.
method Hybrid Gradient Descent (HGE) using random and coordinate-wise gradient estimates.
result The proposed method achieves optimal convergence rates in convex cases and generalizes to non-convex cases.
Paper tackles LDP bandits learning with improved results and sub-linear regret.
problem Contextual bandits learning with LDP privacy constraints.
method Simple black-box reduction frameworks for context-free bandits, extended to GLB.
result First result for BCO with multi-point feedback under LDP, sub-linear regret for GLB.
New framework for resilient bi-criteria optimization under noisy feedback.
problem Bi-criteria combinatorial optimization with noisy function evaluations.
method Introducing -resilience and developing a black-box framework.
result Achieves sublinear regret and constraint violation for bi-criteria bandit problems.
Stochastic convex optimization algorithms are the most popular way to train machine learning models on large-scale data. Scaling up the training process of these models is crucial, but the most popular algorithm, Stochastic Gradient Descent (SGD), is a serial method that is surprisingly hard to parallelize. In this pap…
We present CYCLADES, a general framework for parallelizing stochastic optimization algorithms in a shared memory setting. CYCLADES is asynchronous during shared model updates, and requires no memory locking mechanisms, similar to HOGWILD!-type algorithms. Unlike HOGWILD!, CYCLADES introduces no conflicts during the par…
Through the lens of information-theoretic reductions, we examine a reductions approach to fair optimization and learning where a black-box optimizer is used to learn a fair model for classification or regression. Quantifying the complexity, both statistically and computationally, of making such models satisfy the rigor…
SILBO optimizes high-dimensional Bayesian optimization using semi-supervised embedding learning.
problem Bayesian optimization struggles with high-dimensional search spaces.
method SILBO uses semi-supervised dimension reduction to find a low-dimensional space for iterative optimization.
result SILBO outperforms existing methods on high-dimensional Bayesian optimization tasks.
As application demands for zeroth-order (gradient-free) optimization accelerate, the need for variance reduced and faster converging approaches is also intensifying. This paper addresses these challenges by presenting: a) a comprehensive theoretical analysis of variance reduced zeroth-order (ZO) optimization, b) a nove…