In this paper, a new sequential surrogate-based optimization (SSBO) algorithm is developed, which aims to improve the global search ability and local search efficiency for the global optimization of expensive black-box models. The proposed method involves three basic sub-criteria to infill new samples asynchronously to…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Global sampling”
Improves Bayesian optimization using Gaussian process Thompson sampling.
problem Global optimization of Gaussian process posterior samples.
method Carefully selects starting points for gradient-based multi-start optimizers, identifies all local optima via univariate global rootfinding, and optimizes the posterior sample.
result Dramatic improvements in overall performance of Bayesian optimization.
Combines local and global samplers for efficient sampling.
problem Limited learning accuracy in regions with little data.
method Explore-Exploit Markov chain Monte Carlo strategy ()
result Proves -uniform geometric ergodicity of .
New method optimizes Bayesian optimization for high-dimensional posterior samples.
problem Difficult inner-loop optimization of posterior sample paths in Bayesian optimization.
method Global rootfinding approach with carefully selected starting points.
result The method discovers the global optimum most of the time with just one starting point per set.
Distance and Kernel-Based Measures for Global and Local Two-Sample Conditional Distribution Testingstat.ME
Unified framework for global and local two-sample conditional distribution testing.
problem Testing equality of two conditional distributions.
method Distance and kernel methods, conditional U-statistics, local bootstrap.
result Developed reliable global and local tests.
NERS improves RL by sampling diverse transitions considering local and global contexts.
problem Sampling biases in experience replay lead to redundant transitions.
method Neural Experience Replay Sampler (NERS) that considers both local and global contexts.
result NERS significantly improves RL performance by sampling diverse and meaningful transitions.
New method uses diffusion models to speed up MCMC sampling.
problem Efficiently exploring high-dimensional and multimodal posterior functions.
method Combines Metropolis-Hastings with diffusion models for global sampling.
result Significant reduction in likelihood evaluations for accurate posterior representation.
This paper introduces a scalable benchmark for evaluating local posterior sampling in neural networks.
problem Degeneracy in neural network loss landscapes and its impact on SGMCMC algorithms.
method Development of a scalable benchmark for local posterior sampling.
result RMSProp-preconditioned SGLD is most effective at representing the local geometry of the posterior distribution.
Estimates neural representation dimensionality from small sample sizes.
problem Estimating neural representation dimensionality from limited data.
method Proposed a bias-corrected estimator for participation ratio of eigenvalues.
result The estimator is more accurate with finite samples and noise.
A barrier to the wider adoption of neural networks is their lack of interpretability. While local explanation methods exist for one prediction, most global attributions still reduce neural network decisions to a single set of features. In response, we present an approach for generating global attributions called GAM, w…
Efficiently certifies global robustness of large neural networks with probabilistic guarantees.
problem Certifying robustness of large neural networks in a scalable and efficient manner.
method Sampling an ε-net and invoking a local robustness oracle.
result Certifies a probabilistic relaxation of robustness efficiently and globally.
Efficient binary sampling method for global optimization of univariate functions with low regret.
problem Global optimization of univariate loss functions.
method Binary sampling approach to circumvent hard-to-determine query points in traditional methods.
result At most and regret for -Lipschitz continuous and -Lipschitz smooth functions respectively.
Postprocessing reduces Bayesian optimization steps for global optima.
problem Slow convergence in Bayesian optimization for high-dimensional problems.
method Prohibits duplicated samples in the dataset postprocessing method.
result Significantly reduces the number of sequential steps to find the global optimum.
GCBS improves contrastive learning performance efficiently.
problem Efficiently mining hard negatives for contrastive learning.
method Global Contrastive Batch Sampling (GCBS) approximates batch assignment problem.
result GCBS improves state-of-the-art performance in sentence embedding and code-search tasks.
GUESS improves surrogate model accuracy with adaptive sampling.
problem Creating accurate surrogate models with limited data.
method Gradient and Uncertainty Enhanced Sequential Sampling (GUESS) using predictive uncertainty and Taylor expansion.
result GUESS achieved highest sample efficiency compared to other strategies.
Bayesian optimization has recently emerged as a popular method for the sample-efficient optimization of expensive black-box functions. However, the application to high-dimensional problems with several thousand observations remains challenging, and on difficult problems Bayesian optimization is often not competitive wi…
This paper presents efficient sampling methods for Gaussian processes.
problem High cost of global sensitivity analysis and optimization due to limited high-quality observations.
method Two sampling methods: random Fourier features and pathwise conditioning.
result Efficient generation of posterior samples from Gaussian processes at reduced computational cost.
New method for private density estimation of high-dimensional Gaussian mixtures.
problem Private density estimation for mixtures of unrestricted high-dimensional Gaussians.
method Exploits list global stability to prove upper bound on sample complexity.
result First upper bound on sample complexity for agnostic private density estimation.
New method for symmetric matrix completion using ReLU sampling.
problem Symmetric positive semi-definite low-rank matrix completion with deterministic entry-dependent sampling.
method ReLU sampling, gradient descent with tailored initialization.
result Gradient descent with tailored initialization achieves global minima.
New method uses LP to achieve optimal sample complexity in multi-agent reinforcement learning.
problem Achieving global optimality in multi-agent reinforcement learning with average-cost criterion.
method Randomized Linear Programming and Stochastic Primal-Dual Methods for multi-agent saddle point problems.
result Sample complexity matches tight dependencies on state and action spaces, and scales with network size.
New method optimizes complex models with minimal data, proving global optimality.
problem Optimizing complex models with unknown cost functions and prior distributions.
method Introduces 'coarse learnability' and an iterative MBO algorithm with sample correction.
result Achieves global optimality with polynomial sample complexity.
In this paper is proposed a new heuristic approach belonging to the field of evolutionary Estimation of Distribution Algorithms (EDAs). EDAs builds a probability model and a set of solutions is sampled from the model which characterizes the distribution of such solutions. The main framework of the proposed method is an…
Mixed membership factorization is a popular approach for analyzing data sets that have within-sample heterogeneity. In recent years, several algorithms have been developed for mixed membership matrix factorization, but they only guarantee estimates from a local optimum. Here, we derive a global optimization (GOP) algor…
Global optimization problems whose objective function is expensive to evaluate can be solved effectively by recursively fitting a surrogate function to function samples and minimizing an acquisition function to generate new samples. The acquisition step trades off between seeking for a new optimization vector where the…
In this paper, we propose a stochastic optimization method that adaptively controls the sample size used in the computation of gradient approximations. Unlike other variance reduction techniques that either require additional storage or the regular computation of full gradients, the proposed method reduces variance by …
Analyzes error sources in global feature effect estimation methods.
problem Unexplored error sources in global feature effect estimation methods.
method Systematic, estimator-level analysis of bias and variance.
result Holdout data is theoretically cleanest, but estimation variance depends on sample size and model characteristics.
Optimized AIS scheme reduces bias and MSE for general proposals.
problem Performing Monte Carlo integration with general proposals.
method Global optimization of χ²-divergence using stochastic gradient Langevin dynamics.
result Explicit theoretical guarantees for uniform-in-time MSE reduction.
Compared with global average pooling in existing deep convolutional neural networks (CNNs), global covariance pooling can capture richer statistics of deep features, having potential for improving representation and generalization abilities of deep CNNs. However, integration of global covariance pooling into deep CNNs …
Efficient audio synthesis is an inherently difficult machine learning task, as human perception is sensitive to both global structure and fine-scale waveform coherence. Autoregressive models, such as WaveNet, model local structure at the expense of global latent structure and slow iterative sampling, while Generative A…
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernelstat.ML
Recent works have shown that on sufficiently over-parametrized neural nets, gradient descent with relatively large initialization optimizes a prediction function in the RKHS of the Neural Tangent Kernel (NTK). This analysis leads to global convergence results but does not work when there is a standard regulari…
The paper revisits and improves on a Bayesian relevance vector machine method for small sample sizes.
problem Statistical modeling with small sample sizes relative to the number of covariates.
method Introduces a new class of global-local priors and provides theoretical properties.
result Results on posterior consistency and contraction rates are provided.
The paper proposes a new model for predicting and analyzing economic variables.
problem Predicting and analyzing economic variables in developed regions.
method Time-varying parameter global vector autoregressive (TVP-GVAR) framework combined with machine learning models.
result The proposed model provides high precision out-of-sample predictions and novel insights into economic variable connectedness.
LES optimizes designs by sampling descent sequences, achieving strong sample efficiency.
problem Optimizing large, complex design spaces is infeasible and unnecessary.
method LES uses Bayesian optimization to target solutions reachable by iterative optimizers.
result LES achieves strong sample efficiency compared to existing methods.
Finite Sample Complexity of Sequential Monte Carlo Estimators on Multimodal Target Distributionsstat.CO
New bounds for SMC show its advantage over MCMC in multimodal distributions.
problem Estimating expectations under multimodal distributions with slow global mixing.
method Proves finite sample complexities for SMC with local mixing times, addressing bias through sequential resampling.
result SMC provides fully polynomial time approximation for multimodal problems.
GOLFS selects features for clustering by combining global and local information.
problem Feature selection for high-dimensional clustering without labels.
method Combines global and local information via manifold learning and regularized self-representation.
result Improves feature selection and clustering accuracy.
The structure of the control network of transnational corporations affects global market competition and financial stability. So far, only small national samples were studied and there was no appropriate methodology to assess control globally. We present the first investigation of the architecture of the international …
Improved convergence for actor-critic algorithms in MDPs.
problem Global convergence analysis for actor-critic algorithms in MDPs.
method Introduced an analytical framework to handle complex recursions, established convergence to ε-close globally optimal policy with improved sample complexity.
result Converges to ε-close globally optimal policy with sample complexity of O(ε^(-3)) compared to O(ε^(-2)) for ε-close stationary policy.
Improved BAI under DP reduces gap to constant.
problem Fixed-confidence BAI under global DP for Bernoulli distributions.
method New lower bound, stopping rule, and Top Two sampling rule.
result Reduces gap to a small multiplicative constant.
GAAVI offers anytime-valid tests for CMF global null and contrasts.
problem Inference on the conditional mean function for high confidence decisions.
method Asymptotic anytime-valid tests for CMF global null and contrasts.
result Achieves asymptotic type-I error guarantees, power one, and optimal sample complexity.
Proposes a new method to control FDR using frequentist-assisted horseshoe for high-dimensional testing.
problem Designing tests with frequentist false discovery rate control using horseshoe prior.
method Frequentist-assisted horseshoe procedure for high-dimensional normal means testing.
result Consistently achieves robust finite-sample FDR control in various sparse cases.
Graph neural networks help assess how global changes affect plant-pollinator networks.
problem Interpreting GNN results to understand how global changes impact plant-pollinator networks.
method Simulation study and application on Spipoll dataset to assess effects of global changes on pollination networks.
result GNNs can detect interactive effects between covariates and plant genera on pollination network connectivity.
FS&P uses birth-death process to ensure global convergence of stochastic conic particle gradient descent.
problem Global optimization of non-convex objective functions over measure space.
method Introduces Fast Spawn\&Prune (FS\&P) combining CPGD with birth-death process.
result First theoretical guarantee of global convergence for discrete-time stochastic algorithms.
Large scale optimization problems are ubiquitous in machine learning and data analysis and there is a plethora of algorithms for solving such problems. Many of these algorithms employ sub-sampling, as a way to either speed up the computations and/or to implicitly implement a form of statistical regularization. In this …
SBS uses SVGD to optimize continuous functions globally.
problem Global optimization of continuous Sobolev functions.
method Stein Boltzmann Sampling (SBS) with SVGD.
result SBS and its variants are highly competitive in global optimization.
Online feature selection has been an active research area in recent years. We propose a novel diverse online feature selection method based on Determinantal Point Processes (DPP). Our model aims to provide diverse features which can be composed in either a supervised or unsupervised framework. The framework aims to pro…
A note on learning with agents having global perspectives and a principal optimizing their performance.
problem Learning with dynamic-optimizing principal-agent setting, where agents have global views and the principal optimizes performance.
method Empirical-likelihood estimator under conditional moment restrictions model, considering agents' out-of-sample and private dataset performances.
result A coherent mathematical argument for the learning process in this framework.
Optimizes expensive experiments by incorporating expert knowledge.
problem Expensive experiments require minimizing the number of trials.
method Bayesian optimization with posterior sampling of expert knowledge.
result Demonstrates significant efficiency gains in experiments and hyperparameter tuning.
Enhances BO in high dimensions with Newton methods.
problem Challenges in scaling BO to high-dimensional spaces.
method Construct multiple local quadratic models using gradients and Hessians from a global GP, and select new sample points by solving bound-constrained quadratic programs.
result Outperforms existing high-dimensional BO techniques on synthetic and real-world applications.