Gradient-guided nested sampling improves posterior inference efficiency.
problem Efficiently sampling from complex posterior distributions.
method Gradient-guided nested sampling combining differentiable programming, Hamiltonian slice sampling, clustering, mode separation, dynamic nested sampling, and parallelization.
result Significantly faster mode discovery and more accurate partition function estimates.
Nested Slice Sampling accelerates Nested Sampling for GPU acceleration.
problem Challenging inference for complex, multimodal targets.
method Vectorized Nested Slice Sampling using Hit-and-Run Slice Sampling.
result NSS maintains accurate evidence estimates and high-quality posterior samples, robust on multimodal problems.
Nested sampling improved for arbitrary priors.
problem Technical obstacle to using nested sampling with arbitrary priors.
method Parametric bijectors trained on samples from a desired prior density.
result Nested sampling can be used with arbitrary priors.
Develops a method for learning proposals in nested importance samplers.
problem Improving sampling quality in complex distributions.
method Nested Variational Inference (NVI) using forward or reverse KL divergence.
result Optimizing nested objectives leads to improved sample quality.
Enhances Bayesian model selection for high-dimensional problems.
problem Bayesian model selection for high-dimensional problems.
method Proximal nested sampling with data-driven priors.
result Improves model selection for log-convex likelihood models.
Paper proposes a new estimator for nested expectations with faster convergence.
problem Estimating nested expectations is computationally challenging.
method Nested kernel quadrature estimators with proof of faster convergence rate.
result The proposed method requires fewer samples for accurate estimation.
Improved nested simulation for financial risk measurement.
problem Efficiently estimating nested risk measures in financial engineering.
method Reusing inner simulation outputs to improve efficiency and accuracy.
result The proposed approach outperforms standard nested simulation and regression methods.
Unified SGD method improves convergence for nested optimization problems.
problem Stochastic nested optimization problems.
method ALTERNATE dESCEN (ALSET) method leveraging hidden smoothness.
result Requires O(ε−2) samples to achieve an ε-stationary point. This paper uses Nested Sampling to improve Gaussian Process uncertainty quantification.
problem Underestimating predictive uncertainty and overfitting in Gaussian Process models.
method Marginalises hyperparameters using Nested Sampling for spectral mixture kernels.
result Improves predictive performance and uncertainty quantification across various data sets.
The data torrent unleashed by current and upcoming astronomical surveys demands scalable analysis methods. Many machine learning approaches scale well, but separating the instrument measurement from the physical effects of interest, dealing with variable errors, and deriving parameter uncertainties is often an after-th…
Nested learning improves model performance on multi-granular tasks.
problem Overconfident models and lack of fine-grained confidence in predictions.
method Introducing nested learning with a sequence of nested feature embeddings and explicit combination of outputs.
result Nested learning outperforms standard end-to-end training on various datasets.
Nested sampling is a powerful technique for exploring high-likelihood regions, but its theoretical derivation is complex and involves approximations.
problem Sampling from likelihood-constrained priors in nested sampling
method Providing a comprehensive and detailed exposition of nested sampling derivation and practical challenges
result Deepening understanding of nested sampling and fostering future enhancements
Paper tackles robust model training with a new stochastic algorithm.
problem Training robust models against data distribution shift.
method Derives a novel dual formulation and proposes a nested stochastic gradient descent algorithm.
result Establishes polynomial iteration and sample complexities for large-scale DRO problems.
We propose nested sequential Monte Carlo (NSMC), a methodology to sample from sequences of probability distributions, even where the random variables are high-dimensional. NSMC generalises the SMC framework by requiring only approximate, properly weighted, samples from the SMC proposal distribution, while still resulti…
New method estimates nested expectations with biased and antithetic sampling.
problem Estimating nested expectations with biased and antithetic sampling.
method Nested multilevel Monte Carlo with biased and antithetic sampling.
result Estimator achieves order ε^(-2) asymptotic cost.
Study compares MCMC and nested sampling for high-dimensional physics problems.
problem Efficiently sampling high-dimensional Bayesian posterior distributions in particle physics and cosmology.
method Review and comparison of MCMC and nested sampling techniques on high-dimensional test functions and real physics examples.
result Modern MCMC algorithms can outperform nested sampling in certain cases, highlighting implementation details.
A new method for efficient nested Monte Carlo simulations in financial modeling.
problem Computational challenges in nested stochastic modeling for financial risk assessment.
method Sample recycling approach to speed up inner loop estimations.
result Significantly more efficient than traditional techniques.
We investigate the problem of computing a nested expectation of the form P[E[X∣Y]≥0]=E[H(E[X∣Y])] where H is the Heaviside function. This nested expectation appears, for example, when estimating the probability of a large loss from a financial portfo…
Method estimates Bayesian evidence from posterior samples using normalizing flows.
problem Estimating Bayesian evidence from posterior samples.
method Normalizing flows for evidence estimation.
result Method is more robust to sharp features in posterior distributions, especially in higher dimensions.
Paper tackles robust optimization under uncertainty using nested distance.
problem Optimizing under distributionally robust uncertainty with nested distance.
method Equivalent recursive and dynamic programming reformulations for tractable optimization.
result Optimal robust policies can be found efficiently using convex optimization.
Simple algorithms identify best items or full rankings from choice-based feedback.
problem Learning to identify the best item or full ranking from choice-based feedback.
method Nested Elimination (NE) and Nested Partition (NP) algorithms.
result NE is worst-case asymptotically optimal, NP is optimal up to a constant factor.
Deep learning and genetic algorithms speed up cosmological Bayesian inference.
problem Substantial computational demands in Bayesian inference for cosmological parameter estimation.
method Deep learning using feedforward neural networks to approximate likelihood functions dynamically, optimized with genetic algorithms.
result Significant speed-up in Bayesian inference process for cosmological models and datasets.
New metrics using Laplace approximation improve Gaussian process model selection.
problem Finding a balance between model accuracy, interpretability, and simplicity.
method Introducing multiple metrics based on the Laplace approximation to evaluate Gaussian process models.
result Our metrics provide comparable performance to dynamic nested sampling but are significantly faster.
We accelerate Bayesian inference for neutrino physics experiments by 100-60x.
problem Complex posterior geometries in multi-dimensional parameter spaces.
method GPU acceleration, automatic differentiation, neural-network-guided reparameterization.
result Significant performance improvements in Bayesian inference for direct detection experiments.
Improved MLMC method boosts risk estimation efficiency.
problem Estimating risk measures like Value-at-Risk in financial risk management.
method Novel MLMC parametrization and antithetic sampling.
result Significantly improved performance in practical settings.
The paper proposes an efficient nested simulation design using likelihood ratio method.
problem Designing nested simulations with fixed outer scenarios and minimizing simulation effort.
method Proposes a bi-level optimization problem to decide inner replications and pooling strategies.
result Optimized design achieves $\cO(Γ^{-1})$ mean squared error of estimators.
Parallelizes active learning for Bayesian inference using Nested Sampler.
problem Expensive likelihood evaluations in complex experiments.
method Uses Nested Sampler to generate nearly-optimal batches of candidates in parallel.
result Comparable accuracy to sequential conditioning with efficient parallelization.
Parallelized Bayesian quadrature improves sample efficiency and inference.
problem Efficient Bayesian inference and model evidence calculation.
method Batch Bayesian quadrature with kernel recombination for parallel sampling.
result Empirically, outperforms state-of-the-art methods in various datasets.
Improves test set performance and reduces out-of-sample disappointment for unstable models.
problem Ensuring strong test set performance via cross-validation for unstable models.
method Nested k-fold cross-validation with hyperparameter selection based on a weighted sum of cross-validation metric and model stability measure.
result Improves out-of-sample MSE for sparse ridge regression and CART by 4% and 2% respectively, compared to k-fold cross-validation.
Study cobordisms of nested manifolds and their invariants.
problem Understanding cobordisms of nested manifolds and their invariants.
method Identify a nested analog of the Pontryagin-Thom construction and find spaces homotopy equivalent to nested Pontryagin-Thom spaces.
result Discover nested cobordism invariants and provide an alternative proof of Wall's splitting result.
Develops new optimization techniques for decision-making under uncertainty.
problem Decision-making under uncertainty with complex cost functions and nested expectations.
method Introduces Multistage Conditional Compositional Optimization (MCCO) and develops multilevel Monte Carlo techniques.
result New optimization techniques reduce scenario complexity from exponential to polynomial growth.
We consider probabilistic multinomial probit classification using Gaussian process (GP) priors. The challenges with the multiclass GP classification are the integration over the non-Gaussian posterior distribution, and the increase of the number of unknown latent variables as the number of target classes grows. Expecta…
This work improves confidence intervals for Cox model test error using nested CV.
problem Insufficient understanding of confidence intervals for cross-validation in Cox model.
method Generalized nested cross-validation to Cox proportional hazards model.
result Improved coverage of confidence intervals for Cox model test error.
Quantum algorithm speeds up nested expectation estimation by nearly quadratically.
problem Estimating repeatedly nested expectations with quantum computing.
method Proposes a quantum algorithm achieving nearly quadratic speedup over classical methods.
result Achieves nearly quadratic speedup for RNEs, up to logarithmic factors.
A new algorithm is proposed which accelerates the mini-batch k-means algorithm of Sculley (2010) by using the distance bounding approach of Elkan (2003). We argue that, when incorporating distance bounds into a mini-batch algorithm, already used data should preferentially be reused. To this end we propose using nested …
Scalable tools for nested optimization in deep learning.
problem Solving nested optimization problems on a large scale in deep learning.
method Building scalable tools for bilevel optimization.
result Tools for nested optimization scale to deep learning setups.
Let R be an o-minimal expansion of the real field, and let L(R) be the language consisting of all nested Rolle leaves over R. We call a set nested subpfaffian over R if it is the projection of a boolean combination of definable sets and nested Rolle leaves over R. Assuming that R admits analytic cell decomposition, we …
Proposes methods for online conformal prediction with nested prediction sets across multiple confidence levels.
problem Need for uncertainty quantification with multiple confidence levels in diverse applications.
method Online optimization perspective to enforce nestedness of prediction sets while controlling quantile estimation error.
result Achieves stable coverage across all levels, strictly nested prediction sets, and improved efficiency.
Many problems in machine learning and statistics involve nested expectations and thus do not permit conventional Monte Carlo (MC) estimation. For such problems, one must nest estimators, such that terms in an outer estimator themselves involve calculation of a separate, nested, estimation. We investigate the statistica…
We consider multi-level composite optimization problems where each mapping in the composition is the expectation over a family of random smooth mappings or the sum of some finite number of smooth mappings. We present a normalized proximal approximate gradient (NPAG) method where the approximate gradients are obtained v…
Sequential Monte Carlo (SMC) methods comprise one of the most successful approaches to approximate Bayesian filtering. However, SMC without good proposal distributions struggle in high dimensions. We propose nested sequential Monte Carlo (NSMC), a methodology that generalises the SMC framework by requiring only approxi…
Nested model averaging improves high-dimensional linear regression performance.
problem High-dimensional linear regression with predictor ordering impact.
method Combining model averaging with regularized estimators on the solution path.
result Nested model averaging with lasso and SLOPE outperforms competing methods.
Bayesian neural networks show good correlation between out-of-sample performance and Bayesian evidence.
problem Improving the out-of-sample performance of Bayesian neural networks.
method Numerical sampling of Bayesian posterior, ensembling over architectures, analysis of evidence vs. model size.
result Good correlation between out-of-sample performance and Bayesian evidence; ensembling improves performance.
The latent Dirichlet allocation (LDA) model is a widely-used latent variable model in machine learning for text analysis. Inference for this model typically involves a single-site collapsed Gibbs sampling step for latent variables associated with observations. The efficiency of the sampling is critical to the success o…
New discrete cobordism category for nested manifolds and relations to algebraic structures.
problem Discrete cobordism category for nested manifolds.
method Stratified Morse theory, Cyl-objects, doubling construction, cylindrical bar construction.
result Relations between Cyl-objects and algebraic structures like Temperley-Lieb algebras.
A new sampling method estimates scores without training or nested MCMC.
problem Efficient sampling from complex, unnormalised distributions.
method Multiscale averaging in SDEs for score estimation.
result Empirical results show competitive accuracy and efficiency.
A novel bootstrap method improves concept drift detection in predictive models.
problem Detecting changes in predictive relationships (concept drift) in data-driven applications.
method Developed a nested bootstrap procedure to calibrate control limits using the entire initial sample.
result The method yields more accurate baseline models and faster CL setup times.
Paper introduces IO-NPF for efficient Bayesian experimental design.
problem Efficient Bayesian experimental design in non-exchangeable settings.
method Inside-Out Nested Particle Filter (IO-NPF) for non-Markovian state-space models.
result IO-NPF achieves O(T2) computational complexity, improving efficiency.