Develops a method for learning proposals in nested importance samplers.
problem Improving sampling quality in complex distributions.
method Nested Variational Inference (NVI) using forward or reverse KL divergence.
result Optimizing nested objectives leads to improved sample quality.
Model clusters networks and their communities simultaneously.
problem Clustering networks and their communities in unlabeled, heterogeneous networks.
method Nested Stochastic Block Model (NSBM) with Bayesian approach and NDP prior.
result Model accurately estimates both within and across network clustering structures.
We propose nested sequential Monte Carlo (NSMC), a methodology to sample from sequences of probability distributions, even where the random variables are high-dimensional. NSMC generalises the SMC framework by requiring only approximate, properly weighted, samples from the SMC proposal distribution, while still resulti…
The latent Dirichlet allocation (LDA) model is a widely-used latent variable model in machine learning for text analysis. Inference for this model typically involves a single-site collapsed Gibbs sampling step for latent variables associated with observations. The efficiency of the sampling is critical to the success o…
Parallelizes active learning for Bayesian inference using Nested Sampler.
problem Expensive likelihood evaluations in complex experiments.
method Uses Nested Sampler to generate nearly-optimal batches of candidates in parallel.
result Comparable accuracy to sequential conditioning with efficient parallelization.
GPU-accelerated particle methods outperform neural samplers in LFT benchmarks.
problem High-dimensional multimodal sampling problems in lattice field theory.
method GPU-accelerated particle Monte Carlo methods (Sequential Monte Carlo and nested sampling).
result These methods match or outperform neural samplers in sample quality and wall-clock time.
This work improves Gaussian process inference using mixtures of experts and nested SMC samplers.
problem High computational and memory costs of Gaussian processes.
method Mixtures of Gaussian process experts with nested SMC samplers.
result Significantly improved inference compared to importance sampling.
We develop a fully Bayesian, computationally efficient framework for incorporating model uncertainty into Type II Tobit models and apply this to the investigation of the determinants of Foreign Direct Investment (FDI). While direct evaluation of modelprobabilities is intractable in this setting, we show that by using c…
A tree-based dictionary learning model is developed for joint analysis of imagery and associated text. The dictionary learning may be applied directly to the imagery from patches, or to general feature vectors extracted from patches or superpixels (using any existing method for image feature extraction). Each image is …
Improved MCMC sampling for expensive, irregular likelihoods.
problem Bayesian inference challenges with irregular, expensive likelihoods.
method Adapt subset samplers, introduce data-driven proxies, adaptive controller.
result Improved HINTS algorithm achieves best sampling error in fixed budget.
Unified Bayesian framework for PTA data analysis tackles hierarchical model issues.
problem Hierarchical Bayesian modeling challenges in PTA data analysis.
method Reparameterization strategy using Normalizing Flows (NFs) and i-nessai nested sampler.
result Improved statistical robustness and computational efficiency in PTA analysis.
A new sampling method estimates scores without training or nested MCMC.
problem Efficient sampling from complex, unnormalised distributions.
method Multiscale averaging in SDEs for score estimation.
result Empirical results show competitive accuracy and efficiency.
New algorithm optimizes stochastic optimization with circular dependency.
problem Circular dependency between decision variable and importance sampling.
method Single-loop stochastic approximation algorithm based on Nesterov's dual averaging.
result Achieves minimal asymptotic variance and resolves circular optimization challenge.
Study cobordisms of nested manifolds and their invariants.
problem Understanding cobordisms of nested manifolds and their invariants.
method Identify a nested analog of the Pontryagin-Thom construction and find spaces homotopy equivalent to nested Pontryagin-Thom spaces.
result Discover nested cobordism invariants and provide an alternative proof of Wall's splitting result.
Quantum algorithm speeds up nested expectation estimation by nearly quadratically.
problem Estimating repeatedly nested expectations with quantum computing.
method Proposes a quantum algorithm achieving nearly quadratic speedup over classical methods.
result Achieves nearly quadratic speedup for RNEs, up to logarithmic factors.
Improved nested simulation for financial risk measurement.
problem Efficiently estimating nested risk measures in financial engineering.
method Reusing inner simulation outputs to improve efficiency and accuracy.
result The proposed approach outperforms standard nested simulation and regression methods.
Scalable tools for nested optimization in deep learning.
problem Solving nested optimization problems on a large scale in deep learning.
method Building scalable tools for bilevel optimization.
result Tools for nested optimization scale to deep learning setups.
Paper proposes a new estimator for nested expectations with faster convergence.
problem Estimating nested expectations is computationally challenging.
method Nested kernel quadrature estimators with proof of faster convergence rate.
result The proposed method requires fewer samples for accurate estimation.
Unified SGD method improves convergence for nested optimization problems.
problem Stochastic nested optimization problems.
method ALTERNATE dESCEN (ALSET) method leveraging hidden smoothness.
result Requires O(ε−2) samples to achieve an ε-stationary point. Let R be an o-minimal expansion of the real field, and let L(R) be the language consisting of all nested Rolle leaves over R. We call a set nested subpfaffian over R if it is the projection of a boolean combination of definable sets and nested Rolle leaves over R. Assuming that R admits analytic cell decomposition, we …
Many problems in machine learning and statistics involve nested expectations and thus do not permit conventional Monte Carlo (MC) estimation. For such problems, one must nest estimators, such that terms in an outer estimator themselves involve calculation of a separate, nested, estimation. We investigate the statistica…
Nested model averaging improves high-dimensional linear regression performance.
problem High-dimensional linear regression with predictor ordering impact.
method Combining model averaging with regularized estimators on the solution path.
result Nested model averaging with lasso and SLOPE outperforms competing methods.
Gradient-guided nested sampling improves posterior inference efficiency.
problem Efficiently sampling from complex posterior distributions.
method Gradient-guided nested sampling combining differentiable programming, Hamiltonian slice sampling, clustering, mode separation, dynamic nested sampling, and parallelization.
result Significantly faster mode discovery and more accurate partition function estimates.
Nested Slice Sampling accelerates Nested Sampling for GPU acceleration.
problem Challenging inference for complex, multimodal targets.
method Vectorized Nested Slice Sampling using Hit-and-Run Slice Sampling.
result NSS maintains accurate evidence estimates and high-quality posterior samples, robust on multimodal problems.
Gradient-free framework for Bayesian experimental design in complex systems.
problem Optimal experimental design in systems where gradient information is unavailable.
method Combines EKI and ALDI for optimization and sampling, with approximations for scalable utility estimation.
result Demonstrates robust, accurate, and efficient experimental design in various complex systems.
New discrete cobordism category for nested manifolds and relations to algebraic structures.
problem Discrete cobordism category for nested manifolds.
method Stratified Morse theory, Cyl-objects, doubling construction, cylindrical bar construction.
result Relations between Cyl-objects and algebraic structures like Temperley-Lieb algebras.
Nested sampling improved for arbitrary priors.
problem Technical obstacle to using nested sampling with arbitrary priors.
method Parametric bijectors trained on samples from a desired prior density.
result Nested sampling can be used with arbitrary priors.
Corrected samplers reduce discretization error in discrete flow models without additional computational cost.
problem Discretization error in samplers for discrete flow models.
method Established non-asymptotic error bounds for samplers, proposed time-corrected and location-corrected samplers.
result Location-corrected sampler has lower complexity and better generation quality.
New samplers improve MCMC efficiency in high dimensions.
problem Efficient sampling in high-dimensional problems.
method Affine invariant ensemble samplers, including derivative-free and derivative-based HMC.
result Affine invariant ensemble HMC outperforms standard HMC in high dimensions.
Study of loops in sums of Laplace eigenfunctions on surfaces.
problem Uniform bound for the number of nested loops in sums of Laplace eigenfunctions.
method Real-analytic category analysis and biharmonic function construction.
result Uniform bound for the number of rooted double nests in terms of surface, root, and spectral cutoff.
Enhances Bayesian model selection for high-dimensional problems.
problem Bayesian model selection for high-dimensional problems.
method Proximal nested sampling with data-driven priors.
result Improves model selection for log-convex likelihood models.
Nested learning improves model performance on multi-granular tasks.
problem Overconfident models and lack of fine-grained confidence in predictions.
method Introducing nested learning with a sequence of nested feature embeddings and explicit combination of outputs.
result Nested learning outperforms standard end-to-end training on various datasets.
NEST optimizes deep learning training by placing devices efficiently across networks and memory.
problem Inefficient device placement in distributed deep learning leads to high communication and memory overhead.
method NEST uses network-, compute-, and memory-aware dynamic programming to optimize device placement.
result NEST achieves up to 2.43 times higher throughput and better memory efficiency.
We propose doubly nested network(DNNet) where all neurons represent their own sub-models that solve the same task. Every sub-model is nested both layer-wise and channel-wise. While nesting sub-models layer-wise is straight-forward with deep-supervision as proposed in \cite{xie2015holistically}, channel-wise nesting has…
New study shows Gaussian samplers struggle with heavy-tailed targets, while stable samplers excel.
problem The difficulty of sampling from heavy-tailed distributions using Gaussian versus stable oracles.
method Comparison of Gaussian and stable oracles for proximal samplers.
result Gaussian samplers have a fundamental barrier for high-accuracy guarantees in heavy-tailed sampling, while stable samplers excel.
SRO optimizes decisions against worst-case sampler induced by generative models.
problem Operational uncertainty shifts from explicit probability law to sampler induced by learned generators.
method SRO optimizes decisions against the worst-case sampler induced by perturbing the learned generator.
result Empirical worst-case objective provides high-probability upper certificate for true population objective.
New methods for estimating complex causal effects in econometrics.
problem Estimating causal parameters in short panel data models using nested nonparametric instrumental variable regression.
method Introducing techniques to limit ill-posedness in nested NPIV, providing explicit mean square rates and efficient inference.
result Explicit mean square rates for nested NPIV and efficient inference for causal parameters.
The Gibbs sampler is a particularly popular Markov chain used for learning and inference problems in Graphical Models (GMs). These tasks are computationally intractable in general, and the Gibbs sampler often suffers from slow mixing. In this paper, we study the Swendsen-Wang dynamics which is a more sophisticated Mark…
Discrete diffusion samplers improve sampling from unnormalised densities.
problem Sampling from discrete unnormalised densities efficiently.
method Introduce off-policy training techniques and data-to-energy Schrödinger bridge training for discrete diffusion samplers.
result Improved performance on synthetic and new benchmarks.
We formalize the notion of nesting probabilistic programming queries and investigate the resulting statistical implications. We demonstrate that while query nesting allows the definition of models which could not otherwise be expressed, such as those involving agents reasoning about other agents, existing systems take …
Sharp lower bound on GHHs' representation power of CPWL functions.
problem Proving the minimum number of nestings for GHHs to represent arbitrary CPWL functions.
method Using a key lemma about finite sums of periodic functions, proving necessity of n nestings.
result Proving necessity of n nestings for GHHs to achieve universal representation power.
We develop nested automatic differentiation (AD) algorithms for exact inference and learning in integer latent variable models. Recently, Winner, Sujono, and Sheldon showed how to reduce marginalization in a class of integer latent variable models to evaluating a probability generating function which contains many leve…
Unified analysis for deterministic samplers in diffusion models.
problem Challenges in analyzing deterministic samplers for diffusion models.
method Unified convergence analysis framework.
result Achieved polynomial iteration complexity for DDIM-type samplers.
This paper analyzes MaskGIT sampler and introduces a moment sampler for faster masked diffusion sampling.
problem Efficiently sampling from masked diffusion models.
method Theoretical analysis of MaskGIT sampler, introduction of moment sampler, and two innovations for improving choose-then-sample efficiency.
result The moment sampler is an asymptotically equivalent, more interpretable alternative to MaskGIT.
There is an increasing interest in estimating expectations outside of the classical inference framework, such as for models expressed as probabilistic programs. Many of these contexts call for some form of nested inference to be applied. In this paper, we analyse the behaviour of nested Monte Carlo (NMC) schemes, for w…
PTSD improves neural samplers by combining diffusion models and PT, enhancing efficiency.
problem Efficiency and correlation issues in neural samplers compared to PT.
method Sequential training of diffusion models across temperatures, combining high-temperature models for approximate lower-temperature samples.
result Significantly improved target evaluation efficiency, outperforming diffusion-based samplers.
This paper introduces a neural sampler for scalable sampling from complex distributions.
problem Efficiently sampling from high-dimensional un-normalized distributions.
method Neural implicit sampler trained with KL and Fisher divergence methods.
result The neural sampler generates large batches of samples with low computational costs.
Paper tackles robust model training with a new stochastic algorithm.
problem Training robust models against data distribution shift.
method Derives a novel dual formulation and proposes a nested stochastic gradient descent algorithm.
result Establishes polynomial iteration and sample complexities for large-scale DRO problems.