This paper introduces repulsive Monte Carlo methods for computing the sliced Wasserstein distance.
problem Computing the integral of a function on the unit sphere using Monte Carlo methods.
method The approach involves using determinantal point processes and repelled point processes to create quadratures for the sliced Wasserstein distance.
result The UnifOrtho estimator is recommended for the computation of the sliced Wasserstein distance in large dimensions.
Study shows global invertibility in nonlinear elasticity with vanishing self-repulsion term.
problem Global invertibility in nonlinear elasticity with a vanishing nonlocal self-repulsion term.
method Proves global invertibility in the Γ-limit of elastic energy with a vanishing nonlocal self-repulsion term. result Global invertibility can be obtained in the Γ-limit of the elastic energy with a vanishing nonlocal self-repulsion term. Repulsive ensembles improve uncertainty estimates in PINNs for differential equations.
problem Improving uncertainty estimates in PINNs for differential equations.
method Employing repulsive ensembles (RE-PINN) with a repulsive term in the loss function.
result Repulsive ensembles produce more accurate uncertainty estimates and higher sample diversity.
Neighbor embeddings balance attraction and repulsion to visualize data.
problem Visualizing high-dimensional datasets with trade-offs between continuous and discrete structures.
method Neighbor embeddings combine attractive and repulsive forces to visualize data.
result Changing the exaggeration parameter in t-SNE yields a spectrum of embeddings with a trade-off between continuous and discrete structures.
New quadrature method using randomly pivoted Cholesky outperforms existing techniques.
problem Efficiently approximating integrals of functions in reproducing kernel Hilbert spaces.
method Nodes drawn by randomly pivoted Cholesky algorithm.
result Randomly pivoted Cholesky quadrature is fast and achieves comparable accuracy to more computationally intensive methods.
ARS visualization improves t-SNE dynamics with tunable attraction and repulsion.
problem Improve data visualization techniques for complex data sets.
method ARS framework based on t-SNE dynamics with normalized interactions and tunable kernels.
result ARS visualization provides better control over cluster tightness and spacing.
Bayesian quadrature uses probabilistic models for estimating intractable integrals.
problem Estimating intractable integrals in complex models.
method Probabilistic, model-based approach using Gaussian processes.
result Comprehensive review and systematic taxonomy of Bayesian quadrature methods.
Repulsive deep ensembles improve diversity and Bayesian inference.
problem Challenges in maintaining diversity among ensemble members trained independently.
method Introducing a repulsive term in the update rule of deep ensembles.
result Training dynamics of repulsive ensembles follow a Wasserstein gradient flow of KL divergence with the true posterior.
Computer experiments reveal complex knots that don't simplify.
problem Understanding the dynamics of complex knots under self-repulsion.
method Computer simulations of knot theory, focusing on rational knots and tangles.
result Discovered hard unknots and complexified knots that do not reduce to simpler forms under self-repulsion.
SOBER optimizes and quadrates efficiently in parallel for diverse tasks.
problem Scalability of batch Bayesian optimization and quadrature for expensive functions.
method Reformulates batch selection as a quadrature problem, balancing exploitation and exploration.
result SOBER outperforms 11 baselines on 12 tasks.
New method improves sample diversity and efficiency from complex distributions.
problem Sampling from intractable un-normalized distributions with high auto-correlation.
method Stein self-repulsive dynamics using a repulsive force to push samples away from past trajectories.
result Significantly decreases auto-correlation and increases effective sample size.
This paper presents a convergence analysis of kernel-based quadrature rules in misspecified settings, focusing on deterministic quadrature in Sobolev spaces. In particular, we deal with misspecified settings where a test integrand is less smooth than a Sobolev RKHS based on which a quadrature rule is constructed. We pr…
Improved Gaussian Process regression using TQFF over RFF and Gaussian QFF.
problem Limited performance of Quadrature Fourier Features (QFF) in approximating highly oscillatory functions.
method Developed Trigonometric Quadrature Fourier Features (TQFF) using a novel non-Gaussian quadrature rule.
result TQFF provides better approximation accuracy and fewer features compared to RFF and Gaussian QFF.
FoRDE uses input gradients to improve neural network ensembles.
problem Improving neural network ensembles for robustness and accuracy.
method Proposes FoRDE, an ensemble learning method based on ParVI, which repels function space by input gradients.
result FoRDE significantly outperforms DEs and other ensemble methods in accuracy and calibration.
We study quadrature rules for functions from an RKHS, using nodes sampled from a determinantal point process (DPP). DPPs are parametrized by a kernel, and we use a truncated and saturated version of the RKHS kernel. This link between the two kernels, along with DPP machinery, leads to relatively tight bounds on the qua…
Study identifies stable configurations of intertwined threads with repulsive interactions.
problem Stable configurations of entangled systems with repulsive interactions.
method Analysis of steepest descent flow of an energy functional.
result Existence and uniqueness of stable configuration of two layers drifting apart at t1/3 rate. Bayesian quadrature improves integration efficiency with invariant priors.
problem Efficient numerical integration with known structure.
method Invariance priors for bijective transformations in input domain.
result Superior performance in synthetic and real-world applications.
This work shows dimension regularization can replace skip-gram negative sampling for graph embeddings, improving efficiency and performance.
problem Efficiently enforcing dissimilarity among node embeddings in graph learning.
method Dimension regularization as an alternative to skip-gram negative sampling.
result Dimension regularization is a more efficient approach to enforcing dissimilarity in graph embeddings.
Herding and kernel herding are deterministic methods of choosing samples which summarise a probability distribution. A related task is choosing samples for estimating integrals using Bayesian quadrature. We show that the criterion minimised when selecting samples in kernel herding is equivalent to the posterior varianc…
Herding and kernel herding are deterministic methods of choosing samples which summarise a probability distribution. A related task is choosing samples for estimating integrals using Bayesian quadrature. We show that the criterion minimised when selecting samples in kernel herding is equivalent to the posterior varianc…
Improved kernel quadrature with convex weights using subsampling.
problem Constructing quadrature rules with small worst-case error.
method Combining spectral properties of the kernel with recombination results.
result Effective algorithms for constructing convex quadrature rules with i.i.d. samples.
A new type of quadrature is developed. The Gaussian quadrature, for a given measure, finds optimal values of a function's argument (nodes) and the corresponding weights. In contrast, the Lebesgue quadrature developed in this paper, finds optimal values of function (value-nodes) and the corresponding weights. The Gaussi…
The sigma-point filters, such as the UKF, which exploit numerical quadrature to obtain an additional order of accuracy in the moment transformation step, are popular alternatives to the ubiquitous EKF. The classical quadrature rules used in the sigma-point filters are motivated via polynomial approximation of the integ…
In this study, a numerical quadrature for the generalized inverse Gaussian distribution is derived from the Gauss-Hermite quadrature by exploiting its relationship with the normal distribution. The proposed quadrature is not Gaussian, but it exactly integrates the polynomials of both positive and negative orders. Using…
Adaptive Bayesian quadrature (ABQ) is a powerful approach to numerical integration that empirically compares favorably with Monte Carlo integration on problems of medium dimensionality (where non-adaptive quadrature is not competitive). Its key ingredient is an acquisition function that changes as a function of previou…
The aim of this article is to design a moment transformation for Student- t distributed random variables, which is able to account for the error in the numerically computed mean. We employ Student-t process quadrature, an instance of Bayesian quadrature, which allows us to treat the integral itself as a random variable…
Unified quadrature framework for large-scale kernel machines.
problem Efficiently approximating kernel functions for large-scale machine learning.
method Deterministic and randomized interpolatory rules for numerical integration of kernel functions.
result The proposed method reduces the number of nodes needed for accurate kernel approximation.
Shielded LMC samples from non-convex spaces with repulsive drift.
problem Sampling from non-convex spaces with convex holes.
method Combining adaptive temperature and repulsive drift.
result Advantages over unconstrained sampling in constrained spaces.
New approach improves multi-head attention by making heads less similar.
problem Multi-head attention can lead to similar features, reducing model expressiveness.
method Proposes a non-parametric approach using Bayesian techniques to make heads repel each other.
result Improves feature diversity, leading to better representations and performance.
Paper improves particle variational inference by optimizing generalization error bound.
problem Improving the diversity of models in particle variational inference to enhance generalization.
method Develops a new second-order Jensen inequality with a repulsion term based on the loss function, leading to a tighter generalization error bound.
result The proposed PVI optimizes the generalization error bound directly, improving performance compared to existing methods.
Novel approach for estimating conditional expectations using Bayesian quadrature.
problem Estimating conditional expectations with costly evaluations.
method Probabilistic numerical methods incorporating prior smoothness knowledge.
result Fast convergence rate and uncertainty quantification.
Research examines correlations of complex logarithms of lattice points, showing level repulsion and Poissonian behavior.
problem Analyzing correlations of complex logarithms of lattice points.
method Proving existence of pair correlation functions and examining behavior at various scalings.
result Level repulsion observed at linear scaling, Poissonian behavior at sublinear scalings.
Generative adversarial nets (GANs) are widely used to learn the data sampling process and their performance may heavily depend on the loss functions, given a limited computational budget. This study revisits MMD-GAN that uses the maximum mean discrepancy (MMD) as the loss function for GAN and makes two contributions. F…
New method uses Coulomb gases for Monte Carlo integration with reduced errors.
problem Reducing integration errors in numerical algorithms.
method Using Gibbs measures with a large deviations approach.
result Preserves large deviation principle for improved integration.
New Fourier features improve high-precision approximation in large-scale problems.
problem Designing scalable, high-precision Fourier features for large-scale kernel methods.
method Introducing a new family of quadrature rules that accurately approximate the Gaussian measure in higher dimensions.
result Improved approximation bounds with new Fourier features.
Improved Nyström approximation for kernel quadrature with theoretical guarantees.
problem Efficiently approximating positive definite kernels for large datasets.
method Refined sampling and subspace selection in Nyström approximation.
result Novel theoretical guarantees for non-i.i.d. landmark points in kernel quadrature.
Computation of moments of transformed random variables is a problem appearing in many engineering applications. The current methods for moment transformation are mostly based on the classical quadrature rules which cannot account for the approximation errors. Our aim is to design a method for moment transformation for …
Parallelized Bayesian quadrature improves sample efficiency and inference.
problem Efficient Bayesian inference and model evidence calculation.
method Batch Bayesian quadrature with kernel recombination for parallel sampling.
result Empirically, outperforms state-of-the-art methods in various datasets.
Adaptive quadrature improves Bayesian inference through active learning.
problem Efficiently estimating posterior densities in Bayesian inference.
method Sequential node selection using acquisition functions, combining interpolative surrogate models and quadrature rules.
result Positive estimation of marginal likelihood with improved accuracy.
A new method calculates accurate SABR model option prices and deltas.
problem Inaccurate and arbitrageable SABR model option prices and deltas.
method Gaussian quadrature integration scheme for the normal SABR model.
result Accurate and arbitrage-free SABR model option prices and deltas calculated with 49 points.
DBQPG improves policy gradient estimation with fewer samples.
problem Accurate policy gradient estimation with limited samples.
method Deep Bayesian Quadrature Policy Gradient (DBQPG).
result DBQPG provides more accurate and less variable gradient estimates.
Improved kernel herding algorithm for faster quadrature rule convergence.
problem Slow convergence speed of standard kernel herding algorithm.
method Improved gradient approximation to obtain sparser solutions.
result The cosine of the angle between negative gradient and approximate gradient determines convergence speed.
Bayesian quadrature improves conformal prediction for better risk assessment.
problem Improving risk assessment for machine learning models.
method Revisiting conformal prediction from a Bayesian perspective and proposing Bayesian quadrature.
result Provides interpretable guarantees and a richer representation of likely losses.
RePULSe improves language model alignment by reducing undesired outputs without sacrificing overall performance.
problem Aligning language models with human preferences while minimizing undesired outputs.
method Integrates probabilistic inference into RL training to reduce undesired outputs.
result RePULSe achieves a better balance between expected reward and undesired output probability.
The convergence speed of stochastic gradient descent (SGD) can be improved by actively selecting mini-batches. We explore sampling schemes where similar data points are less likely to be selected in the same mini-batch. In particular, we prove that such repulsive sampling schemes lowers the variance of the gradient est…
Efficiently marginalizes over Gaussian Process kernels for better model flexibility and uncertainty.
problem Inefficient marginalization over Gaussian Process kernels for large datasets.
method Bayesian Quadrature scheme with maximum mean discrepancies and invariances between Spectral Mixture kernels.
result Achieves more accurate predictions and better calibrated uncertainty than state-of-the-art baselines.
The paper improves error bounds for Bayesian quadrature in noisy settings.
problem Improving error bounds for Bayesian quadrature in noisy settings.
method Develops a two-step meta-algorithm to relate average-case quadrature error to L2-function approximation error. result Provides new average-case results for various kernels and noise settings.
Bayesian Quadrature improves ensembling for neural networks with dispersed likelihood peaks.
problem Ensembling neural networks struggles with dispersed, narrow peaks in likelihood surfaces.
method Uses Bayesian Quadrature to construct weighted ensembles of architectures.
result Empirically outperforms state-of-the-art baselines in test likelihood, accuracy, and expected calibration error.