The vast quantity of information brought by big data as well as the evolving computer hardware encourages success stories in the machine learning community. In the meanwhile, it poses challenges for the Gaussian process (GP) regression, a well-known non-parametric and interpretable Bayesian model, which suffers from cu…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “scalable approximations”
New scalable GP approximation using Fourier series decomposition.
problem Scalability and accuracy in Gaussian process approximations.
method Harmonic kernel decomposition (HKD) to decompose kernels orthogonally.
result Significantly outperforms standard variational methods in scalability and accuracy.
A scalable algorithm for GP regression selects relevant covariates efficiently.
problem Scalable variable selection in large GP regression models.
method VGPR algorithm using Vecchia approximation for sparse precision matrix, mini-batch subsampling.
result Improved scalability and accuracy in selecting relevant covariates.
A-NeSI scales approximate inference for probabilistic neurosymbolic learning.
problem Combining neural networks with symbolic reasoning for scalable inference.
method A-NeSI: a new framework for PNL using neural networks for approximate inference.
result A-NeSI achieves scalable approximate inference without semantic changes.
As a non-parametric Bayesian model which produces informative predictive distribution, Gaussian process (GP) has been widely used in various fields, like regression, classification and optimization. The cubic complexity of standard GP however leads to poor scalability, which poses challenges in the era of big data. Hen…
Ribbon: Scalable Approximation and Robust Uncertainty Quantification
problem Reliably quantifying predictive uncertainty for complex models
method Ribbon, a scalable approximation to Dirichlet-reweighted bootstrap uncertainty
result Asymptotically equivalent to a flat-prior Laplace approximation under correct likelihood specification, recovers robust sandwich covariance under misspecification
PSI-LinUCB improves scalability for large recommender systems.
problem Efficiently training and inferring for large action spaces in recommender systems.
method Represent inverse design matrix as diagonal + low-rank correction, derive stable rank-1 and batched updates, use projector-splitting integrator.
result Demonstrated effectiveness on recommender system datasets, achieving scalable training and inference.
Thompson sampling (TS) is a class of algorithms for sequential decision-making, which requires maintaining a posterior distribution over a model. However, calculating exact posterior distributions is intractable for all but the simplest models. Consequently, efficient computation of an approximate posterior distributio…
URSABench benchmarks Bayesian methods for deep learning models.
problem Scalability issues in Bayesian inference for deep learning.
method Open-source benchmark suite for assessing approximate Bayesian inference methods.
result Initial results show promise for addressing uncertainty and robustness in deep learning.
GraphGP: Scalable Gaussian Processes with Vecchia's Approximation
problem Naive Gaussian Process computation limits practical use
method GPU algorithm for Vecchia's approximation
result Linear time and memory requirements for nearly a billion parameters
A new method combines Laplace and Variational Bayes for scalable inference.
problem Complex models and large datasets make exact inference infeasible.
method Low-Rank Variational Bayes Correction (VBC) using Laplace method and Variational Bayes correction in a lower dimension.
result The method ensures scalability in both model complexity and data size.
Bayesian optimization technique scaled using Vecchia approximations.
problem Scalability issue with Gaussian process surrogate models in Bayesian optimization.
method Adapted Vecchia approximation from spatial statistics to Gaussian processes, developed improvements and extensions.
result Methods compared favorably to state-of-the-art on various test functions and reinforcement learning problems.
We develop a scalable method for Bayesian neural networks with stochastic differential equations.
problem Uncertainty quantification in deep neural networks.
method Gradient-based stochastic variational inference in continuous-depth Bayesian neural networks.
result Gradient estimator with zero variance as the approximation improves.
Improving scalability and stability of Stein discrepancies for scalable goodness-of-fit testing
problem Improving scalability and stability of Stein discrepancies for scalable goodness-of-fit testing
method Reformulating Stein discrepancy construction as an explicit SNR^2 maximisation problem
result Avoiding exponential SNR^2 collapse and achieving stable SNR^2
From Local Interactions to Global Operators: Scalable Gaussian Process Operator for Physical Systemsstat.ML
Scalable Gaussian Process Operator tackles high-dimensional PDEs.
problem Scaling Gaussian Process Operators to high-dimensional, data-intensive regimes.
method Nearest-neighbor-based local kernel approximations, sparse kernel approximation, structured Kronecker factorizations, operator-aware kernel structures, task-informed mean functions.
result Consistently achieves high accuracy across varying discretization scales.
STACI uses neural nets to estimate spatio-temporal fields with valid uncertainty quantification.
problem Scalable spatio-temporal deep learning models fail to capture underlying correlation structure.
method Variational Bayesian neural network approximation of non-stationary spatio-temporal Gaussian Process (GP) with conformal inference.
result STACI provides accurate prediction intervals for spatio-temporal processes, outperforming competing methods.
Bayesian tensor train kernel machine uses Laplace approximation for scalable GP regression.
problem Scalability limitations of Gaussian process regression.
method Bayesian tensor train kernel machine with Laplace approximation and variational inference.
result VI replaces cross-validation and offers up to 65x faster training.
We introduce a new structured kernel interpolation (SKI) framework, which generalises and unifies inducing point methods for scalable Gaussian processes (GPs). SKI methods produce kernel approximations for fast computations through kernel interpolation. The SKI framework clarifies how the quality of an inducing point a…
Bayesian learning made scalable with posteriors library.
problem Computational challenges in Bayesian learning with modern models.
method Introducing posteriors library and tempered MCMC.
result Bayesian approximations are useful and scalable.
Gaussian process classification (GPC) provides a flexible and powerful statistical framework describing joint distributions over function space. Conventional GPCs however suffer from (i) poor scalability for big data due to the full kernel matrix, and (ii) intractable inference due to the non-Gaussian likelihoods. Henc…
Scalable method bounds Lipschitz constant of generative models.
problem Bounding the Lipschitz constant of generative models.
method Layerwise convex approximations using zonotopes.
result Efficient and tight bounds on generative models.
Improved Gaussian Process regression using TQFF over RFF and Gaussian QFF.
problem Limited performance of Quadrature Fourier Features (QFF) in approximating highly oscillatory functions.
method Developed Trigonometric Quadrature Fourier Features (TQFF) using a novel non-Gaussian quadrature rule.
result TQFF provides better approximation accuracy and fewer features compared to RFF and Gaussian QFF.
VNNGP uses nearest neighbors to approximate GPs, improving scalability and performance.
problem Scalability issues in Gaussian process approximations.
method Sparse precision structure via nearest neighbors, variational framework.
result VNNGP outperforms low-rank methods and is less prone to overfitting.
Bayesian approach improves performance in Gaussian process models.
problem Scalable posterior estimation in Gaussian process models.
method Revisiting variational inference techniques with Bayesian treatment of inducing variables and hyper-parameters.
result State-of-the-art performance demonstrated across various regression and classification problems.
Efficiently infers latent SDEs with scalable memory and time costs.
problem Inference of latent SDEs with high time and memory complexity.
method Amortized reparametrization of expectations under linear SDEs, coupled with efficient gradient approximation.
result Achieves similar performance to adjoint sensitivities with fewer model evaluations.
New scalable MCMC sampling for nonsymmetric DPPs speeds up computations.
problem Efficient sampling for nonsymmetric DPPs with low-rank kernels.
method Enhanced rejection sampling with efficient proposal distribution construction.
result Sublinear runtime for scalable MCMC sampling of -NDPPs.
A scalable method for BED with implicit models using approximate gradients.
problem Efficiently estimating posterior distribution and maximizing MI for implicit models.
method Stochastic approximate gradient ascent with smoothed variational MI estimator.
result Significantly improves scalability of BED in high-dimensional problems.
New method for scalable set encoding with unbiased gradient approximation.
problem Limited expressive power and large set training issues in set functions.
method Universally MBC (UMBC) class of set functions and efficient MBC training algorithm.
result Unbiased approximation of full set gradient with constant memory overhead.
PPRGo uses approximate PageRank to speed up GNNs on large graphs.
problem Efficiently learning on large graphs using GNNs.
method Approximates PageRank for efficient information diffusion in GNNs.
result PPRGo outperforms other methods in speed and scalability.
Scalable Bayesian Learning for State Space Models using Variational Inference with SMC Samplersstat.ML
We present a scalable approach to performing approximate fully Bayesian inference in generic state space models. The proposed method is an alternative to particle MCMC that provides fully Bayesian inference of both the dynamic latent states and the static parameters of the model. We build up on recent advances in compu…
A new method for efficient Gaussian process inference using sparse approximations.
problem Scalable and accurate inference for latent Gaussian processes.
method Variational approximation with sparse inverse Cholesky factors and double Kullback-Leibler minimization.
result The proposed method can achieve highly accurate approximations with polylogarithmic time complexity.
Vecchia approximations provide the best accuracy-runtime trade-off for Gaussian process approximations.
problem High computational cost of Gaussian processes for large data sets.
method Systematic comparison of different Gaussian process approximations.
result Vecchia approximations consistently provide the best accuracy-runtime trade-off.
The expressive power of a Gaussian process (GP) model comes at a cost of poor scalability in the data size. To improve its scalability, this paper presents a low-rank-cum-Markov approximation (LMA) of the GP model that is novel in leveraging the dual computational advantages stemming from complementing a low-rank appro…
On Practical Reinforcement Learning: Provable Robustness, Scalability, and Statistical Efficiencycs.LG
This thesis improves practical reinforcement learning methods with robustness, scalability, and efficiency.
problem Improving reinforcement learning methods for practical applications.
method Analyzes and develops robust, scalable, and efficient reinforcement learning algorithms.
result Proves the efficiency and robustness of new RL methods.
Connections and Equivalences between the Nyström Method and Sparse Variational Gaussian Processesstat.ML
Clarifies connections between Nyström and SVGP methods for scalable GPs.
problem Lack of understanding between GP and kernel methods communities.
method Investigates Nyström and SVGP methods for scalable Gaussian processes.
result Establishes connections and equivalences between Nyström and SVGP methods.
SoftKI combines SKI and variational methods for scalable GP regression.
problem Scalable Gaussian Process regression on high-dimensional datasets.
method SoftKI approximates kernel via softmax interpolation from a smaller number of learned points.
result SoftKI is competitive with other approximated GP methods for modest data dimensions.
CorePPR combines PageRank and CoreRank for scalable GNNs.
problem Expensive message passing in GNNs for large graphs.
method Learnable convex combination of approximate PPR and CoreRank, dynamic node selection.
result CorePPR outperforms PPRGo on large graphs.
New iterative methods improve scalability of Gaussian process approximations for large data.
problem Scalability issues in Gaussian process approximations for large spatial data.
method Iterative methods combined with preconditioners to reduce computational costs.
result Preconditioners accelerate convergence and improve predictive variances.
This work connects BNNs to GPs, providing scalable inference and identifying key properties.
problem Scaling and inference challenges in Bayesian neural networks.
method General convergence from BNNs to GPs, new covariance function, and scalable Nyström approximation.
result Established a scalable maximum a posterior (MAP) training and prediction procedure.
A scalable ROC-SVM variant reduces training time for imbalanced binary classification.
problem High computational cost of ROC-SVM for imbalanced binary classification.
method Incomplete U-statistics and low-rank kernel approximation.
result Comparable AUC performance with reduced training time.
Despite advances in scalable models, the inference tools used for Gaussian processes (GPs) have yet to fully capitalize on developments in computing hardware. We present an efficient and general approach to GP inference based on Blackbox Matrix-Matrix multiplication (BBMM). BBMM inference uses a modified batched versio…
Combines pseudo-point and state space approximations for scalable GPs.
problem Handling large numbers of off-the-grid spatial data-points and long time-series.
method Combines pseudo-point approximations for spatial data with state space GP approximations for temporal data.
result Combined approach is more scalable and applicable to a greater range of spatio-temporal problems.
There has been recent interest in developing scalable Bayesian sampling methods such as stochastic gradient MCMC (SG-MCMC) and Stein variational gradient descent (SVGD) for big-data analysis. A standard SG-MCMC algorithm simulates samples from a discrete-time Markov chain to approximate a target distribution, thus samp…
Neighbour-Driven Gaussian Process Variational Autoencoders for Scalable Structured Latent Modellingcs.LG
A scalable GPVAE method using local adjacencies to approximate GP inference.
problem Scalability issues in exact GP inference for large-scale GPVAEs.
method Neighbour-driven approximation strategy that confines computations to nearest neighbours.
result Outperforms other GPVAE variants in predictive performance and computational efficiency.
A scalable method for estimating spatial data using VREML.
problem Costly computation of REML for large, sparse precision matrices in spatial data.
method Proposes VREML framework approximating marginal likelihood with Gaussian variational distribution and deriving a coordinate-ascent algorithm.
result Empirically shows VREML outperforms MLE and INLA.
GPNs use unlabeled data to estimate uncertainty in Bayesian problems.
problem Limited training data in high-dimensional problems.
method Generative Posterior Networks (GPNs) that approximate the posterior distribution using unlabeled data.
result GPNs improve epistemic uncertainty estimation and scalability.
A scalable Gaussian process clustering method for large datasets.
problem Infeasibility of Gaussian process clustering on large grids.
method Embedding Vecchia approximation in EM algorithm for scalability.
result Efficient Gaussian process clustering for large environmental applications.
Continuous-time Bayesian networks (CTBNs) constitute a general and powerful framework for modeling continuous-time stochastic processes on networks. This makes them particularly attractive for learning the directed structures among interacting entities. However, if the available data is incomplete, one needs to simulat…