We introduce a new structured kernel interpolation (SKI) framework, which generalises and unifies inducing point methods for scalable Gaussian processes (GPs). SKI methods produce kernel approximations for fast computations through kernel interpolation. The SKI framework clarifies how the quality of an inducing point a…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,694 papers · 148 categories
Trend · papers per month
48 results for “kernel inducing points”
We introduce a kernel approximation strategy that enables computation of the Gaussian process log marginal likelihood and all hyperparameter derivatives in time. Our GRIEF kernel consists of eigenfunctions found using a Nystrom approximation from a dense Cartesian product grid of inducing points. B…
Kernel methods on discrete domains have shown great promise for many challenging data types, for instance, biological sequence data and molecular structure data. Scalable kernel methods like Support Vector Machines may offer good predictive performances but do not intrinsically provide uncertainty estimates. In contras…
Proposes IGN for scalable Gaussian process networks.
problem Scalability and expressivity challenges in Gaussian processes.
method Inducing Gaussian process networks with learned inducing points.
result Significant advances over state-of-the-art methods.
New algorithms improve GP inference without approximations, achieving better results.
problem Inexact stochastic optimization methods in Gaussian Processes leading to biased results.
method Exact stochastic inference for GPs with finite dimensional RKHS, extending to infinite dimensions.
result Achieves better experimental results than existing methods in constrained resource settings.
Gaussian processes (GPs) are flexible models that can capture complex structure in large-scale dataset due to their non-parametric nature. However, the usage of GPs in real-world application is limited due to their high computational cost at inference time. In this paper, we introduce a new framework, \textit{kernel di…
SoftKI combines SKI and variational methods for scalable GP regression.
problem Scalable Gaussian Process regression on high-dimensional datasets.
method SoftKI approximates kernel via softmax interpolation from a smaller number of learned points.
result SoftKI is competitive with other approximated GP methods for modest data dimensions.
This paper analyzes error in SKI for Gaussian Processes, providing conditions for linear time inference.
problem Lack of rigorous theoretical error analysis for SKI.
method Proved error bounds for SKI Gram matrix, examined error effects, provided practical guidelines.
result Identified two dimensionality regimes for SKI's scalability-accuracy trade-offs.
Determinantal point process have recently been used as models in machine learning and this has raised questions regarding the characterizations of conditional independence. In this paper we investigate characterizations of conditional independence. We describe some conditional independencies through the conditions on t…
Deep Kernel Learningcs.LG
We introduce scalable deep kernels, which combine the structural properties of deep learning architectures with the non-parametric flexibility of kernel methods. Specifically, we transform the inputs of a spectral mixture base kernel with a deep architecture, using local kernel interpolation, inducing points, and struc…
New method combines spectral and sparse methods for Gaussian processes.
problem Efficiently fitting Gaussian processes to large datasets.
method Orthogonally decoupled variational Fourier features.
result Competitive performance on synthetic and real-world data.
DGKIP extends KIP for dataset distillation without bi-level optimization.
problem Efficiently distill datasets for various loss functions.
method Leverages duality theory to avoid bi-level optimization.
result DGKIP supports a wider range of loss functions.
Modified group captures braid dynamics, revealing Burau kernel.
problem Detecting the kernel of the Burau representation for braids.
method Defined a modified group and its representation.
result The modified group's representation detects the Burau kernel.
ASkotch solves large-scale KRR faster and better than existing methods.
problem Challenges in scaling full Kernel Ridge Regression (KRR) to large datasets.
method ASkotch: A scalable, accelerated, iterative method for full KRR.
result ASkotch provides better solutions faster than state-of-the-art solvers for full and inducing points KRR.
HIP-GP improves GP inference for inter-domain observations with millions of inducing points.
problem Inference for Gaussian Processes across different domains.
method Hierarchical inducing point Gaussian process with grid structure and stationary kernel assumption.
result Improved approximation accuracy through increased number of inducing points.
Deep kernel learning combines the non-parametric flexibility of kernel methods with the inductive biases of deep learning architectures. We propose a novel deep kernel learning model and stochastic variational inference procedure which generalizes deep kernel learning approaches to enable classification, multi-task lea…
A new model improves uncertainty estimation in deep learning.
problem Deep Kernel Learning (DKL) produces unreliable uncertainty estimates.
method Proposed a bi-Lipschitz constraint to preserve distances in feature space.
result DUE model outperforms previous DKL and other methods in uncertainty quality.
SKI accelerates GP inference with sparse grids to handle higher dimensions.
problem SKI scales poorly in high dimensions due to dense grid size.
method Sparse grids within SKI framework, novel matrix-vector multiplication algorithm.
result SKI can be scaled to higher dimensions while maintaining accuracy.
Exact Gaussian Processes for Massive Datasets via Non-Stationary Sparsity-Discovering Kernelsstat.ML
Exact Gaussian Processes for massive datasets using non-stationary sparsity-discovering kernels.
problem High computational and storage costs for exact GPs in large datasets.
method Develop non-stationary kernels that allow the GP to discover sparse structure naturally.
result Exact Gaussian Processes scalable to over 5 million data points.
The paper proposes a method to detect credit card fraud using sparse Gaussian approximations.
problem Detecting credit card fraud in financial institutions.
method Sparse Gaussian classification method with pseudo or inducing inputs and different kernels and inducing points.
result The RBF kernel with a higher number of inducing points achieved the best accuracy.
We propose a new non-parametric framework for learning incrementally stable dynamical systems x' = f(x) from a set of sampled trajectories. We construct a rich family of smooth vector fields induced by certain classes of matrix-valued kernels, whose equilibria are placed exactly at a desired set of locations and whose …
Paper explores duality in DPPs using embedding structure analysis.
problem Understanding the geometric structure of determinantal point processes.
method Analyzes the exponential family embedding of DPPs and uses the e-embedding curvature tensor.
result Discovers the duality between marginal and L-ensemble kernels.
Accelerates GPR with localized kernels for faster performance.
problem Speeding up Gaussian process regression.
method Localization kernels applied at each data point to down-weight distant points, leading to a sparsified Gram matrix.
result Significant speedups with competitive performance compared to other methods.
A new method compresses conditional distributions of labelled data.
problem No existing method directly compresses the conditional distribution of labelled data.
method Introduce Average Maximum Conditional Mean Discrepancy (AMCMD), derive a closed form estimator, and extend Kernel Herding (KH) to Average Conditional Kernel Herding (ACKH).
result Directly compressing conditional distributions outperforms joint distribution compression and greedy selection.
Neighbour-Driven Gaussian Process Variational Autoencoders for Scalable Structured Latent Modellingcs.LG
A scalable GPVAE method using local adjacencies to approximate GP inference.
problem Scalability issues in exact GP inference for large-scale GPVAEs.
method Neighbour-driven approximation strategy that confines computations to nearest neighbours.
result Outperforms other GPVAE variants in predictive performance and computational efficiency.
Kernel-based SSL creates useful representations without labels.
problem Creating useful representations without labels using self-supervised learning.
method Derive methods for kernel-based SSL, focusing on contrastive and non-contrastive loss functions.
result Kernel-induced representations correlate related points and de-correlate unrelated ones.
The paper introduces a new method for feature selection without explicit sparsification.
problem Feature selection with implicit sparsity-inducing mechanisms.
method Optimization over a family of kernels with gradient descent.
result The method achieves exact sparsity without explicit penalization techniques.
We consider a geometric flow introduced by Gigli and Mantegazza which, in the case of smooth compact manifolds with smooth metrics, is tangen- tial to the Ricci flow almost-everywhere along geodesics. To study spaces with geometric singularities, we consider this flow in the context of smooth manifolds with rough metri…
A new method for privacy-preserving data distillation using wavelet features from ScatterNet.
problem Privacy-preserving data distillation for small datasets.
method Differentially private kernel inducing points using wavelet features from ScatterNet.
result Achieves privacy-preserving data distillation with manageable computational cost.
In this paper, we bound the error induced by using a weighted skeletonization of two data sets for computing a two sample test with kernel maximum mean discrepancy. The error is quantified in terms of the speed in which heat diffuses from those points to the rest of the data, as well as how at the weights on the refere…
Improves learning of spectral mixture kernels with approximate Bayesian inference.
problem Difficult optimization of large number of SM kernel parameters.
method Approximate Bayesian inference using variational distribution of spectral points and random Fourier features.
result Accelerates convergence and leads to better optimal parameters.
Unified framework combines trace-induced quantum kernels for improved machine learning models.
problem Improving performance of quantum machine learning models using trace-induced kernels.
method Developed a unified framework combining various trace-induced quantum kernels, including global fidelity and local projected kernels, as Lego kernels.
result Local projected kernels can achieve comparable performance to global fidelity kernels with fewer quantum resources.
Introducing user-prescribed constraints in Markov chains for nonlinear dimensionality reductioncs.LG
Stochastic kernel based dimensionality reduction approaches have become popular in the last decade. The central component of many of these methods is a symmetric kernel that quantifies the vicinity between pairs of data points and a kernel-induced Markov chain on the data. Typically, the Markov chain is fully specified…
pFedGP uses Gaussian processes for personalized federated learning with improved kernel learning.
problem Learning personalized models across clients with limited data.
method pFedGP uses Gaussian processes with deep kernel learning, including a shared neural network kernel and inducing points.
result pFedGP achieves well-calibrated predictions and significantly outperforms baseline methods.
Solving Support Vector Machines in Reproducing Kernel Banach Spaces with Positive Definite Functionsstat.ML
In this paper we solve support vector machines in reproducing kernel Banach spaces with reproducing kernels defined on nonsymmetric domains instead of the traditional methods in reproducing kernel Hilbert spaces. Using the orthogonality of semi-inner-products, we can obtain the explicit representations of the dual (nor…
We introduce a framework and early results for massively scalable Gaussian processes (MSGP), significantly extending the KISS-GP approach of Wilson and Nickisch (2015). The MSGP framework enables the use of Gaussian processes (GPs) on billions of datapoints, without requiring distributed inference, or severe assumption…
This note explains when neural networks can be seen as Gaussian processes.
problem Understanding the relationship between neural networks and Gaussian processes.
method Formulating a Gaussian process regression based on neural network outputs and analyzing the resulting posterior mean functions.
result The posterior mean functions of neural networks follow a Gaussian process in certain cases, providing an interpretation of reproducing kernel Hilbert spaces.
New scalable GP approximation using Fourier series decomposition.
problem Scalability and accuracy in Gaussian process approximations.
method Harmonic kernel decomposition (HKD) to decompose kernels orthogonally.
result Significantly outperforms standard variational methods in scalability and accuracy.
Deep kernel processesstat.ML
Deep kernel processes unify various models using Gram matrices and kernel functions.
problem Unified representation of various deep learning models.
method Defining deep kernel processes with progressively transformed Gram matrices and sampling from inverse Wishart distributions.
result Deep Gaussian processes, BNNs, infinite BNNs, and infinite BNNs with bottlenecks can all be written as deep kernel processes.
We present a practical way of introducing convolutional structure into Gaussian processes, making them more suited to high-dimensional inputs like images. The main contribution of our work is the construction of an inter-domain inducing point approximation that is well-tailored to the convolutional kernel. This allows …
Algebraic topology methods have recently played an important role for statistical analysis with complicated geometric structured data such as shapes, linked twist maps, and material data. Among them, \textit{persistent homology} is a well-known tool to extract robust topological features, and outputs as \textit{persist…
Deep neural networks for structured prediction using kernel-induced losses.
problem Structured prediction tasks for images and texts.
method Designing a novel family of deep neural architectures that predict in a finite-dimensional subspace derived from the kernel-induced loss.
result Gradient descent algorithms can be used for structured prediction with deep neural networks.
New sparse GP model learns compositional kernels efficiently.
problem Learning accurate Gaussian Process models with complex kernel structures.
method MultiSVGP model with Horseshoe prior for kernel selection.
result Our model provides better fit and faster computation for large-scale data.
Proposes DAK model for improved GP computations.
problem Challenges in high-dimensional GP layers in DKL.
method Additive structure and induced prior approximation for GP units.
result Outperforms state-of-the-art DKL methods in regression and classification.
Decision forests are widely used for classification and regression tasks. A lesser known property of tree-based methods is that one can construct a proximity matrix from the tree(s), and these proximity matrices are induced kernels. While there has been extensive research on the applications and properties of kernels, …
Geometric quantization results for Riemann surfaces with semi-positive line bundles.
problem Analyzing geometric quantization for Riemann surfaces with semi-positive line bundles.
method Exploring the Bergman kernel expansion and related results for induced Fubini-Study metrics, Toeplitz operators, and holomorphic torsion.
result Asymptotic results for holomorphic torsion and random sections.
Sparse Gaussian process hyperparameters optimized using MCMC.
problem Hyperparameter uncertainty leads to biased estimates and underestimation of predictive uncertainty.
method Proposes an MCMC algorithm to sample from the hyperparameter posterior in sparse Gaussian process regression.
result Significantly improves sampling efficiency in the Gaussian likelihood case.
Unified analysis for nonlinear parametric models in Bayesian optimization.
problem Limited theoretical guarantees for nonlinear parametric models in Bayesian optimization.
method Kernel-based framework for analyzing regularized nonlinear parametric models trained on adaptively collected data.
result Unified convergence guarantees for nonlinear acquisition and surrogate models.