Derives kernel PCA with Nyström method for scalability.
problem Scalability of kernel PCA.
method Nyström method for kernel PCA.
result Provides scalable alternative to full kernel PCA.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “scalable kernel methods”
We introduce a new structured kernel interpolation (SKI) framework, which generalises and unifies inducing point methods for scalable Gaussian processes (GPs). SKI methods produce kernel approximations for fast computations through kernel interpolation. The SKI framework clarifies how the quality of an inducing point a…
DKL-KAN combines deep learning and kernel methods for scalable, expressive models.
problem Combining deep learning's depth with kernel methods' flexibility for scalable models.
method DKL-KAN uses Kolmogorov-Arnold Networks (KAN) to optimize kernel attributes within a Gaussian process framework.
result DKL-KAN outperforms DKL-MLP on datasets with a low number of observations and DKL-MLP on large datasets.
Deep Kernel Learningcs.LG
We introduce scalable deep kernels, which combine the structural properties of deep learning architectures with the non-parametric flexibility of kernel methods. Specifically, we transform the inputs of a spectral mixture base kernel with a deep architecture, using local kernel interpolation, inducing points, and struc…
Scalable kernel methods for large datasets using Fourier representations and NUFFT.
problem Cubic complexity in kernel methods limits their use on large-scale datasets.
method Fourier representation of kernels combined with NUFFT for O(n log n) complexity.
result Achieves minimax convergence rates and processes up to tens of billions of samples.
Kernel approximation methods create explicit, low-dimensional kernel feature maps to deal with the high computational and memory complexity of standard techniques. This work studies a supervised kernel learning methodology to optimize such mappings. We utilize the Discriminant Information criterion, a measure of class …
New scalable GP approximation using Fourier series decomposition.
problem Scalability and accuracy in Gaussian process approximations.
method Harmonic kernel decomposition (HKD) to decompose kernels orthogonally.
result Significantly outperforms standard variational methods in scalability and accuracy.
Nyström approximation for scalable operator learning
problem Scalability of operator learning for large datasets
method Nyström subsampling with operator learning
result Minimax-optimal convergence rates for functional outputs
SoftKI combines SKI and variational methods for scalable GP regression.
problem Scalable Gaussian Process regression on high-dimensional datasets.
method SoftKI approximates kernel via softmax interpolation from a smaller number of learned points.
result SoftKI is competitive with other approximated GP methods for modest data dimensions.
Neural-Kernel CME tackles scalability and expressiveness challenges in conditional distribution representation.
problem Scalability and expressiveness challenges in kernel conditional mean embeddings.
method Combines deep learning with CMEs using a neural network optimization framework.
result Achieves competitive and often superior performance in conditional density estimation and RL.
A new Gaussian process framework uses neural feature maps for scalable, accurate inference.
problem Efficient and accurate Gaussian process inference for diverse data types.
method Neural feature maps to construct expressive kernels, with theoretical guarantees and practical scalability.
result The approach outperforms existing methods in accuracy and efficiency across various data modalities.
Kernel methods on discrete domains have shown great promise for many challenging data types, for instance, biological sequence data and molecular structure data. Scalable kernel methods like Support Vector Machines may offer good predictive performances but do not intrinsically provide uncertainty estimates. In contras…
DSoftKI scales GP regression with full derivative observations.
problem Efficiently fitting and predicting full derivative observations in Gaussian Processes.
method Extends SoftKI by using local temperature vectors for interpolation, enabling encoding of local directional sensitivity.
result DSoftKI achieves accurate predictions and scales to larger datasets with full derivative observations.
DBKs enable scalable GPs with tractable inference for large datasets.
problem Scaling Gaussian processes to large and complex datasets while maintaining tractable inference.
method DBKs constructed from neural-network-parameterized basis functions with explicit low-rank structure, enabling linear-complexity inference.
result DBKs provide a unified perspective and improve predictive accuracy, uncertainty quantification, and computational efficiency.
Kernel methods are ubiquitous tools in machine learning. However, there is often little reason for the common practice of selecting a kernel a priori. Even if a universal approximating kernel is selected, the quality of the finite sample estimator may be greatly affected by the choice of kernel. Furthermore, when direc…
Scalable machine learning with path signatures for time series and graphs.
problem Challenges in real-world time series and graph data.
method Combines rough path theory with probabilistic, deep, and kernel methods.
result Scalable models for time series and graph data.
We investigate the capabilities and limitations of Gaussian process models by jointly exploring three complementary directions: (i) scalable and statistically efficient inference; (ii) flexible kernels; and (iii) objective functions for hyperparameter learning alternative to the marginal likelihood. Our approach outper…
From Local Interactions to Global Operators: Scalable Gaussian Process Operator for Physical Systemsstat.ML
Scalable Gaussian Process Operator tackles high-dimensional PDEs.
problem Scaling Gaussian Process Operators to high-dimensional, data-intensive regimes.
method Nearest-neighbor-based local kernel approximations, sparse kernel approximation, structured Kronecker factorizations, operator-aware kernel structures, task-informed mean functions.
result Consistently achieves high accuracy across varying discretization scales.
New kernel enables exact GP analysis of massive datasets.
problem Limited scalability and flexibility of traditional GPs for large data sets.
method Explicitly derived nonstationary kernel for large-scale GP analysis.
result Novel kernel outperforms existing methods in synthetic and real-world applications.
We provide the first mathematically complete derivation of the Nyström method for low-rank approximation of indefinite kernels and propose an efficient method for finding an approximate eigendecomposition of such kernel matrices. Building on this result, we devise highly scalable methods for learning in reproducing ker…
Exact and scalable algorithm for Gaussian process regression with Matérn correlations.
problem Efficient Gaussian process regression with Matérn correlations.
method Novel kernel packet theory and sparse representation of covariance matrix.
result Significantly superior to existing alternatives in computational time and predictive accuracy.
New scalable MCMC sampling for nonsymmetric DPPs speeds up computations.
problem Efficient sampling for nonsymmetric DPPs with low-rank kernels.
method Enhanced rejection sampling with efficient proposal distribution construction.
result Sublinear runtime for scalable MCMC sampling of -NDPPs.
Connections and Equivalences between the Nyström Method and Sparse Variational Gaussian Processesstat.ML
Clarifies connections between Nyström and SVGP methods for scalable GPs.
problem Lack of understanding between GP and kernel methods communities.
method Investigates Nyström and SVGP methods for scalable Gaussian processes.
result Establishes connections and equivalences between Nyström and SVGP methods.
Bayesian tensor train kernel machine uses Laplace approximation for scalable GP regression.
problem Scalability limitations of Gaussian process regression.
method Bayesian tensor train kernel machine with Laplace approximation and variational inference.
result VI replaces cross-validation and offers up to 65x faster training.
New DCD and BDCD methods for K-SVM and K-RR reduce communication costs.
problem Scalability issues in DCD and BDCD methods for kernel problems.
method Developed scalable -step variants of DCD and BDCD methods.
result Achieved strong scaling speedups of up to 9.8x over existing methods.
The application of Gaussian processes (GPs) to large data sets is limited due to heavy memory and computational requirements. A variety of methods has been proposed to enable scalability, one of which is to exploit structure in the kernel matrix. Previous methods, however, cannot easily deal with non-stationary process…
We propose a generic spatiotemporal event forecasting method, which we developed for the National Institute of Justice's (NIJ) Real-Time Crime Forecasting Challenge. Our method is a spatiotemporal forecasting model combining scalable randomized Reproducing Kernel Hilbert Space (RKHS) methods for approximating Gaussian …
Enhances GPLVM for multi-view data with scalable latent representation learning.
problem Limited kernel expressiveness and computational inefficiency in multi-view GPLVM.
method Introduces a new duality between spectral density and kernel function, uses NG-SM kernel, and applies random Fourier feature approximation for scalability.
result Consistently outperforms state-of-the-art models in learning meaningful latent representations across diverse datasets.
Nystrom approximation speeds up kernel model training.
problem Slow convergence in kernel models due to poor conditioning.
method Spectral preconditioning with Nystrom approximation for scalability.
result Nystrom approximation accelerates gradient descent nearly as well as exact preconditioner.
New STKR estimators use unlabeled data for smoother function learning.
problem Leveraging unlabeled data for smoother function learning.
method Spectrally transformed kernel regression (STKR) with scalable implementations.
result STKR can learn any sufficiently smooth function.
Scalable Gaussian processes with latent Kronecker structure for large datasets.
problem Limited scalability of Gaussian processes for large datasets.
method Leveraging latent Kronecker structure, projecting kernel matrix onto latent Kronecker product, using iterative linear system solvers and pathwise conditioning.
result Outperforms state-of-the-art sparse and variational GPs on real-world datasets with up to five million examples.
Scalable methods integrate multiview data for clinical outcomes.
problem Jointly associate and predict outcomes from multiple data sources.
method Randomized Fourier bases for nonlinear mappings, view-independent low-dimensional representations.
result Identified molecular signatures for COVID-19 status and severity.
New method tests conditional independence using spectral representations.
problem Untestable conditional independence in many settings.
method Spectral representations of partial covariance operators, bi-level contrastive learning.
result Asymptotic validity and power guarantees for CI testing.
The general perception is that kernel methods are not scalable, and neural nets are the methods of choice for nonlinear learning problems. Or have we simply not tried hard enough for kernel methods? Here we propose an approach that scales up kernel methods using a novel concept called "doubly stochastic functional grad…
Kernel Density Machinesstat.ML
Kernel Density Machines learn probability densities without structural assumptions.
problem Learning probability densities under minimal assumptions.
method Kernel-based framework, agnostic to structural requirements.
result Consistency and functional central limit theorem for sample estimator.
Proposes IGN for scalable Gaussian process networks.
problem Scalability and expressivity challenges in Gaussian processes.
method Inducing Gaussian process networks with learned inducing points.
result Significant advances over state-of-the-art methods.
A new GP inference method using simplices for high-dimensional data.
problem Scalable Gaussian Processes in high dimensions.
method Developed a Simplex-GP method using a sparse simplicial grid to accelerate MVMs.
result Significantly faster GP inference in high dimensions compared to SKI.
Neighbour-Driven Gaussian Process Variational Autoencoders for Scalable Structured Latent Modellingcs.LG
A scalable GPVAE method using local adjacencies to approximate GP inference.
problem Scalability issues in exact GP inference for large-scale GPVAEs.
method Neighbour-driven approximation strategy that confines computations to nearest neighbours.
result Outperforms other GPVAE variants in predictive performance and computational efficiency.
A fast and scalable method for variable selection in high-dimensional Gaussian processes.
problem Inefficient variable selection in high-dimensional Gaussian processes.
method Developed a fast and scalable variational inference algorithm for spike and slab Gaussian processes.
result Consistently outperforms vanilla and sparse variational GPs while retaining similar runtimes.
A scalable ROC-SVM variant reduces training time for imbalanced binary classification.
problem High computational cost of ROC-SVM for imbalanced binary classification.
method Incomplete U-statistics and low-rank kernel approximation.
result Comparable AUC performance with reduced training time.
Gaussian processes (GPs) are flexible non-parametric models, with a capacity that grows with the available data. However, computational constraints with standard inference procedures have limited exact GPs to problems with fewer than about ten thousand training points, necessitating approximations for larger datasets. …
New methods for scalable causal discovery from complex data.
problem Learning causal structures from nonlinear, continuous or mixed data.
method BF-BIC score and BF-LRT test for scalable causal discovery.
result BF-BIC score and BF-LRT test enable scalable causal discovery with competitive accuracy and runtime.
Kernel methods have great promise for learning rich statistical representations of large modern datasets. However, compared to neural networks, kernel methods have been perceived as lacking in scalability and flexibility. We introduce a family of fast, flexible, lightly parametrized and general purpose kernel learning …
Survival kernets scale deep kernel survival analysis to large datasets with interpretability and theoretical guarantees.
problem Scalable and interpretable deep kernel survival analysis for large datasets.
method Survival kernets use kernel netting for training set compression and XGBoost for warm-starting neural architecture search.
result Survival kernets achieve optimal time-dependent concordance index on various survival analysis datasets.
New classifier combines locally linear kernels for fast and accurate non-linear classification.
problem Developing a fast and accurate non-linear classifier.
method Combines locally linear classifiers using a Multiple Kernel Learning (MKL) problem with scalable MKL training for streaming kernels.
result The resulting classifier achieves high accuracy with fast inference time.
Fast Estimation of Information Theoretic Learning Descriptors using Explicit Inner Product Spacescs.LG
Kernel methods form a theoretically-grounded, powerful and versatile framework to solve nonlinear problems in signal processing and machine learning. The standard approach relies on the \emph{kernel trick} to perform pairwise evaluations of a kernel function, leading to scalability issues for large datasets due to its …
New SMC method for pBNNs improves scalability and predictive performance.
problem Training pBNNs with high-dimensional stochastic parameters.
method Gradient-based proposals within SMC samplers.
result New method outperforms state-of-the-art in predictive performance and training time.
We present a scalable Gaussian process model for identifying and characterizing smooth multidimensional changepoints, and automatically learning changes in expressive covariance structure. We use Random Kitchen Sink features to flexibly define a change surface in combination with expressive spectral mixture kernels to …