Deep kernel learning improves performance on complex tasks.
problem Limited flexibility of classical kernel methods for complicated tasks.
method Random Fourier Features (RFF) integrated into deep architecture.
result Significantly boosts flexibility and richness of kernel machines.
Combines kernels to create flexible priors in BNNs for seasonal and trend data.
problem Creating flexible priors in Bayesian neural networks for complex data.
method Derives BNN architectures from kernel combinations and periodic functions.
result BNNs can produce periodic kernels useful for capturing seasonal and trend data.
A new method for deep Wishart processes improves kernel-based models.
problem Inference in deep Wishart processes is challenging due to the need for flexible distributions over positive semi-definite matrices.
method Developed a novel approach to flexible distributions over positive semi-definite matrices using the Bartlett decomposition of the Wishart probability density. Used this to create an approximate posterior for the DWP.
result Improved performance of inference in the DWP compared to DGP with equivalent prior.
New methods complete multiple incomplete kernel matrices while controlling model flexibility.
problem Incomplete data in multiple kernel learning.
method Parameterized model matrix with restrictions on model covariance and use of LogDet divergence to ensure positive definiteness.
result Proposed methods yield significant improvements in generalization performance.
Proposes a new method for learning flexible nonparametric kernels.
problem Improving model flexibility in margin-based kernel methods.
method Data-adaptive non-parametric kernel learning framework with two constraints.
result Enhanced model flexibility and improved performance on benchmark data sets.
The paper tackles model collapse in GPLVMs by improving kernel flexibility and projection variance.
problem Model collapse in GPLVMs leading to vague latent representations.
method Theoretical analysis of projection variance, integration of SM and RFF kernels, and variational inference.
result The advisedRFLVM outperforms competing models in informative latent representations and missing data imputation.
Kafnets introduces flexible neural network activation functions based on kernels.
problem Lack of flexible activation functions in neural networks.
method Kernel-based non-parametric activation functions (KAFs).
result KAFs can approximate any mapping and are smooth, linear in parameters, and regularizable.
Efficiently marginalizes over Gaussian Process kernels for better model flexibility and uncertainty.
problem Inefficient marginalization over Gaussian Process kernels for large datasets.
method Bayesian Quadrature scheme with maximum mean discrepancies and invariances between Spectral Mixture kernels.
result Achieves more accurate predictions and better calibrated uncertainty than state-of-the-art baselines.
Proposes flexible dilation networks for better time series analysis.
problem Fixed dilation limits flexibility in time series analysis.
method End-to-end learnable dilation layers and independent kernels.
result Improves efficiency and flexibility in training.
We introduce scalable deep kernels, which combine the structural properties of deep learning architectures with the non-parametric flexibility of kernel methods. Specifically, we transform the inputs of a spectral mixture base kernel with a deep architecture, using local kernel interpolation, inducing points, and struc…
SOBER framework optimizes Bayesian optimization tasks efficiently.
problem Challenges in parallel Bayesian optimization.
method Probabilistic Lifting with Kernel Quadrature.
result Versatile and flexible batch Bayesian optimization.
Kernel methods have great promise for learning rich statistical representations of large modern datasets. However, compared to neural networks, kernel methods have been perceived as lacking in scalability and flexibility. We introduce a family of fast, flexible, lightly parametrized and general purpose kernel learning …
Neural Kernel Network (NKN) learns flexible kernels from neural networks.
problem Choosing the right kernel for Gaussian processes is challenging.
method NKN uses a neural network to represent kernels based on composition rules, making them differentiable and trainable.
result NKN can approximate complex kernel structures and is universal for stationary kernels.
A-NICE-MC trains flexible MCMC kernels to efficiently sample from complex distributions.
problem Slow convergence and poor performance of general-purpose MCMC methods.
method Adversarial training with flexible volume-preserving flows to optimize MCMC kernels.
result A-NICE-MC significantly outperforms competing methods like Hamiltonian Monte Carlo.
New method tackles high-dimensional contextual bandits with flexible kernel models.
problem Maximizing rewards in decision-making scenarios with many features.
method Introduces stochastic assumptions and no-regret learning for Gaussian kernels.
result Achieves no-regret learning even with feature dimensions growing with samples.
Improved RNNs with flexible gates using kernel activation functions.
problem Modeling long-term dependencies in sequential data.
method Designed a more flexible architecture with adaptable parameters using kernel activation functions.
result Improved accuracy with negligible computational cost and speed-up in training iterations.
Researchers develop flexible kernels for biological sequences with guaranteed reliability.
problem Challenges in applying machine learning to biological sequences, including unreliable methods.
method Theoretical analysis and development of modified kernels to ensure reliability and accuracy.
result Developed kernels that are universal, characteristic, and metrize the space of distributions for biological sequences.
Paper introduces kernel deformed exponential families for sparse continuous attention.
problem Creating efficient attention mechanisms for sparse data.
method Developed kernel deformed exponential families, theoretically and experimentally.
result Kernel deformed exponential families can attend to multiple compact regions of data.
The NNGP kernel's predictions closely match those of the Matern kernel under certain conditions.
problem Comparing NNGP kernels to Matern kernels in practical applications.
method Demonstrated the necessity of normalization for NNGP kernels, explored numerical challenges, and compared predictions and performance.
result NNGP kernel predictions closely match Matern kernel predictions under specific circumstances.
Flexible Kernels for Protein Property Prediction
problem Predicting protein properties from sparse experimental data
method Sequence kernels using evolutionary substitution matrices and local linearity
result Data-efficient models of protein property landscapes
SRF improves kernel approximation and GP regression performance.
problem Efficient kernel approximation and Bayesian kernel learning in large-scale regression problems.
method Stein variational gradient descent to generate high-quality random features and approximate spectral measure posteriors.
result SRF outperforms traditional approaches in kernel approximation and GP regression.
New Hida-Matérn kernels enable flexible process priors and efficient GP inference.
problem Flexible modeling of stationary processes with oscillatory components.
method Introducing a new class of covariance functions (Hida-Matérn kernels) and their state space representations.
result Efficient Gaussian Process inference and improved numerical stability.
Novel neural GP kernels learn stable, flexible covariance structures.
problem Scalable and flexible covariance kernels for Gaussian processes.
method Directly learn kriging coefficients and conditional standard deviations using deep neural architectures exploiting permutation-equivariant structure.
result Improved training stability and data efficiency with expressive, non-stationary kernels.
Develops a kernel-based framework for dynamic trading strategies.
problem Optimizing portfolios with temporal dependencies in asset dynamics.
method Parameterizes trading strategies as functions in RKHS, enabling flexible, non-Markovian approaches.
result Significantly outperforms classical Markovian methods in synthetic and market-data examples.
AutoGP improves Gaussian process models for classification tasks.
problem Improving Gaussian process models for classification tasks.
method Investigates scalable and statistically efficient inference, flexible kernels, and alternative hyperparameter learning objectives.
result Breaks the 1% error-rate barrier in GP models using the MNIST8M dataset.
Flexible nonstationary Gaussian process with neural network parameters.
problem Limited expressiveness of stationary Gaussian processes.
method Nonstationary kernels with neural network parameters trained jointly.
result Better accuracy and log-score compared to stationary and hierarchical models.
Graph convolutional deep kernel machine learns representations for graph tasks.
problem Limited representation learning in infinite-width neural networks.
method Developed a graph convolutional deep kernel machine as an infinite-width limit.
result Representation learning improves performance for heterophilous node classification tasks.
Exact Gaussian Processes for massive datasets using non-stationary sparsity-discovering kernels.
problem High computational and storage costs for exact GPs in large datasets.
method Develop non-stationary kernels that allow the GP to discover sparse structure naturally.
result Exact Gaussian Processes scalable to over 5 million data points.
Determinantal point processes (DPPs) offer a powerful approach to modeling diversity in many applications where the goal is to select a diverse subset. We study the problem of learning the parameters (the kernel matrix) of a DPP from labeled training data. We make two contributions. First, we show how to reparameterize…
Paper bridges VAEs and KDEs for more flexible posterior estimation.
problem Limitations of Gaussian latent space in VAEs and challenges in KL-divergence estimation.
method Approximate posterior with KDEs and derive upper bound of KL-divergence in ELBO.
result Epanechnikov kernel minimizes KL-divergence upper bound asymptotically.
A flexible variable selection algorithm with sparsistency guarantees.
problem Variable selection in high-dimensional data with theoretical guarantees.
method Kernel-based estimation of regression and gradient functions, followed by hard thresholding.
result Desirable asymptotic sparsistency established for general RKHS.
Empirical Gaussian Processes learn flexible priors from data.
problem Limited effectiveness of standard Gaussian process kernels.
method Estimate mean and covariance functions empirically from data.
result Empirical GPs converge to closest GP to real data generating process.
Flexible band grouping and kernel fusion for hyperspectral image processing.
problem Large dimensionality in hyperspectral imaging.
method Non-contiguous and contiguous band grouping for dimensionality reduction; improved visual clustering; unsupervised clustering algorithms; diverse features via different proximity metrics and kernel functions; l∞-norm multiple kernel learning. result Heterogeneous features and kernels lead to performance gain.
Develops flexible non-parametric ACFs using B-spline kernels.
problem Flexible modelling of the autocovariance function (ACF) in time-series, spatial, and spatio-temporal analysis.
method Derives the inverse Fourier transform of B-spline spectral bases to create a general class of non-parametric ACFs.
result Provides a provably dense, flexible, and general class of non-parametric ACFs for various types of processes.
Paper proposes a method to improve interpretability in kernel learning models.
problem Improving interpretability in flexible machine learning models.
method Proposes a quantitative index for interpretability and a universal learning framework to balance interpretability and generalization performance.
result Demonstrates a method to achieve global optimal solution in balancing interpretability and generalization performance.
Study models weather index insurance pricing by insurers and farmers, finding flexible pricing kernels boost profits.
problem Monopoly pricing of weather index insurance with risk and flexibility considerations.
method Bowley-type sequential game with insurer and farmer, using neural networks for farmer's payoff.
result Flexible pricing kernels increase insurer profits closer to indemnity insurance levels.
Paper shows SVMs can interpolate data in various settings.
problem Understanding SVM performance and generalization.
method Flexible analysis framework for proving SVM interpolation in diverse settings.
result Support vector machines can interpolate data in many cases not previously covered.
Flexible GP model improves wind power prediction accuracy.
problem Accurate probabilistic prediction of wind power for grid stability.
method Heteroscedastic non-stationary Gaussian process with generalised spectral mixture kernel.
result The proposed model outperforms conventional GP models in wind power prediction.
Develops flexible algorithms for estimating heterogeneous treatment effects.
problem Flexible estimation of heterogeneous treatment effects in observational studies.
method Two-step algorithm: 1) Estimate marginal effects and treatment propensities, 2) Optimize data-adaptive objective function.
result Achieves quasi-oracle property, leading to error bounds similar to an oracle.
CPME embeds counterfactual outcomes in RKHS for flexible policy evaluation.
problem Estimating counterfactual policy outcomes for decision-making.
method Counterfactual Policy Mean Embedding (CPME) framework in RKHS, plug-in and doubly robust estimators, kernel test statistic.
result Doubly robust estimator improves convergence rates and asymptotic normality.
We propose graph kernels based on subgraph matchings, i.e. structure-preserving bijections between subgraphs. While recently proposed kernels based on common subgraphs (Wale et al., 2008; Shervashidze et al., 2009) in general can not be applied to attributed graphs, our approach allows to rate mappings of subgraphs by …
Paper proposes a new method for learning kernels that depend on both inputs and outputs.
problem Common kernels are limited in their ability to handle complex tasks.
method Developed a spectral kernel learning framework that uses non-stationary kernels and learns from data.
result Derived a data-dependent generalization error bound and suggested regularization terms.
Improves regression efficiency by separating material and immaterial parts of responses.
problem Improving estimation efficiency in nonlinear multivariate regressions.
method Kernel envelope (KENV) estimator for nonparametric response envelopes in reproducing kernel Hilbert space.
result KENV achieves lower in-sample prediction risk than kernel ridge regression in non-trivial immaterial components.
Efficiently estimates Hawkes process kernels using non-parametric Bayesian methods.
problem Estimating flexible Hawkes process kernels with uncertainty quantification.
method Cluster representation of Hawkes processes, Gibbs sampling, expectation maximization.
result Linear time complexity in both theoretical and empirical settings.
Unified view of score estimators for flexible densities.
problem Estimating the score from unknown distributions.
method Regularized nonparametric regression framework.
result Unified convergence analysis and new estimators with desirable properties.
Improved sentiment analysis with any-gram kernels.
problem Sentiment classification in text data.
method Proposed a new any-gram kernel approach independent of tree kernels, making use of word embeddings more effectively.
result Significantly better performance in sentiment classification.
A new HP model balances interpretability and flexibility for EHR event sequences.
problem Balancing interpretability and flexibility in modeling diagnostic event sequences.
method Proposes a neural network-based HP with flexible impact kernel and transformer layers.
result Accurately recovers impact functions, competitive performance, and clinically meaningful interpretation.
Enhances kernel regression with network data for better predictions.
problem Improving predictive power in high-dimensional data.
method Combines kernel regression with network cohesion data to model nonlinearities.
result Significantly better predictive performances in high-dimensional data.