Sparse Kernel Flows learns dynamical systems from data.
problem Learning dynamical systems from limited data.
method Sparse Kernel Flows: trains optimal kernel from a dictionary of kernels.
result Sparse Kernel Flows can learn from 132 chaotic systems.
New approach to learning kernels from data using AIT principles.
problem Learning kernels from data in machine learning.
method Sparse Kernel Flows method based on AIT principles.
result Sparse Kernel Flows aligns with MDL principle and offers a robust theoretical foundation.
Kernel regression is a popular non-parametric fitting technique. It aims at learning a function which estimates the targets for test inputs as precise as possible. Generally, the function value for a test input is estimated by a weighted average of the surrounding training examples. The weights are typically computed b…
Kernelised flows improve density estimation and generation with fewer parameters.
problem Limited expressiveness of flow-based models due to invertibility constraints.
method Integrates kernels into normalising flows to enhance expressiveness and efficiency.
result Kernelised flows outperform neural network-based flows in parameter efficiency and low-data scenarios.
New GPU kernels boost deep learning speed and memory efficiency.
problem Sparse deep learning matrices are not well-suited for existing sparse kernels.
method Identified favorable properties of sparse matrices from deep learning, developed high-performance GPU kernels for sparse matrix operations.
result 27% of single-precision peak performance on Nvidia V100 GPUs achieved with new kernels.
Sparse Gaussian processes with compact kernels for faster inference.
problem Efficient Gaussian process inference with high computational complexity.
method Parametric families of compactly-supported kernels for sparse matrix representations.
result Sub-quadratic inference complexity and improved performance on real-world tasks.
Paper introduces kernel deformed exponential families for sparse continuous attention.
problem Creating efficient attention mechanisms for sparse data.
method Developed kernel deformed exponential families, theoretically and experimentally.
result Kernel deformed exponential families can attend to multiple compact regions of data.
New sparse GP model learns compositional kernels efficiently.
problem Learning accurate Gaussian Process models with complex kernel structures.
method MultiSVGP model with Horseshoe prior for kernel selection.
result Our model provides better fit and faster computation for large-scale data.
SKI accelerates GP inference with sparse grids to handle higher dimensions.
problem SKI scales poorly in high dimensions due to dense grid size.
method Sparse grids within SKI framework, novel matrix-vector multiplication algorithm.
result SKI can be scaled to higher dimensions while maintaining accuracy.
Canonical correlation analysis (CCA) is a multivariate statistical technique for finding the linear relationship between two sets of variables. The kernel generalization of CCA named kernel CCA has been proposed to find nonlinear relations between datasets. Despite their wide usage, they have one common limitation that…
Efficiently scales continuous kernels with sparse Fourier domain learning.
problem High computational and memory demands, spectral bias in continuous kernels.
method Sparse learning in the Fourier domain.
result Efficient scaling of continuous kernels, reduced computational and memory requirements, mitigated spectral bias.
Signal processing tasks as fundamental as sampling, reconstruction, minimum mean-square error interpolation and prediction can be viewed under the prism of reproducing kernel Hilbert spaces. Endowing this vantage point with contemporary advances in sparsity-aware modeling and processing, promotes the nonparametric basi…
Recent advances suggest that a wide range of computer vision problems can be addressed more appropriately by considering non-Euclidean geometry. This paper tackles the problem of sparse coding and dictionary learning in the space of symmetric positive definite matrices, which form a Riemannian manifold. With the aid of…
Zero-inflated datasets, which have an excess of zero outputs, are commonly encountered in problems such as climate or rare event modelling. Conventional machine learning approaches tend to overestimate the non-zeros leading to poor performance. We propose a novel model family of zero-inflated Gaussian processes (ZiGP) …
Extends heat kernel estimates for super Ricci flow.
problem Heat kernel estimates for super Ricci flow.
method Generalizes Bamler-Zhang's geometric analysis to super Ricci flow.
result Obtains Gaussian heat kernel estimates for super Ricci flow.
In presence of sparse noise we propose kernel regression for predicting output vectors which are smooth over a given graph. Sparse noise models the training outputs being corrupted either with missing samples or large perturbations. The presence of sparse noise is handled using appropriate use of ℓ1-norm along-wi…
Novel confidence intervals improve convergence rates for sparse kernel-based models.
problem High computational cost in kernel-based learning models.
method Novel confidence intervals for Nyström method and sparse variational Gaussian process approximation.
result Improved performance bounds in regression and optimization problems.
Paper explores Fisher-Rao gradient flows and their kernel approximations.
problem Understanding and analyzing approximations of Fisher-Rao gradient flows.
method Rigorous investigation of Fisher-Rao and Wasserstein type gradient flows, focusing on kernel approximations.
result Proves evolutionary Γ-convergence for kernel-approximated Fisher-Rao flows, providing theoretical guarantees.
Paper advances sparse regularisation theory for measures with new kernel insights.
problem Estimating sparse measures from noisy observations using continuous sparse regularisation.
method Develops new continuous sparse regularisation theory on measures with Beurling-LASSO, introduces kernel switch analysis.
result Proves the ``sinc-4'' kernel satisfies a technical LPC assumption for error bounds.
Learning linear combinations of multiple kernels is an appealing strategy when the right choice of features is unknown. Previous approaches to multiple kernel learning (MKL) promote sparse kernel combinations to support interpretability and scalability. Unfortunately, this 1-norm MKL is rarely observed to outperform tr…
The term "CoRE kernel" stands for correlation-resemblance kernel. In many applications (e.g., vision), the data are often high-dimensional, sparse, and non-binary. We propose two types of (nonlinear) CoRE kernels for non-binary sparse data and demonstrate the effectiveness of the new kernels through a classification ex…
Kernel means are frequently used to represent probability distributions in machine learning problems. In particular, the well known kernel density estimator and the kernel mean embedding both have the form of a kernel mean. Unfortunately, kernel means are faced with scalability issues. A single point evaluation of the …
Sparse Bayesian learning algorithm for estimating interaction kernels in Motsch-Tadmor model.
problem Data-driven identification of asymmetric interaction kernels in the Motsch-Tadmor model.
method Variational framework reformulating kernel identification as a subspace identification problem; sparse Bayesian learning algorithm with informative priors.
result Accurate, robust, and interpretable estimation of interaction kernels across various noise levels and data regimes.
Learning can be seen as approximating an unknown function by interpolating the training data. Kriging offers a solution to this problem based on the prior specification of a kernel. We explore a numerical approximation approach to kernel selection/construction based on the simple premise that a kernel must be good if t…
Develops a new method for learning ODEs from sparse data.
problem Learning systems of ODEs from scarce, partial, and noisy data.
method Combines sparse recovery and RKHS techniques.
result Significant gains in accuracy, sample efficiency, and robustness to noise.
The paper constructs optimal confidence bands for kernel gradient flow estimators.
problem Estimating generalization error and constructing confidence bands for kernel gradient flows.
method Established convergence rates and constructed optimal confidence bands under capacity-source condition.
result Optimal confidence bands for kernel gradient flows have shrinkage rates close to minimax optimal rates.
We propose and analyze a novel framework for learning sparse representations, based on two statistical techniques: kernel smoothing and marginal regression. The proposed approach provides a flexible framework for incorporating feature similarity or temporal information present in data sets, via non-parametric kernel sm…
We develop a novel procedure for constructing confidence bands for components of a sparse additive model. Our procedure is based on a new kernel-sieve hybrid estimator that combines two most popular nonparametric estimation methods in the literature, the kernel regression and the spline method, and is of interest in it…
New method for sparse kernel selection improves prediction accuracy.
problem Sparse Multiple Kernel Learning for binary classification.
method Alternating best response algorithm with semidefinite relaxations.
result Method outperforms state-of-the-art MKL approaches in prediction accuracy.
Flow Matching improves statistical guarantees through kernel density estimation.
problem Improving statistical guarantees for generative models.
method Connecting Flow Matching to kernel density estimation and verifying optimal rates of convergence.
result Flow Matching achieves optimal rates up to logarithmic factors for large networks and on lower-dimensional manifolds.
We theoretically investigate the convergence rate and support consistency (i.e., correctly identifying the subset of non-zero coefficients in the large sample limit) of multiple kernel learning (MKL). We focus on MKL with block-l1 regularization (inducing sparse kernel combination), block-l2 regularization (inducing un…
A novel nonstationary permanental process relaxes kernel constraints and captures complex data patterns.
problem Limitations of existing permanental processes in terms of kernel types and stationarity.
method Sparse spectral representation of nonstationary kernels and hierarchical stacking of spectral feature mappings.
result Enhanced model expressiveness and reduced computational complexity.
RG-Flow combines RG and sparse priors for hierarchical image disentanglement.
problem Disentangling and manipulating image representations at different scales.
method Hierarchical flow model using RG and sparse prior distributions.
result RG-Flow enables semantic manipulation and style mixing at different image scales.
Proves upper bounds for heat kernels evolving on manifolds.
problem Bounding heat kernels on evolving manifolds.
method Logarithmic Sobolev inequalities and ultracontractivity estimates.
result Gaussian upper bounds for heat kernels are derived.
Identifies a gradient flow to solve kernel learning problems with noise reduction.
problem Kernel learning problem with Gaussian noise.
method Riemannian gradient flow with continuous Lyapunov functionals.
result Flow reduces noise and finds stationary points.
Efficiently computes sparse signature coefficients using kernels.
problem Lack of efficient methods for sparse signature coefficients.
method Signature kernels and PDE-based methods.
result Sparse groups of signature coefficients can be isolated effectively.
A new method warps inputs to learn nonstationary kernels efficiently.
problem Learning nonstationary patterns in data with varying smoothness.
method Sparse spectrum Gaussian processes with input warping as conditional Gaussian measures.
result Efficient learning of nonstationary patterns with fewer parameters.
In this paper, we study the problem of sparse multiple kernel learning (MKL), where the goal is to efficiently learn a combination of a fixed small number of kernels from a large pool that could lead to a kernel classifier with a small prediction error. We develop an efficient algorithm based on the greedy coordinate d…
Exact Gaussian Processes for massive datasets using non-stationary sparsity-discovering kernels.
problem High computational and storage costs for exact GPs in large datasets.
method Develop non-stationary kernels that allow the GP to discover sparse structure naturally.
result Exact Gaussian Processes scalable to over 5 million data points.
Paper establishes a generalization bound for gradient flow using a data-dependent kernel.
problem Understanding the generalization properties of gradient-based optimization methods.
method Establishes a generalization bound for gradient flow through a data-dependent kernel called the loss path kernel (LPK).
result The LPK captures the entire training trajectory and leads to tighter generalization guarantees.
Quantum stochastic flow computes heat kernel traces for Ricci flat manifolds.
problem Computing heat kernel traces for Ricci flat manifolds.
method Quantum stochastic differential equation (qsde) on Fock space over L2 differential 1-forms, adapted flow construction. result Trace of the connection Laplacian heat kernel can be computed over any compact Ricci-flat Riemannian manifold.
This paper shows equivalence between SVGD and BBVI using kernel gradient flows.
problem Bayesian inference methods and their equivalence.
method Formalizes equivalence between SVGD and BBVI using kernel gradient flows.
result BBVI corresponds precisely to SVGD when using the neural tangent kernel.
The paper proves existence and growth estimates for inverse mean curvature flow and related p-Laplacian Green kernel decay.
problem Existence and growth estimates for inverse mean curvature flow.
method Proving new decay estimates for the Green kernel of the p-Laplacian. result Existence and optimal growth estimates for the weak inverse mean curvature flow.
Clarifies connections between Nyström and SVGP methods for scalable GPs.
problem Lack of understanding between GP and kernel methods communities.
method Investigates Nyström and SVGP methods for scalable Gaussian processes.
result Establishes connections and equivalences between Nyström and SVGP methods.
Linear time algorithm for random walk kernels on sparse graphs.
problem Efficient computation of general random walk kernels for large graphs.
method Sample dependent random walks to compute graph embeddings without direct graph product.
result Up to 27x faster and scalable to 128x larger graphs than previous methods.
Many signal processing and machine learning methods share essentially the same linear-in-the-parameter model, with as many parameters as available samples as in kernel-based machines. Sparse approximation is essential in many disciplines, with new challenges emerging in online learning with kernels. To this end, severa…
Efficiently performs robust and sparse kernel regression.
problem Robust and sparse kernel regression.
method Sign gradient descent and early stopping.
result Sign gradient descent achieves robust and sparse kernel regression efficiently.
New method finds sparse networks without labels, improving performance.
problem Sparse connectivity in neural networks to reduce memory and energy demands.
method Neural Tangent Transfer method to find sparse networks without labels.
result Sparse networks achieve higher classification performance and faster convergence.