Kernel-based algorithms improve integral estimation with near-geometric speed.
problem Estimating integrals with target measures that are nearly atomic.
method Weighted kernel herding and sequential Bayesian quadrature.
result Near-geometric rate of convergence for nearly atomic target measures.
Kernel-based online learning has often shown state-of-the-art performance for many online learning tasks. It, however, suffers from a major shortcoming, that is, the unbounded number of support vectors, making it non-scalable and unsuitable for applications with large-scale datasets. In this work, we study the problem …
Paper proposes an efficient causal discovery method with linear computational complexity.
problem Identifying causal relationships efficiently in large datasets.
method Approximate kernel-based generalized score function with low-rank technique and sampling algorithms.
result Significantly reduces computational costs while maintaining comparable accuracy.
Meta two-sample testing uses auxiliary data to quickly find powerful tests from limited samples.
problem Challenges in identifying powerful kernels for distinguishing complex distributions with limited data.
method Introduces meta two-sample testing (M2ST) to leverage abundant auxiliary data on related tasks.
result Proposed algorithms improve over baselines and identify powerful tests from scarce observations.
Distance-based tests, also called "energy statistics", are leading methods for two-sample and independence tests from the statistics community. Kernel-based tests, developed from "kernel mean embeddings", are leading methods for two-sample and independence tests from the machine learning community. A fixed-point transf…
Laplace kernel feature selection offers statistical guarantees for nonparametric models with few samples.
problem Statistical guarantees for kernel-based feature selection in nonconvex optimization problems.
method Sharp characterization of the gradient of the objective function for Laplace kernel feature selection.
result Model-selection consistency for Laplace kernel-based feature selection in nonparametric settings with n∼logp samples. Kernel-based tests for shape constraints in finance.
problem Enforcing shape relations on latent functions in financial econometrics.
method Kernel-based nonparametric framework for mean-variance optimization.
result Established statistical properties and a joint Wald-type statistic for testing shape constraints.
Novel confidence intervals improve convergence rates for sparse kernel-based models.
problem High computational cost in kernel-based learning models.
method Novel confidence intervals for Nyström method and sparse variational Gaussian process approximation.
result Improved performance bounds in regression and optimization problems.
FastKCI speeds up KCI tests for causal inference on large datasets.
problem Cubic computational complexity of kernel-based conditional independence tests.
method Mixture-of-experts approach with parallel Gaussian process inference.
result Substantial computational speedups with maintained statistical power.
ASVGD accelerates SVGD for efficient sampling.
problem Slow SVGD in high-dimensional sampling.
method Accelerated gradient flow in a metric space of probability densities, using Nesterov's method and momentum-based updates.
result ASVGD outperforms SVGD and other methods in sampling efficiency.
Paper reproduces a kernel-based scan B-statistic for online change-point detection.
problem Continuous detection of distribution changes in online data streams.
method Efficient kernel-based scan B-statistic for online change-point detection.
result Scan B-statistic outperforms parametric methods in challenging scenarios.
New RL method reduces sample complexity for large state-action spaces.
problem Handling large state-action spaces in RL with general Q-functions.
method Nonparametric Q-learning using kernel ridge regression.
result Sample complexity is order optimal with respect to ε and kernel complexity.
Recent developments in system identification have brought attention to regularized kernel-based methods, where, adopting the recently introduced stable spline kernel, prior information on the unknown process is enforced. This reduces the variance of the estimates and thus makes kernel-based methods particularly attract…
High-dimensional U-statistics show surprising phase transitions, impacting kernel-based tests.
problem Understanding phase transitions in high-dimensional U-statistics.
method Proved a convergence theorem for U-statistics of degree two in high dimensions.
result High-dimensional U-statistics can have non-Gaussian limits with larger variance and asymmetry.
Kernel-based reinforcement learning (KBRL) stands out among reinforcement learning algorithms for its strong theoretical guarantees. By casting the learning problem as a local kernel approximation, KBRL provides a way of computing a decision policy which is statistically consistent and converges to a unique solution. U…
Kernel Density Machines learn probability densities without structural assumptions.
problem Learning probability densities under minimal assumptions.
method Kernel-based framework, agnostic to structural requirements.
result Consistency and functional central limit theorem for sample estimator.
New tools evaluate and optimize conditional sequence models in bioinformatics.
problem Evaluating and optimizing conditional sequence models in bioinformatics.
method Kernel-based discrepancy measure (ACMMD) to estimate model fit and tune hyperparameters.
result Rejects the hypothesis that ProteinMPNN fits its data for various protein families and optimizes model temperature.
Conditional independence testing is an important problem, especially in Bayesian network learning and causal discovery. Due to the curse of dimensionality, testing for conditional independence of continuous variables is particularly challenging. We propose a Kernel-based Conditional Independence test (KCI-test), by con…
A permutation-based SW test achieves minimax-optimal power for two-sample testing.
problem Nonparametric two-sample testing using the sliced Wasserstein distance.
method Proposes a permutation-based SW test and analyzes its performance.
result Achieves minimax separation rate n−1/2 over multinomial and bounded-support alternatives. Deep neural nets optimize kernel parameters for non-parametric two-sample tests.
problem Determining if two samples come from the same distribution.
method Deep kernels trained to maximize test power, adapting to distribution smoothness and shape.
result Deep kernels outperform simpler kernels in high dimensions and complex data.
Localized sampler tackles high-dimensional sampling with fewer samples.
problem Sampling from unknown distributions with limited data.
method Combining Schrödinger bridges and plug & play Langevin samplers with localization strategy.
result Localized sampler reduces dimensionality, making sampling more efficient.
Study on testing two populations with confounders.
problem Determining if two populations have the same distribution after accounting for confounding factors.
method Introduce two general frameworks for conditional two-sample testing.
result Demonstrated the power and validity of the proposed frameworks.
Unified framework for global and local two-sample conditional distribution testing.
problem Testing equality of two conditional distributions.
method Distance and kernel methods, conditional U-statistics, local bootstrap.
result Developed reliable global and local tests.
Improved learning theory for kernel distribution regression with two-stage sampling.
problem Distribution regression problem and two-stage sampling setting.
method Kernel methods, near-unbiased condition, new error bounds, convergence rates.
result Strictly improved convergence rates for three important classes of kernels.
Kernel-based test detects differences between two conditional distributions efficiently.
problem Detecting differences between two conditional distributions efficiently.
method Kernel-based measure using nearest-neighbor graphs, consistent estimate with Gaussian limit.
result Asymptotic level control and universal consistency for detecting differences.
This paper introduces Kernel-based Information Criterion (KIC) for model selection in regression analysis. The novel kernel-based complexity measure in KIC efficiently computes the interdependency between parameters of the model using a variable-wise variance and yields selection of better, more robust regressors. Expe…
This paper introduces Bayes Hilbert spaces for efficient posterior approximation.
problem Efficient posterior approximation in Bayesian models for large datasets.
method Develops Bayes Hilbert spaces for posterior approximation and connects them to Bayesian coresets and kernel-based distances.
result Bayes Hilbert spaces provide a novel framework for posterior approximation that is computationally efficient.
GaussDetect-LiNGAM eliminates Gaussianity tests for causal discovery.
problem Causal direction identification without Gaussianity assumptions.
method Leverages the equivalence between noise Gaussianity and residual independence in reverse regression.
result Gaussianity tests replaced with robust kernel-based independence tests.
Independent component analysis (ICA) is a method for recovering statistically independent signals from observations of unknown linear combinations of the sources. Some of the most accurate ICA decomposition methods require searching for the inverse transformation which minimizes different approximations of the Mutual I…
Improved SVMs learn from few samples with composition and multiple scales.
problem Learning with small sample sizes.
method Transformation-invariant SVMs with composition and locality at multiple scales.
result Kernels based on maximum similarity are positive definite and yield superior accuracy.
Signal processing tasks as fundamental as sampling, reconstruction, minimum mean-square error interpolation and prediction can be viewed under the prism of reproducing kernel Hilbert spaces. Endowing this vantage point with contemporary advances in sparsity-aware modeling and processing, promotes the nonparametric basi…
Paper shows robustness of kernel-based pairwise learning without strict assumptions.
problem Statistical robustness of kernel-based pairwise learning under minimal conditions.
method No assumptions on input and output spaces; derives influence function and robustness.
result Qualitative robustness of kernel-based estimator established.
Paper proposes OKGAN to improve GAN training, addressing mode collapse and cycling.
problem Challenges in GAN training, including mode collapse and cycling.
method Kernel-based non-parametric discriminator for online training.
result OKGAN mitigates training issues and performs better than other GAN formulations.
New algorithm improves dynamic mode decomposition for high-dimensional data.
problem Reduced modeling in high-dimensional spaces.
method Low rank constraint optimization and kernel-based computation.
result Gain in approximation accuracy and computational efficiency.
Spectral risk measures (SRMs) belong to the family of coherent risk measures. A natural estimator for the class of SRMs has the form of L-statistics. Various authors have studied and derived the asymptotic properties of the empirical estimator of SRM. We propose a kernel based estimator of SRM. We investigate the large…
DKGM improves image quality by debiasing kernel-based models.
problem Improving image quality in generative models without losing detail.
method Two-stage framework: KDE for density estimation, debiasing algorithm inspired by stochastic approximation.
result DKGM achieves comparable image quality to state-of-the-art models on CIFAR10.
It is shown that bootstrap approximations of support vector machines (SVMs) based on a general convex and smooth loss function and on a general kernel are consistent. This result is useful to approximate the unknown finite sample distribution of SVMs by the bootstrap approach.
Graph-based methods pervade the inference toolkits of numerous disciplines including sociology, biology, neuroscience, physics, chemistry, and engineering. A challenging problem encountered in this context pertains to determining the attributes of a set of vertices given those of another subset at possibly different ti…
Performing exact posterior inference in complex generative models is often difficult or impossible due to an expensive to evaluate or intractable likelihood function. Approximate Bayesian computation (ABC) is an inference framework that constructs an approximation to the true likelihood based on the similarity between …
Kernel-based function approximation improves reinforcement learning performance.
problem Average reward reinforcement learning in infinite horizon settings.
method Optimistic algorithm based on kernel ridge regression.
result No-regret performance guarantees and confidence intervals for kernel-based predictions.
A novel kernel-based test detects equality versus singularity of two probability measures.
problem Detecting equality versus singularity of two probability distributions.
method Combines kernel mean and kernel covariance embeddings to construct a likelihood ratio test statistic.
result The test statistic satisfies a '0/\infty' law, vanishing under the null and diverging under the alternative.
Sequential tests for two-sample and independence testing using betting strategies.
problem Testing sequential data for two-sample and independence without kernel selection issues.
method Prediction-based betting strategies that adaptively determine distribution and joint distribution.
result Prediction-based tests outperform kernel-based approaches in high-dimensional or structured data settings.
A new method for deep Wishart processes improves kernel-based models.
problem Inference in deep Wishart processes is challenging due to the need for flexible distributions over positive semi-definite matrices.
method Developed a novel approach to flexible distributions over positive semi-definite matrices using the Bartlett decomposition of the Wishart probability density. Used this to create an approximate posterior for the DWP.
result Improved performance of inference in the DWP compared to DGP with equivalent prior.
The paper tackles hypothesis testing for likelihood-free inference with a new kernel-based approach.
problem Testing hypotheses with limited labeled data in likelihood-free inference.
method Kernel-based tests using maximum mean discrepancy (MMD) for non-parametric density comparison.
result Existence of an asymmetric trade-off between labeled and unlabeled data samples.
Paper introduces a new test for conditional independence using weighted partial copulas.
problem Testing conditional independence between variables.
method The approach uses a weighted partial copula function and a bootstrap procedure to compute regions of rejection.
result The proposed test has competitive power compared to existing methods.
A new algorithm identifies interpretable network representations via subgraph count statistics.
problem Interpreting network-valued data samples.
method Principal Component Analysis for Networks (PCAN) and its fast sampling-based version (sPCAN).
result The PCAN and sPCAN methods provide informative and discriminatory features for network samples.
Study provides guarantees for kernel clustering under non-parametric mixtures.
problem Statistical guarantees for kernel-based clustering without strong assumptions.
method Non-parametric mixture models, kernel-based clustering, consistency guarantees.
result Necessary and sufficient separability conditions for consistent clustering recovery.
Kernel-based L2-boosting with structure constraints improves regression efficiency.
problem Developing efficient kernel methods for regression.
method Kernel-based re-scaled boosting with truncation (KReBooT).
result KReBooT achieves near overfitting resistance and sparse estimates.