We study strictly proper scoring rules in the Reproducing Kernel Hilbert Space. We propose a general Kernel Scoring rule and associated Kernel Divergence. We consider conditions under which the Kernel Score is strictly proper. We then demonstrate that the Kernel Score includes the Maximum Mean Discrepancy as a special …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “kernel scores”
Generalizes leverage score sampling for neural networks, accelerating kernel methods and deep learning.
problem Accelerating kernel methods and deep learning training.
method Generalizes leverage score sampling to neural networks and proves equivalence to neural tangent kernel ridge regression.
result Equivalence between regularized neural network and neural tangent kernel ridge regression under leverage score sampling initialization.
A new kernel-based nonconformity score improves multivariate prediction regions.
problem Tackling the challenge of compressing multivariate residual vectors into scalars while preserving geometric structure.
method Introducing a Multivariate Kernel Score (MKS) that decomposes into an anisotropic MMD, providing finite-sample coverage guarantees and convergence rates.
result The MKS produces prediction regions that explicitly adapt to geometric structure, reducing volume compared to ellipsoidal baselines.
Kernel-smoothed scores improve diffusion models by reducing memorization.
problem Diffusion models can memorize training data, leading to biased samples.
method Interpret empirical score as noisy version of true score, kernel-smoothed.
result Kernel-smoothing reduces variance and improves generalization.
Proposes SD-KDE for density estimation using debiased kernel density with score-based adjustments.
problem Density estimation with bias in kernel density estimation.
method Adjusts data points by taking a step along the estimated score function, then applies standard KDE with modified bandwidth.
result Significantly reduces mean integrated squared error compared to standard Silverman KDE, especially with noisy score function estimates.
Efficiently approximates statistical leverage scores for faster KRR.
problem Accurately estimating statistical leverage scores for fast KRR.
method Analytic formula for statistical leverage scores, leveraging kernel spectral density.
result Linear time approximation with theoretical guarantees, significantly faster than existing methods.
A new model DKMPP integrates covariates and uses an integration-free method for spatio-temporal point processes.
problem Training intractable deep spatio-temporal point processes with multimodal covariates.
method DKMPP uses a deep kernel to model complex relationships and an integration-free score matching method.
result DKMPP and score-based estimators outperform baseline models in spatio-temporal point processes.
Kernel-Gradient Drifting improves generative modeling for non-Euclidean data.
problem Challenges in generative modeling for non-Euclidean data.
method Replaces Euclidean displacement with kernel-induced directions, exposing score-based structure.
result Kernel-gradient drifting enables state-of-the-art one-step generation for non-Euclidean data.
New test assesses probabilistic model calibration without expensive approximations.
problem Assessing calibration of probabilistic models with scores.
method Kernel Calibration Conditional Stein Discrepancy (KCCSD) test using new score-based kernels.
result Control over type-I error with improved scalability and efficiency.
Statistical leverage scores emerged as a fundamental tool for matrix sketching and column sampling with applications to low rank approximation, regression, random feature learning and quadrature. Yet, the very nature of this quantity is barely understood. Borrowing ideas from the orthogonal polynomial literature, we in…
Paper proposes an efficient causal discovery method with linear computational complexity.
problem Identifying causal relationships efficiently in large datasets.
method Approximate kernel-based generalized score function with low-rank technique and sampling algorithms.
result Significantly reduces computational costs while maintaining comparable accuracy.
Signature Kernel Scoring Rule: A Spatio-Temporal Diagnostic for Probabilistic Weather Forecastingstat.ML
Signature kernel scoring rule improves weather forecasting by capturing temporal and spatial dependencies.
problem Lack of suitable scoring rules for probabilistic weather forecasting.
method Reframe weather variables as continuous paths using iterated integrals (signature kernels) to capture temporal and spatial dependencies.
result Signature kernel scoring rule outperforms conventional methods in weather forecasting, especially for long-term forecasts.
Unified view of score estimators for flexible densities.
problem Estimating the score from unknown distributions.
method Regularized nonparametric regression framework.
result Unified convergence analysis and new estimators with desirable properties.
New machine learning methods for inference from simulated data.
problem Modeling score and likelihood ratio functions from sampled data.
method InferoStatic Networks (ISN), Kernel Score Estimation (KSE), Kernel Likelihood Ratio Estimation (KLRE).
result Improved inference methods for complex models.
The paper develops methods to handle missing data using regularized M-estimation in reproducing kernel Hilbert space.
problem Handling missing data in statistical analysis.
method Kernel ridge regression for imputation and maximum entropy method for propensity score estimation.
result The proposed methods achieve statistical consistency and asymptotic equivalence.
Estimates system parameters from a single observation using kernel-based score.
problem Estimating parameters of a dynamical system from a high-dimensional signal.
method Kernel-based score to compare temporal dependencies between signal and model.
result Accuracy and efficiency demonstrated on chaotic systems.
New method detects changes in high-dimensional Markov processes without explicit likelihood evaluation.
problem Quickest change detection in Markov processes with unknown transition kernels.
method Learn conditional score from sample pairs, develop score-based CUSUM procedure.
result Exponential lower bounds on mean time to false alarm and asymptotic upper bounds on detection delay.
A Novel Framework for Uncertainty Quantification via Proper Scores for Classification and Beyondcs.LG
A novel framework quantifies uncertainty using proper scores for various tasks.
problem Uncertainty quantification in machine learning for reliable applications.
method Proposes a general framework based on proper scores for epistemic, aleatoric uncertainty, and model calibration.
result Achieves state-of-the-art uncertainty estimation for large language models and generative models.
Random features provide a practical framework for large-scale kernel approximation and supervised learning. It has been shown that data-dependent sampling of random features using leverage scores can significantly reduce the number of features required to achieve optimal learning bounds. Leverage scores introduce an op…
We consider the two-group classification problem and propose a kernel classifier based on the optimal scoring framework. Unlike previous approaches, we provide theoretical guarantees on the expected risk consistency of the method. We also allow for feature selection by imposing structured sparsity using weighted kernel…
Unified kernel-based methods improve nonlinear causal discovery.
problem Identifying nonlinear causal relationships between time series variables.
method Unified Kernel Principal Component Regression (KPCR) and Gaussian Process score-based model with Smooth Information Criterion.
result Improved performance in time series nonlinear causal discovery.
Semiparametric KSD test: unifying score and distance-based approaches for goodness-of-fit testingstat.ML
Unified score and distance-based GoF tests for model adequacy.
problem Difficulty in extending score-based GoF tests to nonparametric alternatives.
method Introducing semiparametric kernelized Stein discrepancy (SKSD) test.
result SKSD test is computationally efficient and universally consistent.
Paper introduces non-adversarial training for Neural SDEs using signature kernel scores.
problem Stability and mode collapse issues in adversarial training of Neural SDEs.
method Uses signature kernel scores as objective function for non-adversarial training.
result Non-adversarial training leads to better performance and more stable models.
Leverage score sampling provides an appealing way to perform approximate computations for large matrices. Indeed, it allows to derive faithful approximations with a complexity adapted to the problem at hand. Yet, performing leverage scores sampling is a challenge in its own right requiring further approximations. In th…
Wasserstein proximal operators describe score-based generative models and resolve memorizationstat.ML
Mathematical analysis improves SGMs, resolving memorization issues.
problem Improving performance and avoiding memorization in SGMs.
method Formulated SGMs using Wasserstein proximal operators and mean-field games.
result Improved SGM performance in terms of training samples and time.
The paper investigates the convergence of Vendi scores under finite samples and introduces a truncated version for better performance.
problem The Vendi score's convergence is hindered by computational limitations when using large sample sizes.
method The authors introduce the t-truncated Vendi score to address this issue by truncating the eigenspectrum of the kernel matrix.
result The t-truncated Vendi score converges to its asymptotic limit with a smaller number of samples, improving upon the standard Vendi score.
The paper introduces a new framework to assess generative model uncertainty.
problem Lack of a theoretical framework for assessing generative models' generalization and uncertainty.
method Bias-variance-covariance decomposition for kernel scores, with unbiased and consistent estimators.
result Kernel-based variance and entropy for uncertainty estimation are more predictive than existing methods.
We give the first algorithm for kernel Nyström approximation that runs in *linear time in the number of training points* and is provably accurate for all kernel matrices, without dependence on regularity or incoherence conditions. The algorithm projects the kernel onto a set of landmark points sampled by their *rid…
Strictly proper kernel scores are well-known tool in probabilistic forecasting, while characteristic kernels have been extensively investigated in the machine learning literature. We first show that both notions coincide, so that insights from one part of the literature can be used in the other. We then show that the m…
Three RFF-based methods for nonlinear causal discovery in mixed data.
problem Nonlinear causal discovery in mixed data with computational constraints.
method FFML, TRFF, and FFCI methods for score-based, constraint-based, and hybrid causal discovery.
result FFML and TRFF methods provide complementary performance in causal discovery.
Kernel SVGD improves high-dimensional inference with noise adaptation.
problem Challenges in high-dimensional inference with SVGD.
method Noise Conditional Kernel SVGD (NCK-SVGD) with entropic regularization.
result NCK-SVGD produces samples comparable to GANs and SGLD on computer vision benchmarks.
Study improves KRR for non-i.i.d. data, with applications in denoising.
problem Kernel regression in structured non-i.i.d. settings.
method Developed a blockwise decomposition method for dependent data, deriving excess risk bounds.
result Established generalization guarantees for KRR in non-i.i.d. settings.
SQS uses quantum kernels to improve credit scoring with fewer data points.
problem Credit scoring models struggle with scarce and skewed data.
method Systemic Quantum Score (SQS) leverages quantum kernels for better pattern extraction.
result SQS shows improved performance and pattern extraction with fewer data points.
We illustrate relationships between classical kernel-based dimensionality reduction techniques and eigendecompositions of empirical estimates of reproducing kernel Hilbert space (RKHS) operators associated with dynamical systems. In particular, we show that kernel canonical correlation analysis (CCA) can be interpreted…
New score helps choose PIML model parameters, reducing ambiguity in model quality.
problem Ambiguity in measuring model quality in PIML due to multi-objective fitting.
method Introduces Physics-Informed Log Evidence (PILE) score in Gaussian process framework.
result PILE minimizes ambiguity in model selection, improving hyperparameter choices.
This work analyzes SGD for SGMs, providing convergence rates and error bounds.
problem Optimization dynamics of SGMs trained with stochastic gradients.
method Non-convex convergence rate analysis and Neural Tangent Kernel analysis.
result Theoretical insights into SGD convergence and error bounds for SGMs.
The kernel exponential family is a rich class of distributions, which can be fit efficiently and with statistical guarantees by score matching. Being required to choose a priori a simple kernel such as the Gaussian, however, limits its practical applicability. We provide a scheme for learning a kernel parameterized by …
Novel algorithm identifies nonlinear Granger causal relationships using kernel ridge regression.
problem Identification of nonlinear Granger causal relationships.
method Flexible plug-in architecture with kernel ridge regression using radial basis function.
result Kernel ridge regression in mlcausality achieves competitive AUC scores and more finely calibrated p-values.
Paper presents variational estimates for EBLVMs without structural assumptions.
problem Challenges in learning and evaluating EBLVMs due to intractable true posteriors and partition functions.
method Variational estimates of the score function and its gradient (VaES and VaGES) in a general EBLVM.
result The estimates can be applied to KSD and SM-based methods to learn EBLVMs and estimate Fisher divergence.
This paper analyzes error bounds for biased SMC samplers in conditional sampling.
problem Analyzing error bounds for biased SMC samplers in conditional sampling.
method Develops a non-asymptotic error analysis for SMC samplers with biased mutation kernels.
result Derives the first non-asymptotic error bound for conditional sampling with score-based diffusion models.
SGMs fail to generate samples from complex distributions even when the score function is learned well.
problem Score-based Generative Models fail to produce high-quality samples from complex distributions.
method Score-based Generative Models (SGMs) are evaluated under conditions where the score function is learned well.
result SGMs can only generate Gaussian blurring of training data points, not complex distributions.
Conditional density estimation is a general framework for solving various problems in machine learning. Among existing methods, non-parametric and/or kernel-based methods are often difficult to use on large datasets, while methods based on neural networks usually make restrictive parametric assumptions on the probabili…
A faster method for density estimation using denoising score matching with random Fourier features.
problem Intractability of normalizing constant in density estimation.
method Denoising Score Matching with Random Fourier Features.
result The method is computationally faster and scalable to complex high-dimensional data.
Attention tokens are group elements, negated algebra norms.
problem Attention mechanism for matrix Lie groups
method Lie-Algebra Attention
result Score matches learned kernel on invariant poses, outperforms vector-token baseline
We investigate penalized maximum log-likelihood estimation for exponential family distributions whose natural parameter resides in a reproducing kernel Hilbert space. Key to our approach is a novel technique, doubly dual embedding, that avoids computation of the partition function. This technique also allows the develo…
Improved image generation quality using closed-form discriminator guidance in diffusion models.
problem Enhancing the quality of images generated by diffusion models.
method Theoretical framework to analyze GAN discriminator's effect on Langevin sampling, proposing IPM-GAN optimization as smoothed score-matching.
result Closed-form kernel-based discriminator guidance improves metrics like CLIP-FID and KID.
Paper bridges VAEs and KDEs for more flexible posterior estimation.
problem Limitations of Gaussian latent space in VAEs and challenges in KL-divergence estimation.
method Approximate posterior with KDEs and derive upper bound of KL-divergence in ELBO.
result Epanechnikov kernel minimizes KL-divergence upper bound asymptotically.
Kernel quadrature improves CRPS estimation for probabilistic time-series forecasting.
problem Intractable integrations in CRPS evaluation metrics lead to improper rankings of forecasting models.
method Introduced kernel quadrature approach for unbiased CRPS estimation and scalable computation.
result Our approach consistently outperforms existing CRPS estimators.