Kernel-based function approximation improves reinforcement learning performance.
problem Average reward reinforcement learning in infinite horizon settings.
method Optimistic algorithm based on kernel ridge regression.
result No-regret performance guarantees and confidence intervals for kernel-based predictions.
New algorithm improves dynamic mode decomposition for high-dimensional data.
problem Reduced modeling in high-dimensional spaces.
method Low rank constraint optimization and kernel-based computation.
result Gain in approximation accuracy and computational efficiency.
Kernel-based online learning has often shown state-of-the-art performance for many online learning tasks. It, however, suffers from a major shortcoming, that is, the unbounded number of support vectors, making it non-scalable and unsuitable for applications with large-scale datasets. In this work, we study the problem …
Adaptive rule improves kernel-based gradient descent performance.
problem Improving convergence speed of kernel-based gradient descent algorithms.
method Empirical effective dimension for stopping rule, learning theory analysis, integral operator approach.
result Optimal learning rates and iteration bounds for KGD with adaptive stopping rule.
Study provides guarantees for kernel clustering under non-parametric mixtures.
problem Statistical guarantees for kernel-based clustering without strong assumptions.
method Non-parametric mixture models, kernel-based clustering, consistency guarantees.
result Necessary and sufficient separability conditions for consistent clustering recovery.
Kernel-based L2-boosting with structure constraints improves regression efficiency.
problem Developing efficient kernel methods for regression.
method Kernel-based re-scaled boosting with truncation (KReBooT).
result KReBooT achieves near overfitting resistance and sparse estimates.
Algorithm optimizes collaborative learning among distributed clients using kernel-based bandits.
problem Optimizing personalized objectives in a distributed system with limited global information.
method Kernel-based bandit framework with surrogate Gaussian process models, sparse approximations.
result Order-optimal regret performance (up to polylogarithmic factors) and reduced communication overhead.
Paper proposes an efficient causal discovery method with linear computational complexity.
problem Identifying causal relationships efficiently in large datasets.
method Approximate kernel-based generalized score function with low-rank technique and sampling algorithms.
result Significantly reduces computational costs while maintaining comparable accuracy.
Paper proposes adaptive parameter selection for KGD algorithms.
problem Improving parameter selection for kernel-based gradient descent.
method Integrates bias-variance analysis with splitting method, introduces empirical effective dimension.
result Adaptive parameter selection strategy achieves optimal generalization error bound.
This paper improves kernel-based regression using transfer learning.
problem Improving generalization performance in kernel-based regression.
method Two-step kernel-based estimator for known transferable sources and novel aggregation algorithm for unknown sources.
result Established statistical properties and validated effectiveness of proposed methods.
The rate of convergence of weighted kernel herding (WKH) and sequential Bayesian quadrature (SBQ), two kernel-based sampling algorithms for estimating integrals with respect to some target probability measure, is investigated. Under verifiable conditions on the chosen kernel and target measure, we establish a near-geom…
Convolutional Neural Networks, as most artificial neural networks, are commonly viewed as methods different in essence from kernel-based methods. We provide a systematic translation of Convolutional Neural Networks (ConvNets) into their kernel-based counterparts, Convolutional Kernel Networks (CKNs), and demonstrate th…
Paper reproduces a kernel-based scan B-statistic for online change-point detection.
problem Continuous detection of distribution changes in online data streams.
method Efficient kernel-based scan B-statistic for online change-point detection.
result Scan B-statistic outperforms parametric methods in challenging scenarios.
Laplace kernel feature selection offers statistical guarantees for nonparametric models with few samples.
problem Statistical guarantees for kernel-based feature selection in nonconvex optimization problems.
method Sharp characterization of the gradient of the objective function for Laplace kernel feature selection.
result Model-selection consistency for Laplace kernel-based feature selection in nonparametric settings with n∼logp samples. Developing feature selection algorithms that move beyond a pure correlational to a more causal analysis of observational data is an important problem in the sciences. Several algorithms attempt to do so by discovering the Markov blanket of a target, but they all contain a forward selection step which variables must pas…
Kernel-UCBVI algorithm balances exploration and exploitation in metric state-action spaces.
problem Exploration-exploitation dilemma in finite-horizon reinforcement learning with metric state-action spaces.
method Kernel-UCBVI, leveraging smoothness and kernel estimators of rewards and transitions.
result First regret bound for kernel-based RL using smoothing kernels, O(H3K2d/(2d+1)). DualIV simplifies non-linear IV regression via dual formulation.
problem Non-linear instrumental variable regression with potential first-stage regression bottleneck.
method Dual formulation of non-linear IV regression as a convex-concave saddle-point problem, leading to a kernel-based algorithm with analytic solution.
result Empirical results show competitive performance compared to existing algorithms.
Unified kernel-based methods improve nonlinear causal discovery.
problem Identifying nonlinear causal relationships between time series variables.
method Unified Kernel Principal Component Regression (KPCR) and Gaussian Process score-based model with Smooth Information Criterion.
result Improved performance in time series nonlinear causal discovery.
Kernel-based reinforcement learning (KBRL) stands out among reinforcement learning algorithms for its strong theoretical guarantees. By casting the learning problem as a local kernel approximation, KBRL provides a way of computing a decision policy which is statistically consistent and converges to a unique solution. U…
Kernel-based tests for shape constraints in finance.
problem Enforcing shape relations on latent functions in financial econometrics.
method Kernel-based nonparametric framework for mean-variance optimization.
result Established statistical properties and a joint Wald-type statistic for testing shape constraints.
This paper introduces Kernel-based Information Criterion (KIC) for model selection in regression analysis. The novel kernel-based complexity measure in KIC efficiently computes the interdependency between parameters of the model using a variable-wise variance and yields selection of better, more robust regressors. Expe…
Paper shows robustness of kernel-based pairwise learning without strict assumptions.
problem Statistical robustness of kernel-based pairwise learning under minimal conditions.
method No assumptions on input and output spaces; derives influence function and robustness.
result Qualitative robustness of kernel-based estimator established.
Boosting Nyström improves accuracy of matrix approximations.
problem Generating low-rank approximations of large matrices efficiently.
method Iteratively generate multiple weak Nyström approximations, combine them to form a strong approximation.
result Boosting Nyström yields more efficient and accurate low-rank approximations.
Meta two-sample testing uses auxiliary data to quickly find powerful tests from limited samples.
problem Challenges in identifying powerful kernels for distinguishing complex distributions with limited data.
method Introduces meta two-sample testing (M2ST) to leverage abundant auxiliary data on related tasks.
result Proposed algorithms improve over baselines and identify powerful tests from scarce observations.
We present a novel machine learning approach to understanding conformation dynamics of biomolecules. The approach combines kernel-based techniques that are popular in the machine learning community with transfer operator theory for analyzing dynamical systems in order to identify conformation dynamics based on molecula…
New test for binary treatment effects using kernel methods.
problem Testing distributional effects of binary treatments.
method Kernel-based doubly-robust test, avoiding permutations.
result Valid type-I error with computational efficiency.
Variable selection is central to high-dimensional data analysis, and various algorithms have been developed. Ideally, a variable selection algorithm shall be flexible, scalable, and with theoretical guarantee, yet most existing algorithms cannot attain these properties at the same time. In this article, a three-step va…
ASVGD accelerates SVGD for efficient sampling.
problem Slow SVGD in high-dimensional sampling.
method Accelerated gradient flow in a metric space of probability densities, using Nesterov's method and momentum-based updates.
result ASVGD outperforms SVGD and other methods in sampling efficiency.
A new PU classifier PUAL tackles trifurcate data issues.
problem Training classifiers on trifurcate data containing only labeled-positive instances and unlabeled instances.
method PUAL classifier with asymmetric loss and kernel-based algorithm.
result PUAL achieves satisfactory classification on trifurcate data.
Cellular network configuration plays a critical role in network performance. In current practice, network configuration depends heavily on field experience of engineers and often remains static for a long period of time. This practice is far from optimal. To address this limitation, online-learning-based approaches hav…
We propose graph kernels based on subgraph matchings, i.e. structure-preserving bijections between subgraphs. While recently proposed kernels based on common subgraphs (Wale et al., 2008; Shervashidze et al., 2009) in general can not be applied to attributed graphs, our approach allows to rate mappings of subgraphs by …
In this paper, we address the problem of reconstructing coverage maps from path-loss measurements in cellular networks. We propose and evaluate two kernel-based adaptive online algorithms as an alternative to typical offline methods. The proposed algorithms are application-tailored extensions of powerful iterative meth…
Paper proposes Langevin dynamics for adaptive IRL of stochastic gradient algorithms.
problem Estimating reward functions from noisy gradient estimates of stochastic gradient agents.
method Generalized Langevin dynamics algorithm for IRL.
result Proposed algorithms asymptotically generate samples proportional to exp(R(θ)).
The massive amount of available data potentially used to discover patters in machine learning is a challenge for kernel based algorithms with respect to runtime and storage capacities. Local approaches might help to relieve these issues. From a statistical point of view local approaches allow additionally to deal with …
Kernel clustering algorithm improved for large datasets using incomplete Cholesky factorization.
problem Large memory usage in kernel-based clustering for large-scale datasets.
method Approximate the kernel matrix using incomplete Cholesky factorization and apply linear k-means clustering. result The proposed method achieves similar performance to kernel k-means clustering but handles large-scale datasets efficiently. Novel confidence intervals improve convergence rates for sparse kernel-based models.
problem High computational cost in kernel-based learning models.
method Novel confidence intervals for Nyström method and sparse variational Gaussian process approximation.
result Improved performance bounds in regression and optimization problems.
Paper introduces MinDiff framework for balancing classifier performance and fairness.
problem Balancing classifier performance and fairness in machine learning models.
method MinDiff framework with kernel-based statistical dependency tests.
result Demonstrates real-world improvements in classifier performance and fairness.
MFRDE uses medians of forest estimators to robustly estimate densities in noisy data.
problem Robust density estimation in the presence of outliers.
method MFRDE uses pointwise median operation on forest density estimators fitted on subsampled datasets.
result MFRDE achieves robustness against all outliers while maintaining accuracy for density estimation.
New theoretical tools simplify kernel-based tests analysis.
problem Asymptotic behavior of kernel-based tests in various scenarios.
method Avoids complex expansions and limit theorems, works directly with Hilbert spaces random functionals.
result Framework leads to simpler analysis with minimal regularity conditions.
DKGM improves image quality by debiasing kernel-based models.
problem Improving image quality in generative models without losing detail.
method Two-stage framework: KDE for density estimation, debiasing algorithm inspired by stochastic approximation.
result DKGM achieves comparable image quality to state-of-the-art models on CIFAR10.
A new kernel-based CI test improves on existing methods.
problem Testing conditional independence (CI) in a broad range of dependencies.
method Regression-model-agnostic kernel-based CI test using reproducing kernel Hilbert spaces.
result GKCM outperforms state-of-the-art CI tests in simulations.
This paper proposes a novel kernel-based optimization scheme to handle tasks in the analysis, e.g., signal spectral estimation and single-channel source separation of 1D non-stationary oscillatory data. The key insight of our optimization scheme for reconstructing the time-frequency information is that when a nonparame…
This paper introduces Bayes Hilbert spaces for efficient posterior approximation.
problem Efficient posterior approximation in Bayesian models for large datasets.
method Develops Bayes Hilbert spaces for posterior approximation and connects them to Bayesian coresets and kernel-based distances.
result Bayes Hilbert spaces provide a novel framework for posterior approximation that is computationally efficient.
Paper introduces EO_k for quantifying accuracy-fairness trade-offs in FRL.
problem Tackles the trade-off between accuracy and fairness in FRL.
method Kernel-based formulation of EO criterion for FRL.
result Offers a unified analytical characterization of fairness tradeoffs.
This paper presents a novel kernel-based generative classifier which is defined in a distortion subspace using polynomial series expansion, named Kernel-Distortion (KD) classifier. An iterative kernel selection algorithm is developed to steadily improve classification performance by repeatedly removing and adding kerne…
Optimal kernel improves estimation accuracy in modal statistical methods.
problem Estimation accuracy of kernel-based modal statistical methods depends on the kernel used.
method The study theoretically shows an optimal kernel that minimizes asymptotic error criterion.
result An optimal kernel minimizes the error criterion when using an optimal bandwidth.
KeRNS tackles non-stationary reinforcement learning in metric spaces.
problem Non-stationary reinforcement learning in metric spaces.
method KeRNS uses time-dependent kernels to model non-stationary Markov Decision Processes (MDPs).
result KeRNS achieves a regret bound that scales with the covering dimension and total variation of the MDP.
Algorithm learns to play against unknown opponents in sequential games.
problem Designing strategies for a learner to interact with an unknown opponent in repeated sequential games.
method Kernel-based regularity assumptions and a novel algorithm combining bilevel optimization and online learning.
result Algorithm achieves sublinear regret guarantees and is effective in specific game settings.