Optimal Biweight kernel and computationally efficient Epanechnikov kernel for modal linear regression.
problem Finding the best kernel for modal linear regression.
method Refined analysis of asymptotic statistical behavior and IRLS algorithm convergence.
result Biweight kernel minimizes asymptotic mean squared error, Epanechnikov kernel guarantees IRLS convergence.
Kernel Multigrid accelerates Back-fitting for additive Gaussian Processes.
problem Slow convergence of Back-fitting in training additive Gaussian Processes.
method Kernel Packets (KP) and Sparse Gaussian Process Regression (GPR) to enhance Back-fitting.
result Kernel Multigrid reduces the required iterations to O(logn). New algorithm selects multiple kernels for better GP regression predictions.
problem Improving Gaussian process regression accuracy with multiple kernels.
method Variational Bayesian kernel selection (VBKS) for sparse Gaussian process regression (SGPR).
result VBKS learns uncertainty in kernel selection for better predictions.
A novel KD classifier improves character recognition performance.
problem Improving character recognition accuracy.
method Kernel-based generative classifier in distortion subspace with iterative kernel selection.
result The KD classifier outperforms existing classifiers and has unique recognition capability.
Paper proposes adaptive parameter selection for KGD algorithms.
problem Improving parameter selection for kernel-based gradient descent.
method Integrates bias-variance analysis with splitting method, introduces empirical effective dimension.
result Adaptive parameter selection strategy achieves optimal generalization error bound.
A novel kernel approach for model selection in simulator-based models.
problem Model selection for simulator-based statistical models with limited prior knowledge.
method Iteratively updates model weights and parameters using Bayes' rule and kernel recursive ABC algorithm.
result Demonstrates effectiveness on dynamical systems in ecology and epidemiology.
New algorithm selects best preconditioner for iterative methods.
problem Choosing optimal preconditioner for solving linear systems.
method Sketching methods to estimate preconditioner stability.
result Provable method selects minimal stability preconditioner.
Selecting important features in non-linear or kernel spaces is a difficult challenge in both classification and regression problems. When many of the features are irrelevant, kernel methods such as the support vector machine and kernel ridge regression can sometimes perform poorly. We propose weighting the features wit…
BOOST automates kernel and acquisition function selection in Bayesian optimization.
problem Inappropriate kernel and acquisition function combinations lead to poor performance in Bayesian optimization.
method BOOST uses offline evaluation to predict and select the best kernel-acquisition function pair.
result BOOST consistently improves over fixed-hyperparameter BO and is competitive with state-of-the-art adaptive methods.
iRDM selects unlabeled samples for regression without labels, improving model accuracy.
problem Selecting unlabeled samples for regression without label information.
method Iterative representativeness-diversity maximization (iRDM).
result iRDM significantly outperforms supervised ALR, especially with limited labeled samples.
An incremental SVDD algorithm for online data using Gaussian kernel.
problem Efficiently handling online or large data for SVDD.
method Incremental learning algorithm using Gaussian kernel, focusing on existing support vectors and new data points.
result Significant gains in efficiency with almost no loss in outlier detection accuracy or objective function value.
Changing kernel bandwidth during training improves kernel regression performance.
problem Improving kernel regression performance with varying model complexity.
method Investigated changing the bandwidth of a translational-invariant kernel during training for kernel regression using gradient descent.
result Kernel regression exhibits double descent behavior with decreasing model complexity (bandwidth).
Kernel method outperforms deep neural networks in speech enhancement.
problem Improving single-channel speech enhancement performance.
method Kernel regression with an exponential power kernel and EigenPro iterative method.
result Kernel method consistently outperforms deep neural networks in speech enhancement.
Optimal method detects jumps in jump-diffusion processes.
problem Detecting jumps in jump-diffusion processes with improved finite-sample performance.
method Iterative threshold-kernel method to optimally select threshold parameter.
result Approximate optimal threshold depends on spot volatility, jump intensity, and jump density.
Fisher score is one of the most widely used supervised feature selection methods. However, it selects each feature independently according to their scores under the Fisher criterion, which leads to a suboptimal subset of features. In this paper, we present a generalized Fisher score to jointly select features. It aims …
Proposes a Gradient Boosting method for learning adaptive kernel functions.
problem Learning a versatile ensemble of kernel functions for better performance.
method Approximates kernel functions as a weighted sum of Random Fourier Features and optimizes their barycenter at each iteration.
result Shows improved performance compared to Boosting-based and kernel-learning methods.
KSDAgg combines multiple KSD tests to improve goodness-of-fit testing without splitting data.
problem Improving goodness-of-fit testing without data splitting.
method KSDAgg aggregates multiple KSD tests with different kernels to maximize power.
result KSDAgg achieves the smallest uniform separation rate of the collection, up to a logarithmic term.
Laplace kernel feature selection offers statistical guarantees for nonparametric models with few samples.
problem Statistical guarantees for kernel-based feature selection in nonconvex optimization problems.
method Sharp characterization of the gradient of the objective function for Laplace kernel feature selection.
result Model-selection consistency for Laplace kernel-based feature selection in nonparametric settings with n∼logp samples. AutoStep MCMC adapts step size locally for better sampling efficiency.
problem Challenging step size selection for complex, multiscale targets.
method AutoStep MCMC uses a locally adaptive step size for involutive proposals.
result AutoStep MCMC is π-invariant, irreducible, and aperiodic.
The choice of the kernel is critical to the success of many learning algorithms but it is typically left to the user. Instead, the training data can be used to learn the kernel by selecting it out of a given family, such as that of non-negative linear combinations of p base kernels, constrained by a trace or L1 regular…
Paper optimizes prediction in semi-functional linear models using kernel methods.
problem Optimizing prediction in semi-functional linear models with functional and nonparametric components.
method Double-penalized least squares method in reproducing kernel Hilbert spaces, with regularization parameter selection via generalized cross validation.
result Achieves minimax optimal rates of convergence for both functional and nonparametric components.
Paper introduces RMEN-CCA for multi-view unsupervised learning.
problem Combining multiple data views for unsupervised learning.
method Robust matrix elastic net (RMEN) integrated with canonical correlation analysis (CCA).
result RMEN-CCA outperforms state-of-the-art methods on multiple datasets.
Optimal scoring framework for kernel classification with feature selection.
problem Two-group classification problem.
method Optimal scoring framework, structured sparsity using weighted kernels, automated parameter selection.
result Superior classification performance compared to existing nonparametric classifiers.
Two adaptive kernel selection methods improve the accuracy of Kernelized Diffusion Maps.
problem Selecting an appropriate kernel for Kernelized Diffusion Maps.
method Two complementary approaches: variational outer loop and unsupervised cross-validation.
result Both methods improve the quality and stability of the recovered eigenfunctions.
New active learning methods use statistical leverage scores to select examples efficiently.
problem Efficiently selecting labeled examples for high model accuracy with limited labeled data.
method Proposes ALEVS and DBALEVS methods based on statistical leverage scores.
result DBALEVS selects diverse, representative examples efficiently.
Proposes a method to select diverse kernels for improved clustering performance.
problem Redundancy in selected kernels degrades clustering performance and efficiency.
method Selects diverse subset of kernels as representative kernels, optimizes combination coefficients using alternating minimization.
result Improves clustering performance and efficiency compared to existing methods.
Optimizes kernel discrepancies by selecting subsets efficiently.
problem Improving kernel discrepancies for QMC methods.
method Introduces a novel subset selection algorithm for kernel discrepancies.
result Efficiently generates low-discrepancy samples from various distributions.
Efficient and accurate low-rank approximations of multiple data sources are essential in the era of big data. The scaling of kernel-based learning algorithms to large datasets is limited by the O(n^2) computation and storage complexity of the full kernel matrix, which is required by most of the recent kernel learning a…
Proposes a method for selecting variables in nonparametric learning using power series kernels.
problem Variable selection in nonparametric learning with power series kernels.
method Two-stage estimation: consistent function approximation followed by l1-type penalized variable selection.
result The method achieves variable selection consistency for power series kernels.
A new method for faster bandwidth selection in Gaussian kernel ridge regression.
problem Efficiently selecting the bandwidth in Gaussian kernel ridge regression.
method Formulated an approximate Jacobian expression for bandwidth selection, proposing a closed-form heuristic.
result Our method is as accurate as cross-validation and marginal likelihood maximization but up to six orders of magnitude faster.
This paper uses machine learning to select kernels for machine learning models on various devices.
problem Traditional kernel auto-tuning is limited for machine learning research with changing network topologies and hyperparameters.
method Combines auto-tuning and machine learning to select kernels for SYCL on various devices.
result Initial results show high performance kernel selection with little developer effort.
Method selects features via kernel-based independence measures.
problem Feature selection in high-dimensional data.
method Optimization of conditional covariance trace.
result Method outperforms other feature selection algorithms.
Paper proposes a new method to automatically select Gaussian kernel bandwidth for SVDD.
problem Selecting optimal Gaussian kernel bandwidth for SVDD is crucial but challenging.
method Automatic unsupervised method for selecting Gaussian kernel bandwidth.
result The selected bandwidth is competitive with existing methods and can be computed quickly.
A fast method for selecting Gaussian kernel bandwidth in kernel-based classifiers.
problem High computational complexity in estimating Gaussian kernel bandwidth.
method Developed based on reproducing kernel Hilbert space operators.
result Proposed method outperforms state-of-the-art methods in computational time and performance.
A new method optimizes MMD test power by dynamically selecting kernels, overcoming traditional trade-offs.
problem Fixed kernels fail to distinguish certain distributions, leading to overfitting and variance collapse.
method Complexity-Penalized MMD (CP-MMD) criterion, derived from concentration inequality, optimizes kernel selection.
result CP-MMD maximizes true test power while ensuring unconditional Type-I validity, matching or exceeding state-of-the-art performance.
Two-sample tests using MMD control type I error and achieve optimal power.
problem Developing reliable nonparametric two-sample tests for small sample sizes.
method Maximum Mean Discrepancy (MMD) for constructing novel nonparametric tests, proving non-asymptotic error control and optimality.
result MMDAgg test controls type I error and achieves minimax rate over Sobolev balls, outperforming other tests.
New algorithm optimizes high-dimensional functions with near-constant complexity.
problem Scalability issues in Gaussian process optimization for high-dimensional functions.
method Combines kernelized linear bandit with randomized matrix sketching.
result Achieves near-optimal regret with near-constant per-iteration complexity.
TorchKM: A GPU-Oriented Library for Kernel Learning and Model Selection
problem Kernel learning and model selection
method GPU acceleration
result Competitive predictive performance with speedups
New active learning method for kernel selection improves efficiency and accuracy.
problem Real-world applications where acquiring true labels is costly or time-consuming.
method Active Multiple Kernel Learning (AMKL) with adaptive kernel selection (AMKL-AKS).
result AMKL-AKS achieves optimal sublinear regret and better performance with fewer labeled data.
Proposes a method to select variables for kernel two-sample tests.
problem Determining whether two samples have the same distribution using informative variables.
method A framework based on kernel maximum mean discrepancy (MMD) for selecting a subset of variables.
result The sample size requirements for the three kernels depend on the number of selected variables, not the data dimension.
New method for efficient personalized learning in mobile health.
problem Efficient and personalized learning in mobile health.
method Proposes a novel generative process on kernel composition for online Gaussian Process regression.
result Trajectories of kernel evolutions can be transferred between users to improve learning and kernels are meaningful for mHealth prediction.
A flexible variable selection algorithm with sparsistency guarantees.
problem Variable selection in high-dimensional data with theoretical guarantees.
method Kernel-based estimation of regression and gradient functions, followed by hard thresholding.
result Desirable asymptotic sparsistency established for general RKHS.
Unified kernel approach for similarity and clustering.
problem Challenges in similarity measurement and nonlinear similarity.
method Simultaneously learns cluster indicator matrix and similarity information in kernel spaces.
result Automatic optimization of cluster indicator matrix, similarity relations, and kernel combination.
This paper uses MIO to select features for kernel SVM classification.
problem Feature selection for kernel SVM classification.
method Mixed-integer optimization (MIO) for feature subset selection.
result The MIO approach can often outperform linear-SVM-based methods in prediction performance.
Stein Points improve posterior approximation with minimal points.
problem Approximating posterior distributions with limited points.
method Greedy or conditional gradient method to minimize kernel Stein discrepancy.
result Stein Points enable accurate approximation at low computational cost.
A new method ranks Gaussian process kernels for regression.
problem Choosing between different kernel functions for Gaussian process regression.
method Approximation set coding framework for model selection.
result Approximation set coding shows promise as a model selection criterion.
Paper proposes a new method to solve Schrödinger Bridge Problem using kernel regression.
problem Schrödinger Bridge Problem in the context of entropic optimal transport.
method Forward-reverse iterative Monte Carlo procedure using kernel regression.
result Developed a provably convergent algorithm for approximating Schrödinger potentials.
Paper proposes a new landmark selection method for kernel ridge regression.
problem Efficient landmark selection for scalable kernel methods.
method Two-step approach: first computes importance scores, then clusters them into landmarks.
result Proposed method provides better accuracy and efficiency trade-offs.