Develops multi-kernel regression for graph signal processing.
problem Smoothness of graph signals over a graph.
method Estimates linear weights to learn effective kernel function using graph smoothness.
result Optimization problem is convex and accelerated projected gradient descent solution proposed.
A new kernel-based CI test improves on existing methods.
problem Testing conditional independence (CI) in a broad range of dependencies.
method Regression-model-agnostic kernel-based CI test using reproducing kernel Hilbert spaces.
result GKCM outperforms state-of-the-art CI tests in simulations.
Optimizes learning rates for kernel-based expectile regression.
problem Estimating conditional expectiles efficiently.
method Support vector machine type approach using Gaussian RBF kernels.
result Learning rates are minimax optimal with a logarithmic factor.
Tree ensembles like RF and GBT can be seen as kernels, improving regression and classification performance.
problem Improving kernel methods for tree ensemble based models.
method Investigation of RF and GBT kernels in simulation and real data.
result RF and GBT kernels are competitive to their respective ensembles in higher dimensions, particularly with noisy features.
This paper improves kernel-based regression using transfer learning.
problem Improving generalization performance in kernel-based regression.
method Two-step kernel-based estimator for known transferable sources and novel aggregation algorithm for unknown sources.
result Established statistical properties and validated effectiveness of proposed methods.
Paper develops a new method for distribution regression with indefinite kernels.
problem Distribution regression with indefinite kernels.
method Coefficient-based regularized distribution regression with two-stage sampling.
result Optimal learning rates derived for the algorithm under mild conditions.
Nonlinear kernel regression models are often used in statistics and machine learning because they are more accurate than linear models. Variable selection for kernel regression models is a challenge partly because, unlike the linear regression setting, there is no clear concept of an effect size for regression coeffici…
Kernel method outperforms deep neural networks in speech enhancement.
problem Improving single-channel speech enhancement performance.
method Kernel regression with an exponential power kernel and EigenPro iterative method.
result Kernel method consistently outperforms deep neural networks in speech enhancement.
Mklaren approximates multiple kernel matrices efficiently for regression.
problem Efficiently scaling kernel-based learning to large datasets.
method Geometrical concepts, Incomplete Cholesky decomposition, least-angle regression.
result Significantly lower approximation ranks with equivalent test accuracy.
This paper improves distributed regression by correcting bias in regularization kernel networks.
problem Improving the performance of distributed regression with biased base algorithms.
method Develops a bias-corrected version of regularization kernel network for distributed regression.
result Achieves optimal learning rates in both single and distributed regression settings.
Optimal kernel improves estimation accuracy in modal statistical methods.
problem Estimation accuracy of kernel-based modal statistical methods depends on the kernel used.
method The study theoretically shows an optimal kernel that minimizes asymptotic error criterion.
result An optimal kernel minimizes the error criterion when using an optimal bandwidth.
Unified analysis of kernel ridge regression methods for pairwise learning.
problem Theoretical analysis of kernel-based pairwise learning methods.
method Unified review and analysis of kernel ridge regression methods.
result Kronecker kernel ridge regression unifies and explains existing methods.
Bayesian methods improve kernel and mutual k-nearest neighbor regression.
problem Improving nonparametric regression methods for better accuracy and hyperparameter selection.
method Bayesian extensions of kernel and mutual k-nearest neighbor regression methods based on Gaussian process models.
result The proposed methods asymptotically converge to the original methods and perform better or equally well in simulations.
Novel algorithm identifies nonlinear Granger causal relationships using kernel ridge regression.
problem Identification of nonlinear Granger causal relationships.
method Flexible plug-in architecture with kernel ridge regression using radial basis function.
result Kernel ridge regression in mlcausality achieves competitive AUC scores and more finely calibrated p-values.
Improved learning theory for kernel distribution regression with two-stage sampling.
problem Distribution regression problem and two-stage sampling setting.
method Kernel methods, near-unbiased condition, new error bounds, convergence rates.
result Strictly improved convergence rates for three important classes of kernels.
Efficiently performs robust and sparse kernel regression.
problem Robust and sparse kernel regression.
method Sign gradient descent and early stopping.
result Sign gradient descent achieves robust and sparse kernel regression efficiently.
A new kernel regression method using sparse metric learning improves prediction accuracy.
problem Improving prediction accuracy in kernel regression.
method Sparse metric learning applied to kernel regression model.
result The proposed method leads to better prediction results compared to existing methods.
Kernel Conjugate Gradient achieves fast convergence rates for regression.
problem Statistical rates of convergence for kernel-based regression.
method Kernel Conjugate Gradient algorithm with early stopping for regularization.
result Upper bounds for L2 and Hilbert norms, matching minimax lower bounds. Paper studies distributed kernel regression with imperfect kernels, achieving optimal rates.
problem Optimal rates of distributed regression with imperfect kernels.
method Divide and conquer approach, response weighted base algorithms, leave one out analysis, bias correction.
result Achieves capacity independent optimal rates for distributed kernel regression with imperfect kernels.
This paper introduces Kernel-based Information Criterion (KIC) for model selection in regression analysis. The novel kernel-based complexity measure in KIC efficiently computes the interdependency between parameters of the model using a variable-wise variance and yields selection of better, more robust regressors. Expe…
Paper introduces robust distribution regression using kernel methods.
problem Distribution regression from probability measures to real-valued responses.
method Introduces a robust loss function lσ and a windowing function V for two-stage sampling problems. result Shows improved learning rates and robustness with the robust distribution regression (RDR) scheme.
A new method for distribution regression using sliced Wasserstein distance.
problem Learning functions over spaces of probabilities.
method Proposes an OT-based estimator using the Sliced Wasserstein distance.
result Proves universal consistency and excess risk bounds for the proposed estimator.
Kernel ridge regression imputation with consistent variance estimation for handling missing data.
problem Handling missing data in statistical analysis.
method Kernel ridge regression imputation combined with entropy method for variance estimation.
result Root-n consistency of the imputation estimator in a Sobolev space setting.
Improved prediction and noise removal for dynamical time series.
problem Effective prediction and noise removal for noisy dynamical time series.
method Combination of kernel based regression and smooth splines for denoising and prediction.
result Combination of kernel based regression and smooth splines yields more accurate predictors by a factor of 2 or more.
Paper proposes a method for early stopping in regression using reproducing kernels.
problem Early stopping for iterative learning algorithms in nonparametric regression.
method Data-driven rule based on minimum discrepancy principle, validated by fixed-point analysis of localized Rademacher complexities.
result The proposed rule is minimax-optimal and performs comparably to cross-validation.
DR-ABC uses kernel-based distribution regression for better ABC likelihood approximations.
problem Difficulties in exact posterior inference due to intractable likelihood functions.
method Kernel-based distribution regression for constructing better summary statistics.
result Superior performance compared to related methods on various problems.
Improves regression efficiency by separating material and immaterial parts of responses.
problem Improving estimation efficiency in nonlinear multivariate regressions.
method Kernel envelope (KENV) estimator for nonparametric response envelopes in reproducing kernel Hilbert space.
result KENV achieves lower in-sample prediction risk than kernel ridge regression in non-trivial immaterial components.
Additive models play an important role in semiparametric statistics. This paper gives learning rates for regularized kernel based methods for additive models. These learning rates compare favourably in particular in high dimensions to recent results on optimal learning rates for purely nonparametric regularized kernel …
Kernel-based L2-boosting with structure constraints improves regression efficiency.
problem Developing efficient kernel methods for regression.
method Kernel-based re-scaled boosting with truncation (KReBooT).
result KReBooT achieves near overfitting resistance and sparse estimates.
New method quantifies uncertainty in distributed regression.
problem Large datasets make traditional regression techniques ineffective.
method Data-driven approach to uncertainty quantification for averaged estimator.
result Rigorous theoretical guarantees for sup-norm consistency.
This paper tackles the problem of selecting among several linear estimators in non-parametric regression; this includes model selection for linear regression, the choice of a regularization parameter in kernel ridge regression, spline smoothing or locally weighted regression, and the choice of a kernel in multiple kern…
Paper proposes a novel framework for structure learning using unstructured kernel-based M-regression.
problem Identifying underlying structures of true target functions from observed data.
method General and novel framework using unstructured M-regression in RKHS, inspired by gradient functions.
result Asymptotic results established for a wide range of loss functions, including mean, quantile, likelihood, and margin-based methods.
The paper analyzes the statistical cost of tuning kernel hyperparameters in robust regression.
problem Finding the best interpolant from a class of kernels with unknown hyperparameters under adversarial noise.
method Finite-sample guarantees, subsampling guarantee for linear regression, ε-net argument for discretizing kernel parameterizations.
result Hyperparameter optimization increases sample complexity by just a logarithmic factor, compared to known parameters.
Improved robustness in kernel-based regression via novel loss function and IRLS.
problem Noise sensitivity in kernel-based regression methods.
method Proposed ℓs-loss function and iteratively reweighted least squares (IRLS) optimization. result Improved noise robustness in kernel-based regression methods.
Scalable kernel methods for large datasets using Fourier representations and NUFFT.
problem Cubic complexity in kernel methods limits their use on large-scale datasets.
method Fourier representation of kernels combined with NUFFT for O(n log n) complexity.
result Achieves minimax convergence rates and processes up to tens of billions of samples.
Kernel-based function approximation improves reinforcement learning performance.
problem Average reward reinforcement learning in infinite horizon settings.
method Optimistic algorithm based on kernel ridge regression.
result No-regret performance guarantees and confidence intervals for kernel-based predictions.
The paper examines MAPE's use in regression models and its implications.
problem The use of Mean Absolute Percentage Error (MAPE) as a quality measure for regression models.
method Proves the existence of an optimal MAPE model, shows universal consistency of Empirical Risk Minimization based on MAPE, and demonstrates the equivalence of MAPE model selection to weighted MAE regression.
result Finding the best model under MAPE is equivalent to weighted MAE regression, and this strategy is applied to kernel regression.
SSDKL uses unlabeled data to improve regression models.
problem Lack of labeled data in training deep learning models.
method Semi-supervised deep kernel learning minimizing predictive variance.
result Improvements on real-world regression tasks.
Paper introduces new regression methods for consistent estimation of biophysical parameters.
problem Estimating biophysical parameters while respecting auxiliary variables.
method Linear and nonlinear kernel-based regression models with consistency constraints.
result Models provide closed-form solutions and successfully estimate chlorophyll content.
New method selects variables for nonlinear regression in large datasets.
problem Nonlinear regression with large-scale datasets.
method Kernel-based variable selection with random features.
result Outstanding performance on large-scale synthetic and real datasets.
New tests for binary classification regression functions without distribution assumptions.
problem Testing regression functions in binary classification without distributional assumptions.
method Conditional kernel mean embeddings and resampling-based framework.
result Distribution-free hypothesis tests with exact type I error control.
Gaussian process regression loses locality in high dimensions, affecting molecular energy surface fitting.
problem Loss of locality in high-dimensional Gaussian process regression.
method Analysis of Matern family kernels and multi-zeta basis functions.
result The property of locality disappears in high dimensions, impacting regression quality.
Paper tackles indefinite kernels in logistic regression.
problem Building logistic regression with indefinite kernels.
method Introduces IKLR model in RKKS, uses concave-inexact-convex procedure (CCICP) to solve non-convex optimization.
result Proposed method works effectively under deterministic and stochastic settings.
Kernel for STL formulae enables machine learning in temporal logic.
problem Lack of a kernel for STL formulae.
method Define a kernel for STL formulae and embed them into a Hilbert space.
result Kernel-based machine learning algorithms can now be applied to STL formulae.
Kernel method improves instrumental variable regression rates.
problem Nonparametric instrumental variable regression with weak instruments.
method Kernel-based two-stage least-squares method, strong L2 convergence analysis. result Minimax optimal rates for instrumental regression under standard assumptions.
Diverse sampling improves kernel methods' performance in sparse regions.
problem Improving kernel methods' performance in sparse regions of datasets.
method Using Determinantal Point Processes (DPP) for sampling diverse landmarks in Nyström approximation.
result Nyström kernel regression with diverse landmarks increases accuracy in sparse regions of the dataset.
A new method for faster bandwidth selection in Gaussian kernel ridge regression.
problem Efficiently selecting the bandwidth in Gaussian kernel ridge regression.
method Formulated an approximate Jacobian expression for bandwidth selection, proposing a closed-form heuristic.
result Our method is as accurate as cross-validation and marginal likelihood maximization but up to six orders of magnitude faster.
Paper improves understanding of random Fourier features for kernel ridge regression.
problem Understanding statistical properties of random Fourier features for kernel ridge regression.
method Spectral matrix approximation approach to analyze random Fourier features.
result Proves statistical guarantees for kernel ridge regression using random Fourier features.