A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Enhances KLR for indefinite kernels with -norm regularization.
Efficiently performs robust and sparse kernel regression.
In nonparametric classification and regression problems, regularized kernel methods, in particular support vector machines, attract much attention in theoretical and in applied statistics. In an abstract sense, regularized kernel methods (simply called SVMs here) can be seen as regularized M-estimators for a parameter …
Derives kernel PCA with Nyström method for scalability.
Random feature approximation speeds up spectral methods and improves learning rates.
We theoretically investigate the convergence rate and support consistency (i.e., correctly identifying the subset of non-zero coefficients in the large sample limit) of multiple kernel learning (MKL). We focus on MKL with block-l1 regularization (inducing sparse kernel combination), block-l2 regularization (inducing un…
The paper analyzes how re-weighting helps in reducing variance in high-dimensional kernel methods under covariate shifts.
The paper examines how adversarial training and noise affect neural network performance.
Paper develops a new method for distribution regression with indefinite kernels.
Regularization techniques are widely used to improve the generality, robustness, and efficiency of deep convolutional neural networks (DCNNs). In this paper, we propose a novel approach of regulating DCNN convolutional kernels by a structured filter bank. Comparing with the existing regularization methods, such as $\el…
We propose a new optimization algorithm for Multiple Kernel Learning (MKL) called SpicyMKL, which is applicable to general convex loss functions and general types of regularization. The proposed SpicyMKL iteratively solves smooth minimization problems. Thus, there is no need of solving SVM, LP, or QP internally. SpicyM…
In this paper, we are interested in constructing general graph-based regularizers for multiple kernel learning (MKL) given a structure which is used to describe the way of combining basis kernels. Such structures are represented by sum-product networks (SPNs) in our method. Accordingly we propose a new convex regulariz…
Paper explores how DPP sampling can implicitly regularize kernel regression.
We propose a vector-valued regression problem whose solution is equivalent to the reproducing kernel Hilbert space (RKHS) embedding of the Bayesian posterior distribution. This equivalence provides a new understanding of kernel Bayesian inference. Moreover, the optimization problem induces a new regularization for the …
Random features improve neural operators' generalization properties.
This work closes the theory-practice gap for distributed optimization methods by introducing a new regularity condition.
Diverse sampling improves kernel methods' performance in sparse regions.
In this paper, we discuss how a suitable family of tensor kernels can be used to efficiently solve nonparametric extensions of regularized learning methods. Our main contribution is proposing a fast dual algorithm, and showing that it allows to solve the problem efficiently. Our results contrast recent finding…
Regularized empirical risk minimization including support vector machines plays an important role in machine learning theory. In this paper regularized pairwise learning (RPL) methods based on kernels will be investigated. One example is regularized minimization of the error entropy loss which has recently attracted qu…
New estimator reduces kernel mean estimation error.
Novel method learns memory kernels in Langevin equations.
This paper improves neural network generalization by dynamically learning kernel parameters.
New method learns kernels in nonlocal operators robustly.
Unified analysis of kernel-based and locally adaptive bandit optimization methods.
Kernel methods are among the most popular techniques in machine learning. From a frequentist/discriminative perspective they play a central role in regularization theory as they provide a natural choice for the hypotheses space and the regularization functional through the notion of reproducing kernel Hilbert spaces. F…
A new method for accurately reconstructing signals without knowing the kernel or signal regularity.
Needlets have been recognized as state-of-the-art tools to tackle spherical data, due to their excellent localization properties in both spacial and frequency domains. This paper considers developing kernel methods associated with the needlet kernel for nonparametric regression problems whose predictor variables are de…
RF models implicitly regularize kernel methods as feature count increases.
We investigate if kernel regularization methods can achieve minimax convergence rates over a source condition regularity assumption for the target function. These questions have been considered in past literature, but only under specific assumptions about the decay, typically polynomial, of the spectrum of the the kern…
The paper develops methods to handle missing data using regularized M-estimation in reproducing kernel Hilbert space.
Unified analysis for nonlinear parametric models in Bayesian optimization.
Additive principal components (APCs for short) are a nonlinear generalization of linear principal components. We focus on smallest APCs to describe additive nonlinear constraints that are approximately satisfied by the data. Thus APCs fit data with implicit equations that treat the variables symmetrically, as opposed t…
Error estimates for nonlinear PDEs using kernel/GP methods.
We propose a novel adversarial training method in feature space that improves model robustness and computational efficiency.
Additive models play an important role in semiparametric statistics. This paper gives learning rates for regularized kernel based methods for additive models. These learning rates compare favourably in particular in high dimensions to recent results on optimal learning rates for purely nonparametric regularized kernel …
Modern deep neural networks require a tremendous amount of data to train, often needing hundreds or thousands of labeled examples to learn an effective representation. For these networks to work with less data, more structure must be built into their architectures or learned from previous experience. The learned weight…
Extends random feature analysis to spectral methods and improves learning rates.
Study of regularized least squares in RKKS with indefinite kernels.
New method resolves density ratio estimation saturation issues.
Proposes a new graph kernel framework using regularized Wasserstein distances.
Distributionally robust optimization (DRO) has attracted attention in machine learning due to its connections to regularization, generalization, and robustness. Existing work has considered uncertainty sets based on phi-divergences and Wasserstein distances, each of which have drawbacks. In this paper, we study DRO wit…
We prove rates of convergence in the statistical sense for kernel-based least squares regression using a conjugate gradient algorithm, where regularization against overfitting is obtained by early stopping. This method is directly related to Kernel Partial Least Squares, a regression method that combines supervised dim…
Regularized empirical risk minimization using kernels and their corresponding reproducing kernel Hilbert spaces (RKHSs) plays an important role in machine learning. However, the actually used kernel often depends on one or on a few hyperparameters or the kernel is even data dependent in a much more complicated manner. …
QMME balances cost and speed in convex optimization.
New approach to learning kernels from data using AIT principles.
Due to the growing ubiquity of unlabeled data, learning with unlabeled data is attracting increasing attention in machine learning. In this paper, we propose a novel semi-supervised kernel learning method which can seamlessly combine manifold structure of unlabeled data and Regularized Least-Squares (RLS) to learn a ne…
Proposes practical kernel tests for -divergences with theoretical guarantees.
In this paper, we propose an outlier-robust regularized kernel-based method for linear system identification. The unknown impulse response is modeled as a zero-mean Gaussian process whose covariance (kernel) is given by the recently proposed stable spline kernel, which encodes information on regularity and exponential …