arXiv research

The paper studies convergence rates of Tsallis entropic regularization in optimal transport.

We investigate the learning rate of multiple kernel learning (MKL) with $\ell_1$ and elastic-net regularizations. The elastic-net regularization is a composition of an $\ell_1$-regularizer for inducing the sparsity and an $\ell_2$-regularizer for controlling the smoothness. We focus on a sparse setting where the total …

Study on convergence rates for optimal transport with regularization.

Regularized mixtures improve inflation and interest rate forecasts, especially correcting overconfidence.

Improved fast rates for decision making with forward-KL regularization in contextual bandits.

New theorem for generalized group sparsity improves consistency and convergence rates.

We investigate the learning rate of multiple kernel leaning (MKL) with elastic-net regularization, which consists of an $\ell_1$-regularizer for inducing the sparsity and an $\ell_2$-regularizer for controlling the smoothness. We focus on a sparse setting where the total number of kernels is large but the number of non…

SGD with large learning rates can achieve better test accuracy than expected.

We theoretically investigate the convergence rate and support consistency (i.e., correctly identifying the subset of non-zero coefficients in the large sample limit) of multiple kernel learning (MKL). We focus on MKL with block-l1 regularization (inducing sparse kernel combination), block-l2 regularization (inducing un…

In this paper, we give a new generalization error bound of Multiple Kernel Learning (MKL) for a general class of regularizations, and discuss what kind of regularization gives a favorable predictive accuracy. Our main target in this paper is dense type regularizations including \ellp-MKL. According to the recent numeri…

Optimizes convergence rate of stochastic proximal algorithms for composite convex problems.

Paper improves learning rates for GSC loss functions using iterated Tikhonov regularization.

Dropout regularizes against high-order interactions by canceling interaction rates.

We investigate regularized algorithms combining with projection for least-squares regression problem over a Hilbert space, covering nonparametric regression over a reproducing kernel Hilbert space. We prove convergence results with respect to variants of norms, under a capacity assumption on the hypothesis space and a …

Paper analyzes Nyström regularization for time series forecasting with sequential sub-sampling.

Introduces self-regularization for analyzing learning algorithms.

Deep neural networks without regularization can achieve consistent estimates with good convergence rates.

ERM and RERM minimize error even with malicious label corruptions.

New learning rates derived for Tikhonov-regularized problems without kernel assumptions.

Entropy regularized algorithms such as Soft Q-learning and Soft Actor-Critic, recently showed state-of-the-art performance on a number of challenging reinforcement learning (RL) tasks. The regularized formulation modifies the standard RL objective and thus generally converges to a policy different from the optimal gree…

We consider the stochastic contextual bandit problem with additional regularization. The motivation comes from problems where the policy of the agent must be close to some baseline policy which is known to perform well on the task. To tackle this problem we use a nonparametric model and propose an algorithm splitting t…

Study of regularized least squares in RKKS with indefinite kernels.

TempBalance improves neural network training by balancing learning rates across layers.

New methods tackle statistical inverse problems with random data.

In this paper, we give a new sharp generalization bound of lp-MKL which is a generalized framework of multiple kernel learning (MKL) and imposes lp-mixed-norm regularization instead of l1-mixed-norm regularization. We utilize localization techniques to obtain the sharp learning rate. The bound is characterized by the d…

New insights into network generalization show learning rate affects both norm and sharpness.

A new method for faster optimization in high dimensions.

Study improves estimation of functions from noisy data using convex penalties.

The paper analyzes reg-SGD for convex problems, proving convergence and quantifying the rate of convergence.

VAEs improve generalization by regulating mutual information, contrary to intuition.

Batch Normalization is a commonly used trick to improve the training of deep neural networks. These neural networks use L2 regularization, also called weight decay, ostensibly to prevent overfitting. However, we show that L2 regularization has no regularizing effect when combined with normalization. Instead, regulariza…

LocalDrop uses local Rademacher complexity for neural network regularization.

Deep neural networks achieve optimal learning rates for high-dimensional classification.

Regularization is a well recognized powerful strategy to improve the performance of a learning machine and $l^q$ regularization schemes with $0<q<\infty$ are central in use. It is known that different $q$ leads to different properties of the deduced estimators, say, $l^2$ regularization leads to smooth estimators while…

The paper analyzes the convergence rates of Q-learning with entropy regularization and linear function approximation.

The Sinkhorn-Knopp derivatives converge with linear rate.

Random feature approximation speeds up spectral methods and improves learning rates.

New algorithm improves OT map estimation for semi-discrete settings.

A typical approach in estimating the learning rate of a regularized learning scheme is to bound the approximation error by the sum of the sampling error, the hypothesis error and the regularization error. Using a reproducing kernel space that satisfies the linear representer theorem brings the advantage of discarding t…

We prove rates of convergence in the statistical sense for kernel-based least squares regression using a conjugate gradient algorithm, where regularization against overfitting is obtained by early stopping. This method is directly related to Kernel Partial Least Squares, a regression method that combines supervised dim…

We provide theoretical analysis of the statistical and computational properties of penalized $M$-estimators that can be formulated as the solution to a possibly nonconvex optimization problem. Many important estimators fall in this category, including least squares regression with nonconvex regularization, generalized …

New theoretical framework improves error rates for sparse learning with convex regularization.

New convergence rates found for PnP methods using MMSE denoisers.

We study realizable continual linear regression under random task orderings, a common setting for developing continual learning theory. In this setup, the worst-case expected loss after $k$ learning iterations admits a lower bound of $Ω(1/k)$. However, prior work using an unregularized scheme has only established an up…

We consider the learning algorithms under general source condition with the polynomial decay of the eigenvalues of the integral operator in vector-valued function setting. We discuss the upper convergence rates of Tikhonov regularizer under general source condition corresponding to increasing monotone index function. T…

Proposes reducing random error in stochastic optimization by variance regularization.

Paper develops a new method for distribution regression with indefinite kernels.

New framework assesses regularization norms in ill-posed problems, revealing L2 instability and proposing adaptive fractional RKHS solutions.