arXiv research

New bounds study class-specific generalization error in machine learning.

This paper tightens information-theoretic bounds on generalization errors.

New bound matches exact generalization error for quadratic Gaussian problem.

Paper analyzes ECE bias and provides bounds for its estimation.

Proposes a new bound on generalization error using conditional mutual information.

EBUCB framework achieves optimal regret with bounded approximate inference error.

In this paper, we derive generic bounds on the maximum deviations in prediction errors for sequential prediction via an information-theoretic approach. The fundamental bounds are shown to depend only on the conditional entropy of the data point to be predicted given the previous data points. In the asymptotic case, the…

New bounds on machine learning model generalization error moments.

Novel bounds for SGLD show generalization error decreases with more samples.

Study bounds graph neural networks' over-parameterized error.

This paper improves entropy bounds for ranking time-series complexity.

This paper examines error bounds for deep learning classifiers with noisy labels.

New bound on machine learning model performance using Jensen-Shannon information.

The paper bounds the mean absolute error in DNN vector-to-vector regression.

Careful tuning of a regularization parameter is indispensable in many machine learning tasks because it has a significant impact on generalization performances. Nevertheless, current practice of regularization parameter tuning is more of an art than a science, e.g., it is hard to tell how many grid-points would be need…

The study sets lower bounds on MMSE for inferring sensitive features from noisy data.

The paper develops generalization bounds for deep compound Gaussian neural networks.

In this paper, we obtain generic bounds on the variances of estimation and prediction errors in time series analysis via an information-theoretic approach. It is seen in general that the error bounds are determined by the conditional entropy of the data point to be estimated or predicted given the side information or p…

We consider active, semi-supervised learning in an offline transductive setting. We show that a previously proposed error bound for active learning on undirected weighted graphs can be generalized by replacing graph cut with an arbitrary symmetric submodular function. Arbitrary non-symmetric submodular functions can be…

This work sets theoretical limits on meta-learning performance.

New decision-theoretic calibration error metric improves prediction reliability.

Sharp error bounds derived for bidirectional GANs without restrictive assumptions.

Unified framework for high-dimensional online learning with non-divergent error bounds and adaptive gains.

This work explains how large neural networks generalize well despite overparameterization.

An information-theoretic upper bound on the generalization error of supervised learning algorithms is derived. The bound is constructed in terms of the mutual information between each individual training sample and the output of the learning algorithm. The bound is derived under more general conditions on the loss func…

New method for distributional off-policy evaluation using Bellman residual minimization.

Transformers can learn noisy linear systems with depth and IID data.

The paper analyzes LOCV for high-dimensional risk estimation, proving error bounds.

Crowdsourcing is an effective tool for human-powered computation on many tasks challenging for computers. In this paper, we provide finite-sample exponential bounds on the error rate (in probability and in expectation) of hyperplane binary labeling rules under the Dawid-Skene crowdsourcing model. The bounds can be appl…

Sharp bounds derived for test error of finite-rank kernel ridge regression.

Study improves theoretical understanding of Bayesian deep learning for classification tasks.

Mutual information bounds generalization error in variational classifiers.

The paper bounds generalization error for iterative learning with bounded updates.

The Gibbs algorithm's generalization error is bounded, improving with prior volume in low temperatures.

Empirical error estimates improve graph sparsification reliability.

DALMC provides non-asymptotic error bounds for generative models.

Paper improves risk bounds for nonconvex-strongly-concave minimax problems.

Non-negative matrix factorization (NMF) is a knowledge discovery method that is used in many fields. Variational inference and Gibbs sampling methods for it are also wellknown. However, the variational approximation error has not been clarified yet, because NMF is not statistically regular and the prior distribution us…

The CUR matrix decomposition is an important extension of Nyström approximation to a general matrix. It approximates any data matrix in terms of a small number of its columns and rows. In this paper we propose a novel randomized CUR algorithm with an expected relative-error bound. The proposed algorithm has the advanta…

We propose a general framework for reconstructing and denoising single entries of incomplete and noisy entries. We describe: effective algorithms for deciding if and entry can be reconstructed and, if so, for reconstructing and denoising it; and a priori bounds on the error of each entry, individually. In the noiseless…

Study precise sample covariance error for Gaussian centered data.

Principal component analysis (PCA) has been a prominent tool for high-dimensional data analysis. Online algorithms that estimate the principal component by processing streaming data are of tremendous practical and theoretical interests. Despite its rich applications, theoretical convergence analysis remains largely ope…

A new FFT method for Heston model option pricing with explicit error bounds.

Quantile regression with ReLU networks achieves minimax rates for various function types.

DFM models are analyzed for generating distributions with provable convergence.

The study analyzes transfer learning using information theory.

This work bounds the generalization error of private algorithms for discrete data.

Mutual information has been successfully adopted in filter feature-selection methods to assess both the relevancy of a subset of features in predicting the target variable and the redundancy with respect to other variables. However, existing algorithms are mostly heuristic and do not offer any guarantee on the proposed…