arXiv research

We reformulate data-dependent constraints to ensure they are always met with high probability.

We present a study of generalization for data-dependent hypothesis sets. We give a general learning guarantee for data-dependent hypothesis sets based on a notion of transductive Rademacher complexity. Our main result is a generalization bound for data-dependent hypothesis sets expressed in terms of a notion of hypothe…

Meta-learning bounds derived using PAC-Bayes theory for improved generalization.

The paper improves PAC-Bayes bounds for data-dependent predictors.

The study improves representation learning bounds using data-dependent Gaussian mixtures.

The paper shows robustness and generalization are closely connected via data-dependent bounds.

Optimal kernel in KR can be data-dependent, improving model performance.

Paper introduces data-dependent SSP for private linear and logistic regression.

The Probably Approximately Correct (PAC) Bayes framework (McAllester, 1999) can incorporate knowledge about the learning algorithm and (data) distribution through the use of distribution-dependent priors, yielding tighter generalization bounds on data-dependent posteriors. Using this flexibility, however, is difficult,…

ZNMF improves facial recognition performance using data-dependent penalties.

In this paper, we consider the problem of recovering a graph that represents the statistical data dependency among nodes for a set of data samples generated by nodes, which provides the basic structure to perform an inference task, such as MAP (maximum a posteriori). This problem is referred to as structure learning. W…

Survey on new data-dependent bounds for neural networks.

PAC-Bayesian theory applied to data-dependent hypothesis sets yields uniform generalization bounds.

The randomized-feature approach has been successfully employed in large-scale kernel approximation and supervised learning. The distribution from which the random features are drawn impacts the number of features required to efficiently perform a learning task. Recently, it has been shown that employing data-dependent …

Paper establishes a generalization bound for gradient flow using a data-dependent kernel.

In this work, we improve upon the stepwise analysis of noisy iterative learning algorithms initiated by Pensia, Jog, and Loh (2018) and recently extended by Bu, Zou, and Veeravalli (2019). Our main contributions are significantly improved mutual information bounds for Stochastic Gradient Langevin Dynamics via data-depe…

Framework evaluates privacy cost of non-private pre-processing in DP pipelines.

A new learning rule consistently reduces error over data samples.

Algorithm provides online learning guarantees against general comparators in full and bandit feedback.

Dropout controls model capacity in deep learning and matrix completion.

The Adam algorithm has become extremely popular for large-scale machine learning. Under convexity condition, it has been proved to enjoy a data-dependant $O(\sqrt{T})$ regret bound where $T$ is the time horizon. However, whether strong convexity can be utilized to further improve the performance remains an open problem…

A method to prevent image representation collapse through data-dependent augmentation.

In this paper we analyze a budgeted learning setting, in which the learner can only choose and observe a small subset of the attributes of each training example. We develop efficient algorithms for ridge and lasso linear regression, which utilize the geometry of the data by a novel data-dependent sampling scheme. When …

A new learning method uses data to learn from large model sets.

New algorithm achieves data-dependent regret bounds in MDPs with unknown transitions.

New bounds ensure reliable deep learning performance without model changes.

Ensemble learning, the machine learning paradigm where multiple algorithms are combined, has exhibited promising perfomance in a variety of tasks. The present work focuses on unsupervised ensemble classification. The term unsupervised refers to the ensemble combiner who has no knowledge of the ground-truth labels that …

New data-dependent priors improve PAC-Bayes bounds.

Improves normalizing flows by incorporating data dependencies.

Learning with kernels is an important concept in machine learning. Standard approaches for kernel methods often use predefined kernels that require careful selection of hyperparameters. To mitigate this burden, we propose in this paper a framework to construct and learn a data-dependent kernel based on random features …

Framework evaluates the impact of prior knowledge in deep learning models.

IDK improves anomaly detection for points and groups without explicit learning.

Fast robust subspace tracking in sparse data-dependent noise with near-optimal delay.

Reweighting improves risk bounds in certain data regions.

We improve the robustness of Deep Neural Net (DNN) to adversarial attacks by using an interpolating function as the output activation. This data-dependent activation remarkably improves both the generalization and robustness of DNN. In the CIFAR10 benchmark, we raise the robust accuracy of the adversarially trained Res…

Paper proposes a new method to learn distribution kernels via entropy maximization.

Random features provide a practical framework for large-scale kernel approximation and supervised learning. It has been shown that data-dependent sampling of random features using leverage scores can significantly reduce the number of features required to achieve optimal learning bounds. Leverage scores introduce an op…

Enhances SDR via Hellinger correlation for better data dependency understanding.

The study characterizes diffusion model generalization using data-dependent ridge manifolds.

Directed graphical models (DGMs) are a class of probabilistic models that are widely used for predictive analysis in sensitive domains, such as medical diagnostics. In this paper we present an algorithm for differentially private learning of the parameters of a DGM with a publicly known graph structure over fully obser…

This work proves generalization bounds for neural networks without Lipschitz assumptions.

New NMF algorithm uses Toeplitz matrix for facial recognition.

In this paper, we consider the problem of prediction with expert advice in dynamic environments. We choose tracking regret as the performance metric and develop two adaptive and efficient algorithms with data-dependent tracking regret bounds. The first algorithm achieves a second-order tracking regret bound, which impr…

In this paper, we introduce a novel method to generate interpretable regression function estimators. The idea is based on called data-dependent coverings. The aim is to extract from the data a covering of the feature space instead of a partition. The estimator predicts the empirical conditional expectation over the cel…

This work obtains novel finite sample guarantees for Principal Component Analysis (PCA). These hold even when the corrupting noise is non-isotropic, and a part (or all of it) is data-dependent. Because of the latter, in general, the noise and the true data are correlated. The results in this work are a significant impr…

PriorGrad improves speech synthesis models by using data-dependent adaptive priors.

Kernel methods offer the flexibility to learn complex relationships in modern, large data sets while enjoying strong theoretical guarantees on quality. Unfortunately, these methods typically require cubic running time in the data set size, a prohibitive cost in the large-data setting. Random feature maps (RFMs) and the…

This paper presents eight PAC-Bayes bounds to analyze the generalization performance of multi-view classifiers. These bounds adopt data dependent Gaussian priors which emphasize classifiers with high view agreements. The center of the prior for the first two bounds is the origin, while the center of the prior for the t…