arXiv research

The paper improves PAC-Bayes bounds for data-dependent predictors.

Bayesian neural networks explore rare fluctuations for better feature learning.

Adaptive kernels from neural networks improve model performance.

PAC-Bayes bound requires prior to place mass on high-performing predictors.

We consider using an ensemble of binary classifiers for transductive prediction, when unlabeled test data are known in advance. We derive minimax optimal rules for confidence-rated prediction in this setting. By using PAC-Bayes analysis on these rules, we obtain data-dependent performance guarantees without distributio…

Estimates KRR risk from training data for various kernels and hyperparameters.

The Lasso is a computationally efficient regression regularization procedure that can produce sparse estimators when the number of predictors (p) is large. Oracle inequalities provide probability loss bounds for the Lasso estimator at a deterministic choice of the regularization parameter. These bounds tend to zero if …

Conformal Alignment ensures trustworthy outputs from foundation models.

Paper presents new training methods for neural networks with tighter risk certificates.

New method certifies deep graph classifiers with tighter risk bounds.

Analyzes the complexity of linear hypothesis sets using Rademacher complexity.

Study how depth affects inference in deep Bayesian neural networks.

We reformulate data-dependent constraints to ensure they are always met with high probability.

We present a study of generalization for data-dependent hypothesis sets. We give a general learning guarantee for data-dependent hypothesis sets based on a notion of transductive Rademacher complexity. Our main result is a generalization bound for data-dependent hypothesis sets expressed in terms of a notion of hypothe…

PAC-Bayesian theory applied to data-dependent hypothesis sets yields uniform generalization bounds.

Optimal kernel in KR can be data-dependent, improving model performance.

Paper introduces data-dependent SSP for private linear and logistic regression.

Stochastic gradient descent algorithms for training linear and kernel predictors are gaining more and more importance, thanks to their scalability. While various methods have been proposed to speed up their convergence, the model selection phase is often ignored. In fact, in theoretical works most of the time assumptio…

The paper shows robustness and generalization are closely connected via data-dependent bounds.

The study improves representation learning bounds using data-dependent Gaussian mixtures.

Meta-learning bounds derived using PAC-Bayes theory for improved generalization.

The Probably Approximately Correct (PAC) Bayes framework (McAllester, 1999) can incorporate knowledge about the learning algorithm and (data) distribution through the use of distribution-dependent priors, yielding tighter generalization bounds on data-dependent posteriors. Using this flexibility, however, is difficult,…

Introduces self-regularization for analyzing learning algorithms.

New algorithm achieves data-dependent regret bounds in MDPs with unknown transitions.

ZNMF improves facial recognition performance using data-dependent penalties.

Survey on new data-dependent bounds for neural networks.

Fast robust subspace tracking in sparse data-dependent noise with near-optimal delay.

ScoreStop uses gradient tests to stop gradient boosting early.

Reweighting improves risk bounds in certain data regions.

We improve the robustness of Deep Neural Net (DNN) to adversarial attacks by using an interpolating function as the output activation. This data-dependent activation remarkably improves both the generalization and robustness of DNN. In the CIFAR10 benchmark, we raise the robust accuracy of the adversarially trained Res…

Enhances SDR via Hellinger correlation for better data dependency understanding.

The lasso and related sparsity inducing algorithms have been the target of substantial theoretical and applied research. Correspondingly, many results are known about their behavior for a fixed or optimally chosen tuning parameter specified up to unknown constants. In practice, however, this oracle tuning parameter is …

Paper establishes a generalization bound for gradient flow using a data-dependent kernel.

In this paper, we consider the problem of prediction with expert advice in dynamic environments. We choose tracking regret as the performance metric and develop two adaptive and efficient algorithms with data-dependent tracking regret bounds. The first algorithm achieves a second-order tracking regret bound, which impr…

This work improves multi-task regression performance using approximations of full-conformal prediction.

In this work, we improve upon the stepwise analysis of noisy iterative learning algorithms initiated by Pensia, Jog, and Loh (2018) and recently extended by Bu, Zou, and Veeravalli (2019). Our main contributions are significantly improved mutual information bounds for Stochastic Gradient Langevin Dynamics via data-depe…

Neural networks learn task-specific features, influenced by nonlinearity.

In this paper, we introduce a novel method to generate interpretable regression function estimators. The idea is based on called data-dependent coverings. The aim is to extract from the data a covering of the feature space instead of a partition. The estimator predicts the empirical conditional expectation over the cel…

This work obtains novel finite sample guarantees for Principal Component Analysis (PCA). These hold even when the corrupting noise is non-isotropic, and a part (or all of it) is data-dependent. Because of the latter, in general, the noise and the true data are correlated. The results in this work are a significant impr…

Framework evaluates privacy cost of non-private pre-processing in DP pipelines.

PriorGrad improves speech synthesis models by using data-dependent adaptive priors.

The Adam algorithm has become extremely popular for large-scale machine learning. Under convexity condition, it has been proved to enjoy a data-dependant $O(\sqrt{T})$ regret bound where $T$ is the time horizon. However, whether strong convexity can be utilized to further improve the performance remains an open problem…

Paper introduces new bounds linking data compressibility to generalization error.

In this paper, we consider the problem of recovering a graph that represents the statistical data dependency among nodes for a set of data samples generated by nodes, which provides the basic structure to perform an inference task, such as MAP (maximum a posteriori). This problem is referred to as structure learning. W…

New decentralized KRR algorithm adapts to node-specific data.

Improved pruning method finds winning neural network subnetworks.

We present algorithms for topic modeling based on the geometry of cross-document word-frequency patterns. This perspective gains significance under the so called separability condition. This is a condition on existence of novel-words that are unique to each topic. We present a suite of highly efficient algorithms based…

The problem of forecasting conditional probabilities of the next event given the past is considered in a general probabilistic setting. Given an arbitrary (large, uncountable) set C of predictors, we would like to construct a single predictor that performs asymptotically as well as the best predictor in C, on any data.…