arXiv research

This work introduces a new method for coupling base and target densities in generative models.

A new method generates mixed-type features in tabular data with improved realism and accuracy.

Gaussian processes (GPs) provide a nonparametric representation of functions. However, classical GP inference suffers from high computational cost and it is difficult to design nonstationary GP priors in practice. In this paper, we propose a sparse Gaussian process model, EigenGP, based on the Karhunen-Loeve (KL) expan…

Improved flow-based models capture dependencies better with multi-scale autoregressive priors.

One of the major shortcomings of variational autoencoders is the inability to produce generations from the individual modalities of data originating from mixture distributions. This is primarily due to the use of a simple isotropic Gaussian as the prior for the latent code in the ancestral sampling procedure for the da…

New algorithm reconstructs sparse networks in subquadratic time.

We reformulate data-dependent constraints to ensure they are always met with high probability.

We present a study of generalization for data-dependent hypothesis sets. We give a general learning guarantee for data-dependent hypothesis sets based on a notion of transductive Rademacher complexity. Our main result is a generalization bound for data-dependent hypothesis sets expressed in terms of a notion of hypothe…

PAC-Bayesian theory applied to data-dependent hypothesis sets yields uniform generalization bounds.

Optimal kernel in KR can be data-dependent, improving model performance.

The paper improves PAC-Bayes bounds for data-dependent predictors.

Paper introduces data-dependent SSP for private linear and logistic regression.

The paper shows robustness and generalization are closely connected via data-dependent bounds.

Model-free reinforcement learning algorithms combined with value function approximation have recently achieved impressive performance in a variety of application domains. However, the theoretical understanding of such algorithms is limited, and existing results are largely focused on episodic or discounted Markov decis…

The study improves representation learning bounds using data-dependent Gaussian mixtures.

Meta-learning bounds derived using PAC-Bayes theory for improved generalization.

The Probably Approximately Correct (PAC) Bayes framework (McAllester, 1999) can incorporate knowledge about the learning algorithm and (data) distribution through the use of distribution-dependent priors, yielding tighter generalization bounds on data-dependent posteriors. Using this flexibility, however, is difficult,…

New algorithm achieves data-dependent regret bounds in MDPs with unknown transitions.

ZNMF improves facial recognition performance using data-dependent penalties.

Paper develops neural network approximation for pessimistic offline RL with theoretical guarantees.

Survey on new data-dependent bounds for neural networks.

Fast robust subspace tracking in sparse data-dependent noise with near-optimal delay.

Reweighting improves risk bounds in certain data regions.

We improve the robustness of Deep Neural Net (DNN) to adversarial attacks by using an interpolating function as the output activation. This data-dependent activation remarkably improves both the generalization and robustness of DNN. In the CIFAR10 benchmark, we raise the robust accuracy of the adversarially trained Res…

Enhances SDR via Hellinger correlation for better data dependency understanding.

Paper establishes a generalization bound for gradient flow using a data-dependent kernel.

In this paper, we consider the problem of prediction with expert advice in dynamic environments. We choose tracking regret as the performance metric and develop two adaptive and efficient algorithms with data-dependent tracking regret bounds. The first algorithm achieves a second-order tracking regret bound, which impr…

In this work, we improve upon the stepwise analysis of noisy iterative learning algorithms initiated by Pensia, Jog, and Loh (2018) and recently extended by Bu, Zou, and Veeravalli (2019). Our main contributions are significantly improved mutual information bounds for Stochastic Gradient Langevin Dynamics via data-depe…

In this paper, we introduce a novel method to generate interpretable regression function estimators. The idea is based on called data-dependent coverings. The aim is to extract from the data a covering of the feature space instead of a partition. The estimator predicts the empirical conditional expectation over the cel…

This work obtains novel finite sample guarantees for Principal Component Analysis (PCA). These hold even when the corrupting noise is non-isotropic, and a part (or all of it) is data-dependent. Because of the latter, in general, the noise and the true data are correlated. The results in this work are a significant impr…

Framework evaluates privacy cost of non-private pre-processing in DP pipelines.

PriorGrad improves speech synthesis models by using data-dependent adaptive priors.

The Adam algorithm has become extremely popular for large-scale machine learning. Under convexity condition, it has been proved to enjoy a data-dependant $O(\sqrt{T})$ regret bound where $T$ is the time horizon. However, whether strong convexity can be utilized to further improve the performance remains an open problem…

Paper introduces new bounds linking data compressibility to generalization error.

In this paper, we consider the problem of recovering a graph that represents the statistical data dependency among nodes for a set of data samples generated by nodes, which provides the basic structure to perform an inference task, such as MAP (maximum a posteriori). This problem is referred to as structure learning. W…

New decentralized KRR algorithm adapts to node-specific data.

Improved pruning method finds winning neural network subnetworks.

We present algorithms for topic modeling based on the geometry of cross-document word-frequency patterns. This perspective gains significance under the so called separability condition. This is a condition on existence of novel-words that are unique to each topic. We present a suite of highly efficient algorithms based…

The paper analyzes how re-weighting helps in reducing variance in high-dimensional kernel methods under covariate shifts.

The randomized-feature approach has been successfully employed in large-scale kernel approximation and supervised learning. The distribution from which the random features are drawn impacts the number of features required to efficiently perform a learning task. Recently, it has been shown that employing data-dependent …

In this paper we analyze a budgeted learning setting, in which the learner can only choose and observe a small subset of the attributes of each training example. We develop efficient algorithms for ridge and lasso linear regression, which utilize the geometry of the data by a novel data-dependent sampling scheme. When …

In this dissertation we propose alternative analysis of distributed stochastic gradient descent (SGD) algorithms that rely on spectral properties of the data covariance. As a consequence we can relate questions pertaining to speedups and convergence rates for distributed SGD to the data distribution instead of the regu…

Algorithm provides online learning guarantees against general comparators in full and bandit feedback.

We establish a theoretical link between adversarial training and operator norm regularization for deep neural networks. Specifically, we prove that $\ell_p$-norm constrained projected gradient ascent based adversarial training with an $\ell_q$-norm loss on the logits of clean and perturbed inputs is equivalent to data-…

The study characterizes diffusion model generalization using data-dependent ridge manifolds.

Existing Rademacher complexity bounds for neural networks rely only on norm control of the weight matrices and depend exponentially on depth via a product of the matrix norms. Lower bounds show that this exponential dependence on depth is unavoidable when no additional properties of the training data are considered. We…

New NMF algorithm uses Toeplitz matrix for facial recognition.

Paper analyzes D-SGD convergence with heterogeneous data and proposes topology learning.