arXiv research

Improved MMD test for two-sample testing with random Fourier features.

Markov chains and diffusion processes are indispensable tools in machine learning and statistics that are used for inference, sampling, and modeling. With the growth of large-scale datasets, the computational cost associated with simulating these stochastic processes can be considerable, and many algorithms have been p…

Lasso performs poorly with correlated covariates, but a rescaled approach fixes this.

OTCP extends conformal prediction to multivariate data using optimal transport.

Improved computational efficiency for estimating Wasserstein distance.

Due to the ease of modern data collection, applied statisticians often have access to a large set of covariates that they wish to relate to some observed outcome. Generalized linear models (GLMs) offer a particularly interpretable framework for such an analysis. In these high-dimensional problems, the number of covaria…

We extend the Fourier cosine method to discrete probability distributions, achieving faster convergence rates.

Approximate inference in high-dimensional, discrete probabilistic models is a central problem in computational statistics and machine learning. This paper describes discrete particle variational inference (DPVI), a new approach that combines key strengths of Monte Carlo, variational and search-based techniques. DPVI is…

Study shows computational and statistical gaps in Gaussian Single-Index Models.

TDS provides exact samples for conditional distributions in diffusion models.

We present two paradigms relating algebraic, topological and quantum computational statistics for the topological model for quantum computation. In particular we suggest correspondences between the computational power of topological quantum computers, computational complexity of link invariants and images of braid grou…

We focus on the distribution regression problem: regressing to vector-valued outputs from probability measures. Many important machine learning and statistical tasks fit into this framework, including multi-instance learning and point estimation problems without analytical solution (such as hyperparameter or entropy es…

This monograph presents a class of algorithms called coordinate descent algorithms for mathematicians, statisticians, and engineers outside the field of optimization. This particular class of algorithms has recently gained popularity due to their effectiveness in solving large-scale optimization problems in machine lea…

Efficiently bootstraps massive distributed data without over-resampling.

Study phase transitions in identifying infected individuals using group testing.

We simplify information measure computation using learned features.

Study shows inefficiency of sparse linear regression learning with fewer than Ω(k^2) samples.

New lower bound shows RL with linear approximations is computationally hard.

An important task in computational statistics and machine learning is to approximate a posterior distribution $p(x)$ with an empirical measure supported on a set of representative points $\{x_i\}_{i=1}^n$. This paper focuses on methods where the selection of points is essentially deterministic, with an emphasis on achi…

Neural networks can achieve optimal sample complexity for learning single-index models.

We present a scalable approach to performing approximate fully Bayesian inference in generic state space models. The proposed method is an alternative to particle MCMC that provides fully Bayesian inference of both the dynamic latent states and the static parameters of the model. We build up on recent advances in compu…

Nonparametric two sample testing is a decision theoretic problem that involves identifying differences between two random variables without making parametric assumptions about their underlying distributions. We refer to the most common settings as mean difference alternatives (MDA), for testing differences only in firs…

The TensorFlow Distributions library implements a vision of probability theory adapted to the modern deep-learning paradigm of end-to-end differentiable computation. Building on two basic abstractions, it offers flexible building blocks for probabilistic computation. Distributions provide fast, numerically stable metho…

We study the generalization performance of online learning algorithms trained on samples coming from a dependent source of data. We show that the generalization error of any stable online algorithm concentrates around its regret--an easily computable statistic of the online performance of the algorithm--when the underl…

In certain situations that shall be undoubtedly more and more common in the Big Data era, the datasets available are so massive that computing statistics over the full sample is hardly feasible, if not unfeasible. A natural approach in this context consists in using survey schemes and substituting the "full data" stati…

Develops methods for spectral estimation and rare-event prediction in complex systems.

We provide a general framework for characterizing the trade-off between accuracy and robustness in supervised learning. We propose a method and define quantities to characterize the trade-off between accuracy and robustness for a given architecture, and provide theoretical insight into the trade-off. Specifically we in…

The problem of identifying geometric structure in heterogeneous, high-dimensional data is a cornerstone of representation learning. While there exists a large body of literature on the embeddability of canonical graphs, such as lattices or trees, the heterogeneity of the relational data typically encountered in practic…

A new framework enables real-time task trade-off control.

Complex behaviour in many systems arises from the stochastic interactions of spatially distributed particles or agents. Stochastic reaction-diffusion processes are widely used to model such behaviour in disciplines ranging from biology to the social sciences, yet they are notoriously difficult to simulate and calibrate…

New trade-off found between accuracy and adversarial robustness in regression.

Paper explores robust regression methods and their bias-variance trade-off.

This paper studies trade-offs in private prediction methods.

Efficiently approximates fairness-accuracy trade-offs for diverse datasets.

We consider the weakly supervised binary classification problem where the labels are randomly flipped with probability $1- α$. Although there exist numerous algorithms for this problem, it remains theoretically unexplored how the statistical accuracies and computational efficiency of these algorithms depend on the degr…

New insights into convergence and accuracy trade-offs in federated and meta-learning.

We empirically investigate the best trade-off between sparse and uniformly-weighted multiple kernel learning (MKL) using the elastic-net regularization on real and simulated datasets. We find that the best trade-off parameter depends not only on the sparsity of the true kernel-weight spectrum but also on the linear dep…

Proposes FACT, a diagnostic for understanding group fairness trade-offs.

COPML framework securely trains models across multiple data owners without revealing individual data.

Study shows exponential error reduction in multiclass classification without bias-variance trade-off.

A trade-off between accuracy and fairness is almost taken as a given in the existing literature on fairness in machine learning. Yet, it is not preordained that accuracy should decrease with increased fairness. Novel to this work, we examine fair classification through the lens of mismatched hypothesis testing: trying …

A quantum circuit designed for efficient statistical model preparation and training.

A new method avoids partition function computation for Gibbs density estimation.

Efficient online learning with pairwise loss functions is a crucial component in building large-scale learning system that maximizes the area under the Receiver Operator Characteristic (ROC) curve. In this paper we investigate the generalization performance of online learning algorithms with pairwise loss functions. We…

Vecchia approximations provide the best accuracy-runtime trade-off for Gaussian process approximations.

The paper explores the trade-off between bias and variance in high-dimensional models.

Tensor completion requires fewer samples with weak side information.

New algorithms improve sampling from constrained distributions.