arXiv research

New uniformity tester ensures consistent results across different samples.

Improved uniform convergence bound with fat-shattering dimension reduces sample complexity gap.

New algorithms sample convex bodies using Markov chains and restricted Gaussian oracles.

No algorithm outperforms uniform sampling in A/B testing.

Study shows gap between uniform convergence and test error in random feature models.

New loss function equivalence reveals PER's uniform sampling can be improved.

Random sampling has become a critical tool in solving massive matrix problems. For linear regression, a small, manageable set of data rows can be randomly selected to approximate a tall, skinny data matrix, improving processing time significantly. For theoretical performance guarantees, each row must be sampled with pr…

We design and mathematically analyze sampling-based algorithms for regularized loss minimization problems that are implementable in popular computational models for large data, in which the access to the data is restricted in some way. Our main result is that if the regularizer's effect does not become negligible as th…

New algorithms achieve uniform-PAC guarantees for RL with bounded eluder dimension.

We study primal-dual type stochastic optimization algorithms with non-uniform sampling. Our main theoretical contribution in this paper is to present a convergence analysis of Stochastic Primal Dual Coordinate (SPDC) Method with arbitrary sampling. Based on this theoretical framework, we propose Optimality Violation-ba…

Improved sampling accuracy in SG-MCMC methods via non-uniform gradient subsampling.

The paper tackles uniform sampling from databases with duplicates.

New sampler improves uniform sampling over convex bodies with fewer queries.

SURF steers scalarization weights to uniformly traverse the Pareto front.

Learning ReLU networks to high uniform accuracy requires exponentially many samples.

New sampling bounds improve uniform coverage verification in machine learning.

Improved matrix completion for non-uniformly sampled data.

Wedge Sampling improves tensor completion with nearly-linear sample complexity.

Energy-efficient sampling for machine learning using magnetic tunnel junctions.

We propose a new setting for testing properties of distributions while receiving samples from several distributions, but few samples per distribution. Given samples from $s$ distributions, $p_1, p_2, \ldots, p_s$, we design testers for the following problems: (1) Uniformity Testing: Testing whether all the $p_i$'s are …

Validates conformal prediction for network data under non-uniform sampling.

Adaptive sampling improves graph diffusion models by maintaining uniform information speed.

We show that the sets in a family with finite VC dimension can be uniformly approximated within a given error by a finite partition. Immediate corollaries include the fact that VC classes have finite bracketing numbers, satisfy uniform laws of averages under strong dependence, and exhibit uniform mixing. Our results ar…

In a series of recent works, we have generalised the consistency results in the stochastic block model literature to the case of uniform and non-uniform hypergraphs. The present paper continues the same line of study, where we focus on partitioning weighted uniform hypergraphs---a problem often encountered in computer …

Improved error estimate for SGLD sampling algorithm.

We study distribution testing with communication and memory constraints in the following computational models: (1) The {\em one-pass streaming model} where the goal is to minimize the sample complexity of the protocol subject to a memory constraint, and (2) A {\em distributed model} where the data samples reside at mul…

Study tests uniformity of categorical data against missing-ball alternatives, finding chi-squared test outperforms.

Transforms uniform learners to work under arbitrary distributions efficiently.

Paper tackles estimating initial conditions of spatio-temporal processes from sparse data.

New tester outperforms existing ones in uniformity testing.

Uniform sampling of training data has been commonly used in traditional stochastic optimization algorithms such as Proximal Stochastic Gradient Descent (prox-SGD) and Proximal Stochastic Dual Coordinate Ascent (prox-SDCA). Although uniform sampling can guarantee that the sampled stochastic quantity is an unbiased estim…

Algorithm learns CNF formulas from random solutions under specific conditions.

Investigates principles of generalization in list learning, refutes sample compression conjecture.

New bounds show multicalibration error is close to prediction error.

Matrix completion has been well studied under the uniform sampling model and the trace-norm regularized methods perform well both theoretically and numerically in such a setting. However, the uniform sampling model is unrealistic for a range of applications and the standard trace-norm relaxation can behave very poorly …

New method uses graphene transistors for efficient non-uniform random number generation.

We apply stochastic average gradient (SAG) algorithms for training conditional random fields (CRFs). We describe a practical implementation that uses structure in the CRF gradient to reduce the memory requirement of this linearly-convergent stochastic gradient method, propose a non-uniform sampling scheme that substant…

New method improves active statistical inference by reducing noise.

Generative Adversarial Networks (GAN) training process, in most cases, apply Uniform or Gaussian sampling methods in the latent space, which probably spends most of the computation on examples that can be properly handled and easy to generate. Theoretically, importance sampling speeds up stochastic optimization in supe…

GADD accelerates uniform-rate discrete diffusion models by 2 orders of magnitude.

Residual neural networks don't help overcome sampling complexity issues.

The paper analyzes convergence of neural SDEs as sample size increases.

Sub-sampling is a common and often effective method to deal with the computational challenges of large datasets. However, for most statistical models, there is no well-motivated approach for drawing a non-uniform subsample. We show that the concept of an asymptotically linear estimator and the associated influence func…

New algorithm FLUTE achieves uniform-PAC convergence in RL with linear approx.

This paper introduces time-uniform CLT-based confidence intervals for statistical inference.

Driven by the need for parallelizable hyperparameter optimization methods, this paper studies \emph{open loop} search methods: sequences that are predetermined and can be generated before a single configuration is evaluated. Examples include grid search, uniform random search, low discrepancy sequences, and other sampl…

The method to derive uniform bounds with Gaussian and Rademacher complexities is extended to the case where the sample average is replaced by a nonlinear statistic. Tight bounds are obtained for U-statistics, smoothened L-statistics and error functionals of l2-regularized algorithms.

Uniform consistency proven for spatial distribution and depth estimators in any dimension.