arXiv research

We study the sample complexity of learning one-hidden-layer convolutional neural networks (CNNs) with non-overlapping filters. We propose a novel algorithm called approximate gradient descent for training CNNs, and show that, with high probability, the proposed algorithm with random initialization grants a linear conve…

Study private query release with public data, reducing sample sizes.

We establish a tight characterization of the worst-case rates for the excess risk of agnostic learning with sample compression schemes and for uniform convergence for agnostic sample compression schemes. In particular, we find that the optimal rates of convergence for size-$k$ agnostic sample compression schemes are of…

New method for learning evolving tasks with performance guarantees.

Improved statistical efficiency of Thompson Sampling for combinatorial semi-bandits.

This paper tightens information-theoretic bounds on generalization errors.

We develop coresets for multiple ℓ_p regression problems, improving approximation sizes and efficiency.

Paper introduces VDE, a variance-reduced determinant estimator.

Study clusters distributions with known or unknown clusters using distribution testing.

There has been renewed recent interest in developing effective lower bounds for Dynamic Time Warping (DTW) distance between time series. These have many applications in time series indexing, clustering, forecasting, regression and classification. One of the key time series classification algorithms, the nearest neighbo…

We propose and analyze StoROO, an algorithm for risk optimization on stochastic black-box functions derived from StoOO. Motivated by risk-averse decision making fields like agriculture, medicine, biology or finance, we do not focus on the mean payoff but on generic functionals of the return distribution. We provide a g…

New findings on boosting sample complexity and implications for hardcore theorem.

We design and mathematically analyze sampling-based algorithms for regularized loss minimization problems that are implementable in popular computational models for large data, in which the access to the data is restricted in some way. Our main result is that if the regularizer's effect does not become negligible as th…

In this paper we propose a fast online Kernel SVM algorithm under tight budget constraints. We propose to split the input space using LVQ and train a Kernel SVM in each cluster. To allow for online training, we propose to limit the size of the support vector set of each cluster using different strategies. We show in th…

The Mallows model, introduced in the seminal paper of Mallows 1957, is one of the most fundamental ranking distribution over the symmetric group $S_m$. To analyze more complex ranking data, several studies considered the Generalized Mallows model defined by Fligner and Verducci 1986. Despite the significant research in…

We consider here 6-regular plane graphs whose faces have size 1, 2 or 3. In Section 2 a practical enumeration method is given that allowed us to enumerate them up to 53 vertices. Subsequently, in Section 3 we enumerate all possible symmetry groups of the spheres that showed up. In Section 4 we introduce a new Goldberg-…

We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L_2 regularization: We introduce the γ-adapted-dimension, which is a simple function of the spectrum of a distribution's covariance matrix, and show distribution-specific upper and lower bounds on the s…

Unified complexity bound for sampling logconcave distributions

Optimizes quadratic bandits with tight Hessian-dependent sample complexity bounds.

We give asymptotically tight estimates of tangent space variation on Riemannian submanifolds of Euclidean space with respect to the local feature size of the submanifolds. We show that the result follows directly from structural properties of local feature size of the Riemannian submanifold and some elementary Euclidea…

MRCs minimize worst-case expected 0-1 loss and provide performance guarantees.

Paper introduces MRCs that minimize worst-case 0-1 loss, providing tight performance guarantees.

Recent theoretical work has guaranteed that overparameterized networks trained by gradient descent achieve arbitrarily low training error, and sometimes even low test error. The required width, however, is always polynomial in at least one of the sample size $n$, the (inverse) target error $1/ε$, and the (inverse) fail…

We study the problem of low-rank tensor factorization in the presence of missing data. We ask the following question: how many sampled entries do we need, to efficiently and exactly reconstruct a tensor with a low-rank orthogonal decomposition? We propose a novel alternating minimization based method which iteratively …

We study the problem of identifying correlations in multivariate data, under information constraints: Either on the amount of memory that can be used by the algorithm, or the amount of communication when the data is distributed across several machines. We prove a tight trade-off between the memory/communication complex…

New method for spatiotemporal data regression using Gaussian processes.

New algorithm reduces sample size for robust reinforcement learning.

Paper proposes a cost-sensitive conformal training method with provably controllable learning bounds.

The study tightens bounds on binomial probabilities and minimums using KL-divergence.

The paper provides rigorous guarantees for m-out-of-n bootstrap estimators of sample quantiles.

An $\varepsilon$-coreset for Least-Mean-Squares (LMS) of a matrix $A\in{\mathbb{R}}^{n\times d}$ is a small weighted subset of its rows that approximates the sum of squared distances from its rows to every affine $k$-dimensional subspace of ${\mathbb{R}}^d$, up to a factor of $1\pm\varepsilon$. Such coresets are useful…

We study the problem of high-dimensional linear regression in a robust model where an $ε$-fraction of the samples can be adversarially corrupted. We focus on the fundamental setting where the covariates of the uncorrupted samples are drawn from a Gaussian distribution $\mathcal{N}(0, Σ)$ on $\mathbb{R}^d$. We give near…

Study improves MMD estimation for two distributions with mismeasured data.

FAQ efficiently evaluates LLMs with statistical guarantees using adaptive query selection.

Measuring Mutual Information (MI) between high-dimensional, continuous, random variables from observed samples has wide theoretical and practical applications. Recent work, MINE (Belghazi et al. 2018), focused on estimating tight variational lower bounds of MI using neural networks, but assumed unlimited supply of samp…

Meta learning of optimal classifier error rates allows an experimenter to empirically estimate the intrinsic ability of any estimator to discriminate between two populations, circumventing the difficult problem of estimating the optimal Bayes classifier. To this end we propose a weighted nearest neighbor (WNN) graph es…

Unified framework for SGMoE resolves estimation and selection issues.

GOTabPFN improves tabular model performance with compact tokenization for HDLSS data.

Cer-Eval saves LLM evaluation costs while maintaining accuracy.

In the past decade, sparse principal component analysis has emerged as an archetypal problem for illustrating statistical-computational tradeoffs. This trend has largely been driven by a line of research aiming to characterize the average-case complexity of sparse PCA through reductions from the planted clique (PC) con…

A collection $Δ$ of simple closed curves on an orientable surface is an algebraic $k$-system if the algebraic intersection number $\langle α,β\rangle$ is equal to $k$ in absolute value for every $α, β\in Δ$ distinct. Generalizing a theorem of [MRT14] we compute that the maximum size of an algebraic $k$-system of c…

We introduce a new and improved characterization of the label complexity of disagreement-based active learning, in which the leading quantity is the version space compression set size. This quantity is defined as the size of the smallest subset of the training data that induces the same version space. We show various a…

Develops bounds for deep learning risk via Hilbert coresets.

We propose a minimax concave penalized multi-armed bandit algorithm under generalized linear model (G-MCP-Bandit) for a decision-maker facing high-dimensional data in an online learning and decision-making process. We demonstrate that the G-MCP-Bandit algorithm asymptotically achieves the optimal cumulative regret in t…

New bound matches exact generalization error for quadratic Gaussian problem.

This study tightens bounds on how GD and SGD generalize in smooth convex optimization problems.

A new method for covariate shift adaptation using nearest neighbors.

New method improves performance of Hamiltonian MCMC for log Z estimation.