arXiv research

Study ridge ensembles in proportional feature-to-sample size regime, proving risk equivalence and GCV consistency.

A new algorithm for robust causal discovery in small sample sizes.

Two statistical tasks are shown to have equivalent sample complexity.

Differentiable learning via SGD and GD can simulate various learning problems, depending on precision and minibatch size.

BDeu marginal likelihood score is a popular model selection criterion for selecting a Bayesian network structure based on sample data. This non-informative scoring criterion assigns same score for network structures that encode same independence statements. However, before applying the BDeu score, one must determine a …

We study the sample complexity of private synthetic data generation over an unbounded sized class of statistical queries, and show that any class that is privately proper PAC learnable admits a private synthetic data generator (perhaps non-efficient). Previous work on synthetic data generators focused on the case that …

Seesaw optimizes training by balancing learning rate and batch size, accelerating model pretraining.

SLdisco uses supervised learning to discover causal models from observational data.

Determinantal Point Processes (DPPs) are popular models for point processes with repulsion. They appear in numerous contexts, from physics to graph theory, and display appealing theoretical properties. On the more practical side of things, since DPPs tend to select sets of points that are some distance apart (repulsion…

Privacy affects how much data is needed for CVaR optimization.

The sizes of Markov equivalence classes of directed acyclic graphs play important roles in measuring the uncertainty and complexity in causal learning. A Markov equivalence class can be represented by an essential graph and its undirected subgraphs determine the size of the class. In this paper, we develop a method to …

Recent reports have described that the equivalent sample size (ESS) in a Dirichlet prior plays an important role in learning Bayesian networks. This paper provides an asymptotic analysis of the marginal likelihood score for a Bayesian network. Results show that the ratio of the ESS and sample size determine the penalty…

Directed acyclic graph (DAG) models are popular for capturing causal relationships. From observational and interventional data, a DAG model can only be determined up to its \emph{interventional Markov equivalence class} (I-MEC). We investigate the size of MECs for random DAG models generated by uniformly sampling and o…

Recent reports have described that learning Bayesian networks are highly sensitive to the chosen equivalent sample size (ESS) in the Bayesian Dirichlet equivalence uniform (BDeu). This sensitivity often engenders some unstable or undesirable results. This paper describes some asymptotic analyses of BDeu to explain the …

Study evaluates how much knowledge LLMs have by comparing their prediction accuracy to flexible models.

Optimal portfolios are formed by combining momentum, size, and volatility characteristics, enhancing utility for all investors.

RBMs learn archetypes when trained on blurred copies of them, revealing a critical sample size.

Develops privacy-preserving methods for equivalence testing in healthcare.

Reduces multiclass and regression compression schemes to binary ones.

We consider the problem of approximating the set of eigenvalues of the covariance matrix of a multivariate distribution (equivalently, the problem of approximating the "population spectrum"), given access to samples drawn from the distribution. The eigenvalues of the covariance of a distribution contain basic informati…

We shrink confidence sets for equivalent discrete distributions using permutation equivalence.

The paper shows that relaxing assumptions about causal graphs can lead to exponentially large equivalence classes.

GaussDetect-LiNGAM eliminates Gaussianity tests for causal discovery.

New discriminant analysis using GDS projection improves face recognition.

Probabilistic clustering models (or equivalently, mixture models) are basic building blocks in countless statistical models and involve latent random variables over discrete spaces. For these models, posterior inference methods can be inaccurate and/or very slow. In this work we introduce deep network architectures tra…

Study risk-sensitive reinforcement learning with optimized certainty equivalents.

Paper proposes a method to use in silico experiments with foundation models to reduce sample size.

Optimized nonlinearities enhance generalization in random feature models.

High-dimensional data analysis has motivated a spectrum of regularization methods for variable selection and sparse modeling, with two popular classes of convex ones and concave ones. A long debate has been on whether one class dominates the other, an important question both in theory and to practitioners. In this pape…

This article establishes the performance of stochastic blockmodels in addressing the co-clustering problem of partitioning a binary array into subsets, assuming only that the data are generated by a nonparametric process satisfying the condition of separate exchangeability. We provide oracle inequalities with rate of c…

Unified scaling laws reveal how model size and training time impact neural network performance.

We call an Ising model tractable when it is possible to compute its partition function value (statistical inference) in polynomial time. The tractability also implies an ability to sample configurations of this model in polynomial time. The notion of tractability extends the basic case of planar zero-field Ising models…

RAPID efficiently samples SVDD subsets for better anomaly detection.

Develops exact convex optimization formulations for neural networks.

In this paper we study a family of variance reduction methods with randomized batch size---at each step, the algorithm first randomly chooses the batch size and then selects a batch of samples to conduct a variance-reduced stochastic update. We give the linear convergence rate for this framework for composite functions…

Algorithm learns CNF formulas from random solutions under specific conditions.

Optimal ridge regularization computed iteratively from generative parameters.

The article examines in some detail the convergence rate and mean-square-error performance of momentum stochastic gradient methods in the constant step-size and slow adaptation regime. The results establish that momentum methods are equivalent to the standard stochastic gradient method with a re-scaled (larger) step-si…

This paper studies a class of exponential family models whose canonical parameters are specified as linear functionals of an unknown infinite-dimensional slope function. The optimal minimax rates of convergence for slope function estimation are established. The estimators that achieve the optimal rates are constructed …

Estimates network structure from Gaussian Graphical Models and Gaussian Free Fields.

We study Exo-MDPs to reduce sample complexity in reinforcement learning.

We develop efficient algorithms to train $\ell_1$-regularized linear classifiers with large dimensionality $d$ of the feature space, number of classes $k$, and sample size $n$. Our focus is on a special class of losses that includes, in particular, the multiclass hinge and logistic losses. Our approach combines several…

We study a new aggregation operator for gradients coming from a mini-batch for stochastic gradient (SG) methods that allows a significant speed-up in the case of sparse optimization problems. We call this method AdaBatch and it only requires a few lines of code change compared to regular mini-batch SGD algorithms. We p…

A fast algorithm for counting Markov equivalent DAGs and designing experiments.

This paper addresses the complexity of labeled datasets using topological methods.

The paper improves theoretical bounds on deep neural networks' convergence.

Improved convergence for actor-critic algorithms in MDPs.

Study shows how feature weighting affects neural network regularization.