arXiv research

RBMs learn archetypes when trained on blurred copies of them, revealing a critical sample size.

A new sampler tackles critical phenomena by leveraging scale invariance.

Improved convergence for actor-critic algorithms in MDPs.

High-dimensional models become unstable when sample size falls below a critical level, leading to a phase transition.

The classical asymptotic theory for parametric $M$-estimators guarantees that, in the limit of infinite sample size, the excess risk has a chi-square type distribution, even in the misspecified case. We demonstrate how self-concordance of the loss allows to characterize the critical sample size sufficient to guarantee …

How does missing data affect our ability to learn signal structures? It has been shown that learning signal structure in terms of principal components is dependent on the ratio of sample size and dimensionality and that a critical number of observations is needed before learning starts (Biehl and Mietzner, 1993). Here …

Study evaluates synthetic data augmentation for small datasets, highlighting inconsistencies in traditional metrics.

SGD's performance improves with critical batch size, minimizing SFO complexity.

The problem of estimation error in portfolio optimization is discussed, in the limit where the portfolio size N and the sample size T go to infinity such that their ratio is fixed. The estimation error strongly depends on the ratio N/T and diverges for a critical value of this parameter. This divergence is the manifest…

The paper analyzes the expected size of conformal prediction sets.

Study shows DNNs can recover functions with fewer samples than model parameters at overparameterization.

Study non-monotonic loss functions in CRC, achieving valid risk control with large calibration samples.

The paper analyzes SGD in high-dimensional networks, revealing new scaling limits.

We determine the critical batch size for large language models and find it scales with data size, not model size.

The study reveals a transition in neural network performance from infinite-width to variance-limited behavior as dataset size increases.

Increasing the batch size is a popular way to speed up neural network training, but beyond some critical batch size, larger batch sizes yield diminishing returns. In this work, we study how the critical batch size changes based on properties of the optimization algorithm, including acceleration and preconditioning, thr…

A large portfolio of independent returns is optimized under the variance risk measure with a ban on short positions. The no-short selling constraint acts as an asymmetric $\ell_1$ regularizer, setting some of the portfolio weights to zero and keeping the out of sample estimator for the variance bounded, avoiding the di…

Gradient descent with large steps leads to chaotic parameter space and unpredictable outcomes.

The paper investigates model collapse in language models from a probabilistic perspective.

A neural network model predicts the critical point of the Ising phase transition.

We discuss a simple model based on the Minority Game which reproduces the main stylized facts of anomalous fluctuations in finance. We present the analytic solution of the model in the thermodynamic limit and show that stylized facts arise only close to a line of critical points with non-trivial properties. By a simple…

Paper introduces a new sampling method combining Consistency Models with importance sampling.

Enhanced Sampling Scheme improves masked generative modeling.

Web crawling, snowball sampling, and respondent-driven sampling (RDS) are three types of network sampling techniques used to contact individuals in hard-to-reach populations. This paper studies these procedures as a Markov process on the social network that is indexed by a tree. Each node in this tree corresponds to an…

In this paper we study the support recovery problem for single index models $Y=f(\boldsymbol{X}^{\intercal} \boldsymbolβ,\varepsilon)$, where $f$ is an unknown link function, $\boldsymbol{X}\sim N_p(0,\mathbb{I}_{p})$ and $\boldsymbolβ$ is an $s$-sparse unit vector such that $\boldsymbolβ_{i}\in \{\pm\frac{1}{\sqrt{s}}…

Neural causal discovery methods fail to accurately uncover causal structures due to the faithfulness property.

Study SGD dynamics in high-dimensional models, revealing consistent behavior across different batch sizes and learning rates.

Seesaw optimizes training by balancing learning rate and batch size, accelerating model pretraining.

Motivated by safety-critical applications, test-time attacks on classifiers via adversarial examples has recently received a great deal of attention. However, there is a general lack of understanding on why adversarial examples arise; whether they originate due to inherent properties of data or due to lack of training …

FedFaiREE addresses fairness in decentralized learning with small samples.

The ability to accurately predict the fit of fashion items and recommend the correct size is key to reducing merchandise returns in e-commerce. A critical prerequisite of fit prediction is size normalization, the mapping of product sizes across brands to a common space in which sizes can be compared. At present, size n…

The study uses AI to optimize trading in FX markets by considering size-dependent fees and risk-aversion.

Bayesian DDR models complex multivariate distributions.

This work studies the impact of intra-/inter-class diversity on pre-training datasets and finds a balance for optimal performance.

A new method reduces feature screening cost from $O(np)$ to $O(\sqrt{n}p)$.

Improved sample complexity for actor-critic algorithms in MDPs.

This work studies scaling laws for low-precision training in high-dimensional linear regression.

FaiREE provides fair classification with guarantees for small datasets.

New adaptive test for NPIV models controls size and has superior power.

AdAdaGrad optimizes batch sizes for deep learning models, reducing the generalization gap.

New analysis shows actor-critic method converges efficiently in practical settings.

HollowFlow speeds up likelihood evaluation for large-scale models.

New method makes machine learning approximations unbiased and efficient.

Study on neural networks' performance in sequential task learning.

Transfer learning improves causal model estimates in small samples.

Optimistic actor-critic tackles linear MDPs with parametric policies.

A simple model economy with locally interacting producers and consumers is introduced. When driven by extremal dynamics, the model self-organizes {\em not} to an attractor state, but to an asymptote, on which the economy has a constant rate of deflation, is critical, and exhibits avalanches of activity with power-law d…

This work analyzes actor-critic methods for faster convergence.