This paper identifies drift Lipschitz budget K as key to diffusion policy expressivity and statistical trade-offs.
problem Understanding and maximizing the expressivity of diffusion policies while managing statistical limitations.
method Identifying drift Lipschitz budget K as central, quantifying expressivity and statistical behavior, proving lower bounds, and providing practical implementation guidelines.
result Balancing expressivity and statistical complexity yields a finite-sample performance gap, with rates depending on sample size and drift type.
We analyze the impact of the sampling interval on the estimation of Kramers-Moyal coefficients. We obtain the finite-time expressions of these coefficients for several standard processes. We also analyze extreme situations such as the independence and no-fluctuation limits that constitute useful references. Our results…
The study tightens bounds on binomial probabilities and minimums using KL-divergence.
problem Tightening bounds on binomial probabilities and minimums of i.i.d. Binomials.
method Applied Sanov's theorem to derive upper and lower bounds on binomial tail probabilities and minimums, expressed in terms of KL-divergence.
result High probability upper and lower bounds on the minimum of i.i.d. Binomial random variables, finite sample, asymptotically tight.
Estimates causal effects in Gaussian Linear SCMs with finite data.
problem Estimating causal effects from observational data with latent confounders.
method Centralized Gaussian Linear SCMs (CGL-SCMs) and EM-based estimation algorithm.
result Learned CGL-SCM parameters accurately recover causal distributions from finite observational samples.
In Bayesian inference, the posterior distributions are difficult to obtain analytically for complex models such as neural networks. Variational inference usually uses a parametric distribution for approximation, from which we can easily draw samples. Recently discrete approximation by particles has attracted attention …
Paper develops a method to identify minimal sample subspaces from limited data.
problem Challenging task of subspace segmentation with minimal sample subspaces.
method Develops a theoretical framework and optimization algorithms for MSS.
result The MSS model can retrieve minimal sample subspaces even when they are heavily intersected.
New algorithm reduces sample complexity for planning in MDPs.
problem Planning in MDPs with unknown transitions.
method MDP-GapE, a trajectory-based MCTS algorithm.
result Proves upper bound on sample complexity in terms of sub-optimality gaps.
The study sets lower bounds on MMSE for inferring sensitive features from noisy data.
problem Estimating sensitive features from noisy observations of correlated features.
method Adversarial evaluation framework based on MMSE estimation with theoretical lower bounds.
result Derives closed-form bounds for linear models, showing optimality in noise variance.
3-layer ReLU networks with Ω(√N) nodes can memorize most datasets.
problem Understanding memorization capacity of ReLU networks.
method Analyzing depth and width requirements for memorization.
result Width Θ(√N) is necessary and sufficient for memorizing N data points.
This work closes the gap between theory and practice for nICA identifiability.
problem Identifying latent components in nonlinearly mixed data.
method Finite-sample analysis of GCL-based nICA, combining GCL properties, statistical generalization, and numerical differentiation.
result Establishes a trade-off between function learner complexity and expressiveness.
Unified approach to sampling and inference for generative models with latent diffusions.
problem Efficient sampling and inference in generative models with latent diffusions.
method Unified stochastic control viewpoint, multilayer feedforward neural nets for drift, unbiased simulation scheme.
result Efficient sampling from a wide class of terminal target distributions with minimal KL divergence.
An emerging design principle in deep learning is that each layer of a deep artificial neural network should be able to easily express the identity transformation. This idea not only motivated various normalization techniques, such as \emph{batch normalization}, but was also key to the immense success of \emph{residual …
We formalize AURC and develop estimators for SC systems.
problem Evaluation of SC systems' performance.
method Formal statistical formulation, Monte Carlo methods, plug-in estimators.
result Plug-in estimators are consistent, with low bias and bounded MSE.
Optimizes sample reweighting to match laws under covariate shift using Wasserstein distance.
problem Matching laws of samples with different distributions under covariate shift.
method Minimizes Wasserstein distance between empirical measures of samples using Nearest Neighbors weights.
result Consistent reweighting leads to asymptotic convergence of empirical measures.
Study on HyperGNNs' expressiveness and structural generalization.
problem Understanding HyperGNNs' ability to solve graph problems and generalize to larger graphs.
method Fine-grained analysis of expressiveness and learning properties, with theoretical and empirical support.
result HyperGNNs can solve a hierarchy of graph problems defined by hyperparameters.
Dual neural network architecture improves accuracy and interpretability.
problem Improving neural network interpretability and accuracy.
method Stacked recurrent and feedforward layers, binary activation function.
result Binary activation leads to simpler, more interpretable models with higher accuracy.
Learning rates for least-squares regression are typically expressed in terms of L2-norms. In this paper we extend these rates to norms stronger than the L2-norm without requiring the regression function to be contained in the hypothesis space. In the special case of Sobolev reproducing kernel Hilbert spaces used …
Implicit probabilistic models are models defined naturally in terms of a sampling procedure and often induces a likelihood function that cannot be expressed explicitly. We develop a simple method for estimating parameters in implicit models that does not require knowledge of the form of the likelihood function or any d…
Bayesian models overestimate clusters, but practical summaries can correct this.
problem Bayesian mixture models overestimate the number of clusters.
method Simulations and gene expression data analysis using MCMC summarisation.
result Overestimation is limited in finite samples and can be corrected, but misspecification leads to significant overestimation.
A new permutation method improves two-sample testing power.
problem Two-sample testing with improved power and validity.
method Structured block-restricted cross-swaps.
result Block-restricted permutations achieve higher power than full permutations.
Proposes VCNet for estimating ADRFs of continuous treatments.
problem Estimating ADRFs of continuous treatments from observational data.
method VCNet for improved model expressiveness and continuity; targeted regularization for finite sample performance.
result Improves model expressiveness and continuity of ADRFs.
We present a general method to detect and extract from a finite time sample statistically meaningful correlations between input and output variables of large dimensionality. Our central result is derived from the theory of free random matrices, and gives an explicit expression for the interval where singular values are…
New embedding method in function spaces improves expressiveness.
problem Enhancing expressiveness in knowledge graph embeddings.
method Employing polynomial functions and neural networks with varying layer complexities.
result Improved expressiveness and more degrees of freedom in entity representation.
Study clarifies variance of stratification estimators for causal effects.
problem Estimating average causal effects with discrete covariates.
method Combines insights from potential outcomes, causal diagrams, and structural models.
result Derives expressions for the variance of stratification estimators.
We present an interactive version of an evidence-driven state-merging (EDSM) algorithm for learning variants of finite state automata. Learning these automata often amounts to recovering or reverse engineering the model generating the data despite noisy, incomplete, or imperfectly sampled data sources rather than optim…
We show that there is no algorithm deciding whether the maximal residually free quotient of a given finitely presented group is finitely presentable or not. Given a finitely generated subgroup G of a finite product of limit groups, we discuss the possibility of finding an explicit set of defining equations (i.e. of exp…
Identifying latent structure in large data matrices is essential for exploring biological processes. Here, we consider recovering gene co-expression networks from gene expression data, where each network encodes relationships between genes that are locally co-regulated by shared biological mechanisms. To do this, we de…
PPI++ outperforms gold-standard labels only if pseudo-labels are highly correlated.
problem Optimizing statistical estimation using noisy pseudo-labels.
method Exact finite-sample analysis of PPI++ on mean estimation problem.
result PPI++ has provably worse estimation error than gold-standard labels alone in some settings.
Due to their heterogeneity, insurance risks can be properly described as a mixture of different fixed models, where the weights assigned to each model may be estimated empirically from a sample of available data. If a risk measure is evaluated on the estimated mixture instead of the (unknown) true one, then it is impor…
Bayesian approach improves portfolio optimization using VaR and CVaR.
problem Optimizing portfolio weights using VaR and CVaR for risk management.
method Bayesian perspective, posterior predictive distribution, observed data.
result Bayesian approach yields more accurate optimal portfolio weights.
Stochastic gradient descent procedures have gained popularity for parameter estimation from large data sets. However, their statistical properties are not well understood, in theory. And in practice, avoiding numerical instability requires careful tuning of key parameters. Here, we introduce implicit stochastic gradien…
In this paper, we study the schrodinger equation and wave equation with the Dirichlet boundary condition on a connected finite graph. The explicit expressions for solutions are given and the energy conservations are derived. Applications to the corresponding nonlinear problems are indicated.
CoT improves transformer sample efficiency by reducing input token dependencies and attention sparsity.
problem Transformer sample inefficiency in simple tasks.
method Demonstrated through parity-learning setup, showing CoT reduces required samples from exponential to polynomial.
result Transformer learns function within polynomial samples with CoT, requiring exponential samples without CoT.
Kernel-based tests for shape constraints in finance.
problem Enforcing shape relations on latent functions in financial econometrics.
method Kernel-based nonparametric framework for mean-variance optimization.
result Established statistical properties and a joint Wald-type statistic for testing shape constraints.
Motivation: Algorithms that discover variables which are causally related to a target may inform the design of experiments. With observational gene expression data, many methods discover causal variables by measuring each variable's degree of statistical dependence with the target using dependence measures (DMs). Howev…
Develops a new model for synthesizing and analyzing probability measures.
problem Synthesis and analysis of probability measures.
method Linear barycentric coding model (LBCM) using linear optimal transport (LOT) metric.
result Closed-form solution to 2-Wasserstein barycenters for compatible measures.
Bias and flexibility trade off in learning algorithms.
problem Understanding the trade-off between bias and expressivity in learning algorithms.
method Measuring expressivity using entropy on algorithm outcome distributions, and deriving bounds on bias and expressivity.
result There is a necessary trade-off between bias and expressivity in learning algorithms.
This study examines a single attention layer's capabilities using random features.
problem Understanding the learning and generalization of a single multi-head attention layer.
method Random feature setting with large number of heads, frozen query and key matrices, and trainable value matrices.
result Random-feature attention layer can express a broad class of permutation-invariant target functions.
Flow Matching models help generative models stay within the subspace of real data.
problem How do generative models stay within the subspace of real data?
method Flow Matching models using a learned velocity field to transform a simple prior into a complex target distribution.
result Generated samples memorize real data points and represent the sample data subspace exactly.
Next-generation sequencing technologies provide a revolutionary tool for generating gene expression data. Starting with a fixed RNA sample, they construct a library of millions of differentially abundant short sequence tags or "reads", which constitute a fundamentally discrete measure of the level of gene expression. A…
This work introduces efficient sampling methods for Gaussian processes by focusing on pathwise conditioning.
problem Intractable mathematical expressions in Gaussian process posteriors limit practical applications.
method Investigates a pathwise interpretation of conditioning to derive efficient sampling methods.
result Derives a general family of approximations that allow for efficient sampling of Gaussian process posteriors.
IDS algorithm optimizes sequential decisions in various monitoring settings.
problem Optimizing sequential decisions in complex monitoring scenarios.
method Information-directed sampling (IDS) algorithm for linear partial monitoring.
result IDS achieves nearly worst-case rate optimality in finite-action games.
The most important aspect of any classifier is its error rate, because this quantifies its predictive capacity. Thus, the accuracy of error estimation is critical. Error estimation is problematic in small-sample classifier design because the error must be estimated using the same data from which the classifier has been…
New framework analyzes SGD dynamics in large samples and dimensions.
problem Analyzing stochastic gradient descent in large-scale settings.
method Inspired by random matrix theory, new framework for fixed stepsize and finite sum settings.
result SGD dynamics become deterministic in the large sample and dimensional limit, governed by a Volterra integral equation.
Algorithm learns stochastic system dynamics from data.
problem Recovering interpretable symbolic expressions for stochastic systems.
method Data-driven, trajectory averaging, drift-informed correction.
result Recover coefficients and densities to within 5% and 0.01 in total variation, respectively.
We study the geometry of jets of submanifolds with special interest in the relationship with the calculus of variations. We give a new proof of the fact that higher order jets of submanifolds are affine bundles; as a by-product we obtain a new expression for the associated vector bundles. We use Green-Vinogradov formul…
This work explores test-time scaling strategies for LLMs, improving sample efficiency and expressiveness.
problem Understanding the sample efficiency and expressiveness of test-time scaling strategies for LLMs.
method Established separation and expressiveness results for self-consistency, best-of-n, and self-correction strategies. result Self-correction enables Transformers to simulate online learning over multiple tasks without prior knowledge.
New graph-based discriminators improve sample complexity and expressiveness in learning theory.
problem Identifying if two distributions are identical with limited samples.
method Introducing k-ary based discriminators, which use families of Boolean k-ary functions to distinguish between distributions.
result Having k-ary functions (k ≥ 2) improves distinguishability and sample complexity compared to classical hypothesis classes.