Generative models use latent abstractions to create images.
problem Understanding how generative models create high-dimensional data like images.
method Developed a theoretical framework using SDE and information theory.
result Diffusion models can be seen as a non-linear filter driven by latent abstractions.
We analyze the probability density function (PDF) of waiting times between financial loss exceedances. The empirical PDFs are fitted with the self-excited Hawkes conditional Poisson process with a long power law memory kernel. The Hawkes process is the simplest extension of the Poisson process that takes into account h…
The paper tackles learning to control systems with unknown parameters using Brownian noise.
problem Learning to control systems with unknown parameters.
method Proposes algorithms based on moving empirical averages and integrates statistical methods with stochastic control theory.
result Achieves a logarithmic expected regret rate.
The paper offers efficient algorithms for combinatorial and linear bandits using empirical process theory.
problem Optimal algorithms for combinatorial and linear bandits with practical sample complexity.
method Empirical process theory, Gaussian-width, minimizing experimental design objective.
result Sample complexity matches lower bounds, especially for combinatorial classes.
Paper connects neural networks to Gaussian processes for understanding double-descent.
problem Understanding the double-descent phenomenon in neural networks.
method Uses techniques from random matrix theory and Gaussian processes.
result Establishes a connection between NNGP and random matrix theory for neural networks.
Unified theory explains market impact using a simplified supply-demand parameter.
problem Understanding the market impact of metaorders and excess volatility.
method Coarse-grained approach with a single parameter ρ to model supply-demand equilibrium and market impact.
result Establishes a connection between excess volatility and order-driven markets through the square-root law.
Tree structures are ubiquitous in data across many domains, and many datasets are naturally modelled by unobserved tree structures. In this paper, first we review the theory of random fragmentation processes [Bertoin, 2006], and a number of existing methods for modelling trees, including the popular nested Chinese rest…
Study uniform learnability of binary classification networks with communication.
problem Learning a network with communication between vertices from uniform ergodic Random Graph Process.
method Introduced structural Rademacher complexity and used martingale method and Marton's coupling.
result Uniform learnability as worst-case theoretical limits for binary classification problems.
The paper studies convergence of kernel autocovariance operators for stationary processes.
problem Estimating autocovariance operators of stationary processes on Polish spaces.
method Investigates convergence of empirical estimates of autocovariance operators under various conditions.
result Provides consistency results for kernel PCA and spectral analysis methods.
New empirical process bounds reveal trade-off between dependence and complexity in nonparametric learning.
problem Understanding generalization in nonparametric learning with temporal dependencies.
method Developed bounds on expected supremum of empirical processes under β/ρ-mixing assumptions. result Achieved rates similar to i.i.d. setting under long-range dependence with complex function classes.
Neural networks estimate statistical divergences with performance guarantees.
problem Estimating statistical divergences with theoretical performance guarantees.
method Parametrizing empirical variational form by a neural network and optimizing over parameter space.
result Established non-asymptotic absolute error bounds for neural estimators of four f-divergences. Time-subordinated Brownian motion models improve financial market stochastic distribution.
problem Improving stochastic distribution modeling in financial markets.
method Fourier theory and methodology for time-subordinated Brownian motion models, extending real domain to complex plane.
result Characterization and direct study of stochastic time-change from full process.
The data processing inequality doesn't always hold in practice, showing benefits in low-level tasks.
problem The data processing inequality suggests no benefit in pre-processing for classification.
method Theoretical and empirical study of binary classification setup with deep neural networks.
result Pre-classification processing can improve classification accuracy for any finite number of training samples.
The paper improves Monte Carlo methods for optimization problems.
problem Efficiently solving optimization problems with biased Monte Carlo estimators.
method Introduces Multilevel Monte Carlo (MLMC) within Sample Average Approximation (SAA).
result Establishes uniform convergence and sample complexity for MLMC in SAA.
Kernel-based methods improve policy evaluation in MRP models.
problem Estimating value functions in infinite-horizon discounted MRP models.
method Kernel-based temporal difference methods using reproducing kernel Hilbert spaces.
result Optimal error bounds derived for the kernel-based LSTD estimate.
New online GP algorithm offers performance guarantees for streaming data.
problem Training and inference of GPs require all historic data, limiting online decision-making.
method Developed a new theoretical framework based on PAC-Bayes theory, optimizing empirical risk and parameter divergence.
result Offers both a guarantee of generalized performance and good accuracy.
This paper provides performance guarantees for neural estimation of statistical distances.
problem Developing performance guarantees for neural estimation of statistical distances.
method Non-asymptotic error bounds using function approximation theorems and empirical process theory.
result Established a fundamental tradeoff between approximation and estimation errors in neural estimation of statistical distances.
New theory explains consistency of kernel methods with non-i.i.d. data.
problem Consistency of kernel methods under non-i.i.d. data.
method Empirical weak convergence (EWC) as a general assumption.
result Established consistency of SVMs, kernel mean embeddings, and CKMEs with EWC data.
RL approach for continuous-time mean-variance portfolio selection with empirical validation.
problem Continuous-time mean-variance portfolio selection in unknown market coefficients.
method Reinforcement learning for diffusion processes, sublinear regret bound derivation.
result RL strategy consistently outperforms model-based counterparts, especially in volatile markets.
Study improves ERM for heavy-tailed data with dependent inputs.
problem Empirical Risk Minimization with dependent and heavy-tailed data.
method Extending risk bounds for ERM with heavy-tailed, dependent data.
result Established risk bounds for ERM with dependent and heavy-tailed data.
The developments of Rademacher complexity and PAC-Bayesian theory have been largely independent. One exception is the PAC-Bayes theorem of Kakade, Sridharan, and Tewari (2008), which is established via Rademacher complexity theory by viewing Gibbs classifiers as linear operators. The goal of this paper is to extend thi…
Investigates financial and economic systems using statistical mechanics and information theory.
problem Complexity, asymmetry, stochasticity, and non-linearity in financial and economic systems.
method Model-based and empirical analyses using statistical mechanics and information theory.
result Derives probability distribution functions for better understanding of financial and economic dynamics.
Modeling joint log-volatility dynamics with multivariate fractional Ornstein-Uhlenbeck process.
problem Empirical evidence of joint behavior in realized volatility time series.
method Multivariate fractional Ornstein-Uhlenbeck process with different Hurst exponents and non-trivial interdependencies.
result Model accurately captures asymmetries and spillover effects in realized-volatility time series.
Machine can learn its own bias from related tasks.
problem Machine learning bias through hand-crafted features.
method Introduces two models: PAC-based and hierarchical Bayes.
result Machine can learn bias from multiple tasks.
We propose an empirical Bayes estimator based on Dirichlet process mixture model for estimating the sparse normalized mean difference, which could be directly applied to the high dimensional linear classification. In theory, we build a bridge to connect the estimation error of the mean difference and the misclassificat…
Matrix H-theory models stock market fluctuations using hierarchical multivariate distributions.
problem Understanding collective behavior in stock market fluctuations.
method Matrix H-theory framework for multivariate stochastic processes with hierarchical structure.
result Matrix H-theory effectively describes stock market fluctuations using Meijer G-functions.
RLHF performs well despite violating social choice theory axioms.
problem RLHF's empirical success contradicts social choice theory axioms.
method Showed RLHF satisfies pairwise majority and Condorcet consistency under mild assumptions, and introduced new alignment criteria.
result RLHF satisfies pairwise majority and Condorcet consistency under mild assumptions, explaining its practical success.
This paper investigates the supervised learning problem with observations drawn from certain general stationary stochastic processes. Here by \emph{general}, we mean that many stationary stochastic processes can be included. We show that when the stochastic processes satisfy a generalized Bernstein-type inequality, a u…
Unified asymptotic theory and tests for ACD models reveal infinite-mean durations in cryptocurrency trading.
problem Challenges in asymptotic theory for ACD models, especially for integrated ACD.
method Unified asymptotic theory for quasi-maximum likelihood estimator, hypothesis testing framework.
result Infinite-mean durations in cryptocurrency trading, rejected integrated ACD hypothesis.
Majorizing measures control sequential complexities for online learning.
problem Extending classical empirical processes theory to sequential cases.
method Generic chaining, majorizing measures, fractional covering numbers.
result Sharp control of worst-case sequential Rademacher complexity.
New empirical PAC-Bayes bound for Markov chains with finite state space.
problem Lack of empirical bounds for Markov chains with temporal dependence.
method Proved a new PAC-Bayes bound for Markov chains, providing an empirical pseudo-spectral gap.
result First fully empirical PAC-Bayes bound for Markov chains with finite state space.
New bounds show empirical EOT adapts to simpler measure.
problem Statistical performance of empirical EOT estimators.
method Novel statistical bounds, empirical process theory, dual formulation.
result Empirical EOT and its unregularized version follow lower complexity adaptation.
New framework improves learning across multiple distributions.
problem Modeling uncertainty in sensitive machine learning applications.
method Inspired by multi-armed bandits, provides distribution-dependent guarantees.
result Enhanced dependence on suboptimality gaps and sample size.
New method improves long-term forecasting of stochastic dynamical systems.
problem Improving long-term forecasting accuracy for stochastic dynamical systems.
method Combining Koopman and transfer operator theory with feature centering.
result Learning bounds ensure uniform performance on future distributions.
We introduce closed-form transition density expansions for multivariate affine jump-diffusion processes. The expansions rely on a general approximation theory which we develop in weighted Hilbert spaces for random variables which possess all polynomial moments. We establish parametric conditions which guarantee existen…
The study provides theoretical guarantees for the statistical performance of optimal decision trees.
problem Theoretical limits on the statistical performance of globally optimal decision trees.
method Sharp oracle inequalities and uniform concentration framework based on Rademacher complexity.
result Derivation of minimax optimal rates for piecewise sparse heterogeneous anisotropic Besov space.
Consider a sample of n points taken i.i.d from a submanifold Σ of Euclidean space. We show that there is a way to estimate the Ricci curvature of Σ with respect to the induced metric from the sample. Our method is grounded in the notions of Carré du Champ for diffusion semi-groups, the theory of Empirical process…
Study improves L∞ estimates and extreme value behavior in stochastic differential games.
problem Analyzing the mean-field limit of diffusive games through master equation.
method Using the Master Equation to approximate state processes and establishing L∞ estimates for the total error. result Established No∞ asymptotic behavior of upper order statistics of Nash states, initiating Extreme Value Theory for stochastic differential games. We show two novel concentration inequalities for suprema of empirical processes when sampling without replacement, which both take the variance of the functions into account. While these inequalities may potentially have broad applications in learning theory in general, we exemplify their significance by studying the t…
Bayesian nonparametrics improves data-driven risk optimization under distributional uncertainty.
problem Improving out-of-sample performance in machine learning models due to distributional uncertainty.
method Combining Bayesian nonparametric theory and decision-theoretic preferences to propose a robust optimization criterion.
result The proposed robust optimization procedure provides favorable statistical guarantees and tractable approximations.
New theory improves diffusion models' convergence rates.
problem Understanding and optimizing diffusion models for faster data generation.
method Developed non-asymptotic theory for diffusion models with minimal assumptions.
result Established convergence rates for two diffusion models.
Random Matrix Theory (RMT) is applied to analyze weight matrices of Deep Neural Networks (DNNs), including both production quality, pre-trained models such as AlexNet and Inception, and smaller models trained from scratch, such as LeNet5 and a miniature-AlexNet. Empirical and theoretical results clearly indicate that t…
A scalable factorized Gaussian process VAE for faster inference.
problem Inference bottlenecks in Gaussian process VAEs.
method Factorizes latent kernel across auxiliary features, leveraging independence.
result Significant speed-up in inference time (in theory and practice).
Neural models price financial options without assuming underlying price forms.
problem Pricing financial options under flexible price processes.
method Apply neural SDEs as universal approximators, use Wasserstein distance for training.
result Error in option prices bounded by Wasserstein distance used for training.
Paper analyzes time series prediction using empirical risk minimization.
problem Optimizing 1-step-ahead prediction for time series.
method Empirical risk minimization applied to recursive algorithms for time series forecasting.
result Empirical risk minimization achieves optimal predictive performance.
This paper establishes a theoretical foundation for consistency training in diffusion models.
problem Lack of a comprehensive theoretical understanding of consistency training in diffusion models.
method Demonstrates the necessity of a number of steps in consistency learning exceeding d5/2/ε for generating samples within ε proximity to the target distribution. result Establishes rigorous insights into the validity and efficacy of consistency models, offering theoretical underpinnings for their utility.
The paper analyzes local minima in high-dimensional empirical risk minimization.
problem Understanding local minima in high-dimensional data models.
method Using Kac-Rice formula and proportional asymptotics, the paper derives bounds on local minima.
result Sharp asymptotics on estimation and prediction errors are derived.
Study improves financial risk assessment using ARMA-APARCH-EVT models with HACs.
problem Improving risk assessment in financial portfolios.
method ARMA-APARCH-EVT-HAC model for volatility and extreme value forecasting.
result Empirical analysis shows the model's effectiveness in international stock market data.