Limit order books (LOBs) match buyers and sellers in more than half of the world's financial markets. This survey highlights the insights that have emerged from the wealth of empirical and theoretical studies of LOBs. We examine the findings reported by statistical analyses of historical LOB data and discuss how severa…
A simple model explains deep learning phenomena like grokking and gradient boosting.
problem Understanding the unexpected behaviors of deep learning models.
method A telescoping sequence of first-order approximations to explain neural network performance.
result Empirical insights into neural network performance and training process.
New insights into how data transformations affect self-supervised clustering.
problem Impact of data transformations on self-supervised clustering convergence.
method Theoretical and empirical analysis of various data transformations.
result Certain transformations help in faster convergence of self-supervised clustering.
Proposes OBS, a method to adaptively combine Bayesian models online.
problem Learning optimal combinations of Bayesian models in online learning.
method Empirical Bayes lens, Online Bayesian Stacking (OBS).
result Establishes a novel connection between OBS and portfolio selection.
New insights into using momentum for non-convex optimization.
problem Improving training of non-convex models like deep neural networks.
method Developed a Lyapunov analysis of SGD with momentum using stochastic primal averaging.
result Precise conditions under which SGD+M outperforms SGD and optimal hyper-parameter schedules.
Tab-Shapley identifies top-k anomalies in tabular data quality insights.
problem Challenges in identifying anomalies in unlabeled tabular datasets.
method Cooperative game theory using Shapley values to quantify attribute contributions.
result Efficiently identifies top-k tabular data quality insights using closed-form Shapley values.
Theoretical framework explains why few epochs are enough for LLM fine-tuning.
problem Understanding why few epochs are sufficient for LLM fine-tuning.
method Combining early stopping theory with attention-based Neural Tangent Kernel (NTK) for LLMs.
result Formalizes convergence rate of attention-based fine-tuning with respect to sample size.
Bayesian nonparametrics improves data-driven risk optimization under distributional uncertainty.
problem Improving out-of-sample performance in machine learning models due to distributional uncertainty.
method Combining Bayesian nonparametric theory and decision-theoretic preferences to propose a robust optimization criterion.
result The proposed robust optimization procedure provides favorable statistical guarantees and tractable approximations.
Investigates model selection challenges in heterogeneous treatment effect estimation.
problem Lack of validation metrics for choosing the best model in treatment effect estimation.
method Empirical investigation of different model selection criteria.
result Complex interplay between selection strategies, estimators, and data.
Deep convolutional networks can be understood through kernel methods, providing insights into their inductive bias.
problem Understanding the functional space and inductive bias of deep convolutional networks.
method Using kernel methods to analyze simple hierarchical kernels with convolution and pooling layers.
result The RKHS consists of additive models of interaction terms between patches, and pooling layers encourage spatial similarities.
New research shows common ID estimators in neural representations are inaccurate.
problem Inaccurate estimation of intrinsic dimensions in neural representations.
method Theoretical and empirical investigation of ID estimators in neural representations.
result Common ID estimators do not accurately reflect the true underlying ID of neural representations.
Boosted additive models reveal new insights and potential pathologies.
problem Theoretical understanding of boosted additive models (BAMs) and their convergence behavior.
method Study of solution paths of BAMs and derivation of convergence results.
result Uncovering pathologies of boosting for certain additive model classes.
Spectral features of the empirical moment matrix constitute a resourceful tool for unveiling properties of a cloud of points, among which, density, support and latent structures. It is already well known that the empirical moment matrix encodes a great deal of subtle attributes of the underlying measure. Starting from …
Reinforcement learning is a promising approach to learning robotics controllers. It has recently been shown that algorithms based on finite-difference estimates of the policy gradient are competitive with algorithms based on the policy gradient theorem. We propose a theoretical framework for understanding this phenomen…
Model uses statistical physics principles to predict financial market volatility and returns.
problem Predicting price volatility and expected returns in financial markets.
method Inspired by statistical physics, the study introduces a physical model using Level 3 order book data to measure kinetic energy and momentum.
result The model outperforms traditional and machine learning approaches in forecasting volatility and expected returns.
The paper explores theoretical insights into WGANs for better understanding and stability.
problem Stabilizing the training process of GANs.
method Theoretical analysis and statistical convergence study of WGANs.
result Theoretical properties and convergence of WGANs are clarified.
The paper improves semi-supervised learning using f-divergences and α-Rényi divergences.
problem Improving semi-supervised learning with noisy pseudo-labels.
method Inspired by f-divergences and α-Rényi divergences, the paper develops new empirical risk functions and regularization techniques. result The new methods show better performance than traditional self-training methods, especially in noisy pseudo-label scenarios.
We reconsider the multivariate Kyle model in a risk-neutral setting with a single, perfectly informed rational insider and a rational competitive market maker, setting the price of n correlated securities. We prove the unicity of a symmetric, positive definite solution for the impact matrix and provide insights on its …
The paper explains how neural networks learn less salient frequency components during training.
problem Understanding the grokking phenomenon in neural networks.
method Empirical frequency analysis of training data.
result Neural networks initially learn less salient frequency components of the test data.
COF algorithm minimizes cost in multi-armed bandits with known costs and reward constraints.
problem Minimizing cost while meeting a minimum reward requirement in uncertain environments.
method COF algorithm that intelligently combines samples from all arms to gauge feasibility and minimize cost.
result COF achieves instance-dependent upper bounds on cumulative cost and quality regret.
Cryptocurrencies use blockchain tech for secure transactions, offering new research opportunities.
problem Misunderstanding of cryptocurrency technology and lack of empirical data.
method Analyzing detailed transaction data and summarizing statistics.
result Opportunity for academic research in financial economics.
EDML is a recently proposed algorithm for learning MAP parameters in Bayesian networks. In this paper, we present a number of new advances and insights on the EDML algorithm. First, we provide the multivalued extension of EDML, originally proposed for Bayesian networks over binary variables. Next, we identify a simplif…
Noise-ignorant empirical risk minimization achieves state-of-the-art performance on noisy data.
problem Learning with noisy labels in multi-class classification problems.
method Introducing relative signal strength (RSS) to quantify transferability and applying Noise Ignorant Empirical Risk Minimization (NI-ERM).
result NI-ERM achieves state-of-the-art performance on CIFAR-N data challenge.
DAGnosis uses DAGs to identify and localize data inconsistencies.
problem Handling data inconsistencies in machine learning models at deployment time.
method Directed acyclic graphs (DAGs) to encode feature probability distribution and independencies.
result Localization of inconsistencies and insights into their causes.
Extends causal additive models to include higher-order interactions.
problem Inferring causal insights from data with higher-order mechanisms.
method Introduces directed acyclic hypergraphs to represent higher-order interactions in causal structure learning.
result Learning more complex hypergraphs can lead to better empirical results.
A new method uses algebraic insights to create approximately equivariant networks without complex architectures.
problem Designing equivariant neural networks with complex architectures and high computational cost.
method Imposes the group's regular representation as an inductive bias via an auxiliary loss, adding no learnable parameters.
result Matches or outperforms specialized models in several cases, even for infinite groups.
Empirical analysis serves as an important complement to theoretical analysis for studying practical Bayesian optimization. Often empirical insights expose strengths and weaknesses inaccessible to theoretical analysis. We define two metrics for comparing the performance of Bayesian optimization methods and propose a ran…
The study examines how the number of noise samples affects diffusion models' performance.
problem Understanding the balance between generalization and memorization in diffusion models.
method Theoretical analysis and empirical experiments with Denoising Score Matching (DSM) using random features.
result Precise expressions for test and train errors under specific conditions reveal the mechanisms of generalization and memorization.
We present a framework for analyzing the exact dynamics of a class of online learning algorithms in the high-dimensional scaling limit. Our results are applied to two concrete examples: online regularized linear regression and principal component analysis. As the ambient dimension tends to infinity, and with proper tim…
We propose a distributed approach to train deep neural networks (DNNs), which has guaranteed convergence theoretically and great scalability empirically: close to 6 times faster on instance of ImageNet data set when run with 6 machines. The proposed scheme is close to optimally scalable in terms of number of machines, …
We analyze the dynamics of an online algorithm for independent component analysis in the high-dimensional scaling limit. As the ambient dimension tends to infinity, and with proper time scaling, we show that the time-varying joint empirical measure of the target feature vector and the estimates provided by the algorith…
We present in this paper an empirical framework motivated by the practitioner point of view on stability. The goal is to both assess clustering validity and yield market insights by providing through the data perturbations we propose a multi-view of the assets' clustering behaviour. The perturbation framework is illust…
Revisits PCA with new formulations and insights.
problem Improving PCA formulations and understanding.
method Difference-of-convex (DC) framework, kernelizability, out-of-sample applicability, simultaneous iteration, DCA perspective.
result PCA-like problems are kernelizable and have new optimization perspectives.
Study detects and explains positional bias in financial LLMs.
problem Positional bias in financial decision-making using LLMs.
method Unified framework and benchmark for detecting and quantifying bias in Qwen2.5 models.
result Positional bias is pervasive, scale-sensitive, and resurfaces under nuanced prompt designs.
Paper analyzes Hit-and-Run's convergence rates and applies similar methods to randomized Kaczmarz.
problem Quantifying advantages of Hit-and-Run's coordinate-free property.
method Sharp estimates via coupling methods and mixing time bounds.
result Ballistic and superdiffusive convergence rates in certain settings.
Researchers analyze inverse optimal transport, deriving theoretical and empirical insights.
problem Understanding the inverse problem of inferring cost matrices from optimal couplings.
method Formalized and analyzed using entropy-regularized optimal transport, with theoretical and empirical contributions.
result Characterization of the manifold of cross-ratio equivalent costs and derivation of an MCMC sampler.
Formalizes concepts as latent variables in hierarchical models for high-dimensional data.
problem Lack of formalization and theoretical insights for learning discrete concepts from high-dimensional data.
method Formalizes concepts as latent causal variables in a hierarchical model, formulates conditions for concept identification.
result Conditions for identifying latent hierarchical models in unsupervised data, handling complex structures and high-dimensional data.
Empirical data on real complex systems are becoming increasingly available. Parallel to this is the need for new methods of reconstructing (inferring) the topology of networks from time-resolved observations of their node-dynamics. The methods based on physical insights often rely on strong assumptions about the proper…
Study integrates climate and text data to improve credit default prediction.
problem Improving credit risk assessment for mSEs with limited financial histories.
method Multimodal framework using LSTM, GRU, and transformer models.
result Integration of multiple data modalities improves credit default prediction.
A new autoencoder method uses empirical beta copulas for generating data.
problem Creating a generative model from an autoencoder's latent space.
method Empirical Beta Copula Autoencoder method.
result The Empirical Beta Copula Autoencoder outperforms other methods in simplicity and effectiveness.
DAL enhances black-box bandit algorithms for non-stationary environments.
problem Non-stationary environments in bandit problems.
method DAL combines any stationary bandit algorithm with a change detector.
result DAL consistently outperforms state-of-the-art methods in various non-stationary scenarios.
New datasets improve fairness research by revealing UCI Adult's limitations.
problem Limitations of UCI Adult dataset in fairness research.
method Reconstructed a superset of UCI Adult data from US Census sources.
result New datasets reveal trade-offs between fairness criteria and performance.
This paper explores SSL for graph neural networks, improving performance on real-world datasets.
problem Leveraging unlabeled data for graph neural networks to improve deep learning performance.
method Empirical study of various SSL pretext tasks on graphs and proposing a new approach called SelfTask.
result Proposes SelfTask, achieving state-of-the-art performance on real-world datasets.
Graph embeddings have become a key and widely used technique within the field of graph mining, proving to be successful across a broad range of domains including social, citation, transportation and biological. Graph embedding techniques aim to automatically create a low-dimensional representation of a given graph, whi…
Unified calibration metrics improve forecast sharpness and accuracy.
problem Improving the sharpness of probabilistic forecasts while maintaining calibration.
method Kernel-based calibration metrics that unify and generalize existing methods for classification and regression.
result Enhanced calibration, sharpness, and decision-making across various tasks.
The paper explores how model complexity affects OOD detection performance.
problem Ensuring reliability and safety of machine learning systems through OOD detection.
method Investigates the relationship between model capacity and OOD detection performance using empirical and theoretical analysis.
result The Double Descent phenomenon is observed in post-hoc OOD detection, indicating that overparameterization can enhance OOD detection.
This work explores the generalization properties of diffusion models, providing theoretical and empirical insights.
problem Theoretical understanding of diffusion models' generalization capabilities remains underdeveloped.
method Theoretical exploration and quantitative analysis of generalization gaps in diffusion models.
result Established polynomially small generalization error (O(n−2/5+m−4/5)) for diffusion models, avoiding the curse of dimensionality. This work analyzes Fréchet regression using comparison geometry, providing theoretical and practical insights.
problem Analyzing data on complex structures like manifolds and graphs.
method Theoretical analysis through comparison geometry, focusing on existence, uniqueness, and stability of the Fréchet mean.
result Key results on the existence, uniqueness, and stability of the Fréchet mean, along with statistical guarantees for nonparametric regression.