A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New statistical theory explains contrastive learning effectiveness.
Paper introduces data-dependent SSP for private linear and logistic regression.
Neural networks help create summary statistics for complex models.
Paper develops a theory explaining contrastive pre-training for multimodal AI.
We uncover a fairly general principle in online learning: If regret can be (approximately) expressed as a function of certain "sufficient statistics" for the data sequence, then there exists a special Burkholder function that 1) can be used algorithmically to achieve the regret bound and 2) only depends on these suffic…
Generalized linear models (GLMs) -- such as logistic regression, Poisson regression, and robust regression -- provide interpretable models for diverse data types. Probabilistic approaches, particularly Bayesian ones, allow coherent estimates of uncertainty, incorporation of prior information, and sharing of power acros…
This paper provides a method for noise-calibrated inference from DP synthetic data.
This paper introduces SS-MAMP to address convergence issues in AMP algorithms.
New method for separating mixed signals with nonlinear functions.
New statistics are introduced that maintain the Fisher metric structure closely, akin to sufficient statistics.
We propose a novel approach for density estimation with exponential families for the case when the true density may not fall within the chosen family. Our approach augments the sufficient statistics with features designed to accumulate probability mass in the neighborhood of the observed points, resulting in a non-para…
New lower bound shows RL with linear approximations is computationally hard.
The paper explores intersectional fairness in machine learning, proving bounds on it.
New method uses sufficient statistics to infer causal relationships from observational data.
This research designs a data-driven partition to test independence between continuous variables.
Efficiently tests two distributions using Nyström approximation of MMD.
This article suggests that deterministic Gradient Descent, which does not use any stochastic gradient approximation, can still exhibit stochastic behaviors. In particular, it shows that if the objective function exhibit multiscale behaviors, then in a large learning rate regime which only resolves the macroscopic but n…
The paper corrects bias in fluid approximation for better decision-making in stochastic optimization.
New method uses neural exponential families for likelihood-free inference.
Modern deep learning methods provide effective means to learn good representations. However, is a good representation itself sufficient for sample efficient reinforcement learning? This question has largely been studied only with respect to (worst-case) approximation error, in the more classical approximate dynamic pro…
Complicated generative models often result in a situation where computing the likelihood of observed data is intractable, while simulating from the conditional density given a parameter value is relatively easy. Approximate Bayesian Computation (ABC) is a paradigm that enables simulation-based posterior inference in su…
Squared families are a new model class derived from linear transformations, offering convenient properties and universal approximation.
Privacy preserving mechanisms such as differential privacy inject additional randomness in the form of noise in the data, beyond the sampling mechanism. Ignoring this additional noise can lead to inaccurate and invalid inferences. In this paper, we incorporate the privacy mechanism explicitly into the likelihood functi…
We propose an efficient algorithm for approximate computation of the profile maximum likelihood (PML), a variant of maximum likelihood maximizing the probability of observing a sufficient statistic rather than the empirical sample. The PML has appealing theoretical properties, but is difficult to compute exactly. Inspi…
We study sequential change-point detection procedures based on linear sketches of high-dimensional signal vectors using generalized likelihood ratio (GLR) statistics. The GLR statistics allow for an unknown post-change mean that represents an anomaly or novelty. We consider both fixed and time-varying projections, deri…
The paper develops time-uniform inference methods for stochastic approximation parameters.
New KSDs control moments in approximations, improving diagnostics and tests.
Information geometry provides a geometric approach to families of statistical models. The key geometric structures are the Fisher quadratic form and the Amari-Chentsov tensor. In statistics, the notion of sufficient statistic expresses the criterion for passing from one model to another without loss of information. Thi…
One of the most fundamental questions one can ask about a pair of random variables X and Y is the value of their mutual information. Unfortunately, this task is often stymied by the extremely large dimension of the variables. We might hope to replace each variable by a lower-dimensional representation that preserves th…
This paper develops embeddings that preserve likelihood-based statistical inference.
Boosting improves data fitting while maintaining fairness guarantees.
Cross validation (CV) and the bootstrap are ubiquitous model-agnostic tools for assessing the error or variability of machine learning and statistical estimators. However, these methods require repeatedly re-fitting the model with different weighted versions of the original dataset, which can be prohibitively time-cons…
Paper analyzes online tensorial ICA convergence with stochastic approximation.
The study provides conditions for approximating Riemannian manifolds with polyhedral metrics.
The concept of sample mean in dynamic time warping (DTW) spaces has been successfully applied to improve pattern recognition systems and generalize centroid-based clustering algorithms. Its existence has neither been proved nor challenged. This article presents sufficient conditions for existence of a sample mean in DT…
Reduces IB problem to a simpler, lower-dimensional problem.
In this paper, a Bayesian inference technique based on Taylor series approximation of the logarithm of the likelihood function is presented. The proposed approximation is devised for the case, where the prior distribution belongs to the exponential family of distributions. The logarithm of the likelihood function is li…
We introduce Minimal Achievable Sufficient Statistic (MASS) Learning, a training method for machine learning models that attempts to produce minimal sufficient statistics with respect to a class of functions (e.g. deep networks) being optimized over. In deriving MASS Learning, we also introduce Conserved Differential I…
We introduce a model-independent approximation for the branching ratio of Hawkes self-exciting point processes. Our estimator requires knowing only the mean and variance of the event count in a sufficiently large time window, statistics that are readily obtained from empirical data. The method we propose greatly simpli…
Paper uses statistical depth to create DP estimators for regression.
We study the problem of nonparametric dependence detection. Many existing methods may suffer severe power loss due to non-uniform consistency, which we illustrate with a paradox. To avoid such power loss, we approach the nonparametric test of independence through the new framework of binary expansion statistics (BEStat…
The paper distinguishes between conditional and marginal processes in language models and discusses conditions for usefulness.
Unified framework for FDR control in knockoffs, validating Gaussian knockoffs.
Data augmentation can achieve the same statistical benefits as full augmentation up to an approximation error.
We show that, for an affine submersion with horizontal distribution, is a statistical manifold with the metric and connection induced from the statistical manifold . The concept of conformal submersion with horizontal distribution is introduced, which i…
Traditionally, the field of computational Bayesian statistics has been divided into two main subfields: variational methods and Markov chain Monte Carlo (MCMC). In recent years, however, several methods have been proposed based on combining variational Bayesian inference and MCMC simulation in order to improve their ov…
Deep, wide ConvResNets can approximate functions and their smoothness.
Meta-algorithm selects optimal model complexity in RL with function approximation.