A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
The paper addresses the gap between theoretical and practical confidence set widths in universal inference.
We introduce a method for using deep neural networks to amortize the cost of inference in models from the family induced by universal probabilistic programming languages, establishing a framework that combines the strengths of probabilistic programming and deep learning methods. We call what we do "compilation of infer…
NP-iMCMC algorithm for nonparametric models in universal PPLs.
We present a novel framework for estimation and inference with the broad class of universal approximators. Estimation is based on the decomposition of model predictions into Shapley values. Inference relies on analyzing the bias and variance properties of individual Shapley components. We show that Shapley value estima…
Universal probabilistic programming systems (PPSs) provide a powerful framework for specifying rich probabilistic models. They further attempt to automate the process of drawing inferences from these models, but doing this successfully is severely hampered by the wide range of non--standard models they can express. As …
Probabilistic programming languages (PPLs) are powerful modelling tools which allow to formalise our knowledge about the world and reason about its inherent uncertainty. Inference methods used in PPL can be computationally costly due to significant time burden and/or storage requirements; or they can lack theoretical g…
A universal framework for constructing confidence sets using sequential likelihood mixing.
Universally valid ground truth is almost impossible to obtain or would come at a very high cost. For supervised learning without universally valid ground truth, a recommended approach is applying crowdsourcing: Gathering a large data set annotated by multiple individuals of varying possibly expertise levels and inferri…
Transformer pretraining yields strong EB performance without explicit adaptation.
Graph neural networks help infer causal effects from partially observable data.
We consider the problem of identifying universal low-dimensional features from high-dimensional data for inference tasks in settings involving learning. For such problems, we introduce natural notions of universality and we show a local equivalence among them. Our analysis is naturally expressed via information geometr…
Modified relative universality for unbiasedness and consistency in dimension reduction.
DebiNet uses over-parameterized neural networks to improve linear model performance and debiasing.
Transfer learning improves chaotic dynamics predictions with less data.
We propose a general method for constructing hypothesis tests and confidence sets that have finite sample guarantees without regularity conditions. We refer to such procedures as "universal." The method is very simple and is based on a modified version of the usual likelihood ratio statistic, that we call "the split li…
Understanding large-scale patterns in student course enrollment is a problem of great interest to university administrators and educational researchers. Yet important decisions are often made without a good quantitative framework of the process underlying student choices. We propose a probabilistic approach to modellin…
Generalizes causal inference to high-dimensional outcomes.
NP-HMC extends HMC for nonparametric models in probabilistic programming.
New methods for inferring, predicting, and estimating continuous-time, discrete-event processes.
This research formalizes uncertainty quantification for Universal Differential Equations models.
Market sectors play a key role in the efficient flow of capital through the modern Global economy. We analyze existing sectorization heuristics, and observe that the most popular - the GICS (which informs the S&P 500), and the NAICS (published by the U.S. Government) - are not entirely quantitatively driven, but rather…
New theorem for generalized group sparsity improves consistency and convergence rates.
Probabilistic graphical models are powerful tools which allow us to formalise our knowledge about the world and reason about its inherent uncertainty. There exist a considerable number of methods for performing inference in probabilistic graphical models; however, they can be computationally costly due to significant t…
Universal AI seeks high-optionality states through empowerment and curiosity.
BFNs use Bayesian inference and neural networks for generative modeling.
Paper relaxes symmetry conditions for universal feature selection in noisy data.
RealUID distills matching models using real data without GANs.
We present a new approach to automatic amortized inference in universal probabilistic programs which improves performance compared to current methods. Our approach is a variation of inference compilation (IC) which leverages deep neural networks to approximate a posterior distribution over latent variables in a probabi…
VFG model embeds flow-based models with hierarchical structures using variational inference.
FibQuant improves KV-cache compression for long-context inference.
The paper describes a method to infer the signal-to-noise ratio in portfolio optimization.
Variational inference is a powerful tool for approximate inference, and it has been recently applied for representation learning with deep generative models. We develop the variational Gaussian process (VGP), a Bayesian nonparametric variational family, which adapts its shape to match complex posterior distributions. T…
Mesoscopic pattern extraction (MPE) is the problem of finding a partition of the nodes of a complex network that maximizes some objective function. Many well-known network inference problems fall in this category, including, for instance, community detection, core-periphery identification, and imperfect graph coloring.…
CATVI improves variational inference for Bayesian nonparametric models by reducing divergence and improving prediction accuracy.
Unified framework for accelerating DNNs on resource-limited platforms.
We consider the problem of inference in a causal generative model where the set of available observations differs between data instances. We show how combining samples drawn from the graphical model with an appropriate masking function makes it possible to train a single neural network to approximate all the correspond…
LIC compiles probabilistic models to generate efficient MCMC proposals.
Boosting variational inference (BVI) approximates an intractable probability density by iteratively building up a mixture of simple component distributions one at a time, using techniques from sparse convex optimization to provide both computational scalability and approximation error guarantees. But the guarantees hav…
Naive approaches to amortized inference in probabilistic programs with unbounded loops can produce estimators with infinite variance. This is particularly true of importance sampling inference in programs that explicitly include rejection sampling as part of the user-programmed generative procedure. In this paper we de…
LDTA expands LDA's topic modeling capacity with tree-structured priors.
As inductive inference and machine learning methods in computer science see continued success, researchers are aiming to describe ever more complex probabilistic models and inference algorithms. It is natural to ask whether there is a universal computational procedure for probabilistic inference. We investigate the com…
Celeste is a procedure for inferring astronomical catalogs that attains state-of-the-art scientific results. To date, Celeste has been scaled to at most hundreds of megabytes of astronomical images: Bayesian posterior inference is notoriously demanding computationally. In this paper, we report on a scalable, parallel v…
Unified theory for semi-implicit variational inference, bridging approximation and optimization.
We introduce a flexible framework for making inferences about general linear forms of a large matrix based on noisy observations of a subset of its entries. In particular, under mild regularity conditions, we develop a universal procedure to construct asymptotically normal estimators of its linear forms through double-…
Fast Bayesian inference with adaptable priors for real-time applications.
DISTANA improves weather prediction by inferring hidden factors from temperature data.
In this paper, we study two aspects of the variational autoencoder (VAE): the prior distribution over the latent variables and its corresponding posterior. First, we decompose the learning of VAEs into layerwise density estimation, and argue that having a flexible prior is beneficial to both sample generation and infer…
Single-head transformers with a single self-attention layer can approximate any sequence-to-sequence function and are efficient under certain conditions.