The relative -entropy is the Rényi analog of relative entropy and arises prominently in information-theoretic problems. Recent information geometric investigations on this quantity have enabled the generalization of the Cramér-Rao inequality, which provides a lower bound for the variance of an estimator of an escort…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Bayesian Cramer-Rao inequality”
The paper explores how information geometry impacts classical CR inequalities.
problem Deriving and generalizing CR inequalities using information geometry.
method Examining Eguchi's theory and applying Amari-Nagoaka's theory to KL-divergence, and then extending to other divergences.
result Generalized CR inequalities derived from various divergences.
Paper extends Bayesian Cramér-Rao bound with geometric considerations.
problem Estimation of covariance matrices with geometric structures.
method Intrinsic Bayesian Cramér-Rao bound with Riemannian geometry.
result Performance bounds for covariance matrix estimation.
Study on conditions for achieving optimal robustness in statistical estimators.
problem Achieving the optimal robustness of estimators in statistical models.
method Developed a Wasserstein analogue of the Cramer-Rao inequality and investigated conditions for achieving the Wasserstein-Cramer-Rao lower bound.
result Conditions for the existence of asymptotically efficient estimators in one-parameter models and location-scale families.
Proposes a learned Bayesian Cramér-Rao bound for unknown measurement models.
problem Computing the Bayesian Cramér-Rao bound requires full knowledge of priors and measurement distributions.
method Introduces a Physics-encoded score neural network to learn priors and measurements.
result Demonstrates improved sample complexity and interpretability through domain knowledge incorporation.
In this communication, we describe some interrelations between generalized -entropies and a generalized version of Fisher information. In information theory, the de Bruijn identity links the Fisher information and the derivative of the entropy. We show that this identity can be extended to generalized versions of en…
This work introduces a new data-driven estimator for the Bayesian Cramér-Rao bound using score matching.
problem Benchmarking the performance of statistical estimators and providing a principled metric for system design and optimization.
method Score matching technique to estimate the Bayesian Cramér-Rao bound from training data.
result Developed novel non-asymptotic bounds on the score matching error and the Bayesian CRB estimator.
Optimal Algorithms in Linear Regression under Covariate Shift: On the Importance of Preconditionstat.ML
The paper explores optimal algorithms for linear regression under covariate shift, proving the optimality of certain transformations and SGD variants.
problem Optimal algorithms for linear regression under covariate shift with ellipse-shaped constraints.
method Establishes a tight lower generalization bound via Bayesian Cramer-Rao inequality, proves the optimality of certain transformations, and analyzes SGD variants.
result Optimal estimators and SGD variants achieve optimality under specific conditions.
We propose a modified -divergence, give some of its properties, and show that this leads to the definition of a generalized Fisher information. We give generalized Cramér-Rao inequalities, involving this Fisher information, an extension of the Fisher information matrix, and arbitrary norms and power of the estimat…
We study the geometry of probability distributions with respect to a generalized family of Csiszár -divergences. A member of this family is the relative -entropy which is also a Rényi analog of relative entropy in information theory and known as logarithmic or projective power divergence in statistics. We apply E…
Cartan meets Cramér-Raomath.ST
Unified geometric interpretation of statistical estimation inequalities.
problem Curvature corrections in parametric statistical estimation.
method Cartan-geometric jet bundle formulation and jet prolongations.
result Unified geometric interpretation of higher-order information inequalities.
Posterior Cramer-Rao Lower Bound based Adaptive State Estimation for Option Price Forecastingq-fin.CP
Novel method uses Bayesian filters and PCRLB for state estimation of option prices.
problem Estimating unobserved latent variables from option prices.
method Posterior Cramer-Rao Lower Bound (PCRLB) based adaptive state estimation using various Bayesian filters.
result Proposed method outperforms individual filters and improves forecasting.
This work uses Sylvester normalizing flows for more accurate metabolite quantification in MRS.
problem Challenges in accurate metabolite quantification in MRS due to spectral overlap, low SNR, and artifacts.
method Bayesian inference framework with physics-informed Sylvester normalizing flows.
result Accurate metabolite quantification, well-calibrated uncertainties, and insights into parameter correlations and multi-modal distributions.
The paper explores quantum statistical manifolds and their autoparallelity, providing estimation-theoretical characterizations.
problem Quantum statistical manifolds and their geometric properties.
method Study of autoparallelity w.r.t. the e-connection, using quantum estimation theory.
result Characterizations of e-autoparallel submanifolds as statistical models with efficient estimators.
Paper defines Fisher co-metric on cotangent bundle and clarifies its relation to variance.
problem Defines Fisher co-metric on cotangent bundle and clarifies its relation to variance.
method Defines Fisher co-metric directly from Fisher metric without going through tangent bundle, using a natural correspondence between cotangent vectors and random variables.
result Clarifies the relation between Fisher co-metric and variance/covariance, trivializing the Cramér-Rao inequality.
The paper sets bounds on how much regret is unavoidable in adaptive LQR with unknown B-matrix.
problem Understanding the limits of adaptive LQR with unknown B-matrix.
method Local asymptotic minimax regret lower bounds using van Trees' inequality and Bellman error representation.
result Logarithmic regret is impossible if the parametrization induces an uninformative optimal policy.
The paper derives Cramer-Rao bounds for Laplacian matrix estimation under various constraints.
problem Estimating Laplacian matrices with structural constraints and sparsity.
method Linear reparametrization and closed-form expressions for Cramer-Rao bounds tailored to Laplacian matrix estimation.
result The derived CRBs provide performance limits for Laplacian matrix estimation and are validated in various applications.
This paper uses Bayesian optimization to efficiently identify stochastic dynamical systems.
problem Efficiently identifying linear stochastic dynamical systems with unknown coefficients and noise variances.
method Adaptive Bayesian optimization with ensemble Gaussian processes (EGP) and Kalman filter recursion.
result BO-based estimator achieves RMSE below the Cramer-Rao bound, improving robustness and consistency.
In this paper, we derive Hybrid, Bayesian and Marginalized Cramér-Rao lower bounds (HCRB, BCRB and MCRB) for the single and multiple measurement vector Sparse Bayesian Learning (SBL) problem of estimating compressible vectors and their prior distribution parameters. We assume the unknown vector to be drawn from a compr…
Paper provides a new lower bound on MMSE using Poincaré inequality.
problem Estimating X from noisy Y in exponential family noise.
method Alternative MMSE representation + Poincaré inequality.
result New lower bound on MMSE holds for all distributions.
New theory of sensitivity for unbiased estimators using Wasserstein geometry.
problem Estimating the instability of estimators under small perturbations.
method Developed a new theory based on Wasserstein geometry, analogous to classical Cramér-Rao theory.
result Wasserstein-Cramér-Rao lower bound for sensitivity of unbiased estimators.
Inverse Particle Filtermath.OC
Develops an inverse particle filter for cognitive systems.
problem Tracking cognitive adversaries in counter-adversarial applications.
method Global filtering approach using Monte Carlo methods and differentiable I-PF.
result Demonstrates convergence to optimal inverse filter and improved estimation performance.
Geometrically refines Cramér-Rao bound using extrinsic manifold curvature.
problem Improving estimator efficiency in non-asymptotic settings.
method Incorporates curvature-aware corrections based on extrinsic geometry of statistical model manifold.
result Meaningful tightening of estimator variance bounds.
Abstract: Geometrically reformulates estimation theory for finite-dimensional C*-algebras.
problem Estimation theory for finite-dimensional C*-algebras.
method Geometrical formulation of estimation theory.
result Derivation of Cramer-Rao and Helstrom bounds.
We introduce several novel change of measure inequalities for two families of divergences: -divergences and -divergences. We show how the variational representation for -divergences leads to novel change of measure inequalities. We also present a multiplicative change of measure inequality for -divergences …
We prove semi-empirical concentration inequalities for random variables which are given as possibly nonlinear functions of independent random variables. These inequalities describe concentration of random variable in terms of the data/distribution-dependent Efron-Stein (ES) estimate of its variance and they do not requ…
In this paper, we derive a useful lower bound for the Kullback-Leibler divergence (KL-divergence) based on the Hammersley-Chapman-Robbins bound (HCRB). The HCRB states that the variance of an estimator is bounded from below by the Chi-square divergence and the expectation value of the estimator. By using the relation b…
Non asymptotic estimation lower bounds for LTI state space models with Cramér-Rao and van Treesmath.ST
Study non-asymptotic estimation bounds for LTI models with Gaussian noise.
problem Estimating parameters of LTI models with non-asymptotic error bounds.
method Sharp non-asymptotic lower bounds using Cramér-Rao and van Trees inequalities, concentration results, and differential geometric constructions.
result Sharp and rate-optimal lower bounds for mean square estimation risk.
The aim of this paper is to provide some theoretical understanding of quasi-Bayesian aggregation methods non-negative matrix factorization. We derive an oracle inequality for an aggregated estimator. This result holds for a very general class of prior distributions and shows how the prior affects the rate of convergenc…
On the Dynamics of Inference and Learningcond-mat.dis-nn
Bayesian updating is modeled as a dynamical system, revealing learning rate laws.
problem Modeling Bayesian inference as a dynamical system.
method Formulated Bayesian updating as a continuous dynamical system, solving for trajectories in information geometry.
result Learning rate is governed by a power-law when the Cramér-Rao bound is saturated.
CD algorithm achieves near-optimal convergence rate for unnormalized models.
problem Training unnormalized models with high efficiency.
method Non-asymptotic analysis of contrastive divergence algorithm.
result CD can achieve convergence rate under regularity assumptions.
New bound relaxes uniform gradient norm assumptions for PAC-Bayesian bounds.
problem Generalization bounds with strict assumptions like uniformly bounded loss.
method Relax uniform bounds assumptions to on-average bounded loss and gradient norm.
result Proposes a new generalization bound with a surrogate of model complexity.
We present a set of high-probability inequalities that control the concentration of weighted averages of multiple (possibly uncountably many) simultaneously evolving and interdependent martingales. Our results extend the PAC-Bayesian analysis in learning theory from the i.i.d. setting to martingales opening the way for…
New CRB derived for curved models using extrinsic geometry.
problem Estimate curved statistical families accurately.
method Vector generalization of CRB with curvature correction using SDP and SOS relaxations.
result Directional curvature correction provides more accurate estimation.
We develop a novel advanced Particle Markov chain Monte Carlo algorithm that is capable of sampling from the posterior distribution of non-linear state space models for both the unobserved latent states and the unknown model parameters. We apply this novel methodology to five population growth models, including models …
Convolutional Bayesian filtering generalizes state estimation by incorporating inequality conditions.
problem Standard Bayesian filtering assumes exact conditional probabilities, limiting its applicability.
method Introducing inequality conditions transforms conditional probabilities into convolutional forms, expanding the filtering framework.
result Convolutional Bayesian filtering encompasses standard Bayesian filtering and allows for more nuanced model consideration.
The Fisher-Rao geometry is applied to elliptical distributions for optimization and classification.
problem Optimizing and classifying covariance matrices using geometric tools.
method Riemannian optimization and intrinsic Cramér-Rao bounds.
result Geometric tools enhance covariance matrix estimation and classification.
Adaptive Heston model calibration using PCRLB and switching filters.
problem Estimating volatility in stochastic volatility models like Heston.
method Bayesian filtering (EKF, UKF, PF) with PCRLB for parameter estimation.
result Adaptive estimation of Heston model parameters improves volatility estimation.
Improved Bayesian inference via variational approximations of generalized rho-posteriors.
problem Robust Bayesian inference under model misspecification and data contamination.
method Introducing a modified -posterior and using PAC-Bayesian analysis with variational approximations.
result Theoretical guarantees for tractable inference with competitive robustness and computational efficiency.
The paper establishes conditions for Bayesian consistency in supremum metric.
problem Ensuring Bayesian consistency in the supremum metric.
method Using a triangle inequality and weak convergence, the paper establishes conditions for Bayesian consistency.
result Demonstrates supremum consistency with weaker conditions than previously used.
Unified framework for deriving generalization bounds in supervised learning.
problem Generalization error bounds in supervised learning.
method Data Processing Inequality PAC-Bayesian framework.
result Unified bounds on binary Kullback-Leibler generalization gap for various divergences.
Estimates chirp signal frequencies using probabilistic models.
problem Estimating instantaneous frequencies of chirp signals when true forms are unknown.
method Non-linear Gaussian processes and stochastic filters/smothers for posterior estimation.
result The method outperforms state-of-the-art methods on synthetic and real-world datasets.
We improve maximum likelihood for location estimation in finite samples.
problem Estimating a parameter from samples with unknown or varying distribution.
method Use smoothed Fisher information for finite sample size and varying distributions.
result Recover optimal estimation theory for finite and arbitrary .
Algorithm improves variational inference in Wasserstein distance.
problem Improving variational inference methods for complex models.
method Wasserstein contraction analysis of coordinate ascent.
result General and sharp convergence guarantees for various models.
When I first encountered PAC-Bayesian concentration inequalities they seemed to me to be rather disconnected from good old-fashioned results like Hoeffding's and Bernstein's inequalities. But, at least for one flavour of the PAC-Bayesian bounds, there is actually a very close relation, and the main innovation is a cont…
Bayesian model selection consistency and oracle inequality with intractable marginal likelihoodmath.ST
In this article, we investigate large sample properties of model selection procedures in a general Bayesian framework when a closed form expression of the marginal likelihood function is not available or a local asymptotic quadratic approximation of the log-likelihood function does not exist. Under appropriate identifi…
One of the goals of probabilistic inference is to decide whether an empirically observed distribution is compatible with a candidate Bayesian network. However, Bayesian networks with hidden variables give rise to highly non-trivial constraints on the observed distribution. Here, we propose an information-theoretic appr…
We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This …