arXiv research

A note on extending Chernoff bound for unit interval random variables.

New bounds on continuous random variables' right-tail probabilities.

Valid p-value for bounded random variables without distributional assumptions.

Note improves confidence bounds for random variables.

The paper sets limits on the accuracy of macroeconomic forecasts based on statistical moments and trade volumes.

We prove semi-empirical concentration inequalities for random variables which are given as possibly nonlinear functions of independent random variables. These inequalities describe concentration of random variable in terms of the data/distribution-dependent Efron-Stein (ES) estimate of its variance and they do not requ…

New framework for tracking varying bounds in time series forecasting.

Near-optimal confidence intervals for bounded data.

We establish general versions of a variety of results for quasiconvex, lower-semicontinuous, and law-invariant functionals. Our results extend well-known results from the literature to a large class of spaces of random variables. We sometimes obtain sharper versions, even for the well-studied case of bounded random var…

Diversification improves profits for heavy-tailed investments.

Novel Orlicz regrets consistently bound environmental variable statistics.

The paper examines bounds for stop-loss payoffs using transformed random variables.

Paper extends tail bounds to high-dimensional random objects on Riemannian manifolds.

We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This …

We propose an approach to the aggregation of risks which is based on estimation of simple quantities (such as covariances) associated to a vector of dependent random variables, and which avoids the use of parametric families of copulae. Our main result demonstrates that the method leads to bounds on the worst case Valu…

New findings on how certain functionals behave in random variable spaces.

Sharp concentration bounds for i.i.d. variables.

New bound on machine learning model performance using Jensen-Shannon information.

A refinement of Bennett's inequality is introduced which is strictly tighter than the classical bound. The new bound establishes the convergence of the average of independent random variables to its expected value. It also carefully exploits information about the potentially heterogeneous mean, variance, and ceiling of…

Sharp comparison for sub-Gaussian random variables in convex order.

Improved guarantees for sparse random embeddings with explicit bounds and empirical superiority.

In this paper, we study the stochastic combinatorial multi-armed bandit (CMAB) framework that allows a general nonlinear reward function, whose expected value may not depend only on the means of the input random variables but possibly on the entire distributions of these variables. Our framework enables a much larger c…

The paper analyzes graph Laplacians on manifolds with curvature bounds and applies to non-collapsed spaces.

We study covariance matrix estimation for the case of partially observed random vectors, where different samples contain different subsets of vector coordinates. Each observation is the product of the variable of interest with a $0-1$ Bernoulli random variable. We analyze an unbiased covariance estimator under this mod…

SUMO provides unbiased log marginal likelihood estimation for latent variable models.

New concentration inequalities for tensors with heavy-tailed coefficients.

We prove non-asymptotic lower bounds on the expectation of the maximum of $d$ independent Gaussian variables and the expectation of the maximum of $d$ independent symmetric random walks. Both lower bounds recover the optimal leading constant in the limit. A simple application of the lower bound for random walks is an (…

Random walk constructs Morse functions on surfaces.

Random feature matrices' singular values concentrate near their full expectation in high dimensions.

Given a Gaussian Markov random field, we consider the problem of selecting a subset of variables to observe which minimizes the total expected squared prediction error of the unobserved variables. We first show that finding an exact solution is NP-hard even for a restricted class of Gaussian Markov random fields, calle…

In this paper, we study the problem of expected utility maximization of an agent who, in addition to an initial capital, receives random endowments at maturity. Contrary to previous studies, we treat as the variables of the optimization problem not only the initial capital but also the number of units of the random end…

Given a monotone convex function on the space of essentially bounded random variables with the Lebesgue property (order continuity), we consider its extension preserving the Lebesgue property to as big solid vector space of random variables as possible. We show that there exists a maximum such extension, with explicit …

Coordinate ascent variational inference is an important algorithm for inference in probabilistic models, but it is slow because it updates only a single variable at a time. Block coordinate methods perform inference faster by updating blocks of variables in parallel. However, the speed and stability of these algorithms…

New PAC-Bayes bounds for unbounded losses using Cramér-Chernoff techniques.

The study improves the perceptron's storage capacity by optimizing variable selection.

This note displays an interesting phenomenon for percentiles of independent but non-identical random variables. Let $X_1,\cdots,X_n$ be independent random variables obeying non-identical continuous distributions and $X^{(1)}\geq \cdots\geq X^{(n)}$ be the corresponding order statistics. For any $p\in(0,1)$, we investig…

The standard margin-based structured prediction commonly uses a maximum loss over all possible structured outputs. The large-margin formulation including latent variables not only results in a non-convex formulation but also increases the search space by a factor of the size of the latent space. Recent work has propose…

We give the proof of a tight lower bound on the probability that a binomial random variable exceeds its expected value. The inequality plays an important role in a variety of contexts, including the analysis of relative deviation bounds in learning theory and generalization bounds for unbounded loss functions.

Gibbs sampling is a Markov Chain Monte Carlo sampling technique that iteratively samples variables from their conditional distributions. There are two common scan orders for the variables: random scan and systematic scan. Due to the benefits of locality in hardware, systematic scan is commonly used, even though most st…

Study improves error bounds for sparse regression with heavy-tailed covariates.

The paper provides bounds on the CDF of a variable under nonstationary conditions.

Improved concentration inequalities for sub-Weibull variables enhance statistical and machine learning applications.

The belief propagation (BP) algorithm is widely applied to perform approximate inference on arbitrary graphical models, in part due to its excellent empirical properties and performance. However, little is known theoretically about when this algorithm will perform well. Using recent analysis of convergence and stabilit…

Paper improves CI and CS for bounded means using betting and mixtures.

KSG mutual information estimator, which is based on the distances of each sample to its k-th nearest neighbor, is widely used to estimate mutual information between two continuous random variables. Existing work has analyzed the convergence rate of this estimator for random variables whose densities are bounded away fr…

Stochastic gradient Langevin dynamics (SGLD) is a computationally efficient sampler for Bayesian posterior inference given a large scale dataset. Although SGLD is designed for unbounded random variables, many practical models incorporate variables with boundaries such as non-negative ones or those in a finite interval.…

Information bottleneck (IB) is a technique for extracting information in one random variable $X$ that is relevant for predicting another random variable $Y$. IB works by encoding $X$ in a compressed "bottleneck" random variable $M$ from which $Y$ can be accurately decoded. However, finding the optimal bottleneck variab…

The paper bounds solutions to complex optimization problems with uncertain data.