arXiv research

Rule-based classifiers quantify uncertainty using Bernoulli random variables.

In this paper, we consider the multivariate Bernoulli distribution as a model to estimate the structure of graphs with binary nodes. This distribution is discussed in the framework of the exponential family, and its statistical properties regarding independence of the nodes are demonstrated. Importantly the model can e…

We consider $N$ Bernoulli random variables, which are independent conditional on a common random factor determining their probability distribution. We show that certain expected functionals of the proportion $L_N$ of variables in a given state converge at rate $1/N$ as $N\rightarrow \infty$. Based on these results, we …

Paper proposes a new estimator for generic discrete distributions.

A note on extending Chernoff bound for unit interval random variables.

We study covariance matrix estimation for the case of partially observed random vectors, where different samples contain different subsets of vector coordinates. Each observation is the product of the variable of interest with a $0-1$ Bernoulli random variable. We analyze an unbiased covariance estimator under this mod…

The paper proves ML estimators are strongly consistent for identifying edge weights in BAR models.

We present and study models of adversarial online learning where the feedback observed by the learner is noisy, and the feedback is either full information feedback or bandit feedback. Specifically, we consider binary losses xored with the noise, which is a Bernoulli random variable. We consider both a constant noise r…

Characterizes symmetric Bernoulli distributions with minimal convex sums.

A very simple event frequency approximation algorithm that is sensitive to event timeliness is suggested. The algorithm iteratively updates categorical click-distribution, producing (path of) a random walk on a standard $n$-dimensional simplex. Under certain conditions, this random walk is self-similar and corresponds …

We introduce an infectious default and recovery model for N obligors. Obligors are assumed to be exchangeable and their states are described by N Bernoulli random variables S_{i} (i=1,...,N). They are expressed by multiplying independent Bernoulli variables X_{i},Y_{ij},Y'_{ij}, and default and recovery infections are …

A new method for efficient nonlinear process monitoring using random Bernoulli features.

Improved regret bounds for DP-KLUCB and DP-IMED in Bernoulli bandits.

The structure of a Bayesian network encodes most of the information about the probability distribution of the data, which is uniquely identified given some general distributional assumptions. Therefore it's important to study the variability of its network structure, which can be used to compare the performance of diff…

The paper cleans label noise in supervised classification using Bernoulli sampling.

Proposes a non-parametric method for deep discrete latent variable models.

Let $X_{λ_1},\ldots,X_{λ_n}$ be dependent non-negative random variables and $Y_i=I_{p_i} X_{λ_i}$, $i=1,\ldots,n$, where $I_{p_1},\ldots,I_{p_n}$ are independent Bernoulli random variables independent of $X_{λ_i}$'s, with ${\rm E}[I_{p_i}]=p_i$, $i=1,\ldots,n$. In actuarial sciences, $Y_i$ corresponds to the claim amo…

We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This …

The structure of a Bayesian network includes a great deal of information about the probability distribution of the data, which is uniquely identified given some general distributional assumptions. Therefore it's important to study its variability, which can be used to compare the performance of different learning algor…

Let $X_{λ_1},\ldots,X_{λ_n}$ be a set of dependent and non-negative random variables share a survival copula and let $Y_i= I_{p_i}X_{λ_i}$, $i=1,\ldots,n$, where $I_{p_1},\ldots,I_{p_n}$ be independent Bernoulli random variables independent of $X_{λ_i}$'s, with ${\rm E}[I_{p_i}]=p_i$, $i=1,\ldots,n$. In actuarial scie…

We show how to analyze and interpret the correlation structures, the conditional expectation values and correlation coefficients of exchangeable Bernoulli random variables. We study implied default distributions for the iTraxx-CJ tranches and some popular probabilistic models, including the Gaussian copula model, Beta …

Hybrid continuous-discrete models naturally represent many real-world applications in robotics, finance, and environmental engineering. Inference with large-scale models is challenging because relational structures deteriorate rapidly during inference with observations. The main contribution of this paper is an efficie…

Sequence-to-sequence text-to-speech (TTS) is dominated by soft-attention-based methods. Recently, hard-attention-based methods have been proposed to prevent fatal alignment errors, but their sampling method of discrete alignment is poorly investigated. This research investigates various combinations of sampling methods…

Adaptive learning method identifies and corrects corrupted data.

The Straight-Through (ST) estimator is a widely used technique for back-propagating gradients through discrete random variables. However, this effective method lacks theoretical justification. In this paper, we show that ST can be interpreted as the simulation of the projected Wasserstein gradient flow (pWGF). Based on…

We develop constructions for exchangeable sequences of point processes that are rendered conditionally-i.i.d. negative binomial processes by a (possibly unknown) random measure called the base measure. Negative binomial processes are useful in Bayesian nonparametrics as models for random multisets, and in applications …

Improved spectral method recovers sparse vectors in random subspaces.

LCBM model improves image classification without human supervision.

New method compresses non-Gaussian distributions exponentially.

The issue of model risk in default modeling has been known since inception of the Academic literature in the field. However, a rigorous treatment requires a description of all the possible models, and a measure of the distance between a single model and the alternatives, consistent with the applications. This is the pu…

Variational autoencoders (VAE) have quickly become a central tool in machine learning, applicable to a broad range of data types and latent variable models. By far the most common first step, taken by seminal papers and by core software libraries alike, is to model MNIST data using a deep network parameterizing a Berno…

We introduce a novel multivariate random process producing Bernoulli outputs per dimension, that can possibly formalize binary interactions in various graphical structures and can be used to model opinion dynamics, epidemics, financial and biological time series data, etc. We call this a Bernoulli Autoregressive Proces…

Feature selection problems have been extensively studied for linear estimation, for instance, Lasso, but less emphasis has been placed on feature selection for non-linear functions. In this study, we propose a method for feature selection in high-dimensional non-linear function estimation problems. The new procedure is…

Study explores geometric structure and prior for beta-logistic distribution.

Optimal transport is #P-hard when components are independent, even with approximate solutions.

We study in detail and explicitly solve the version of Kyle's model introduced in a specific case in \cite{BB}, where the trading horizon is given by an exponentially distributed random time. The first part of the paper is devoted to the analysis of time-homogeneous equilibria using tools from the theory of one-dimensi…

We present a new explicit formula for the $m$-th Bernoulli number $B_m$, which involves two integer parameters $a$ and $n$ with $0\le a\le m\le n$. If we set $a=0$ and $n=m$, then the formula reduces to the celebrated Kronecker formula for $B_m$. We give two proofs of our formula. One is analytic and uses a certain fun…

This paper describes a recursive estimation procedure for multivariate binary densities (probability distributions of vectors of Bernoulli random variables) using orthogonal expansions. For $d$ covariates, there are $2^d$ basis coefficients to estimate, which renders conventional approaches computationally prohibitive …

The beta-Bernoulli process provides a Bayesian nonparametric prior for models involving collections of binary-valued features. A draw from the beta process yields an infinite collection of probabilities in the unit interval, and a draw from the Bernoulli process turns these into binary-valued features. Recent work has …

New ZIPLN model accounts for zero-inflation in multivariate count data.

A new neural subsampling method reduces data volume for deep models.

Probabilistic learning for binary classification with categorical variables.

Regularization for matrix factorization (MF) and approximation problems has been carried out in many different ways. Due to its popularity in deep learning, dropout has been applied also for this class of problems. Despite its solid empirical performance, the theoretical properties of dropout as a regularizer remain qu…

Multi-layer neural networks have lead to remarkable performance on many kinds of benchmark tasks in text, speech and image processing. Nonlinear parameter estimation in hierarchical models is known to be subject to overfitting and misspecification. One approach to these estimation and related problems (local minima, co…

In this paper, we are concerned with obtaining distribution-free concentration inequalities for mixture of independent Bernoulli variables that incorporate a notion of variance. Missing mass is the total probability mass associated to the outcomes that have not been seen in a given sample which is an important quantity…

A new method prunes neural networks efficiently without losing effectiveness.

Dropout is a simple yet effective algorithm for regularizing neural networks by randomly dropping out units through Bernoulli multiplicative noise, and for some restricted problem classes, such as linear or logistic regression, several theoretical studies have demonstrated the equivalence between dropout and a fully de…

A Bernoulli Mixture Model (BMM) is a finite mixture of random binary vectors with independent dimensions. The problem of clustering BMM data arises in a variety of real-world applications, ranging from population genetics to activity analysis in social networks. In this paper, we analyze the clusterability of BMMs from…