arXiv research

The resilience of low-degree Rademacher chaos is studied, providing probabilistic lower bounds.

We show that the log-likelihood of several probabilistic graphical models is Lipschitz continuous with respect to the lp-norm of the parameters. We discuss several implications of Lipschitz parametrization. We present an upper bound of the Kullback-Leibler divergence that allows understanding methods that penalize the …

A new diffusion model tackles brightness issues with a probabilistic approach.

With deep neural networks providing state-of-the-art machine learning models for numerous machine learning tasks, quantifying the robustness of these models has become an important area of research. However, most of the research literature merely focuses on the \textit{worst-case} setting where the input of the neural …

We present a probabilistic viewpoint to multiple kernel learning unifying well-known regularised risk approaches and recent advances in approximate Bayesian inference relaxations. The framework proposes a general objective function suitable for regression, robust regression and classification that is lower bound of the…

We consider a problem of stochastic online learning with general probabilistic graph feedback, where each directed edge in the feedback graph has probability $p_{ij}$. Two cases are covered. (a) The one-step case, where after playing arm $i$ the learner observes a sample reward feedback of arm $j$ with independent prob…

PAGE is a simple gradient estimator for nonconvex optimization problems.

In this work, we investigate Batch Normalization technique and propose its probabilistic interpretation. We propose a probabilistic model and show that Batch Normalization maximazes the lower bound of its marginalized log-likelihood. Then, according to the new probabilistic model, we design an algorithm which acts cons…

Study probabilistic safety of BNNs under adversarial attacks.

In this paper, we introduce Deep Probabilistic Ensembles (DPEs), a scalable technique that uses a regularized ensemble to approximate a deep Bayesian Neural Network (BNN). We do so by incorporating a KL divergence penalty term into the training objective of an ensemble, derived from the evidence lower bound used in var…

Employing probabilistic techniques we compute best possible upper and lower bounds on the price of an option on one or two assets with continuous piecewise linear payoff function based on prices of simple call options of possibly distinct maturities and the no-arbitrage condition, but without any assumption on the pric…

Estimates chirp signal frequencies using probabilistic models.

PAC-Bayesian bounds for MLPs with cross entropy loss validated.

We present a general method for deriving collapsed variational inference algo- rithms for probabilistic models in the conjugate exponential family. Our method unifies many existing approaches to collapsed variational inference. Our collapsed variational inference leads to a new lower bound on the marginal likelihood. W…

Deep Gaussian processes provide a flexible approach to probabilistic modelling of data using either supervised or unsupervised learning. For tractable inference approximations to the marginal likelihood of the model must be made. The original approach to approximate inference in these models used variational compressio…

How can we perform efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distributions, and large datasets? We introduce a stochastic variational inference and learning algorithm that scales to large datasets and, under some mild dif…

We build on auto-encoding sequential Monte Carlo (AESMC): a method for model and proposal learning based on maximizing the lower bound to the log marginal likelihood in a broad family of structured probabilistic models. Our approach relies on the efficiency of sequential Monte Carlo (SMC) for performing inference in st…

Combines offline and online learning for identifying the best arm in bandits.

A framework to boost the efficiency of Bayesian inference in probabilistic programs is introduced by embedding a sampler inside a variational posterior approximation. We call it the refined variational approximation. Its strength lies both in ease of implementation and automatically tuning of the sampler parameters to …

We study the problem of estimating, in the sense of optimal transport metrics, a measure which is assumed supported on a manifold embedded in a Hilbert space. By establishing a precise connection between optimal transport metrics, optimal quantization, and learning theory, we derive new probabilistic bounds for the per…

VCL adds uncertainty to contrastive learning models.

ProbDR framework interprets DR algorithms as probabilistic inference.

Variational inference approximates the posterior distribution of a probabilistic model with a parameterized density by maximizing a lower bound for the model evidence. Modern solutions fit a flexible approximation with stochastic gradient descent, using Monte Carlo approximation for the gradients. This enables variatio…

We consider log-supermodular models on binary variables, which are probabilistic models with negative log-densities which are submodular. These models provide probabilistic interpretations of common combinatorial optimization tasks such as image segmentation. In this paper, we focus primarily on parameter estimation in…

Probabilistic bounds on neuron death in deep networks, showing depth can be increased indefinitely.

Paper presents estimators for entropy and information in probabilistic models.

We study two-layer belief networks of binary random variables in which the conditional probabilities Pr[childlparents] depend monotonically on weighted sums of the parents. In large networks where exact probabilistic inference is intractable, we show how to compute upper and lower bounds on many probabilities of intere…

New method calibrates DPMs to improve likelihood bounds.

Improved upper bound for online calibrated forecasting of binary sequences.

New bounds show complex neural networks need many queries to learn.

The paper reviews recent statistical methods for financial markets, focusing on jumps, volatility, and microstructure noise.

Paper tests DPPs for diversity models, distinguishing them from other distributions.

Derives VMP for LDA, simplifying inference for topic modeling.

We study the fundamental tradeoffs between computational tractability and statistical accuracy for a general family of hypothesis testing problems with combinatorial structures. Based upon an oracle model of computation, which captures the interactions between algorithms and data, we establish a general lower bound tha…

Paper develops a scalable distributed inference algorithm for sensor networks.

Simplified explanation of DDPMs for machine learning.

In the problem of optimal investment with utility function defined on $(0,\infty)$, we formulate sufficient conditions for the dual optimizer to be a uniformly integrable martingale. Our key requirement consists of the existence of a martingale measure whose density process satisfies the probabilistic Muckenhoupt $(A_p…

Survey of factor analysis, PCA, variational inference, and VAE.

We study combinatorial multi-armed bandit with probabilistically triggered arms (CMAB-T) and semi-bandit feedback. We resolve a serious issue in the prior CMAB-T studies where the regret bounds contain a possibly exponentially large factor of $1/p^*$, where $p^*$ is the minimum positive probability that an arm is trigg…

Generative models' ELBOs converge to entropy sums, proving for various models.

Probabilistic models are a critical part of the modern deep learning toolbox - ranging from generative models (VAEs, GANs), sequence to sequence models used in machine translation and speech processing to models over functional spaces (conditional neural processes, neural processes). Given the size and complexity of th…

DO-EM framework for quantum models improves generative tasks.

We present a queuing model of parking dynamics and a model-based prediction method to provide real-time probabilistic forecasts of future parking occupancy. The queuing model has a non-homogeneous arrival rate and time-varying service time distribution. All statistical assumptions of the model are verified using data f…

Quantum circuit models learn better with specific initialization strategies.

Next-gen reservoir computers fail to predict complex processes, highlighting need for better architectures.

Efficient RL in PRMs with improved regret bound.

Energy-based models (EBMs) are powerful probabilistic models, but suffer from intractable sampling and density evaluation due to the partition function. As a result, inference in EBMs relies on approximate sampling algorithms, leading to a mismatch between the model and inference. Motivated by this, we consider the sam…

We can compare the expressiveness of neural networks that use rectified linear units (ReLUs) by the number of linear regions, which reflect the number of pieces of the piecewise linear functions modeled by such networks. However, enumerating these regions is prohibitive and the known analytical bounds are identical for…