The hidden tail of empirical distributions is analyzed using extreme value theory.
problem Understanding the bias between in-sample mean and true statistical mean for large n. method Extreme value theory applied to empirical distributions and their moments.
result The hidden moment of order 0 for power law distributions follows an exponential distribution with expectation 1/n. RBM models reveal how hidden unit tail behavior affects pattern reconstruction.
problem Understanding how the tail behavior of hidden units in RBMs influences pattern reconstruction.
method Identified an effective energy function for RBMs and studied its local minima.
result The ability to reconstruct patterns depends on the tail behavior of the hidden unit prior distribution.
RBMs model binary interactions with hidden node activation effects.
problem Understanding how RBM hidden node activation affects binary variable distributions.
method Investigated RBM marginal distributions with different hidden node activation functions.
result Found exact expressions for RBM marginals as interacting binary variables.
New method accounts for hidden context in preference learning for RLHF models.
problem Incomplete data with hidden context affects RLHF model outcomes.
method Distributional Preference Learning (DPL) methods estimate hidden context distributions.
result DPL methods reduce RLHF vulnerabilities by accounting for hidden context.
Hidden tree Markov models allow learning distributions for tree structured data while being interpretable as nondeterministic automata. We provide a concise summary of the main approaches in literature, focusing in particular on the causality assumptions introduced by the choice of a specific tree visit direction. We w…
New method estimates HMM hidden states efficiently.
problem Inaccurate posterior predictive distribution in HMMs.
method Autoregressive-flow for estimating hidden states.
result Estimates comparable to SMC algorithm.
Neural networks learn more efficiently with hidden factorial structures.
problem Challenges in high-dimensional statistical learning.
method Controlled experimental framework to test neural networks' ability to exploit hidden factorial structures.
result Neural networks can leverage hidden factorial structures to learn discrete distributions more efficiently.
Detects anomalies in multiple processes using hidden Markov models.
problem Detecting an anomalous process among many with hidden states.
method Sequential search strategy using ADHM algorithm.
result ADHM algorithm effectively leverages temporal correlations.
The paper introduces FMCI and hybrid decoding for hidden Markov models.
problem Computing distributions and decoding hidden state sequences in HMMs.
method Finite Markov chain imbedding (FMCI) and hybrid decoding.
result Hybrid decoding improves performance over traditional methods.
We provide a classification of graphical models according to their representation as subfamilies of exponential families. Undirected graphical models with no hidden variables are linear exponential families (LEFs), directed acyclic graphical models and chain graphs with no hidden variables, including Bayesian networks …
This paper tackles hidden state inference for HMMs using particle filtering.
problem Inference for hidden states under HMMs is challenging due to unavailable true labels.
method Adaptive conformal inference framework using particle filtering.
result The framework produces prediction sets with specific aggregated coverage levels.
In unsupervised classification, Hidden Markov Models (HMM) are used to account for a neighborhood structure between observations. The emission distributions are often supposed to belong to some parametric family. In this paper, a semiparametric modeling where the emission distributions are a mixture of parametric distr…
We present a non-parametric Bayesian approach to structure learning with hidden causes. Previous Bayesian treatments of this problem define a prior over the number of hidden causes and use algorithms such as reversible jump Markov chain Monte Carlo to move between solutions. In contrast, we assume that the number of hi…
Expands Hidden Markov Model to include Markov chain observations.
problem Handling Markov chain observations in Hidden Markov Models.
method Developed Expectation-Maximization algorithm and Viterbi algorithm analogs.
result Estimates transition probabilities for hidden states and observations.
Machine learning promises methods that generalize well from finite labeled data. However, the brittleness of existing neural net approaches is revealed by notable failures, such as the existence of adversarial examples that are misclassified despite being nearly identical to a training example, or the inability of recu…
To infer a multilayer representation of high-dimensional count vectors, we propose the Poisson gamma belief network (PGBN) that factorizes each of its layers into the product of a connection weight matrix and the nonnegative real hidden units of the next layer. The PGBN's hidden layers are jointly trained with an upwar…
New methods for estimating causal effects in hidden variable DAGs.
problem Estimating causal effects in models with hidden variables.
method Influence function based estimators for causal effects in hidden variable DAGs.
result Achieves semiparametric efficiency bounds for identifiable effects.
Optimizes e-commerce traffic sales by incorporating hidden costs into auction mechanisms.
problem Hidden costs from unexpected advertising items in search results.
method Dynamic reserve price design with distributed solving algorithm.
result Ensures a balanced relationship between revenue and user experience.
Hidden regular variation defines a subfamily of distributions satisfying multivariate regular variation on E=[0,∞]d\{(0,0,...,0)} and models another regular variation on the sub-cone E(2)=E\∪i=1dLi, where Li is the $i…
Paper introduces MLRH, a probabilistic model for multilabel classification.
problem Multilabel classification challenges in various domains.
method Introduces hidden variables to relax one-hot-encoding in logistic regression.
result Probabilistic model achieves competitive performance compared to other algorithms.
Paper presents a fast method for estimating hidden states in Bayesian models.
problem Estimating hidden states in Bayesian state space models efficiently.
method Amortized simulation-based inference with pretraining.
result The method achieves sufficient accuracy and fast inference times.
Hidden Markov Neural Networks balance adaptation and forgetting in time-series data.
problem Balancing adaptation to new data and forgetting outdated information in time-series forecasting.
method Modeling weights as hidden states of a Hidden Markov model, using a filtering algorithm for learning a variational approximation of the posterior distribution over weights, and employing sequential Bayes by Backprop with variational DropConnect for regularization.
result Achieves strong predictive performance and effective uncertainty quantification on various tasks.
This work presents entropic constraints from DAGs with hidden variables.
problem Characterizing causal relations in systems with hidden variables.
method Entropic inequality constraints derived from e-separation relations. result These constraints can learn about true causal models from observed data.
New algorithm clusters multiple samples from hidden distributions.
problem Traditional clustering algorithms cannot handle multiple samples from non-Gaussian distributions.
method Proposes a general framework for multiple sample clustering, generating various algorithms.
result Sufficient statistics improve clustering accuracy and stability.
Bounds neural network output distribution to Gaussian for random initialization.
problem Quantifying the distribution of randomly initialized deep neural networks.
method Quantitative Gaussian approximation using quadratic Wasserstein distance.
result Explicit inequalities show how network sizes affect Gaussian behavior.
The hidden Markov model (HMM) is a generative model that treats sequential data under the assumption that each observation is conditioned on the state of a discrete hidden variable that evolves in time as a Markov chain. In this paper, we derive a novel algorithm to cluster HMMs through their probability distributions.…
Proposes BATer for improved adversarial example detection.
problem Detecting adversarial examples in neural networks.
method Introduces a Bayesian adversarial example detector (BATer) using random components in a Bayesian neural network.
result BATer outperforms state-of-the-art detectors in adversarial example detection.
A new framework predicts hidden Markov model regimes online.
problem Efficiently identify hidden Markov model regimes in streaming data.
method Develops a predictive-first optimisation framework for streaming HMMs, approximating the full posterior predictive distribution.
result The method provides competitive prequential performance compared to Online EM and Sequential Monte Carlo.
Deep neural networks converge to Gaussian mixtures as layer width increases.
problem Understanding the distribution of outputs from deep neural networks.
method Proof and experiments with a simple model showing the convergence of neural network outputs to Gaussian mixtures.
result Neural networks converge to Gaussian mixtures as the width of the last hidden layer increases.
New model for bandit problem with linear rewards and side information.
problem Hidden Markovian bandit problem with linear rewards and side information.
method Presented a model and algorithm with regret analysis for the problem.
result Logarithmic regret achieved even in high-dimensional problems with structural side information.
In this letter we borrow from the inference techniques developed for unbounded state-cardinality (nonparametric) variants of the HMM and use them to develop a tuning-parameter free, black-box inference procedure for Explicit-state-duration hidden Markov models (EDHMM). EDHMMs are HMMs that have latent states consisting…
The paper studies recovering hidden nearest neighbor graphs in large networks.
problem Discovering strong ties in social networks and assembling genome subsequences.
method Maximum likelihood estimator for recovering hidden 2k-nearest neighbor graphs. result The maximum likelihood estimator achieves asymptotic recovery guarantees under specific conditions.
We search for digital biomarkers from Parkinson's Disease by observing approximate repetitive patterns matching hypothesized step and stride periodic cycles. These observations were modeled as a cycle of hidden states with randomness allowing deviation from a canonical pattern of transitions and emissions, under the hy…
Boosted Control Functions improve prediction under distributional shifts.
problem Prediction under distributional shifts in the presence of hidden confounding.
method Boosted Control Function (BCF) and ControlTwicing algorithm.
result BCF allows for distribution generalization and invariance under nonlinear, non-identifiable structural functions.
New model for time series classification from single example.
problem Classifying time series patterns from limited data.
method Developed a Hidden semi-Markov Model with variable state duration.
result Different representations of state duration have distinct strengths and weaknesses.
Polynomial delay algorithm tests causal models with hidden variables.
problem Testing causal models with hidden variables in polynomial delay.
method c-component local Markov property (C-LMP) and polynomial delay algorithm.
result First algorithm for poly-delay testing of CIs in causal graphs with hidden variables.
The paper extends mean field results to three-layer neural networks using SGD.
problem Understanding the dynamics of training three-layer neural networks with SGD.
method Extending mean field results from two-layer networks to three-layer networks with two hidden layers, using non-linear partial differential equations.
result The distributions of weights in the two hidden layers are independent.
Price movements of stock market are not totally random. In fact, what drives the financial market and what pattern financial time series follows have long been the interest that attracts economists, mathematicians and most recently computer scientists [17]. This paper gives an idea about the trend analysis of stock mar…
Hidden Markov Chains and Linear-chain CRFs are equivalent.
problem Comparing Hidden Markov Chains and Conditional Random Fields.
method Constructing an HMC with the same posterior distribution as a CRF.
result HMCs and linear-chain CRFs are equivalent models.
Generative adversarial models are powerful tools to model structure in complex distributions for a variety of tasks. Current techniques for learning generative models require an access to samples which have high quality, and advanced generative models are applied to generate samples from noisy training data through amb…
The paper improves asset allocation using a skew-normal distribution in the Black-Litterman model.
problem Improving asset allocation under skewed return distributions.
method Using the Black-Litterman model with hidden truncation skew-normal distribution and Simaan's three-moment risk model.
result Optimal portfolios have less risk and higher skewness compared to classical BL model.
Deep networks have achieved impressive results across a variety of important tasks. However a known weakness is a failure to perform well when evaluated on data which differ from the training distribution, even if these differences are very small, as is the case with adversarial examples. We propose Fortified Networks,…
The inequality of wealth distribution is a universal phenomenon in the civilized nations, and it is often imputed to the Matthew effect, that is, the rich get richer and the poor get poorer. Some philosophers unjustified this phenomenon and tried to put the human civilization upon the evenness of wealth. Noticing the f…
We provide high-probability sample complexity guarantees for exact structure recovery and accurate predictive learning using noise-corrupted samples from an acyclic (tree-shaped) graphical model. The hidden variables follow a tree-structured Ising model distribution, whereas the observable variables are generated by a …
Bayesian networks with hidden variables help identify causal relationships obscured by confounding.
problem Identifying causal relationships obscured by unobserved confounders.
method Use finite k-mixtures of Bayesian networks with hidden variables to recover the joint probability distribution and identify causal relationships. result First algorithm to learn mixtures of non-empty DAGs, recovering identifiable causal relationships.
Stochastic Volatility in Mean models with heavy-tailed distributions using Hidden Markov Models
problem Accurate inference for Stochastic Volatility in Mean models with heavy-tailed distributions
method Numerically stable estimation procedure and parallel computing
result Significant reduction in computational times
Proposes DWMD for better matching of hidden representations across domains.
problem Measuring data distribution discrepancy between semantically related domains for feature representation matching.
method DWMD, a moment-based probability distribution metric that explicitly orders and weights higher-order moments.
result DWMD is error-free and can strictly reflect distribution differences without feature distribution assumptions.
We consider the problem where an agent wants to find a hidden object that is randomly located in some vertex of a directed acyclic graph (DAG) according to a fixed but possibly unknown distribution. The agent can only examine vertices whose in-neighbors have already been examined. In this paper, we address a learning s…