arXiv research

New method amplifies hidden structure in complex networks.

This paper reviews recent advances in Bayesian nonparametric techniques for constructing and performing inference in infinite hidden Markov models. We focus on variants of Bayesian nonparametric hidden Markov models that enhance a posteriori state-persistence in particular. This paper also introduces a new Bayesian non…

Neural networks learn more efficiently with hidden factorial structures.

We introduce a new paradigm that is important for community detection in the realm of network analysis. Networks contain a set of strong, dominant communities, which interfere with the detection of weak, natural community structure. When most of the members of the weak communities also belong to stronger communities, t…

We present a non-parametric Bayesian approach to structure learning with hidden causes. Previous Bayesian treatments of this problem define a prior over the number of hidden causes and use algorithms such as reversible jump Markov chain Monte Carlo to move between solutions. In contrast, we assume that the number of hi…

Paper tackles hidden game problem in AI alignment and language games.

New method learns graph structure with hidden causes from observational data.

In this work, we propose the marginal structured SVM (MSSVM) for structured prediction with hidden variables. MSSVM properly accounts for the uncertainty of hidden variables, and can significantly outperform the previously proposed latent structured SVM (LSSVM; Yu & Joachims (2009)) and other state-of-art methods, espe…

In recent years, there is a growing interest in learning Bayesian networks with continuous variables. Learning the structure of such networks is a computationally expensive procedure, which limits most applications to parameter learning. This problem is even more acute when learning networks with hidden variables. We p…

Language models based on deep neural networks and traditional stochastic modelling have become both highly functional and effective in recent times. In this work, a general survey into the two types of language modelling is conducted. We investigate the effectiveness of the Hidden Markov Model (HMM), and the Long Short…

In this paper, we present an infinite hierarchical non-parametric Bayesian model to extract the hidden factors over observed data, where the number of hidden factors for each layer is unknown and can be potentially infinite. Moreover, the number of layers can also be infinite. We construct the model structure that allo…

Researchers calculate Shannon entropy rates of hidden Markov processes efficiently.

CgNN uses network structure as IVs to estimate causal effects in networks.

The paper explores hidden torus symmetries in integrable systems and their stability.

We provide high probability finite sample complexity guarantees for hidden non-parametric structure learning of tree-shaped graphical models, whose hidden and observable nodes are discrete random variables with either finite or countable alphabets. We study a fundamental quantity called the (noisy) information threshol…

Study symmetric linear bandits with hidden symmetry, achieving improved regret bounds.

A serious problem in learning probabilistic models is the presence of hidden variables. These variables are not observed, yet interact with several of the observed variables. Detecting hidden variables poses two problems: determining the relations to other variables in the model and determining the number of states of …

Estimates effects of multiple interventions with hidden confounders using single-variable interventions.

Hidden regular variation defines a subfamily of distributions satisfying multivariate regular variation on $\mathbb{E} = [0, \infty]^d \backslash \{(0,0, ..., 0) \}$ and models another regular variation on the sub-cone $\mathbb{E}^{(2)} = \mathbb{E} \backslash \cup_{i=1}^d \mathbb{L}_i$, where $\mathbb{L}_i$ is the $i…

We solve a high-dimensional model where nonlinear autoencoders detect hidden structure missed by PCA.

New clustering method recovers hidden tree structure from data.

New insights into hidden minima in neural networks.

Effectively modelling hidden structures in a network is very practical but theoretically challenging. Existing relational models only involve very limited information, namely the binary directional link data, embedded in a network to learn hidden networking structures. There is other rich and meaningful information (e.…

A new method detects hidden driving forces in systems with multiple observables.

Hidden tree Markov models allow learning distributions for tree structured data while being interpretable as nondeterministic automata. We provide a concise summary of the main approaches in literature, focusing in particular on the causality assumptions introduced by the choice of a specific tree visit direction. We w…

We provide high-probability sample complexity guarantees for exact structure recovery and accurate predictive learning using noise-corrupted samples from an acyclic (tree-shaped) graphical model. The hidden variables follow a tree-structured Ising model distribution, whereas the observable variables are generated by a …

In this paper we propose a unified framework for structured prediction with latent variables which includes hidden conditional random fields and latent structured support vector machines as special cases. We describe a local entropy approximation for this general formulation using duality, and derive an efficient messa…

Group invariant and equivariant Multilayer Perceptrons (MLP), also known as Equivariant Networks, have achieved remarkable success in learning on a variety of data structures, such as sequences, images, sets, and graphs. Using tools from group theory, this paper proves the universality of a broad class of equivariant M…

Study spectral settings of generalized Laplacians on homogeneous spaces.

This paper compares HMC and RNN expressivity using SRT.

The paper offers simple, near-optimal algorithms for multi-group learning.

Some natural hidden symmetries in the Verma modules over the Virasoro algebra are constructed in terms of geometric quantization. Their differential geometric meaning is established and their expression via $q_R$-conformal symmetries in the Verma modules over the Lie algebra $sl(2,C)$ is found. The analysis and the unr…

We present an attention mechanism inspired from definition of screened Coulomb potential. This attention mechanism was used to interpret the Graph Attention (GAT) model layers and training dataset by using a flexible and scalable framework (CoulGAT) developed for this purpose. Using CoulGAT, a forest of plain and resne…

New method extracts hidden phases in binary mixtures using tubular tilings.

Generalizes bits back coding for time-series models with latent Markov structures.

Generative adversarial models are powerful tools to model structure in complex distributions for a variety of tasks. Current techniques for learning generative models require an access to samples which have high quality, and advanced generative models are applied to generate samples from noisy training data through amb…

This work speeds up fHMM analysis by tensor algebra.

The partially observable hidden Markov model is an extension of the hidden Markov Model in which the hidden state is conditioned on an independent Markov chain. This structure is motivated by the presence of discrete metadata, such as an event type, that may partially reveal the hidden state but itself emanates from a …

In this paper we present a model for the hidden Markovian bandit problem with linear rewards. As opposed to current work on Markovian bandits, we do not assume that the state is known to the decision maker before making the decision. Furthermore, we assume structural side information where the decision maker knows in a…

New findings on computational limits for estimating hidden structures.

Gradient Descent Ascent converges to von-Neumann solution in hidden zero-sum games.

This paper extends depth separation results to piece-wise oscillatory functions.

Structural RBM reduces parameters for image denoising and classification.

Proposes a method to estimate sparse Gaussian graphical models with hidden clustering structure.

In this paper, we introduce transformations of deep rectifier networks, enabling the conversion of deep rectifier networks into shallow rectifier networks. We subsequently prove that any rectifier net of any depth can be represented by a maximum of a number of functions that can be realized by a shallow network with a …

Study evaluates initialization strategies for infinite hidden Markov models.

To infer a multilayer representation of high-dimensional count vectors, we propose the Poisson gamma belief network (PGBN) that factorizes each of its layers into the product of a connection weight matrix and the nonnegative real hidden units of the next layer. The PGBN's hidden layers are jointly trained with an upwar…

Flood extent mapping plays a crucial role in disaster management and national water forecasting. Unfortunately, traditional classification methods are often hampered by the existence of noise, obstacles and heterogeneity in spectral features as well as implicit anisotropic spatial dependency across class labels. In thi…