arXiv research

This paper reviews recent advances in Bayesian nonparametric techniques for constructing and performing inference in infinite hidden Markov models. We focus on variants of Bayesian nonparametric hidden Markov models that enhance a posteriori state-persistence in particular. This paper also introduces a new Bayesian non…

Two-hidden-layer networks can approximate any continuous function.

A new method uses triplet embeddings to improve human annotation for hidden constructs.

We define a Hidden Markov Model (HMM) in which each hidden state has time-dependent $\textit{activity levels}$ that drive transitions and emissions, and show how to estimate its parameters. Our construction is motivated by the problem of inferring human mobility on sub-daily time scales from, for example, mobile phone …

We generalize the symmetry superalgebras of isometries and geometric Killing spinors on a manifold to include all the hidden symmetries of the manifold generated by Killing spinors in all dimensions. We show that bilinears of geometric Killing spinors produce special Killing-Yano and special conformal Killing-Yano form…

A neural network with a single hidden layer can't represent certain multivariable functions.

Study spectral settings of generalized Laplacians on homogeneous spaces.

Some natural hidden symmetries in the Verma modules over the Virasoro algebra are constructed in terms of geometric quantization. Their differential geometric meaning is established and their expression via $q_R$-conformal symmetries in the Verma modules over the Lie algebra $sl(2,C)$ is found. The analysis and the unr…

A stochastic model with hidden discrete Markov processes is constructed to understand the behavior of debtors.

Investor selects portfolios based on news attention in a hidden Markov model.

This paper tackles hidden state inference for HMMs using particle filtering.

Combining Bayesian nonparametrics and a forward model selection strategy, we construct parsimonious Bayesian deep networks (PBDNs) that infer capacity-regularized network architectures from the data and require neither cross-validation nor fine-tuning when training the model. One of the two essential components of a PB…

This paper continues our exploration of homology cobordism of 3-manifolds using our recent results on Cheeger-Gromov rho-invariants associated to amenable representations. We introduce a new type of torsion in 3-manifold groups we call hidden torsion, and an algebraic approximation we call local hidden torsion. We cons…

This work speeds up fHMM analysis by tensor algebra.

The objective of this paper is to investigate how noisy and incomplete observations can be integrated in the process of building a reduced-order model. This problematic arises in many scientific domains where there exists a need for accurate low-order descriptions of highly-complex phenomena, which can not be directly …

The hidden M-algebra is integrated into a super-Lie group, allowing for compactification of extra dimensions.

This paper examines number theoretic and topological properties of fully augmented pretzel link complements. In particular, we determine exactly when these link complements are arithmetic and exactly which are commensurable with one another. We show these link complements realize infinitely many CM-fields as invariant …

Flood extent mapping plays a crucial role in disaster management and national water forecasting. Unfortunately, traditional classification methods are often hampered by the existence of noise, obstacles and heterogeneity in spectral features as well as implicit anisotropic spatial dependency across class labels. In thi…

The Viterbi process can be extended indefinitely in a pairwise Markov model.

We present a new method for nonlinear prediction of discrete random sequences under minimal structural assumptions. We give a mathematical construction for optimal predictors of such processes, in the form of hidden Markov models. We then describe an algorithm, CSSR (Causal-State Splitting Reconstruction), which approx…

In this paper, we present an infinite hierarchical non-parametric Bayesian model to extract the hidden factors over observed data, where the number of hidden factors for each layer is unknown and can be potentially infinite. Moreover, the number of layers can also be infinite. We construct the model structure that allo…

The paper studies symmetries and hidden symmetries of twisted knot complements.

We briefly review the hierarchy for the hyper-Kähler equations and define a notion of symmetry for solutions of this hierarchy. A four-dimensional hyper-Kähler metric admits a hidden symmetry if it embeds into a hierarchy with a symmetry. It is shown that a hyper-Kähler metric admits a hidden symmetry if it admits a ce…

The concept of SCN offers a fast framework with universal approximation guarantee for lifelong learning of non-stationary data streams. Its adaptive scope selection property enables for proper random generation of hidden unit parameters advancing conventional randomized approaches constrained with a fixed scope of rand…

Study on neural networks' sample complexity with one hidden layer.

The paper constructs minimizers for deep learning networks and analyzes their geometric structure.

Ensemble unsupervised anomaly detection using IRT for hidden ground truth.

This paper presents a constructive algorithm that achieves successful one-shot learning of hidden spike-patterns in a competitive detection task. It has previously been shown (Masquelier et al., 2008) that spike-timing-dependent plasticity (STDP) and lateral inhibition can result in neurons competitively tuned to repea…

Hidden Markov Chains and Linear-chain CRFs are equivalent.

In this paper we investigate the family of functions representable by deep neural networks (DNN) with rectified linear units (ReLU). We give an algorithm to train a ReLU DNN with one hidden layer to *global optimality* with runtime polynomial in the data size albeit exponential in the input dimension. Further, we impro…

In this paper, we explore different ways to extend a recurrent neural network (RNN) to a \textit{deep} RNN. We start by arguing that the concept of depth in an RNN is not as clear as it is in feedforward neural networks. By carefully analyzing and understanding the architecture of an RNN, however, we find three points …

Graph Ricci flow reveals hidden hierarchies in stock market correlations.

This paper extends stable blanket theory to models with hidden variables and causal cycles.

Study optimal adjustment sets for causal policies with hidden variables.

Introduces HMM to model data structure in neural networks.

There is much interest in the Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM) as a natural Bayesian nonparametric extension of the ubiquitous Hidden Markov Model for learning from sequential and time-series data. However, in many settings the HDP-HMM's strict Markovian constraints are undesirable, particul…

We present a variation of the Autoencoder (AE) that explicitly maximizes the mutual information between the input data and the hidden representation. The proposed model, the InfoMax Autoencoder (IMAE), by construction is able to learn a robust representation and good prototypes of the data. IMAE is compared both theore…

Neural networks with random hidden nodes have gained increasing interest from researchers and practical applications. This is due to their unique features such as very fast training and universal approximation property. In these networks the weights and biases of hidden nodes determining the nonlinear feature mapping a…

Study tail risk aggregation under dependence uncertainty.

We study finite sample expressivity, i.e., memorization power of ReLU networks. Recent results require $N$ hidden nodes to memorize/interpolate arbitrary $N$ data points. In contrast, by exploiting depth, we show that 3-layer ReLU networks with $Ω(\sqrt{N})$ hidden nodes can perfectly memorize most datasets with $N$ po…

Recurrent neural networks (RNNs) are notoriously difficult to train. When the eigenvalues of the hidden to hidden weight matrix deviate from absolute value 1, optimization becomes difficult due to the well studied issue of vanishing and exploding gradients, especially when trying to learn long-term dependencies. To cir…

Improved online learning for hidden-convex losses achieves optimal regret.

Lipschitz RNNs improve stability and performance in various tasks.

Traditionally, deep learning algorithms update the network weights whereas the network architecture is chosen manually, using a process of trial and error. In this work, we propose two novel approaches that automatically update the network structure while also learning its weights. The novelty of our approach lies in o…

This paper introduces a new approach to finding knots and links with hidden symmetries using "hidden extensions", a class of hidden symmetries defined here. We exhibit a family of tangle complements in the ball whose boundaries have symmetries with hidden extensions, then we further extend these to hidden symmetries of…

We solve a high-dimensional model where nonlinear autoencoders detect hidden structure missed by PCA.

Consider a finite connected graph possibly with multiple edges and loops. In discrete geometric analysis, Kotani and Sunada constructed the crystal associated to the graph as a standard realization of the maximal abelian covering of the graph. As an application of what the author showed in an earlier paper with Seshadr…

Shallow supervised 1-hidden layer neural networks have a number of favorable properties that make them easier to interpret, analyze, and optimize than their deep counterparts, but lack their representational power. Here we use 1-hidden layer learning problems to sequentially build deep networks layer by layer, which ca…