New method uses almost orthonormal bases to prove low-degree lower bounds in complex statistical models.
problem Proving statistical-computational gaps in high-dimensional models with planted structures.
method Constructing an almost orthonormal polynomial basis under the planted distribution.
result Established new low-degree lower bounds for various complex models.
This paper introduces a new approach to finding knots and links with hidden symmetries using "hidden extensions", a class of hidden symmetries defined here. We exhibit a family of tangle complements in the ball whose boundaries have symmetries with hidden extensions, then we further extend these to hidden symmetries of…
RBM models reveal how hidden unit tail behavior affects pattern reconstruction.
problem Understanding how the tail behavior of hidden units in RBMs influences pattern reconstruction.
method Identified an effective energy function for RBMs and studied its local minima.
result The ability to reconstruct patterns depends on the tail behavior of the hidden unit prior distribution.
Expands Hidden Markov Model to include Markov chain observations.
problem Handling Markov chain observations in Hidden Markov Models.
method Developed Expectation-Maximization algorithm and Viterbi algorithm analogs.
result Estimates transition probabilities for hidden states and observations.
This paper reviews recent advances in Bayesian nonparametric techniques for constructing and performing inference in infinite hidden Markov models. We focus on variants of Bayesian nonparametric hidden Markov models that enhance a posteriori state-persistence in particular. This paper also introduces a new Bayesian non…
Supervised learning frequently boils down to determining hidden and bright parameters in a parameterized hypothesis space based on finite input-output samples. The hidden parameters determine the attributions of hidden predictors or the nonlinear mechanism of an estimator, while the bright parameters characterize how h…
Randomly chosen primary hidden units and derived secondary units reduce neural network complexity.
problem Large number of hidden units in neural networks.
method Introducing primary and secondary hidden units with random weights for primary units and derived weights for secondary units.
result Significant reduction in the number of hidden units without compromising accuracy.
New MBL hidden Born machine learns various tasks.
problem Learning from quantum many-body systems.
method MBL dynamics and hidden units for training.
result Enhanced trainability and stability in learning.
New insights into hidden minima in neural networks.
problem Identifying hidden minima in two-layer ReLU networks.
method Analyzing curves along which loss is minimized, focusing on eigenvalue contributions.
result Distinctive structural and symmetry properties of arcs emanating from hidden minima.
Paper tackles hidden game problem in AI alignment and language games.
problem Hidden game problem in AI alignment and language games.
method Developed a composition of regret minimization techniques to discover and exploit hidden structures.
result Achieved optimal external and swap regret bounds for rapid convergence to correlated equilibria.
Hidden regular variation is a sub-model of multivariate regular variation and facilitates accurate estimation of joint tail probabilities. We generalize the model of hidden regular variation to what we call hidden domain of attraction. We exhibit examples that illustrate the need for a more general model and discuss de…
We present a non-parametric Bayesian approach to structure learning with hidden causes. Previous Bayesian treatments of this problem define a prior over the number of hidden causes and use algorithms such as reversible jump Markov chain Monte Carlo to move between solutions. In contrast, we assume that the number of hi…
Hidden Markov Neural Networks balance adaptation and forgetting in time-series data.
problem Balancing adaptation to new data and forgetting outdated information in time-series forecasting.
method Modeling weights as hidden states of a Hidden Markov model, using a filtering algorithm for learning a variational approximation of the posterior distribution over weights, and employing sequential Bayes by Backprop with variational DropConnect for regularization.
result Achieves strong predictive performance and effective uncertainty quantification on various tasks.
The paper introduces FMCI and hybrid decoding for hidden Markov models.
problem Computing distributions and decoding hidden state sequences in HMMs.
method Finite Markov chain imbedding (FMCI) and hybrid decoding.
result Hybrid decoding improves performance over traditional methods.
This paper is concerned with the sparsification of the input-hidden weights of ELM (Extreme Learning Machine). For ordinary feedforward neural networks, the sparsification is usually done by introducing certain regularization technique into the learning process of the network. But this strategy can not be applied for E…
A serious problem in learning probabilistic models is the presence of hidden variables. These variables are not observed, yet interact with several of the observed variables. Detecting hidden variables poses two problems: determining the relations to other variables in the model and determining the number of states of …
The paper proves limitations on hyperbolic knot complements with hidden symmetries.
problem Identifying hyperbolic knot complements with hidden symmetries.
method Obstructions to infinitely many fillings producing knot complements with hidden symmetries.
result The figure-eight knot complement is the only hyperbolic knot complement with hidden symmetries.
This paper tackles hidden state inference for HMMs using particle filtering.
problem Inference for hidden states under HMMs is challenging due to unavailable true labels.
method Adaptive conformal inference framework using particle filtering.
result The framework produces prediction sets with specific aggregated coverage levels.
Extends causal inference to hidden mediators with proxies.
problem Identifying causal effects with hidden mediators and error-prone proxies.
method Established causal hidden mediation analysis and hidden front-door criterion.
result Identification of population intervention indirect effect possible with hidden mediators.
New method amplifies hidden structure in complex networks.
problem Difficulty in uncovering hidden structure in complex networks.
method Iterative weakening of dominant structure through randomization.
result Theoretical support for the effectiveness of structure amplification.
New findings on hidden symmetries in ReLU networks.
problem Understanding the redundancy and symmetries in ReLU network parameter space.
method Analyzing parameter settings and function classes for various network architectures.
result For certain network architectures, there are no hidden symmetries.
The hidden M-algebra is integrated into a super-Lie group, allowing for compactification of extra dimensions.
problem Integrating the hidden M-algebra into a super-Lie group to model super-exceptional spacetimes.
method Left-invariant extension of the decomposed M-theory 3-form, providing a computer-checked re-derivation and streamlined conception of super-Lie groups.
result Lattice subgroups of the hidden M-group allow toroidal compactification of hidden dimensions, akin to topological T-duality.
Formula found for neural network error with fixed weights.
problem Understanding error in neural networks with fixed weights.
method Provided an explicit formula for approximation error.
result Explicit formula for neural network error with fixed weights.
Multilabel classification is an important problem in a wide range of domains such as text categorization and music annotation. In this paper, we present a probabilistic model, Multilabel Logistic Regression with Hidden variables (MLRH), which extends the standard logistic regression by introducing hidden variables. Hid…
New framework identifies hidden risks and optionality in American options.
problem Underestimation of flexibility and convexity in early-exercise features.
method Introducing stochasticity into underlying determinants to quantify hidden risks and optionality.
result Remedies conventional pricing systems that underestimate optionality.
This work speeds up fHMM analysis by tensor algebra.
problem Scalability issues in analyzing factorial hidden Markov models.
method Tensorized algorithms and scalable filtering methods.
result Significant improvement in computational performance.
Hidden regular variation defines a subfamily of distributions satisfying multivariate regular variation on E=[0,∞]d\{(0,0,...,0)} and models another regular variation on the sub-cone E(2)=E\∪i=1dLi, where Li is the $i…
Study examines dependence properties of Bayesian neural network units in finite-width networks.
problem Understanding dependence properties of hidden units in practical finite-width Bayesian neural networks.
method Theoretical analysis and empirical evaluation of depth and width impacts.
result Hidden units in finite-width Bayesian neural networks are dependent, contrary to the infinite-width limit assumption.
Modified asymmetric hidden Markov models for time series with autoregressive components.
problem Dynamic relationships between variables in time series data.
method Introducing an asymmetric autoregressive component to recent asymmetric hidden Markov models.
result The model can choose the optimal autoregressive order for better likelihood.
Study on hidden units in finite Bayesian neural networks and their tail properties.
problem Understanding the behavior of hidden units in finite Bayesian neural networks.
method Introduced a generalized Weibull-tail property to describe hidden units tails.
result Unit priors become heavier-tailed going deeper, providing insights into finite Bayesian neural networks.
A novel RNN model with shuffled hidden states.
problem Improving recurrent neural network performance.
method Permuting hidden states and applying learned functions.
result Competitive results compared to literature baselines.
Detects anomalies in multiple processes using hidden Markov models.
problem Detecting an anomalous process among many with hidden states.
method Sequential search strategy using ADHM algorithm.
result ADHM algorithm effectively leverages temporal correlations.
New method accounts for hidden context in preference learning for RLHF models.
problem Incomplete data with hidden context affects RLHF model outcomes.
method Distributional Preference Learning (DPL) methods estimate hidden context distributions.
result DPL methods reduce RLHF vulnerabilities by accounting for hidden context.
Detect hidden confounding in observational data using multiple environments.
problem Detect hidden confounding in observational data.
method Theoretical framework and simulation studies to test for hidden confounding.
result The proposed procedure correctly predicts hidden confounding, especially when bias is large.
New methods for estimating causal effects in hidden variable DAGs.
problem Estimating causal effects in models with hidden variables.
method Influence function based estimators for causal effects in hidden variable DAGs.
result Achieves semiparametric efficiency bounds for identifiable effects.
A neural network with a single hidden layer can't represent certain multivariable functions.
problem Representing certain multivariable functions with a neural network having only one hidden layer.
method Developed a continuum version of a one-hidden-layer neural network with ReLU activation, and proved constraints on its parameters and second derivative.
result Existence of a smooth binary function that cannot be precisely represented by any such neural network.
We define a Hidden Markov Model (HMM) in which each hidden state has time-dependent activity levels that drive transitions and emissions, and show how to estimate its parameters. Our construction is motivated by the problem of inferring human mobility on sub-daily time scales from, for example, mobile phone …
A "hidden symmetry" of a Riemannian manifold M is an isometry of a d-sheeted, 1<d<\infty, Riemannian cover of M which is not the lift of any isometry. In this paper we characterize the locally symmetric metric(s) on a closed, arithmetic manifold as the unique metric with infinitely many hidden symmetries.
Paper analyzes coexisting hidden and self-excited attractors in an economic system.
problem Existence of coexisting hidden and self-excited attractors in economic systems.
method Integer and fractional order analysis of an economic system.
result Integer-order system exhibits multiple combinations of coexisting hidden and self-excited attractors.
The hidden tail of empirical distributions is analyzed using extreme value theory.
problem Understanding the bias between in-sample mean and true statistical mean for large n. method Extreme value theory applied to empirical distributions and their moments.
result The hidden moment of order 0 for power law distributions follows an exponential distribution with expectation 1/n. Language models based on deep neural networks and traditional stochastic modelling have become both highly functional and effective in recent times. In this work, a general survey into the two types of language modelling is conducted. We investigate the effectiveness of the Hidden Markov Model (HMM), and the Long Short…
Group invariant and equivariant Multilayer Perceptrons (MLP), also known as Equivariant Networks, have achieved remarkable success in learning on a variety of data structures, such as sequences, images, sets, and graphs. Using tools from group theory, this paper proves the universality of a broad class of equivariant M…
Since learning is typically very slow in Boltzmann machines, there is a need to restrict connections within hidden layers. However, the resulting states of hidden units exhibit statistical dependencies. Based on this observation, we propose using l1/l2 regularization upon the activation possibilities of hidden unit…
Neural networks learn more efficiently with hidden factorial structures.
problem Challenges in high-dimensional statistical learning.
method Controlled experimental framework to test neural networks' ability to exploit hidden factorial structures.
result Neural networks can leverage hidden factorial structures to learn discrete distributions more efficiently.
Deep networks have achieved impressive results across a variety of important tasks. However a known weakness is a failure to perform well when evaluated on data which differ from the training distribution, even if these differences are very small, as is the case with adversarial examples. We propose Fortified Networks,…
The partially observable hidden Markov model is an extension of the hidden Markov Model in which the hidden state is conditioned on an independent Markov chain. This structure is motivated by the presence of discrete metadata, such as an event type, that may partially reveal the hidden state but itself emanates from a …
NoMoPy models noise as HMM/FHMM in Python.
problem Modeling noise in data.
method Approximate and exact EM algorithms, cross-validation, confidence region estimation.
result Validated on example problems.
This paper compares HMM and LSTM for time series forecasting.
problem Comparing HMM and LSTM for time series forecasting.
method Discretization techniques and various training methods.
result HMM can outperform LSTM with limited labeled data.