As deep neural networks continue to revolutionize various application domains, there is increasing interest in making these powerful models more understandable and interpretable, and narrowing down the causes of good and bad predictions. We focus on recurrent neural networks (RNNs), state of the art models in speech re…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “hybrid HMM”
Language models based on deep neural networks and traditional stochastic modelling have become both highly functional and effective in recent times. In this work, a general survey into the two types of language modelling is conducted. We investigate the effectiveness of the Hidden Markov Model (HMM), and the Long Short…
Hidden Markov Model (HMM) combined with Gaussian Process (GP) emission can be effectively used to estimate the hidden state with a sequence of complex input-output relational observations. Especially when the spectral mixture (SM) kernel is used for GP emission, we call this model as a hybrid HMM-GPSM. This model can e…
As deep neural networks continue to revolutionize various application domains, there is increasing interest in making these powerful models more understandable and interpretable, and narrowing down the causes of good and bad predictions. We focus on recurrent neural networks, state of the art models in speech recogniti…
Hybrid model improves traffic flow prediction accuracy.
problem Predicting traffic flow with high accuracy in short-term future.
method A hybrid model combining hidden Markov model and LSTM.
result Significant performance gains over conventional methods.
Improved neural speaker embeddings enhance ASR performance.
problem Few studies have explored neural speaker embeddings for ASR.
method Integrating improved neural speaker embeddings into a conformer-based hybrid HMM ASR system.
result Improved neural embeddings achieve on-par performance with i-vectors.
Objective: A clinical decision support tool that automatically interprets EEGs can reduce time to diagnosis and enhance real-time applications such as ICU monitoring. Clinicians have indicated that a sensitivity of 95% with a specificity below 5% was the minimum requirement for clinical acceptance. We propose a highper…
Hybrid model improves synthetic equity data generation.
problem Generating realistic synthetic financial time series.
method Discretized excess growth rates into states with Poisson jumps, estimating parameters directly.
result Framework achieved high pass rates for distributional and volatility clustering tests.
The paper introduces FMCI and hybrid decoding for hidden Markov models.
problem Computing distributions and decoding hidden state sequences in HMMs.
method Finite Markov chain imbedding (FMCI) and hybrid decoding.
result Hybrid decoding improves performance over traditional methods.
GMM-HMMs improve malware classification compared to discrete HMMs.
problem Improving malware classification accuracy using GMM-HMM models.
method Used GMM-HMM models for opcode and entropy-based features comparison with discrete HMMs.
result GMM-HMMs generally improve significantly on entropy-based features classification results.
Develops a more flexible HDP-HMM for temporal data segmentation.
problem Limited expressiveness of sticky HDP-HMM due to stationary self-persistence probability.
method Introduces recurrent sticky HDP-HMM with a novel Gibbs sampling strategy.
result RS-HDP-HMM outperforms other models in segmentation tasks.
A new model separates persistence and transition priors in HDP-HMM.
problem Limitation of sticky HDP-HMM in expressing different persistence strengths.
method Developed a disentangled sticky HDP-HMM (DS-HDP-HMM) with novel Gibbs sampling algorithms.
result DS-HDP-HMM outperforms sticky HDP-HMM and HDP-HMM on synthetic and real data.
We examine the effect of the Group Lasso (gLasso) regularizer in selecting the salient nodes of Deep Neural Network (DNN) hidden layers by applying a DNN-HMM hybrid speech recognizer to TED Talks speech data. We test two types of gLasso regularization, one for outgoing weight vectors and another for incoming weight vec…
Speech Translation has always been about giving source text or audio input and waiting for system to give translated output in desired form. In this paper, we present the Acoustic Dialect Decoder (ADD) - a voice to voice ear-piece translation device. We introduce and survey the recent advances made in the field of Spee…
Motivated by the unceasing interest in hidden Markov models (HMMs), this paper re-examines hidden path inference in these models, using primarily a risk-based framework. While the most common maximum a posteriori (MAP), or Viterbi, path estimator and the minimum error, or Posterior Decoder (PD), have long been around, …
HMRNN combines HMMs and neural networks for Alzheimer's disease forecasting.
problem Improving disease progression modeling with hidden states not fully known.
method Developed HMRNN combining HMMs and recurrent neural networks.
result HMRNN improves disease forecasting and offers novel clinical interpretation.
Convolutional Neural Networks (CNNs) are effective models for reducing spectral variations and modeling spectral correlations in acoustic features for automatic speech recognition (ASR). Hybrid speech recognition systems incorporating CNNs with Hidden Markov Models/Gaussian Mixture Models (HMMs/GMMs) have achieved the …
We study the problem of learning overcomplete HMMs---those that have many hidden states but a small output alphabet. Despite having significant practical importance, such HMMs are poorly understood with no known positive or negative results for efficient learning. In this paper, we present several new results---both po…
The paper uses HMM and LSTM for stock market trend analysis.
problem Predicting stock market trends using machine learning.
method Apply Hidden Markov Model and Long Short Term Memory to stock market data.
result The combination of GMM-HMM+LSTM and XGB-HMM+LSTM outperformed other models.
The fundamental frequency (F0) contour of speech is a key aspect to represent speech prosody that finds use in speech and spoken language analysis such as voice conversion and speech synthesis as well as speaker and language identification. This work proposes new methods to estimate the F0 contour of speech using deep …
LC-CRFs are equivalent to HMMs, and MPM/MAP classifiers can be reformulated as CRFs.
problem Comparing and reformulating HMMs and CRFs.
method Demonstrating equivalence and reformulation of classifiers.
result LC-CRFs are equivalent to HMMs, and MPM/MAP classifiers can be reformulated as CRFs.
The hidden Markov model (HMM) is a generative model that treats sequential data under the assumption that each observation is conditioned on the state of a discrete hidden variable that evolves in time as a Markov chain. In this paper, we derive a novel algorithm to cluster HMMs through their probability distributions.…
When learning a hidden Markov model (HMM), sequen- tial observations can often be complemented by real-valued summary response variables generated from the path of hid- den states. Such settings arise in numerous domains, includ- ing many applications in biology, like motif discovery and genome annotation. In this pape…
New method estimates HMM hidden states efficiently.
problem Inaccurate posterior predictive distribution in HMMs.
method Autoregressive-flow for estimating hidden states.
result Estimates comparable to SMC algorithm.
Paper tackles HMM learning with unknown missing observation locations.
problem Learning HMMs with missing data locations unknown.
method Proposes reconstruction algorithms without structural assumptions.
result Can reconstruct process dynamics as if missing locations were known.
Improved speech recognition with language model integration in sequence-to-sequence models.
problem Improving word error rate in speech recognition models.
method Log-linear combination of acoustic and language models with per-token renormalization.
result The proposed method shows good improvements over standard model combination on Librispeech system.
Paper tackles efficient HMM learning with conditional samples.
problem Cryptographic hardness in learning HMMs from i.i.d. samples.
method Interactive access model, polynomial-time algorithms for conditional probabilities and latent low rank structures.
result Efficient algorithms for HMM learning in both exact and approximate conditional settings.
Paper develops a deforestation detection system using optical and SAR data.
problem Detecting tree-loss in dense forests using satellite data.
method Combines optical and SAR data, uses KL expansion for anomaly detection, and Hidden Markov Model for classification.
result Hybrid method achieves high accuracy and robustness in sparse optical data.
Suppose that we are given a time series where consecutive samples are believed to come from a probabilistic source, that the source changes from time to time and that the total number of sources is fixed. Our objective is to estimate the distributions of the sources. A standard approach to this problem is to model the …
Hidden Markov Models (HMMs) are a ubiquitous tool to model time series data, and have been widely used in two main tasks of Automatic Music Transcription (AMT): note segmentation, i.e. identifying the played notes after a multi-pitch estimation, and sequential post-processing, i.e. correcting note segmentation using tr…
The hidden Markov model (HMM) is a widely-used generative model that copes with sequential data, assuming that each observation is conditioned on the state of a hidden Markov chain. In this paper, we derive a novel algorithm to cluster HMMs based on the hierarchical EM (HEM) algorithm. The proposed algorithm i) cluster…
As one of Bayesian analysis tools, Hidden Markov Model (HMM) has been used to in extensive applications. Most HMMs are solved by Baum-Welch algorithm (BWHMM) to predict the model parameters, which is difficult to find global optimal solutions. This paper proposes an optimized Hidden Markov Model with Particle Swarm Opt…
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuningcs.LG
Analysis of pretrained models' effectiveness in downstream tasks.
problem Understanding why pretrained models perform well in NLP tasks.
method Analyzed head and prompt tuning approaches using latent variable models.
result Prompt tuning provides stronger guarantees than head tuning.
We propose a framework, named Aggregated Wasserstein, for computing a dissimilarity measure or distance between two Hidden Markov Models with state conditional distributions being Gaussian. For such HMMs, the marginal distribution at any time spot follows a Gaussian mixture distribution, a fact exploited to softly matc…
This paper addresses the issue of model selection for hidden Markov models (HMMs). We generalize factorized asymptotic Bayesian inference (FAB), which has been recently developed for model selection on independent hidden variables (i.e., mixture models), for time-dependent hidden variables. As with FAB in mixture model…
We consider the smoothing probabilities of hidden Markov model (HMM). We show that under fairly general conditions for HMM, the exponential forgetting still holds, and the smoothing probabilities can be well approximated with the ones of double sided HMM. This makes it possible to use ergodic theorems. As an applicatio…
EMHMM Simulation Studystat.ML
Eye Movement analysis with Hidden Markov Models (EMHMM) is a method for modeling eye fixation sequences using hidden Markov models (HMMs). In this report, we run a simulation study to investigate the estimation error for learning HMMs with variational Bayesian inference, with respect to the number of sequences and the …
We propose a framework, named Aggregated Wasserstein, for computing a dissimilarity measure or distance between two Hidden Markov Models with state conditional distributions being Gaussian. For such HMMs, the marginal distribution at any time position follows a Gaussian mixture distribution, a fact exploited to softly …
In this paper, we explore the effectiveness of dynamic analysis techniques for identifying malware, using Hidden Markov Models (HMMs) and Profile Hidden Markov Models (PHMMs), both trained on sequences of API calls. We contrast our results to static analysis using HMMs trained on sequences of opcodes, and show that dyn…
We consider learning parameters of Binomial Hidden Markov Models, which may be used to model DNA methylation data. The standard algorithm for the problem is EM, which is computationally expensive for sequences of the scale of the mammalian genome. Recently developed spectral algorithms can learn parameters of latent va…
Efficiently models tree-like data with coupled branches using HMMs.
problem Modeling sequential data with coupled branches in biological systems.
method Developed a dynamic programming algorithm for tree-based HMMs with coupled branches.
result Efficiently solves likelihood, decoding, and parameter learning problems for tree-based HMMs with coupled branches.
New algorithm for aggregate inference in HMMs with continuous observations.
problem Inference in large populations with indistinguishable individuals and continuous measurements.
method Continuous observation collective forward-backward algorithm extending existing discrete case algorithm.
result Efficacy demonstrated through numerical experiments.
Bayesian HMM for protein alignment state estimation.
problem Estimating the MAP state sequence for a Bayesian HMM.
method Training data with Dirichlet priors, iterative algorithms for MAP path estimation.
result Bayesian setup outperforms frequentist estimation in protein alignment.
Hidden Markov Models (HMM) have been used for several years in many time series analysis or pattern recognitions tasks. HMM are often trained by means of the Baum-Welch algorithm which can be seen as a special variant of an expectation maximization (EM) algorithm. Second-order training techniques such as Variational Ba…
Since the early days of digital communication, Hidden Markov Models (HMMs) have now been routinely used in speech recognition, processing of natural languages, images, and in bioinformatics. An HMM assumes observations to be conditionally independent given an "explanotary" Markov proc…
The technological applications of hidden Markov models have been extremely diverse and successful, including natural language processing, gesture recognition, gene sequencing, and Kalman filtering of physical measurements. HMMs are highly non-linear statistical models, and just as linear models are amenable to linear a…
Exploiting Low-dimensional Structures to Enhance DNN Based Acoustic Modeling in Speech Recognitioncs.CL
We propose to model the acoustic space of deep neural network (DNN) class-conditional posterior probabilities as a union of low-dimensional subspaces. To that end, the training posteriors are used for dictionary learning and sparse coding. Sparse representation of the test posteriors using this dictionary enables proje…
This paper demonstrates two novel methods to estimate the global SNR of speech signals. In both methods, Deep Neural Network-Hidden Markov Model (DNN-HMM) acoustic model used in speech recognition systems is leveraged for the additional task of SNR estimation. In the first method, the entropy of the DNN-HMM output is c…