Supervised learning frequently boils down to determining hidden and bright parameters in a parameterized hypothesis space based on finite input-output samples. The hidden parameters determine the attributions of hidden predictors or the nonlinear mechanism of an estimator, while the bright parameters characterize how h…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “hidden parameters”
New method models unknown systems with hidden parameters using neural networks.
problem Modeling unknown dynamical systems with hidden parameters.
method Training a deep neural network (DNN) model using trajectory data of the unknown system.
result DNN model accurately predicts unknown dynamical systems with new initial conditions.
New findings on hidden symmetries in ReLU networks.
problem Understanding the redundancy and symmetries in ReLU network parameter space.
method Analyzing parameter settings and function classes for various network architectures.
result For certain network architectures, there are no hidden symmetries.
Study on the limits of learning HMM parameters under various conditions.
problem Understanding the conditions under which hidden Markov model parameters can be learned.
method Nonasymptotic minimax upper and lower bounds, thresholds analysis.
result Nonasymptotic minimax bounds match up to constants, showing learnable thresholds.
A neural network with a single hidden layer can't represent certain multivariable functions.
problem Representing certain multivariable functions with a neural network having only one hidden layer.
method Developed a continuum version of a one-hidden-layer neural network with ReLU activation, and proved constraints on its parameters and second derivative.
result Existence of a smooth binary function that cannot be precisely represented by any such neural network.
We extend the Bayesian Information Criterion (BIC), an asymptotic approximation for the marginal likelihood, to Bayesian networks with hidden variables. This approximation can be used to select models given large samples of data. The standard BIC as well as our extension punishes the complexity of a model according to …
NoMoPy models noise as HMM/FHMM in Python.
problem Modeling noise in data.
method Approximate and exact EM algorithms, cross-validation, confidence region estimation.
result Validated on example problems.
This is a technical report which explores the estimation methodologies on hyper-parameters in Markov Random Field and Gaussian Hidden Markov Random Field. In first section, we briefly investigate a theoretical framework on Metropolis-Hastings algorithm. Next, by using MH algorithm, we simulate the data from Ising model…
Estimates dependent parameters using Markovian dependence with shrinkage.
problem Estimating dependent parameters from a hidden Markov model.
method Developed a novel non-parametric shrinkage algorithm combining Tweedie-based ideas and efficient state estimation.
result Superior performance compared to non-shrinkage methods in hidden Markov models.
Modified asymmetric hidden Markov models for time series with autoregressive components.
problem Dynamic relationships between variables in time series data.
method Introducing an asymmetric autoregressive component to recent asymmetric hidden Markov models.
result The model can choose the optimal autoregressive order for better likelihood.
A new approach for signal parametrization, which consists of a specific regression model incorporating a discrete hidden logistic process, is proposed. The model parameters are estimated by the maximum likelihood method performed by a dedicated Expectation Maximization (EM) algorithm. The parameters of the hidden logis…
We develop a new approach to learn the parameters of regression models with hidden variables. In a nutshell, we estimate the gradient of the regression function at a set of random points, and cluster the estimated gradients. The centers of the clusters are used as estimates for the parameters of hidden units. We justif…
We define a Hidden Markov Model (HMM) in which each hidden state has time-dependent that drive transitions and emissions, and show how to estimate its parameters. Our construction is motivated by the problem of inferring human mobility on sub-daily time scales from, for example, mobile phone …
Direct approach for handling contextual bandits with latent state dynamics.
problem Handling contextual bandits with latent state dynamics, especially when rewards depend on posterior probabilities of hidden states.
method Direct reduction to standard linear contextual bandits, extended analysis of HMM parameters, periodic update of reward-model parameters.
result Periodic update of reward-model parameters allows handling complex dependencies in hidden states.
HMRNN combines HMMs and neural networks for Alzheimer's disease forecasting.
problem Improving disease progression modeling with hidden states not fully known.
method Developed HMRNN combining HMMs and recurrent neural networks.
result HMRNN improves disease forecasting and offers novel clinical interpretation.
We introduce a new formulation of the Hidden Parameter Markov Decision Process (HiP-MDP), a framework for modeling families of related tasks using low-dimensional latent embeddings. Our new framework correctly models the joint uncertainty in the latent parameters and the state space. We also replace the original Gaussi…
The paper examines utility maximization in markets with hidden Gaussian drift, finding restrictions on model parameters.
problem Utility maximization problems in markets with hidden Gaussian drift mean-reverting processes.
method Derives sufficient conditions for bounded maximum expected utility of terminal wealth for models with full and partial information.
result Restrictions on model parameters for bounded maximum expected utility.
The generic identification problem is to decide whether a stochastic process is a hidden Markov process and if yes to infer its parameters for all but a subset of parametrizations that form a lower-dimensional subvariety in parameter space. Partial answers so far available depend on extra assumptions on the pro…
A new approach for feature extraction from time series is proposed in this paper. This approach consists of a specific regression model incorporating a discrete hidden logistic process. The model parameters are estimated by the maximum likelihood method performed by a dedicated Expectation Maximization (EM) algorithm. …
We consider a binary sequence generated by thresholding a hidden continuous sequence. The hidden variables are assumed to have a compound symmetry covariance structure with a single parameter characterizing the common correlation. We study the parameter estimation problem under such one-parameter models. We demonstrate…
RBMs model binary interactions with hidden node activation effects.
problem Understanding how RBM hidden node activation affects binary variable distributions.
method Investigated RBM marginal distributions with different hidden node activation functions.
result Found exact expressions for RBM marginals as interacting binary variables.
Paper shows how to infer hidden states in neural networks analytically.
problem Intractability of Bayesian inference for neural networks.
method Leverage tractable approximate Gaussian inference (TAGI) for hidden states inference.
result Demonstrates inference of hidden states through constraints for various applications.
New method improves parameter estimation in complex stochastic models.
problem Parameter calibration in stochastic models with unavailable analytical likelihood.
method Gradient-based simulated parameter estimation with multi-time scale stochastic approximation.
result Enhanced estimation accuracy and reduced computational costs.
Deep nonlinear models pose a challenge for fitting parameters due to lack of knowledge of the hidden layer and the potentially non-affine relation of the initial and observed layers. In the present work we investigate the use of information theoretic measures such as mutual information and Kullback-Leibler (KL) diverge…
This work concerns testing the number of parameters in one hidden layer multilayer perceptron (MLP). For this purpose we assume that we have identifiable models, up to a finite group of transformations on the weights, this is for example the case when the number of hidden units is know. In this framework, we show that …
Deep Stacked Stochastic Configuration Networks for Lifelong Learning of Non-Stationary Data Streamscs.LG
The concept of SCN offers a fast framework with universal approximation guarantee for lifelong learning of non-stationary data streams. Its adaptive scope selection property enables for proper random generation of hidden unit parameters advancing conventional randomized approaches constrained with a fixed scope of rand…
Traditionally, deep learning algorithms update the network weights whereas the network architecture is chosen manually, using a process of trial and error. In this work, we propose two novel approaches that automatically update the network structure while also learning its weights. The novelty of our approach lies in o…
The paper tackles estimation of hidden state LTI systems of unknown order.
problem Estimation of Markov parameters and minimal realization of unknown order LTI systems.
method Hankel penalized least square estimator, Ho-Kalman algorithm, and a combined algorithm.
result Statistical guarantees for estimation error, rank recovery, and sample complexity.
Estimating hidden processes from non-linear noisy observations is particularly difficult when the parameters of these processes are not known. This paper adopts a machine learning approach to devise variational Bayesian inference for such scenarios. In particular, a random process generated by the autoregressive moving…
The technological applications of hidden Markov models have been extremely diverse and successful, including natural language processing, gesture recognition, gene sequencing, and Kalman filtering of physical measurements. HMMs are highly non-linear statistical models, and just as linear models are amenable to linear a…
New method estimates HMM hidden states efficiently.
problem Inaccurate posterior predictive distribution in HMMs.
method Autoregressive-flow for estimating hidden states.
result Estimates comparable to SMC algorithm.
In this work, a method of random parameters generation for randomized learning of a single-hidden-layer feedforward neural network is proposed. The method firstly, randomly selects the slope angles of the hidden neurons activation functions from an interval adjusted to the target function, then randomly rotates the act…
New algorithm learns HMM parameters on Riemannian manifolds.
problem Learning hidden Markov models on non-Euclidean spaces.
method Geometric method of moments algorithm for Riemannian manifolds.
result Significantly improved speed and accuracy compared to existing methods.
In this paper we propose a method to model speaker and session variability and able to generate likelihood ratios using neural networks in an end-to-end phrase dependent speaker verification system. As in Joint Factor Analysis, the model uses tied hidden variables to model speaker and session variability and a MAP adap…
Structural RBM reduces parameters for image denoising and classification.
problem High parameter count in RBMs limits their applicability to large datasets.
method Introduces SRBM with constrained connections to reduce parameters.
result SRBM achieves better performance and faster training than vanilla RBM.
Hidden Markov Models analyze mobile health data to identify APNS states.
problem Subjective self-report measures of APNS lead to errors and biases.
method Exploratory hidden Markov factor models and Stabilized Expectation-Maximization algorithm.
result Identified homogeneous APNS states and dynamic transitions.
Paper derives constraints for Bayesian Knowledge Tracing parameters.
problem Issues with EM algorithm in BKT parameter estimation.
method From first principles, derives constraints on BKT parameter space.
result Novel algorithm respects derived constraints for parameter estimation.
The paper introduces FMCI and hybrid decoding for hidden Markov models.
problem Computing distributions and decoding hidden state sequences in HMMs.
method Finite Markov chain imbedding (FMCI) and hybrid decoding.
result Hybrid decoding improves performance over traditional methods.
There is broad interest in creating RL agents that can solve many (related) tasks and adapt to new tasks and environments after initial training. Model-based RL leverages learned surrogate models that describe dynamics and rewards of individual tasks, such that planning in a good surrogate can lead to good control of t…
George Cybenko's landmark 1989 paper showed that there exists a feedforward neural network, with exactly one hidden layer (and a finite number of neurons), that can arbitrarily approximate a given continuous function on the unit hypercube. The paper did not address how to find the weight/parameters of such a networ…
Develops a model for analyzing cryptocurrency returns focusing on extreme values.
problem Analyzing extreme returns in cryptocurrency time series.
method Linear expectile hidden Markov model with time-dependent coefficients.
result The method effectively captures the temporal evolution of extreme returns.
The standard method of generating random weights and biases in feedforward neural networks with random hidden nodes, selects them both from the uniform distribution over the same fixed interval. In this work, we show the drawbacks of this approach and propose a new method of generating random parameters. This method en…
An adaptive simulated annealing EM algorithm for inference on non-homogeneous hidden Markov modelsstat.ML
Non-homogeneous hidden Markov models (NHHMM) are a subclass of dependent mixture models used for semi-supervised learning, where both transition probabilities between the latent states and mean parameter of the probability distribution of the responses (for a given state) depend on the set of covariates. A priori w…
Model-based clustering with Hidden Markov Model regression for time series with regime changesstat.ML
This paper introduces a novel model-based clustering approach for clustering time series which present changes in regime. It consists of a mixture of polynomial regressions governed by hidden Markov chains. The underlying hidden process for each cluster activates successively several polynomial regimes during time. The…
This paper addresses the issue of model selection for hidden Markov models (HMMs). We generalize factorized asymptotic Bayesian inference (FAB), which has been recently developed for model selection on independent hidden variables (i.e., mixture models), for time-dependent hidden variables. As with FAB in mixture model…
In this paper, we consider regression problems with one-hidden-layer neural networks (1NNs). We distill some properties of activation functions that lead to in the neighborhood of the ground-truth parameters for the 1NN squared-loss objective. Most popular nonlinear activation function…
As one of Bayesian analysis tools, Hidden Markov Model (HMM) has been used to in extensive applications. Most HMMs are solved by Baum-Welch algorithm (BWHMM) to predict the model parameters, which is difficult to find global optimal solutions. This paper proposes an optimized Hidden Markov Model with Particle Swarm Opt…
New model for time series classification from single example.
problem Classifying time series patterns from limited data.
method Developed a Hidden semi-Markov Model with variable state duration.
result Different representations of state duration have distinct strengths and weaknesses.