Several machine learning problems arising in natural language processing can be modeled as a sequence labeling problem. We provide Gaussian process models based on pseudo-likelihood approximation to perform sequence labeling. Gaussian processes (GPs) provide a Bayesian approach to learning in a kernel based framework. …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Gaussian sequence model”
Empirical Bayes method improves Gaussian sequence model inference.
problem Estimating parameters in correlated Gaussian sequence models.
method Maximum Composite Marginal Likelihood (CML) estimator, leveraging geometric Brascamp-Lieb inequality.
result CML estimator converges at rate \( n_*^{-1/2} \) in weighted Hellinger distance.
Improves sequence modeling with a flow-based recurrent mixture density network.
problem Sequence modeling and sequence-to-sequence mapping applications.
method Generalized recurrent mixture density networks using normalized flow transformations.
result Significantly improved fit to image sequences measured by log-likelihood.
IAE extracts innovations sequences for non-Gaussian processes.
problem Extracting innovations sequences for non-Gaussian processes.
method Causal convolutional neural network.
result IAE effectively detects anomalies in non-Gaussian data.
Scalable hybrid HMM with Gaussian Process for time-series data clustering.
problem Large number of parameters and long sequences in time-series data make HMM-GPSM training difficult.
method Stochastic Variational Inference (SVI) for long sequences and reparameterized random Fourier features (R-RFF) for large data points.
result Significant reduction in training time and improved hidden-state estimation accuracy.
Study on convergence of transformed metric spaces as dimensions grow.
problem Conditions for convergence of transformed metric spaces.
method Clarifying conditions for convergence of transformed spaces from original sequence and vice versa.
result Spheres and projective spaces converge to Gaussian space and its quotient as dimensions increase.
A novel multi-resolution Gaussian process model for efficient time traversal.
problem Inference for long sequences with fast and slow transitions is difficult.
method A novel Gaussian process state-space architecture composed of multiple components, each trained on a different resolution.
result The combined model allows efficient inference for arbitrarily long sequences with complex dynamics.
Study on signal detection in heteroscedastic Gaussian sequences with sparse alternatives.
problem Signal detection in heterogeneous Gaussian sequences with unknown means and known covariance.
method Characterization of minimax separation radius and derivation of matching upper and lower bounds.
result Matching minimax upper and lower bounds for signal detection in heteroscedastic Gaussian sequences.
Study connects Gaussian processes and regularization for sequence-function mappings.
problem Understanding and interpreting sequence-function maps in biology.
method Relates Gaussian process priors, regularization, and gauge fixing in overparameterized weight space.
result Established the relationship between regularized regression and Gaussian processes in function space.
Optimizes sliding window approach for tracking Gaussian densities.
problem Improving tracking performance of Gaussian density estimation.
method Theoretical analysis of sliding window Gaussian Kernel Density Estimators.
result Empirical evidence shows improved tracking performance with optimal weight sequence.
Ellipsoids approach Gaussian distribution in high dimensions.
problem Understanding convergence of high-dimensional ellipsoids to Gaussian spaces.
method Proof of convergence in Gromov's concentration topology.
result Solid ellipsoids converge to Gaussian space in high dimensions.
High dimensional time series are endemic in applications of machine learning such as robotics (sensor data), computational biology (gene expression data), vision (video sequences) and graphics (motion capture data). Practical nonlinear probabilistic approaches to this data are required. In this paper we introduce the v…
In order to better model high-dimensional sequential data, we propose a collaborative multi-output Gaussian process dynamical system (CGPDS), which is a novel variant of GPDSs. The proposed model assumes that the output on each dimension is controlled by a shared global latent process and a private local latent process…
Variational Bayesian Sequence-to-Sequence Networks for Memory-Efficient Sign Language Translationstat.ML
A new memory-efficient sign language translation model reduces weight usage.
problem Memory constraints in real-time sign language translation.
method Variational Bayesian sequence-to-sequence network with Gaussian posterior and Indian Buffet Process prior.
result The proposed model achieves substantial weight compression without compromising performance.
A new model uses attention and Gaussian processes for efficient time-series generation.
problem Computational inefficiency and uncertainty underestimation in sequence transduction.
method Attention-based Gaussian process network for real-valued sequence generation.
result The model improves training efficiency and learns factorized generative distribution.
Bayesian nonparametric method segments multi-sequence time series data.
problem Temporal segmentation of multi-sequence time series data into stationary segments.
method Gaussian process priors and nonparametric distribution for segment partitioning.
result Model effectively segments synthetic and real-time series data.
Develops new e-processes and confidence sequences for Gaussian means with unknown variance.
problem Constructing valid t-tests and confidence sequences for Gaussian means with unknown variance.
method Explores generalized nonintegrable martingales and extended Ville's inequality, developing two new e-processes and confidence sequences.
result Analyzes the width of resulting confidence sequences with a polynomial dependence on error probability, proving it to be unavoidable and even better than classical fixed-sample t-tests.
This thesis improves deep sequence models by integrating probabilistic methods for uncertainty quantification.
problem Lack of uncertainty awareness in deep sequence models limits their deployment.
method Develops approximate Bayesian inference methods for Transformers and HiPPOs, leveraging inductive biases.
result Improves predictive and generative performance of deep sequence models by incorporating probabilistic structures.
Expectation Maximization (EM) is among the most popular algorithms for estimating parameters of statistical models. However, EM, which is an iterative algorithm based on the maximum likelihood principle, is generally only guaranteed to find stationary points of the likelihood objective, and these points may be far from…
Langevin Quasi-Monte Carlostat.CO
LMC improves sampling from complex distributions using quasi-random sequences.
problem Sampling from complex high-dimensional distributions with high accuracy.
method Using completely uniformly distributed (CUD) sequences in Langevin Monte Carlo (LMC) to generate Gaussian perturbations.
result LMC with low-discrepancy CUD sequences achieves smaller estimation error than standard LMC.
Generative Distribution Embeddings learn multiscale representations of distributions.
problem Learning representations of entire distributions for multiscale reasoning.
method Introducing GDE framework that lifts autoencoders to the space of distributions, using conditional generative models and distributional invariance.
result GDEs learn predictive sufficient statistics embedded in Wasserstein space, recovering distances and trajectories for Gaussian and Gaussian mixture distributions.
New method improves Robbins-Monro algorithm convergence with prior information.
problem Improving convergence speed of Robbins-Monro algorithm.
method Integrates prior information into Robbins-Monro iteration without regression model.
result Prior-information Robbins-Monro sequence converges faster than standard.
We introduce the truncated Gaussian graphical model (TGGM) as a novel framework for designing statistical models for nonlinear learning. A TGGM is a Gaussian graphical model (GGM) with a subset of variables truncated to be nonnegative. The truncated variables are assumed latent and integrated out to induce a marginal m…
GMM-HMMs improve malware classification compared to discrete HMMs.
problem Improving malware classification accuracy using GMM-HMM models.
method Used GMM-HMM models for opcode and entropy-based features comparison with discrete HMMs.
result GMM-HMMs generally improve significantly on entropy-based features classification results.
Bayesian graphical models are a useful tool for understanding dependence relationships among many variables, particularly in situations with external prior information. In high-dimensional settings, the space of possible graphs becomes enormous, rendering even state-of-the-art Bayesian stochastic search computationally…
We present a probabilistic model for unsupervised alignment of high-dimensional time-warped sequences based on the Dirichlet Process Mixture Model (DPMM). We follow the approach introduced in (Kazlauskaite, 2018) of simultaneously representing each data sequence as a composition of a true underlying function and a time…
For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence. This problem has been formalized as a sequence extrapolation problem, where a number of observations are used to predict the sequence into the future. Real-world scenarios …
Study on predicting sequences with Gaussian constraints, linking to intrinsic volumes and metric complexity.
problem Predicting sequences almost as well as the best Gaussian distribution with mean in a given subset.
method Expressed minimax regret in terms of intrinsic volumes, established comparison inequality for Wills functional, characterized global covering numbers and local Gaussian widths.
result Sharp estimates on the log-Laplace transform of intrinsic volume sequence for a general nonconvex set.
We consider the problem of learning high-dimensional Gaussian graphical models. The graphical lasso is one of the most popular methods for estimating Gaussian graphical models. However, it does not achieve the oracle rate of convergence. In this paper, we propose the graphical nonconvex optimization for optimal estimat…
We develop a Bayesian approach to learning from sequential data by using Gaussian processes (GPs) with so-called signature kernels as covariance functions. This allows to make sequences of different length comparable and to rely on strong theoretical results from stochastic analysis. Signatures capture sequential struc…
We investigate the geometric characteristics of constant gaussian curvature surfaces obtained from solutions of the sigma model. Most of these solutions are related to the Veronese sequence. We show that we can distinguish surfaces with the same gaussian curvature using additional quantities like the topologic…
Analyzing the underlying structure of multiple time-sequences provides insights into the understanding of social networks and human activities. In this work, we present the \emph{Bayesian nonparametric Poisson process allocation} (BaNPPA), a latent-function model for time-sequences, which automatically infers the numbe…
LaMBO optimizes biological sequences using autoencoders and Bayesian optimization.
problem Bayesian optimization for drug design is limited by discrete, high-dimensional decision variables.
method Jointly trains denoising autoencoder with a Gaussian process head for gradient-based optimization in latent space.
result LaMBO outperforms genetic optimizers and requires no large pretraining corpus.
Regularized Estimation of High-Dimensional Vector AutoRegressions with Weakly Dependent Innovationsmath.ST
Study LASSO for high-dimensional VAR models with weakly dependent innovations.
problem Understanding sparse regularization in high-dimensional VAR models with weakly dependent innovations.
method LASSO estimation for weakly sparse VAR models with heavy tailed innovations, under mixingale condition.
result Oracle properties of LASSO estimation in high-dimensional VAR models with weakly dependent innovations.
Time series models generalize ARMA and ARFIMA with non-Gaussian dependence.
problem Modeling non-Gaussian serial dependence in time series data.
method Infinite-order partial copula dependence in s-vine processes.
result Rich class of models that generalize linear processes.
Kernel methods on discrete domains have shown great promise for many challenging data types, for instance, biological sequence data and molecular structure data. Scalable kernel methods like Support Vector Machines may offer good predictive performances but do not intrinsically provide uncertainty estimates. In contras…
Prediction of future states of the environment and interacting agents is a key competence required for autonomous agents to operate successfully in the real world. Prior work for structured sequence prediction based on latent variable models imposes a uni-modal standard Gaussian prior on the latent variables. This indu…
Algorithm balances learning and coverage for multi-robots over unknown fields.
problem Balancing learning and coverage for multi-robots over unknown, nonuniform sensory fields.
method DSLC algorithm that schedules learning and coverage epochs, using Gaussian Process modeling and coverage regret analysis.
result Upper bound on expected cumulative coverage regret provided for DSLC.
Transformer-MGK replaces redundant heads with Gaussian key mixtures, improving efficiency and performance.
problem Redundant attention heads in transformers degrade performance and efficiency.
method Transformer-MGK replaces redundant heads with a mixture of Gaussian keys.
result Transformer-MGK accelerates training and inference, reduces parameters and FLOPs, and achieves comparable or better accuracy.
Finite Mixtures of Multivariate Poisson-Log Normal Factor Analyzers for Clustering Count Datastat.ME
Flexible models cluster RNA sequencing data.
problem Clustering discrete data from RNA sequencing studies.
method Finite mixtures of multivariate Poisson-log normal factor analyzers with constraints.
result Models give favorable clustering performance on real and simulated data.
Two algorithms learn Gaussian graphical models from Glauber dynamics trajectories, achieving optimal performance.
problem Learning Gaussian graphical models from a single trajectory of a dependent stochastic process.
method Two algorithms based on dueling-neighborhood search and local statistics built from the update sequence of Glauber dynamics.
result Achieve dependence of the information-theoretic lower bounds, mixing-free and signal-optimal.
Hybridizes CEM and gradient descent for efficient model-predictive control.
problem Efficiently planning optimal action sequences in high-dimensional spaces.
method Interleaves Cross-Entropy Method (CEM) and gradient descent steps.
result Faster convergence and avoidance of local optima compared to CEM.
New method improves GP uncertainty quantification for misspecified priors.
problem Uncertainty quantification for GPs under incorrect priors.
method Constructs a confidence sequence using martingale techniques.
result Empirically outperforms standard GP methods in robustness and utility for Bayesian Optimization.
Transformers tend to learn more symmetric functions in sequence data.
problem Understanding inductive bias in Transformers with infinitely over-parameterized models.
method Analyzing Transformers in the Gaussian process limit, using representation theory of the symmetric group.
result Transformers are biased towards more permutation symmetric functions, and this can be quantitatively predicted.
The study proves a central limit theorem for Gaussian holomorphic sections on Kähler manifolds.
problem Understanding statistical properties of zeros of random holomorphic sections.
method Proves a central limit theorem for smooth linear statistics of zero divisors of Gaussian sections in line bundles over Kähler manifolds.
result Derives first-order asymptotics and upper decay estimates for Bergman kernels.
Given a stationary state-space model that relates a sequence of hidden states and corresponding measurements or observations, Bayesian filtering provides a principled statistical framework for inferring the posterior distribution of the current state given all measurements up to the present time. For example, the Apoll…
Study convergence and approximations of entropic regularized Wasserstein distances for Gaussian and RKHS measures.
problem Convergence and approximations of entropic regularized Wasserstein distances in Gaussian and RKHS settings.
method Analysis of convergence and finite sample approximations of entropic regularized Wasserstein distances in Gaussian and RKHS settings.
result Strictly weaker convergence in 2-Sinkhorn divergence for Gaussian measures compared to exact 2-Wasserstein distance.
New algorithm learns HMM parameters on Riemannian manifolds.
problem Learning hidden Markov models on non-Euclidean spaces.
method Geometric method of moments algorithm for Riemannian manifolds.
result Significantly improved speed and accuracy compared to existing methods.