arXiv research

Corrects pseudo log-likelihood method issues in various applications.

The pseudo-likelihood method is one of the most popular algorithms for learning sparse binary pairwise Markov networks. In this paper, we formulate the $L_1$ regularized pseudo-likelihood problem as a sparse multiple logistic regression problem. In this way, many insights and optimization procedures for sparse logistic…

Many algorithms have been proposed for fitting network models with communities, but most of them do not scale well to large networks, and often fail on sparse networks. Here we propose a new fast pseudo-likelihood method for fitting the stochastic block model for networks, as well as a variant that allows for an arbitr…

This paper shows how to perform likelihood inference for complex graphical models efficiently.

Sparse high dimensional graphical model selection is a popular topic in contemporary machine learning. To this end, various useful approaches have been proposed in the context of $\ell_1$-penalized estimation in the Gaussian framework. Though many of these inverse covariance estimation approaches are demonstrably scala…

Bayesian inference in the presence of an intractable likelihood function is computationally challenging. When following a Markov chain Monte Carlo (MCMC) approach to approximate the posterior distribution in this context, one typically either uses MCMC schemes which target the joint posterior of the parameters and some…

New algorithm detects communities in weighted networks, improving on binary ones.

TPSQRs model longitudinal event data, detecting ADRs from EHRs.

Sparse high dimensional graphical model selection is a topic of much interest in modern day statistics. A popular approach is to apply l1-penalties to either (1) parametric likelihoods, or, (2) regularized regression/pseudo-likelihoods, with the latter having the distinct advantage that they do not explicitly assume Ga…

Undirected graphical models known as Markov networks are popular for a wide variety of applications ranging from statistical physics to computational biology. Traditionally, learning of the network structure has been done under the assumption of chordality which ensures that efficient scoring methods can be used. In ge…

Several machine learning problems arising in natural language processing can be modeled as a sequence labeling problem. We provide Gaussian process models based on pseudo-likelihood approximation to perform sequence labeling. Gaussian processes (GPs) provide a Bayesian approach to learning in a kernel based framework. …

New method improves community detection for large networks.

Pseudo-Likelihood Inference improves ABC for high-dimensional Bayesian inference.

A new MCMC method for GPs tackles computational burden and intractable likelihoods.

A new method scores contextual Markov networks without assuming chordality.

Estimating statistical models within sensor networks requires distributed algorithms, in which both data and computation are distributed across the nodes of the network. We propose a general approach for distributed learning based on combining local estimators defined by pseudo-likelihood components, encompassing a num…

Learning the undirected graph structure of a Markov network from data is a problem that has received a lot of attention during the last few decades. As a result of the general applicability of the model class, a myriad of methods have been developed in parallel in several research fields. Recently, as the size of the c…

Paper proposes MMD-Bayes for robust Bayesian estimation in misspecified models.

New algorithms learn staged trees from incomplete data.

We explain SSL objectives as log-likelihoods in a data curation model.

Paper proposes a method to handle linear regression with partially shuffled data.

Speeding up Markov Chain Monte Carlo (MCMC) for datasets with many observations by data subsampling has recently received considerable attention. A pseudo-marginal MCMC method is proposed that estimates the likelihood by data subsampling using a block-Poisson estimator. The estimator is a product of Poisson estimators,…

StepMix estimates mixture models with covariates for social science applications.

We propose a Bayesian approximate inference method for learning the dependence structure of a Gaussian graphical model. Using pseudo-likelihood, we derive an analytical expression to approximate the marginal likelihood for an arbitrary graph structure without invoking any assumptions about decomposability. The majority…

Optimal downsampling improves GLM performance in imbalanced classification.

Method for factor analysis in short panels without assuming sphericity or Gaussianity.

The paper presents a new method to represent directed graphs using pseudo-Riemannian manifolds.

Bayesian Pseudo Label Selection reduces overfitting in semi-supervised learning.

Kernel methods have revolutionized the fields of pattern recognition and machine learning. Their success, however, critically depends on the choice of kernel parameters. Using Gaussian process (GP) classification as a working example, this paper focuses on Bayesian inference of covariance (kernel) parameters using Mark…

Paper proposes an end-to-end learning method for state estimation in robotics.

Sparse Gaussian Processes simplify GP inference for large datasets.

We introduce a Bayesian framework for inference with a supervised version of the Gaussian process latent variable model. The framework overcomes the high correlations between latent variables and hyperparameters by using an unbiased pseudo estimate for the marginal likelihood that approximately integrates over the late…

The paper improves self-training in semi-supervised learning by selecting more robust pseudo-labeled data.

MLMs score better than autoregressive models without finetuning.

Study higher-order spin glass models for social network behavior with peer-group effects.

We propose Subsampling MCMC, a Markov Chain Monte Carlo (MCMC) framework where the likelihood function for $n$ observations is estimated from a random subset of $m$ observations. We introduce a highly efficient unbiased estimator of the log-likelihood based on control variates, such that the computing cost is much smal…

Paper presents a new probabilistic approach for high-dimensional quantile prediction.

The Whittle likelihood is a widely used and computationally efficient pseudo-likelihood. However, it is known to produce biased parameter estimates for large classes of models. We propose a method for de-biasing Whittle estimates for second-order stationary stochastic processes. The de-biased Whittle likelihood can be …

Tensor decomposition methods allow us to learn the parameters of latent variable models through decomposition of low-order moments of data. A significant limitation of these algorithms is that there exists no general method to regularize them, and in the past regularization has mostly been performed using bespoke modif…

We improve likelihood-free inference using distillation of importance sampling.

Develops first and second-order pseudo-mirror descent methods for nonnegative function estimation.

Probabilistic pseudo knots model uncertain knot diagrams.

We propose a generic Markov Chain Monte Carlo (MCMC) algorithm to speed up computations for datasets with many observations. A key feature of our approach is the use of the highly efficient difference estimator from the survey sampling literature to estimate the log-likelihood accurately using only a small fraction of …

A new method learns continuous occupancy fields efficiently using the Ising model.

Financial econometrics has become an increasingly popular research field. In this paper we review a few parametric and nonparametric models and methods used in this area. After introducing several widely used continuous-time and discrete-time models, we study in detail dependence structures of discrete samples, includi…

Paper offers a framework for estimating symmetric properties efficiently.

We propose a new inferential framework for constructing confidence regions and testing hypotheses in statistical models specified by a system of high dimensional estimating equations. We construct an influence function by projecting the fitted estimating equations to a sparse direction obtained by solving a large-scale…

This work optimizes statistical inference with neural networks for high-energy physics data.