Log-concavity proven for multinomial likelihoods under specific constraints.
problem Log-concavity of multinomial likelihoods under interval censoring constraints.
method Proved log-concavity by showing M-convex subsets of the discrete simplex.
result Likelihood function is completely log-concave.
Recently there has been sustained interest in modifying prediction algorithms to satisfy fairness constraints. These constraints are typically complex nonlinear functionals of the observed data distribution. Focusing on the path-specific causal constraints proposed by Nabi and Shpitser (2018), we introduce new theoreti…
Unified framework for unlearning in diffusion models using KL divergence and likelihood constraints.
problem Removing undesirable data or concepts while preserving utility of pretrained models.
method Constrained optimization framework based on reverse and forward KL divergences, and likelihood constraints.
result Our KL-constrained approach achieves superior retention-unlearning tradeoffs compared to weight-based baselines.
Optimization results are one method for understanding neural computation from Nature's perspective and for defining the physical limits on neuron-like engineering. Earlier work looks at individual properties or performance criteria and occasionally a combination of two, such as energy and information. Here we make use …
While most approaches to the problem of Inverse Reinforcement Learning (IRL) focus on estimating a reward function that best explains an expert agent's policy or demonstrated behavior on a control task, it is often the case that such behavior is more succinctly represented by a simple reward combined with a set of hard…
Synthesizes sensor likelihoods to enforce accuracy constraints in uncertain systems.
problem Designing sensing architectures for systems with uncertain or unavailable sensor models and accuracy requirements.
method Inverts the design flow, synthesizing measurement likelihoods that minimize Kullback-Leibler divergence from the prior while enforcing an accuracy bound.
result The method synthesizes a maximum-entropy posterior and induced likelihood, accommodating various discrepancy metrics.
Develops efficient method for nonconvex problems using Regula Falsi.
problem Nonconvex inverse problems with likelihood constraints.
method Regula Falsi root-finding techniques applied to level-set formulations.
result Proves extension of level-set methods to nonconvex problems.
We investigate the problem of semi-parametric maximum likelihood under constraints on summary statistics. Such a procedure results in a discrete probability distribution that maximises the likelihood among all such distributions under the specified constraints (called estimating equations), and is an approximation to t…
Paper extends LME models to allow sign constraints on coefficients with SDTN random effects.
problem Inference with sign constraints on random effects in LME models.
method Proposes SDTN distribution for random effects and develops likelihood-based approaches for estimation.
result Proposed constrained model improves real-world interpretations and achieves satisfactory performance.
Likelihood-free (a.k.a. simulation-based) inference problems are inverse problems with expensive, or intractable, forward models. ODE inverse problems are commonly treated as likelihood-free, as their forward map has to be numerically approximated by an ODE solver. This, however, is not a fundamental constraint but jus…
Normalizing flows optimize Jacobian determinant for unique likelihood objective.
problem Optimizing normalizing flows for unique likelihood.
method Showed Jacobian determinant is unique for given distributions, leading to a unique global optimum. Used eigenvalues of auto-correlation matrix for explicit likelihood expression.
result Explicit expression of likelihood for flows, independent of neural network parameterization, with theoretical optimal value.
Novel approach for SEM in small samples with p>n.
problem Small sample size and p>n issues in factor-based SEM. method Reformulates covariance structure into self-covariance and cross-covariance, defines a feasible set with relative error constraint.
result Improved stability and directional information in small-sample settings.
Researchers develop methods for inference in hierarchical models using neural simulations.
problem Inference in hierarchical models with intractable likelihoods.
method Construct neural estimators for likelihood-ratio or posterior, accounting for hierarchical structure.
result Explicitly accounting for hierarchical structure leads to tighter parameter constraints.
Paper studies sparsity and DAG constraints for learning linear DAGs.
problem Learning DAGs from data is challenging due to the large search space.
method Formulates structure learning as a constrained optimization problem with soft sparsity and DAG constraints.
result Soft sparsity and DAG constraints lead to an easier optimization problem.
A new Bayesian framework simplifies stochastic optimization by focusing on key parameters.
problem Bayesian methods struggle with complex structural constraints.
method Minimalist Bayesian framework that eliminates nuisance parameters via profile likelihood.
result Near-optimal regret guarantees for multi-armed bandits and convex optimization.
Geodesic descent optimizes likelihood in dually flat spaces.
problem Maximum likelihood estimation in exponential families.
method m-geodesic and e-geodesic updates on dually flat spaces.
result Geodesic updates can reach maximum likelihood estimator in one step.
Proposes a method to optimize neural network initialization using marginal likelihood maximization.
problem Optimizing hyperparameters for neural network initialization.
method Leverages the connection between neural networks and Gaussian processes to infer optimal hyperparameters.
result Marginal likelihood maximization provides near-optimal prediction performance on MNIST classification tasks.
AFTNet uses a network-constrained Weibull model for biomarker discovery.
problem Discovering biomarkers from survival data with correlated predictors.
method Survival analysis method based on Weibull AFT model, incorporating network constraints and penalized likelihood for variable selection.
result Theoretical consistency and efficient algorithm for AFTNet estimator validated on synthetic and real data.
Estimation in generalized linear models (GLM) is complicated by the presence of constraints. One can handle constraints by maximizing a penalized log-likelihood. Penalties such as the lasso are effective in high dimensions, but often lead to unwanted shrinkage. This paper explores instead penalizing the squared distanc…
Unified framework for DRO using OT with constraints.
problem Handling ambiguity in likelihood ratios and outcomes.
method Unified framework leveraging optimal transport with conditional moment constraints.
result Unified approach enables adversarial perturbation of likelihood ratios and outcomes.
We present an improved Bayesian framework for performing inference of affine transformations of constrained functions. We focus on quadrature with nonnegative functions, a common task in Bayesian inference. We consider constraints on the range of the function of interest, such as nonnegativity or boundedness. Although …
Bayesian evidence helps compare models but can overfit.
problem Comparing hypotheses consistent with observations.
method Marginal likelihood, Occam's razor, PAC-Bayes bounds.
result Marginal likelihood can negatively correlate with generalization.
New MMM captures hierarchical marketing effects and sign restrictions.
problem Measuring effectiveness of marketing activities with hierarchical structure and sign constraints.
method Proposes a constrained maximum likelihood approach using Hamiltonian Monte Carlo algorithm.
result Demonstrates superior performance on real datasets compared to multi-stage methods.
R package for Bayesian empirical likelihood sampling using HMC.
problem Sampling from non-convex Bayesian empirical likelihood posteriors.
method Hamiltonian Monte Carlo (HMC) algorithm for numerical integration.
result Efficient HMC sampling from BayesEL posteriors.
The Schwartz-Smith model parameters are estimated using Kalman Filter with additional constraints.
problem Estimating parameters of the Schwartz-Smith model for risk-neutral pricing of futures contracts.
method Kalman Filter method with additional constraints to address parameter identification problem.
result The obtained parameter estimates are the conditional Maximum Likelihood Estimators (MLEs) evaluated within the Kalman Filter.
New framework improves generative models with prediction and consistency constraints.
problem Improving generative models with sparse labeled data.
method Optimizes variational autoencoders with prediction and consistency constraints.
result Promising image classification performance, especially in semi-supervised scenarios.
Mixed likelihood GPs improve model performance in human-in-the-loop experiments.
problem Lack of auxiliary information in traditional GPs for human responses.
method Propose mixed likelihood variational GPs to leverage auxiliary information.
result Modeling performance improvements across diverse applications.
Framework for Bayesian inference using GP emulated MH sampler for noisy likelihoods.
problem Approximate Bayesian inference with limited noisy log-likelihood evaluations.
method Gaussian process emulates MH sampler for log-likelihood evaluations; sequential experimental design selects evaluation points.
result Approximate sampler is sample-efficient and robust to GP assumptions.
Many models of interest in the natural and social sciences have no closed-form likelihood function, which means that they cannot be treated using the usual techniques of statistical inference. In the case where such models can be efficiently simulated, Bayesian inference is still possible thanks to the Approximate Baye…
We address the problem of likelihood based inference for correlated diffusion processes using Markov chain Monte Carlo (MCMC) techniques. Such a task presents two interesting problems. First, the construction of the MCMC scheme should ensure that the correlation coefficients are updated subject to the positive definite…
MINTS uses a minimalist Bayesian framework to tackle multi-armed bandits with structural constraints.
problem Sequential decision-making under uncertainty with complex structural constraints.
method Minimalist Bayesian framework with profile likelihood to eliminate nuisance parameters.
result MINTS achieves near-optimal regret guarantees and adapts to unimodal structure.
We derive an explicit formula for likelihood function for Gaussian VARMA model conditioned on initial observables where the moving-average (MA) coefficients are scalar. For fixed MA coefficients the likelihood function is optimized in the autoregressive variables Φ's by a closed form formula generalizing regression c…
Proposes learning invariances in neural networks using a weight-space approach.
problem Learning invariances from data in neural networks remains an open problem.
method Minimizes a lower bound on the marginal likelihood in weight space.
result Results in higher performing models with naturally learned invariances.
New method trains any neural network as a generative model.
problem Constrained design of normalizing flows due to analytical invertibility.
method Efficient gradient estimator for non-analytically invertible networks.
result Any dimension-preserving neural network can be used as a generative model.
We consider robust covariance estimation with group symmetry constraints. Non-Gaussian covariance estimation, e.g., Tyler scatter estimator and Multivariate Generalized Gaussian distribution methods, usually involve non-convex minimization problems. Recently, it was shown that the underlying principle behind their succ…
Algorithm improves movie recommendation efficiency with fairness constraints.
problem Improving movie recommendation efficiency with fairness constraints in combinatorial semi-bandits.
method Adopted Thompson Sampling with beta priors and Bernoulli likelihoods to handle fairness constraints.
result Time-averaged regret upper bounded by $\frac{N}{2η} + O\left(\frac{\sqrt{mNT\ln T}}{T}
ight)$, with fairness constraints satisfied.
There has been much recent, exciting work on combining the complementary strengths of latent variable models and deep learning. Latent variable modeling makes it easy to explicitly specify model constraints through conditional independence properties, while deep learning makes it possible to parameterize these conditio…
A Bayesian approach termed BAyesian Least Squares Optimization with Nonnegative L1-norm constraint (BALSON) is proposed. The error distribution of data fitting is described by Gaussian likelihood. The parameter distribution is assumed to be a Dirichlet distribution. With the Bayes rule, searching for the optimal parame…
Extends likelihood ratio exponential families to analyze various optimization methods.
problem Analyzing optimization methods like rate-distortion and information bottleneck.
method Linking geometric mixture paths to exponential families and using hypothesis testing.
result Provides a common mathematical framework for understanding these methods.
PIPA aligns preferences without reinforcement learning, improving language model performance.
problem Aligning preferences in language models efficiently and without reinforcement learning.
method Formulates preference alignment as a Maximum Likelihood Estimation problem with prior constraints.
result PIPA algorithms achieve up to 10% performance improvement on benchmarks.
We propose flow-based likelihoods to accurately capture non-Gaussian data.
problem Bypassing the Gaussian assumption in scientific analyses.
method Use optimization targets of flow-based generative models to reconstruct likelihoods.
result Flow-based likelihoods can accurately capture non-Gaussian data, improving parameter constraints.
SurVAE Flows combine VAEs and flows using surjective transformations.
problem Combining the strengths of VAEs and flows to model complex densities.
method Modular framework of composable deterministic and stochastic transformations.
result Exact likelihood computation and lower bound on likelihood.
The paper tackles sparse graph learning under Laplacian-related constraints, improving upon existing methods.
problem Learning a sparse undirected graph from multivariate data under Laplacian-related constraints.
method Modifications to penalized log-likelihood approaches to enforce total positivity and lasso/adaptive lasso penalties using ADMM.
result The proposed constrained adaptive lasso approach significantly outperforms existing Laplacian-based approaches.
In decision-making systems, it is important to have classifiers that have calibrated uncertainties, with an optimisation objective that can be used for automated model selection and training. Gaussian processes (GPs) provide uncertainty estimates and a marginal likelihood objective, but their weak inductive biases lead…
Automatically learns flexible symmetry constraints in neural networks using gradients.
problem Fixed hard constraints on neural network functions that cannot be adapted.
method Improves parameterisations of soft equivariance and optimizes marginal likelihood using differentiable Laplace approximations.
result Achieves equivalent or improved performance on image classification tasks compared to baselines with hard-coded symmetry.
Clinical patient records are an example of high-dimensional data that is typically collected from disparate sources and comprises of multiple likelihoods with noisy as well as missing values. In this work, we propose an unsupervised generative model that can learn a low-dimensional representation among the observations…
Probabilistic learning is increasingly being tackled as an optimization problem, with gradient-based approaches as predominant methods. When modelling multivariate likelihoods, a usual but undesirable outcome is that the learned model fits only a subset of the observed variables, overlooking the rest. In this work, we …
Bayesian method detects Markov order in network paths more reliably.
problem Detecting Markov order in constrained categorical sequences.
method Multi-order Bayesian modelling framework.
result Bayesian method detects correct Markov order more reliably than competing methods.