arXiv research

SOLBP extends efficient inference to uncertain Bayesian networks.

Paper proposes a second-order method for faster SVI convergence.

We introduce TrustVI, a fast second-order algorithm for black-box variational inference based on trust-region optimization and the reparameterization trick. At each iteration, TrustVI proposes and assesses a step based on minibatches of draws from the variational distribution. The algorithm provably converges to a stat…

A method for inferring motility models and heterogeneity from particle trajectories.

Enhances SMC² with Hessian info for more efficient posterior approximation.

NMC improves MCMC convergence by analyzing gradients to determine optimal proposal densities.

New pruning method captures global correlations for efficient neural network inference.

New method improves online covariance estimation for SGD.

We propose a second-order (Hessian or Hessian-free) based optimization method for variational inference inspired by Gaussian backpropagation, and argue that quasi-Newton optimization can be developed as well. This is accomplished by generalizing the gradient computation in stochastic backpropagation via a reparametriza…

We present efficient differentiable implementations of second-order multi-hop reasoning using a large symbolic knowledge base (KB). We introduce a new operation which can be used to compositionally construct second-order multi-hop templates in a neural model, and evaluate a number of alternative implementations, with d…

Paper improves particle variational inference by optimizing generalization error bound.

We present Ordinary Differential Equation Variational Auto-Encoder (ODE$^2$VAE), a latent second order ODE model for high-dimensional sequential data. Leveraging the advances in deep generative models, ODE$^2$VAE can simultaneously learn the embedding of high dimensional trajectories and infer arbitrarily complex conti…

Stein variational gradient descent (SVGD) was recently proposed as a general purpose nonparametric variational inference algorithm [Liu & Wang, NIPS 2016]: it minimizes the Kullback-Leibler divergence between the target distribution and its approximation by implementing a form of functional gradient descent on a reprod…

The paper improves ODE solvers by integrating diverse information types.

The paper extends logistic regression for unbounded majority classes and derives asymptotic properties.

This work develops a learning theory for inferring interaction kernels in complex agent systems.

A new method for faster optimization on statistical manifolds.

Paper proposes an online covariance estimator for sketched Newton methods.

We propose a method to infer causal structures containing both discrete and continuous variables. The idea is to select causal hypotheses for which the conditional density of every variable, given its causes, becomes smooth. We define a family of smooth densities and conditional densities by second order exponential mo…

New method for estimating high-dimensional binary time series coefficients.

Agents learn state ambiguity from non-linear sensor data using Gaussian approximations.

Scattering representations simplify SBI for images without extra compression.

DADVI improves ADVI by using deterministic approximation for faster, more accurate posterior estimation.

We present a novel statistical inference framework for convex empirical risk minimization, using approximate stochastic Newton steps. The proposed algorithm is based on the notion of finite differences and allows the approximation of a Hessian-vector product from first-order information. In theory, our method efficient…

Local approach learns causal structure of linear Gaussian polytree models from interventional data.

This paper presents the nonparametric inference for nonlinear volatility functionals of general multivariate Itô semimartingales, in high-frequency and noisy setting. Pre-averaging and truncation enable simultaneous handling of noise and jumps. Second-order expansion reveals explicit biases and a pathway to bias correc…

Second-order economic theory considers new variables to improve price volatility predictions.

SLIM efficiently solves overidentified models in a scalable manner.

C-kNN-LSH identifies similar patient histories for causal inference in longitudinal data.

Second-order optimizers retain residual information after data deletion, affecting machine unlearning.

New statistical inference method for high-dimensional Hawkes processes.

Exact second-order optimization for deep learning reduces computational cost and improves performance.

Developed a theory of local convexity for second order differential equations on Lie algebroids.

A new method improves SVI for high-dimensional, poorly-conditioned distributions.

Paper examines risk measure expansions under FGM dependence, improving accuracy at extreme levels.

Negative step sizes improve second-order methods for neural networks.

Simplified argument for second order estimate in quaternionic Calabi-Yau problem.

Paper studies second order tail probabilities in risk models.

A dynamical system on the total space of the fibre bundle of second order accelerations, $T^2M$, is defined as a third order vector field $S$ on $T^2M$, called semispray, which is mapped by the second order tangent structure into one of the Liouville vector field. For a regular Lagrangian of second order we prove that …

Paper studies second order symmetric parallel tensors in generalized f.pk-space forms.

Improved neural network ensembles using Stein Variational Newton updates.

Paper generalizes connections between Lie groups and affine connections.

In this present paper, we study geometric structures of rank two prolongations of implicit second-order partial differential equations (PDEs) for two independent and one dependent variables and characterize the type of these PDEs by the topology of fibers of the rank two prolongations. Moreover, by using properties of …

New method improves inference for discrete diffusion models, achieving better quality and efficiency.

In this paper, we study stochastic non-convex optimization with non-convex random functions. Recent studies on non-convex optimization revolve around establishing second-order convergence, i.e., converging to a nearly second-order optimal stationary points. However, existing results on stochastic non-convex optimizatio…

Geometric framework analyzes bias in variational inference for posterior functionals.

AdamQLR optimizes Adam with K-FAC heuristics, achieving comparable performance to tuned benchmarks.

The quantification of diversification benefits due to risk aggregation plays a prominent role in the (regulatory) capital management of large firms within the financial industry. However, the complexity of today's risk landscape makes a quantifiable reduction of risk concentration a challenging task. In the present pap…