A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
This paper introduces a neural sampler for scalable sampling from complex distributions.
Improved UIVI method shows better performance than state-of-the-art SIVI methods.
Proposes LDIDPs for efficient sequential data generation from latent dynamical models.
Adaptive sampler improves recommendation for implicit feedback data.
Recent works propose using the discriminator of a GAN to filter out unrealistic samples of the generator. We generalize these ideas by introducing the implicit Metropolis-Hastings algorithm. For any implicit probabilistic model and a target distribution represented by a set of samples, implicit Metropolis-Hastings oper…
Improves BED scalability for implicit models.
IDAC improves reinforcement learning efficiency by modeling implicit distributions.
To combine explicit and implicit generative models, we introduce semi-implicit generator (SIG) as a flexible hierarchical model that can be trained in the maximum likelihood framework. Both theoretically and experimentally, we demonstrate that SIG can generate high quality samples especially when dealing with multi-mod…
Paper explores how DPP sampling can implicitly regularize kernel regression.
Generative Latent Implicit Conditional Optimization (GLICO) learns from small samples.
Stochastic gradient descent procedures have gained popularity for parameter estimation from large data sets. However, their statistical properties are not well understood, in theory. And in practice, avoiding numerical instability requires careful tuning of key parameters. Here, we introduce implicit stochastic gradien…
A scalable method for BED with implicit models using approximate gradients.
Implicit models, which allow for the generation of samples but not for point-wise evaluation of probabilities, are omnipresent in real-world problems tackled by machine learning and a hot topic of current research. Some examples include data simulators that are widely used in engineering and scientific research, genera…
Gradient descent converges to a global minimum in nonlinear ReLU implicit networks with linear width.
Recently there have been increasing interests in learning and inference with implicit distributions (i.e., distributions without tractable densities). To this end, we develop a gradient estimator for implicit distributions based on Stein's identity and a spectral decomposition of kernel operators, where the eigenfuncti…
We propose an adversarial training procedure for learning a causal implicit generative model for a given causal graph. We show that adversarial training can be used to learn a generative model with true observational and interventional distributions if the generator architecture is consistent with the given causal grap…
Stochastic recurrent neural networks with latent random variables of complex dependency structures have shown to be more successful in modeling sequential data than deterministic deep models. However, the majority of existing methods have limited expressive power due to the Gaussian assumption of latent variables. In t…
Improved method for unbiased causal discovery in presence of unobserved confounding.
There are two types of deep generative models: explicit and implicit. The former defines an explicit density form that allows likelihood inference; while the latter targets a flexible transformation from random noise to generated samples. While the two classes of generative models have shown great power in many applica…
For sampling from a log-concave density, we study implicit integrators resulting from -method discretization of the overdamped Langevin diffusion stochastic differential equation. Theoretical and algorithmic properties of the resulting sampling methods for and a range of step sizes are established. Ou…
Over-parameterized neural networks generalize well in practice without any explicit regularization. Although it has not been proven yet, empirical evidence suggests that implicit regularization plays a crucial role in deep learning and prevents the network from overfitting. In this work, we introduce the gradient gap d…
This paper proves SGD converges to global minimum for over-parameterized ReLU networks.
Implicit probabilistic models are models defined naturally in terms of a sampling procedure and often induces a likelihood function that cannot be expressed explicitly. We develop a simple method for estimating parameters in implicit models that does not require knowledge of the form of the likelihood function or any d…
iDAD uses neural networks to quickly adapt experiments without likelihoods.
New Hamiltonian Monte Carlo method for non-canonical dynamics.
Generative adversarial networks (GANs) have given us a great tool to fit implicit generative models to data. Implicit distributions are ones we can sample from easily, and take derivatives of samples with respect to model parameters. These models are highly expressive and we argue they can prove just as useful for vari…
Recently, there has been a growing interest in the problem of learning rich implicit models - those from which we can sample, but can not evaluate their density. These models apply some parametric function, such as a deep network, to a base measure, and are learned end-to-end using stochastic optimization. One strategy…
New method trains generative models without discriminators, improving stability and accuracy.
EnCF improves data assimilation for implicit, non-smooth observations.
Gradient descent implicitly favors group sparsity in neural networks.
Novel framework optimizes experiments for implicit models using mutual information.
Gradient flow on ReLU networks converges to a simple model with few regions.
This work investigates implicit bias in multiclass separable data using a novel geometry-aware optimizer.
Bayesian experimental design involves the optimal allocation of resources in an experiment, with the aim of optimising cost and performance. For implicit models, where the likelihood is intractable but sampling from the model is possible, this task is particularly difficult and therefore largely unexplored. This is mai…
Implicit stochastic models, where the data-generation distribution is intractable but sampling is possible, are ubiquitous in the natural sciences. The models typically have free parameters that need to be inferred from data collected in scientific experiments. A fundamental question is how to design the experiments so…
Optimizes experimental designs for intractable models using mutual information bounds.
Kernel methods have achieved very good performance on large scale regression and classification problems, by using the Nyström method and preconditioning techniques. The Nyström approximation -- based on a subset of landmarks -- gives a low rank approximation of the kernel matrix, and is known to provide a form of impl…
VRSMD improves SMD convergence and has implicit regularization.
Semi-supervised method enhances recommendation from sparse data.
New method uses deep learning to solve linear inverse problems.
In this paper, we propose an implicit gradient descent algorithm for the classic -means problem. The implicit gradient step or backward Euler is solved via stochastic fixed-point iteration, in which we randomly sample a mini-batch gradient in every iteration. It is the average of the fixed-point trajectory that is c…
Double descent refers to the phase transition that is exhibited by the generalization error of unregularized learning models when varying the ratio between the number of parameters and the number of training samples. The recent success of highly over-parameterized machine learning models such as deep neural networks ha…
Semi-implicit variational inference (SIVI) is introduced to expand the commonly used analytic variational distribution family, by mixing the variational parameter with a flexible distribution. This mixing distribution can assume any density function, explicit or not, as long as independent random samples can be generat…
New method removes pseudo-label bias for unsupervised domain adaptation.
In implicit models, one often interpolates between sampled points in latent space. As we show in this paper, care needs to be taken to match-up the distributional assumptions on code vectors with the geometry of the interpolating paths. Otherwise, typical assumptions about the quality and semantics of in-between points…
Graph contrastive learning reveals unique inductive biases.
Adam's bias shifts from full-batch to max-margin of different norms for separable data.
Interpolates between SPG and NeuRD with Capped Implicit Exploration.