arXiv research

WALNUTS improves sampling efficiency and robustness for multi-scale distributions.

Adaptive HMC improves sampling efficiency by optimizing mass matrix.

We present a novel technique for learning the mass matrices in samplers obtained from discretized dynamics that preserve some energy function. Existing adaptive samplers use Riemannian preconditioning techniques, where the mass matrices are functions of the parameters being sampled. This leads to significant complexiti…

Model predicts alternating market dominance for two competing firms.

Lower bounds on MALA and HMC for well-conditioned distributions.

GIST adapts HMC by tuning parameters based on position and momentum.

HMC with leapfrog integrator mixes faster than MALA under certain smoothness conditions.

We introduce a recent symplectic integration scheme derived for solving physically motivated systems with non-separable Hamiltonians. We show its relevance to Riemannian manifold Hamiltonian Monte Carlo (RMHMC) and provide an alternative to the currently used generalised leapfrog symplectic integrator, which relies on …

A new clustering method improves recovery guarantees by re-embedding data.

Hamiltonian Monte Carlo on ReLU networks is inefficient due to large local error.

ATLAS adapts HMC step size and trajectory length for complex geometries.

Hamiltonian Monte Carlo (HMC) is a state-of-the-art Markov chain Monte Carlo sampling algorithm for drawing samples from smooth probability densities over continuous spaces. We study the variant most widely used in practice, Metropolized HMC with the Störmer-Verlet or leapfrog integrator, and make two primary contribut…

Hamiltonian Monte Carlo (HMC) is a widely deployed method to sample from high-dimensional distributions in Statistics and Machine learning. HMC is known to run very efficiently in practice and its popular second-order "leapfrog" implementation has long been conjectured to run in $d^{1/4}$ gradient evaluations. Here we …

The hybrid Monte Carlo algorithm (HMCA) is applied for Bayesian parameter estimation of the realized stochastic volatility (RSV) model. Using the 2nd order minimum norm integrator (2MNI) for the molecular dynamics (MD) simulation in the HMCA, we find that the 2MNI is more efficient than the conventional leapfrog integr…

Hamiltonian Monte Carlo (HMC) exploits Hamiltonian dynamics to construct efficient proposals for Markov chain Monte Carlo (MCMC). In this paper, we present a generalization of HMC which exploits \textit{non-canonical} Hamiltonian dynamics. We refer to this algorithm as magnetic HMC, since in 3 dimensions a subset of th…

New HMC method uses asymmetrical momentum distributions and improves performance.

This paper analyzes the convergence of dynamic HMC and NUTS methods.

New method improves sampling efficiency for complex distributions.

MALA mixes optimally in κ√d steps for log-concave sampling.

A new method improves actor-critic RL by integrating HMC, enhancing policy distribution and exploration.

LAVA values data without needing a specific learning algorithm.

Stochastic methods with coordinate-wise adaptive stepsize (such as RMSprop and Adam) have been widely used in training deep neural networks. Despite their fast convergence, they can generalize worse than stochastic gradient descent. In this paper, by revisiting the design of Adagrad, we propose to split the network par…

Sequence-to-sequence (seq2seq) based ASR systems have shown state-of-the-art performances while having clear advantages in terms of simplicity. However, comparisons are mostly done on speaker independent (SI) ASR systems, though speaker adapted conventional systems are commonly used in practice for improving robustness…

AdaPTS adapts univariate FMs for multivariate time series forecasting.

We propose a new concept named adaptive submodularity ratio to study the greedy policy for sequential decision making. While the greedy policy is known to perform well for a wide variety of adaptive stochastic optimization problems in practice, its theoretical properties have been analyzed only for a limited class of p…

Adaptive sequential decision making is one of the central challenges in machine learning and artificial intelligence. In such problems, the goal is to design an interactive policy that plans for an action to take, from a finite set of $n$ actions, given some partial observations. It has been shown that in many applicat…

New algorithm reduces interventional strategy complexity for causal graph discovery.

Automation of machine learning model development is increasingly becoming an established research area. While automated model selection and automated data pre-processing have been studied in depth, there is, however, a gap concerning automated model adaptation strategies when multiple strategies are available. Manually…

Paper presents a Transformer model for automatic domain adaptation.

cKAM improves adaptive sampling by incorporating a cyclical stepsize scheme.

Adaptive variational Bayes framework improves inference adaptively.

New approach shows AI can adapt like toddlers by correcting old knowledge.

This paper improves neural network generalization by dynamically learning kernel parameters.

Study on distributed nonparametric function estimation with optimal rate and cost of adaptation.

This paper explains why Adam generalizes worse than SGD by analyzing its components.

New adaptive importance samplers improve stability and accuracy.

Adaptive networks improve model robustness through conditional normalization.

In domain adaptation, classifiers with information from a source domain adapt to generalize to a target domain. However, an adaptive classifier can perform worse than a non-adaptive classifier due to invalid assumptions, increased sensitivity to estimation errors or model misspecification. Our goal is to develop a doma…

FLAP adapts policies quickly to new tasks using shared linear representations.

Learn to automatically plug domain-specific modules into a common network.

FLoE adapts LLMs by selectively deploying LoRA adapters based on layer importance and task requirements.

The paper improves generalization bounds for domain adaptation.

New protocols show 1-bit mean estimation can be order-optimal without interaction.

New adaptive attacks bypass many defenses to adversarial examples.

We study methods for aggregating pairwise comparison data in order to estimate outcome probabilities for future comparisons among a collection of n items. Working within a flexible framework that imposes only a form of strong stochastic transitivity (SST), we introduce an adaptivity index defined by the indifference se…

AdapTable adapts tabular models to shifts without source data, improving HELOC performance.

The paper addresses statistical inference issues in adaptive experiments.

Enhances physics-informed neural networks with adaptive sampling and weighting.