arXiv research

New study on No-U-Turn Sampler for accelerated mixing in Hamiltonian Monte Carlo.

Accelerates sampling from Gibbs distributions using ARWP method.

RHMC accelerates sampling from log-concave distributions.

Paper tackles NAS problem by modeling it as a sparse supernet.

The paper proposes an efficient method to scale Bayesian inference for mixed multinomial logit models to very large datasets.

In this paper, we propose a novel technique to implement stochastic gradient methods, which are beneficial for learning from large datasets, through accelerated stochastic dynamics. A stochastic gradient method is based on mini-batch learning for reducing the computational cost when the amount of data is large. The sto…

Continuized Nesterov acceleration accelerates stochastic gradient descent and gossip algorithms.

In this paper we explore acceleration techniques for large scale nonconvex optimization problems with special focuses on deep neural networks. The extrapolation scheme is a classical approach for accelerating stochastic gradient descent for convex optimization, but it does not work well for nonconvex optimization typic…

Inference of latent feature models in the Bayesian nonparametric setting is generally difficult, especially in high dimensional settings, because it usually requires proposing features from some prior distribution. In special cases, where the integration is tractable, we can sample new feature assignments according to …

Anderson acceleration (or Anderson mixing) is an efficient acceleration method for fixed point iterations $x_{t+1}=G(x_t)$, e.g., gradient descent can be viewed as iteratively applying the operation $G(x) \triangleq x-α\nabla f(x)$. It is known that Anderson acceleration is quite efficient in practice and can be viewed…

Mixed likelihood GPs improve model performance in human-in-the-loop experiments.

Mixed RL improves RL efficiency with dual representations.

A fundamental problem in Bayesian inference and statistical machine learning is to efficiently sample from multimodal distributions. Due to metastability, multimodal distributions are difficult to sample using standard Markov chain Monte Carlo methods. We propose a new sampling algorithm based on a birth-death mechanis…

This paper presents novel mixed-type Bayesian optimization (BO) algorithms to accelerate the optimization of a target objective function by exploiting correlated auxiliary information of binary type that can be more cheaply obtained, such as in policy search for reinforcement learning and hyperparameter tuning of machi…

Efficient Bitwidth Search optimizes neural network quantization for better performance.

Post-training quantization method using multiple low-precision points achieves higher precision for critical weights.

We present a new framework for Hermite fractional financial markets, generalizing the fractional Brownian motion and fractional Rosenblatt markets. Considering pure and mixed Hermite markets, we introduce a strategy-specific arbitrage tax on the rate of transaction volume acceleration of the hedging portfolio as the pr…

We present a novel algorithm for overcomplete independent components analysis (ICA), where the number of latent sources k exceeds the dimension p of observed variables. Previous algorithms either suffer from high computational complexity or make strong assumptions about the form of the mixing matrix. Our algorithm does…

Bayesian optimization reduces materials design costs by 10x.

Convolutional neural networks (CNNs) are commonly trained using a fixed spatial image size predetermined for a given model. Although trained on images of aspecific size, it is well established that CNNs can be used to evaluate a wide range of image sizes at test time, by adjusting the size of intermediate feature maps.…

L2O uses ML to optimize traditional optimization techniques.

This paper analyzes and improves monotonic accelerated algorithms like M-NAG and M-FISTA.

MixML unifies analysis of weakly consistent parallel learning.

Adaptive tuning of latent space for non-stationary data.

Pseudo-marginal Metropolis-Hastings (pmMH) is a powerful method for Bayesian inference in models where the posterior distribution is analytical intractable or computationally costly to evaluate directly. It operates by introducing additional auxiliary variables into the model and form an extended target distribution, w…

This paper uses MIO to select features for kernel SVM classification.

Mixed-precision CA-SGD for generalized linear models on GPUs

GADD accelerates uniform-rate discrete diffusion models by 2 orders of magnitude.

New method for mixed-variable GSA improves material design efficiency.

Accelerates Birkhoff projection for manifold-constrained hyper-connections with high accuracy and speed.

Faster convergence of kernel mean embeddings using variance information.

New PDMP samplers improve BNN inference with accelerated computation.

Although Bayesian Optimization (BO) has been employed for accelerating materials design in computational materials engineering, existing works are restricted to problems with quantitative variables. However, real designs of materials systems involve both qualitative and quantitative design variables representing materi…

Recently developed techniques have made it possible to quickly learn accurate probability density functions from data in low-dimensional continuous space. In particular, mixtures of Gaussians can be fitted to data very quickly using an accelerated EM algorithm that employs multiresolution kd-trees (Moore, 1999). In thi…

Study reviews tree-based methods and introduces new ensemble strategies.

MER algorithm speeds up VI solving with Markovian data.

Exploiting sparsity enables hardware systems to run neural networks faster and more energy-efficiently. However, most prior sparsity-centric optimization techniques only accelerate the forward pass of neural networks and usually require an even longer training process with iterative pruning and retraining. We observe t…

Spectral clustering approaches have led to well-accepted algorithms for finding accurate clusters in a given dataset. However, their application to large-scale datasets has been hindered by computational complexity of eigenvalue decompositions. Several algorithms have been proposed in the recent past to accelerate spec…

2D-PT improves sampling in constrained optimization problems.

Sparse learning has recently received increasing attention in many areas including machine learning, statistics, and applied mathematics. The mixed-norm regularization based on the l1q norm with q>1 is attractive in many applications of regression and classification in that it facilitates group sparsity in the model. T…

Despite their exceptional flexibility and popularity, the Monte Carlo methods often suffer from slow mixing times for challenging statistical physics problems. We present a general strategy to overcome this difficulty by adopting ideas and techniques from the machine learning community. We fit the unnormalized probabil…

Bayesian optimization speeds up bioprocess development across scales.

Unified framework for constrained diffusion models on nonconvex sets with efficient landing mechanism.

Generalizes neural network verification by adding arbitrary cutting planes.

Neural-g models mixtures of densities with flexibility and accuracy.

Accelerates Riemannian gradient methods with extrapolation.

We propose a Markov chain Monte Carlo (MCMC) algorithm based on third-order Langevin dynamics for sampling from distributions with log-concave and smooth densities. The higher-order dynamics allow for more flexible discretization schemes, and we develop a specific method that combines splitting with more accurate integ…

AutoScale improves LLM pre-training by adjusting data mixtures at different scales.