BONAS accelerates NAS while maintaining reliability.
problem Computational inefficiency in sample-based NAS.
method Bayesian Optimized Neural Architecture Search (BONAS) using weight-sharing.
result BONAS accelerates sample-based NAS significantly while maintaining reliability.
New method for Bayesian neural networks reduces inference difficulty.
problem Difficulty in sample-based inference for Bayesian neural networks.
method Embracing mode-connectedness to link overparameterization and sampling difficulty.
result Practical guidelines and deep ensemble approach for effective SBI.
A scalable method for Bayesian inference in large linear models.
problem High computational cost in Bayesian linear models for large networks.
method Sample-based inference and g-prior for hyperparameter selection.
result Linearised neural network inference on large datasets (ResNet-18, ResNet-50, U-Net).
Posterior sampling-based EI achieves sublinear regret bounds for expensive function optimization.
problem Theoretical analysis of expected improvement (EI) in Bayesian optimization.
method Randomized posterior sampling of EI.
result Achieves sublinear Bayesian cumulative regret bounds.
We study primal-dual type stochastic optimization algorithms with non-uniform sampling. Our main theoretical contribution in this paper is to present a convergence analysis of Stochastic Primal Dual Coordinate (SPDC) Method with arbitrary sampling. Based on this theoretical framework, we propose Optimality Violation-ba…
Given a data matrix X∈Rn×d and a response vector y∈Rn, suppose n>d, it costs O(nd2) time and O(nd) space to solve the least squares regression (LSR) problem. When n and d are both large, exactly solving the LSR problem is very expensive. When n≫d, one feasible approach to spee…
This work compares and evaluates various sampling methods for neural language models.
problem Lack of systematic comparison and myths about sampling methods.
method Monte Carlo sampling, importance sampling, compensated partial summation, noise contrastive estimation.
result All sampling methods can perform equally well if posterior probabilities are corrected.
Improves generalization with few samples using a new regularization method.
problem Training deep neural networks with limited data leads to overfitting.
method Sample-based regularization (SBR) to improve generalization without relying on source model knowledge.
result SBR outperformed existing methods in various configurations.
Improves generative models for cost-sensitive decisions.
problem Generative models lack awareness of decision costs.
method Integrates a decision loss into the training objective.
result Improves cost-sensitive forecast accuracy.
A new method improves SVI for high-dimensional, poorly-conditioned distributions.
problem Challenges in existing SVI methods for high-dimensional, poorly-conditioned distributions.
method Trust-region optimization approach leveraging conditional independences and second-order information.
result Superior numerical performance and better scalability in high-dimensional distributions.
The study proves sampling-based GNNs can approximate training on full graphs with small subgraphs.
problem Training Graph Neural Networks (GNNs) on large graphs is computationally expensive.
method Theoretical framework using graph local limits to prove approximation of GNN training on small samples.
result Parameters learned from sampling-based GNNs on small subgraphs are close to those on full graphs.
Domain adaptation has become a prominent problem setting in machine learning and related fields. This review asks the question: how can a classifier learn from a source domain and generalize to a target domain? We present a categorization of approaches, divided into, what we refer to as, sample-based, feature-based and…
This paper presents sampling-based speech parameter generation using moment-matching networks for Deep Neural Network (DNN)-based speech synthesis. Although people never produce exactly the same speech even if we try to express the same linguistic and para-linguistic information, typical statistical speech synthesis pr…
Deep unfolding accelerates MCMC-based COP solvers.
problem Optimizing combinatorial problems with MCMC and gradient descent.
method Combines MCMC and gradient descent, trains step sizes, uses variance estimation for non-differentiable MCMC.
result Significantly accelerates convergence speed for COPs.
Novel method decomposes configuration space for improved collision checking.
problem Improving collision checking in high-degree-of-freedom robot motion planning.
method Proposes a configuration space decomposition method to build a composite classifier.
result Composite classifier outperforms state-of-the-art single classifier methods.
A framework for hypothesis testing on attributed graphs using sampling.
problem Statistical testing on graph data, especially large attributed graphs.
method Sampling-based framework with PHASE and PHASEopt for accurate and efficient hypothesis testing.
result PHASE and PHASEopt improve accuracy and efficiency of hypothesis testing in attributed graphs.
Bayesian model-based reinforcement learning is a formally elegant approach to learning optimal behaviour under model uncertainty, trading off exploration and exploitation in an ideal way. Unfortunately, finding the resulting Bayes-optimal policies is notoriously taxing, since the search space becomes enormous. In this …
Efficient algorithm finds fast Transformer models.
problem Slow inference time of Transformer models.
method Decompose Transformer architecture into components, use sampling-based one-shot search.
result Achieved 10% to 30% speedup on pre-trained BERT and 70% on top of a previous state-of-the-art model.
New sampling-based approach for filtering problems using multiplicative Gaussian functions.
problem Approximate inference in filtering problems.
method Approximates distribution with a weighted sum of continuous functions using sampling for multiplications.
result Preliminary experiments show potential of the new method compared to particle filters.
Purpose: Machine learning is broadly used for clinical data analysis. Before training a model, a machine learning algorithm must be selected. Also, the values of one or more model parameters termed hyper-parameters must be set. Selecting algorithms and hyper-parameter values requires advanced machine learning knowledge…
Quantum algorithm improves portfolio construction accuracy.
problem Efficiently constructing portfolios with real-world constraints.
method Sampling-based CVaR Variational Quantum Algorithm (VQA) combined with local-search post-processing.
result Achieved a relative solution error of 0.49% on IBM Heron processors.
New method learns graph structure and uncertainty from data.
problem Learning latent graph structures and their uncertainty from data.
method Proposes a sampling-based method to learn latent graph structure and uncertainty simultaneously.
result Proves that suitable loss functions on stochastic model outputs solve both learning latent graph structure and achieving optimal predictions.
We propose a relaxation-based approximate inference algorithm that samples near-MAP configurations of a binary pairwise Markov random field. We experiment on MAP inference tasks in several restricted Boltzmann machines. We also use our underlying sampler to estimate the log-partition function of restricted Boltzmann ma…
To meet the standard of differential privacy, noise is usually added into the original data, which inevitably deteriorates the predicting performance of subsequent learning algorithms. In this paper, motivated by the success of improving predicting performance by ensemble learning, we propose to enhance privacy-preserv…
Efficiently certifies global robustness of large neural networks with probabilistic guarantees.
problem Certifying robustness of large neural networks in a scalable and efficient manner.
method Sampling an ε-net and invoking a local robustness oracle.
result Certifies a probabilistic relaxation of robustness efficiently and globally.
Paper proposes robust methods for estimating optimal treatment rules with censored survival data.
problem Estimating optimal treatment rules for censored survival data.
method Developed two robust criteria and a sampling-based difference-of-convex algorithm for learning optimal treatment rules.
result Proposed methods show improved performance compared to existing methods in simulations and real data.
Efficient clustering for large datasets using a sampling-based approach.
problem Clustering high-dimensional data with a large number of clusters efficiently.
method A simple and efficient clustering method that evaluates distances of data points with a subset of cluster centers.
result Optimal solutions of the approximation are the same as in the exact solution, but more efficient at extracting clusters.
We present an embedding of stochastic optimal control problems, of the so called path integral form, into reproducing kernel Hilbert spaces. Using consistent, sample based estimates of the embedding leads to a model free, non-parametric approach for calculation of an approximate solution to the control problem. This fo…
DGFS improves sampling from complex densities by optimizing partial trajectories.
problem Sampling from intractable high-dimensional density functions.
method DGFS uses a flow function to break down the training process into short partial trajectory segments, leveraging intermediate learning signals.
result DGFS achieves more accurate estimates of the normalization constant.
SDPG algorithm improves sample efficiency and reward in DRL for continuous action spaces.
problem Improving sample efficiency and reward in distributional reinforcement learning for continuous action spaces.
method SDPG algorithm models return distribution using samples via reparameterization technique.
result SDPG shows better sample efficiency and higher reward in OpenAI Gym environments.
GraphSAINT improves GCN training efficiency and accuracy with graph sampling.
problem Neighbor explosion problem in minibatch training of GCNs.
method GraphSAINT constructs minibatches by sampling the training graph, ensuring fixed well-connected nodes in all layers.
result GraphSAINT achieves new state-of-the-art F1 scores for PPI and Reddit.
HH-VAEM improves imputation and acquisition of missing data using hierarchical models and Hamiltonian Monte Carlo.
problem Imputation and acquisition of missing heterogeneous data.
method Hierarchical VAE model with Hamiltonian Monte Carlo and automatic hyper-parameter tuning.
result HH-VAEM outperforms existing methods in imputation and supervised learning tasks.
Neural solver computes Wasserstein geodesics and velocity fields efficiently.
problem Computing Wasserstein geodesics and velocity fields efficiently.
method Sample-based neural network approach to solve the minimax problem.
result Directly samples from target distribution and estimates velocity field.
EMAP finds minimal perturbations to change model predictions, combining feature weighting and counterfactuals.
problem Improving model explanations for black box classifiers.
method Neural network approach that returns minimal adversarial perturbations.
result EMAP provides more interpretable explanations and faster than sampling-based methods.
MPPN network improves long-term time series forecasting accuracy.
problem Inaccurate long-term time series forecasting due to noise and lack of interpretability.
method MPPN network constructs context-aware multi-resolution semantic units and employs multi-periodic pattern mining and channel adaptive module.
result MPPN significantly outperforms state-of-the-art methods on nine real-world benchmarks.
This paper tackles Sinkhorn DRO by reformulating it as a bilevel program and proposes sampling-based algorithms.
problem Distributionally robust optimization with ambiguity sets defined via the Sinkhorn discrepancy.
method Primal perspective reformulation as a bilevel program, double-loop and single-loop sampling-based algorithms.
result Simultaneously obtain the optimal robust decision and the worst-case distribution.
Learning in restricted Boltzmann machine is typically hard due to the computation of gradients of log-likelihood function. To describe the network state statistics of the restricted Boltzmann machine, we develop an advanced mean field theory based on the Bethe approximation. Our theory provides an efficient message pas…
A new online method estimates mode from samples using ODE analysis.
problem Estimating mode from samples when analytical density is unknown.
method Online iterative algorithm using ODE analysis for asymptotic convergence and stability.
result Asymptotic convergence and stability of estimates demonstrated.
Support Vector Data Description (SVDD) is a popular outlier detection technique which constructs a flexible description of the input data. SVDD computation time is high for large training datasets which limits its use in big-data process-monitoring applications. We propose a new iterative sampling-based method for SVDD…
New method for efficient marginalization of discrete latent variables in neural networks.
problem Computational challenges in training models with discrete latent variables.
method Parameterizing discrete distributions using sparse mappings (sparsemax and structured variants) to reduce support and enable efficient marginalization.
result Achieved good performance in various tasks with efficient and practical training.
Paper proposes first unlearning algorithm for MCMC models.
problem Enforcing right to be forgotten in AI causes high costs for data deletion.
method Converts MCMC unlearning to explicit optimization problem, designs MCMC influence function.
result MCMC unlearning does not compromise generalizability of models.
SBBO optimizes complex spaces using sampling-based models.
problem Optimizing complex spaces with discrete variables.
method Simulation Based Bayesian Optimization (SBBO) using sampling-based surrogate models.
result Empirical effectiveness of SBBO in combinatorial optimization.
Likelihood-free inference refers to inference when a likelihood function cannot be explicitly evaluated, which is often the case for models based on simulators. Most of the literature is based on sample-based `Approximate Bayesian Computation' methods, but recent work suggests that approaches based on deep neural condi…
Improved statistical computation through efficient matrix sampling.
problem Reducing computational cost in large-scale statistical methods.
method Accumulative sub-sampling method to improve statistical efficiency.
result Effective matrix size control improves computational efficiency.
Differentiable learning via SGD and GD can simulate various learning problems, depending on precision and minibatch size.
problem Understanding the power of differentiable learning via SGD and GD compared to statistical query (SQ) learning.
method Comparing the learning power of SGD and GD on population and empirical losses with statistical query learning.
result The learning power of SGD and GD depends on the precision of gradient calculations relative to the minibatch size or sample size.
MIRA scores assess conditional distribution accuracy using joint samples.
problem Assessing the accuracy of candidate conditional distributions.
method Analytic expression for Mira score based on equal probability mass regions.
result Mira enables Bayesian model comparison by quantifying alignment with true process.
Two formulae estimate sensitivity of random vectors to distributional parameters.
problem Estimating sensitivity of random vectors to distributional parameters.
method Two analytical formulae and four numerical algorithms.
result Validated numerical algorithms and demonstrated effectiveness.
Bounded rationality investigates utility-optimizing decision-makers with limited information-processing power. In particular, information theoretic bounded rationality models formalize resource constraints abstractly in terms of relative Shannon information, namely the Kullback-Leibler Divergence between the agents' pr…