Adaptive workflow combines fast amortized inference with MCMC for many datasets.
problem Trade-off between computational speed and sampling accuracy in Bayesian inference.
method Adaptive workflow integrating amortized inference and MCMC with principled diagnostics.
result Efficiency gains with high posterior quality on tens of thousands of datasets.
PPI++ uses machine learning predictions to improve inference from small datasets.
problem Efficient inference from small labeled datasets with high-quality predictions.
method Adapts prediction-powered inference (PPI) to compute confidence sets for any parameter dimensionality.
result Improves classical intervals using only labeled data, always yielding better results.
The paper proposes an efficient method to scale Bayesian inference for mixed multinomial logit models to very large datasets.
problem Efficiency in Bayesian inference for mixed multinomial logit models on large datasets.
method Amortized Variational Inference with stochastic backpropagation, automatic differentiation, and GPU acceleration.
result The proposed method achieves significant computational speedups over traditional methods for large datasets.
Deep learning method infers causal interactions from data.
problem Causal inference from observational data.
method Transform input vectors to NEPDFs, train CNN on NEPDFs.
result Improves upon prior methods for causal inference.
A tutorial on variational inference for high-dimensional models.
problem Approximating marginal likelihood and posterior in Bayesian models.
method Parametric approach to variational inference.
result Variational inference is now preferred for high-dimensional models and large datasets.
Inferring a decision tree from a given dataset is one of the classic problems in machine learning. This problem consists of buildings, from a labelled dataset, a tree such that each node corresponds to a class and a path between the tree root and a leaf corresponds to a conjunction of features to be satisfied in this c…
Paper introduces attacks to infer GAN training dataset properties.
problem Security and privacy risks of generative models like GANs.
method Proposes a general attack pipeline for two attack scenarios.
result Demonstrates strong performance in inferring GAN training dataset properties.
This paper speeds up inference in large hierarchical models.
problem Inference in large hierarchical models is slow and difficult.
method Amortized variational inference with shared parameters.
result Amortized inference is as accurate as a full-rank Gaussian but much faster.
New method for efficient inference in large datasets.
problem Statistical inference in massive datasets.
method Combines divide-and-conquer method and empirical likelihood.
result Reduces computation burden and demonstrates effectiveness.
Poisoning datasets can reveal private details of other users' training points.
problem Integrity and privacy of machine learning training data.
method Active inference attacks that poison a small fraction of the training dataset.
result Poisoning as little as 0.1% of the training dataset can significantly boost inference attacks.
Stochastic variational inference for collapsed models has recently been successfully applied to large scale topic modelling. In this paper, we propose a stochastic collapsed variational inference algorithm in the sequential data setting. Our algorithm is applicable to both finite hidden Markov models and hierarchical D…
Exact selective inference with randomization for Gaussian regression models.
problem Exact selective inference in Gaussian regression models.
method Introduces a pivot for exact selective inference with randomization, reducing the problem to a bivariate truncated Gaussian distribution.
result Our pivot leads to exact inference and produces narrower confidence intervals than related methods.
Bayesian inference corrects bias in supervised learning datasets.
problem Correcting bias in supervised machine learning datasets.
method Bayesian inference framework to adjust posterior distribution.
result Improved model fitting to original dataset.
Valid inference from data and predictions.
problem Valid statistical inference with machine learning predictions.
method Framework for valid inference using machine learning predictions.
result Valid confidence intervals without assumptions on predictions.
Method scales Bayesian inference to large datasets and robustifies against outliers.
problem Scalability and robustness to outliers in Bayesian inference.
method Variational inference with β-divergence and Riemannian coresets. result Efficiently constructs cleansed data summaries robust to outliers.
Improves inference from sparse data with hybrid summary statistics.
problem Robust simulation-based inference from limited data.
method Augment traditional summary statistics with neural network outputs to maximize mutual information.
result Improves information extraction and makes inference robust in low-data settings.
New method learns causal models from data efficiently.
problem Learning Structural Causal Models from data is challenging.
method Amortized inference via Conditional Fixed-Point Iterations with transformer embeddings.
result Single model predicts causal mechanisms conditioned on data and graph.
ADASAP accelerates GP inference for large datasets.
problem Scaling Gaussian process inference to large datasets.
method Approximate sketch-and-project algorithm for solving linear systems.
result Sketch-and-project rapidly converges to true posterior mean.
Proposes a new model for better speech segmentation.
problem Improving speech segmentation accuracy.
method Integrates recurrent explicit duration variables into rSLDS and uses Pólya-gamma augmentation for inference.
result Demonstrates improved segmentation on various datasets.
CInA method uses attention to improve causal inference.
problem Challenges in causal inference, especially in complex tasks.
method CInA method utilizes self-supervised causal learning with multiple unlabeled datasets and transformer-type architecture.
result CInA effectively generalizes to out-of-distribution datasets and various real-world datasets.
Federated learning for Bayesian clustering of large datasets.
problem Bayesian model-based clustering of large-scale binary and categorical data.
method Federated variational inference with local merge and delete moves in parallel batches, followed by global merge moves.
result Empirical validation shows superior performance compared to existing algorithms.
Dataset inference defends against model stealing by identifying stolen training data.
problem Model stealing through machine learning models trained on proprietary data.
method Combines statistical testing with decision boundary estimation to identify stolen training data.
result Model owners can claim with high confidence that their model was stolen, even with limited stolen data.
We quantitatively investigate how machine learning models leak information about the individual data records on which they were trained. We focus on the basic membership inference attack: given a data record and black-box access to a model, determine if the record was in the model's training dataset. To perform members…
New algorithm speeds up IRT model fitting for large datasets.
problem Inefficient algorithms for fitting Item Response Theory models to large datasets.
method Stochastic Variational Inference for IRT models.
result Higher log likelihoods and accuracy in imputing missing data.
How can we perform efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distributions, and large datasets? We introduce a stochastic variational inference and learning algorithm that scales to large datasets and, under some mild dif…
Cross-prediction improves inference from small labeled datasets.
problem Valid inference from small labeled datasets with imperfect predictions.
method Imputes missing labels via machine learning and debiases predictions.
result Inferences achieve desired error probability and are more powerful.
Algorithm reduces variance in causal effect estimation from multiple datasets.
problem Unidentifiable average treatment effect in observational data due to selection bias.
method Constructs control variates using datasets where ATE is not identifiable to reduce variance.
result Significant reduction in variance of ATE estimate using control variates.
Proposes Causal-Batle for estimating treatment effects in small high-dimensional datasets.
problem Estimating treatment effects with small high-dimensional datasets.
method Adopts transfer learning techniques for causal inference.
result Improves treatment effect estimates in small high-dimensional datasets.
VTIRT speeds up IRT inference for dynamic learner proficiency.
problem Expensive and slow inference algorithms for dynamic IRT models.
method Variational Temporal IRT (VTIRT) for fast, accurate inference.
result Orders of magnitude speedup in inference runtime with accurate results.
Develops SGP-VAE for efficient sparse GP inference in multi-dimensional datasets.
problem Sparse GP approximations and missing data in multi-dimensional spatio-temporal datasets.
method Leverages partial inference networks for sparse GP approximations and amortized variational inference.
result Outperforms multi-output GPs and structured VAEs in various experiments.
Many decisions in healthcare, business, and other policy domains are made without the support of rigorous evidence due to the cost and complexity of performing randomized experiments. Using observational data to answer causal questions is risky: subjects who receive different treatments also differ in other ways that a…
SpaCE tackles spatial confounding in scientific studies.
problem Spatial confounding influences treatment and outcome, leading to spurious associations.
method Introduces SpaCE toolkit for benchmark datasets and tools to evaluate causal inference methods.
result Facilitates automated evaluation of machine learning and causal inference models.
Improves likelihood-free inference by using a new sampling approach to avoid biased data collection.
problem Efficient Bayesian inference without likelihood evaluation for real-world datasets.
method Introduces Neural Proposal (NP) to sample simulation inputs i.i.d. for unbiased posterior inference.
result Demonstrates improved performance, especially for multi-modal posteriors, through experiments.
Bayesian coresets improve scalable Bayesian inference.
problem Efficiently approximating posterior inference with a subset of data.
method Sparsity constrained optimization and accelerated optimization methods.
result Explicit convergence rate guarantees and superior performance compared to state-of-the-art.
We introduce a novel uncertainty estimation for classification tasks for Bayesian convolutional neural networks with variational inference. By normalizing the output of a Softplus function in the final layer, we estimate aleatoric and epistemic uncertainty in a coherent manner. The intractable posterior probability dis…
BBCI uses meta prediction to estimate causal effects from datasets.
problem Estimating causal effects from observed data.
method Meta prediction to learn causal effect estimation.
result BBCI accurately estimates ATEs and CATEs across various causal inference problems.
FastKCI speeds up KCI tests for causal inference on large datasets.
problem Cubic computational complexity of kernel-based conditional independence tests.
method Mixture-of-experts approach with parallel Gaussian process inference.
result Substantial computational speedups with maintained statistical power.
PClean automates Bayesian data cleaning for specific datasets.
problem Bayesian inference for diverse and complex data cleaning.
method Domain-specific probabilistic programming language with custom models and inference.
result PClean programs outperform general-purpose PPLs in accuracy and runtime.
Deep SCMs with deep learning infer counterfactuals from noisy data.
problem Inference of counterfactuals from noisy data in causal models.
method Normalizing flows and variational inference for deep SCMs.
result Tractable inference of exogenous noise variables for counterfactuals.
Bayesian methods improve inference for cumulative probit models on large datasets.
problem Challenges in Bayesian inference for large cumulative probit models.
method Proposed scalable algorithms using Variational Bayes and Expectation Propagation.
result Superior computational performance and accuracy compared to MCMC.
Membership inference attacks seek to infer the membership of individual training instances of a privately trained model. This paper presents a membership privacy analysis and evaluation system, called MPLens, with three unique contributions. First, through MPLens, we demonstrate how membership inference attack methods …
Federated learning leaks participant dataset quality even with secure aggregation.
problem Leakage of participant dataset quality in federated learning with secure aggregation.
method Image recognition experiments to infer and attribute dataset quality.
result Relative quality ordering of participants can be inferred and used for various purposes.
New method for efficient probabilistic inference using masked language modeling.
problem Efficient posterior inference in probabilistic programs with many hyper-parameters.
method Formulate inference as masked language modeling, train a neural network to unmask random values.
result Foundation posterior for zero-shot inference and fine-tuning across a range of programs.
Sparse Markovian Gaussian processes improve probabilistic model inference for large datasets.
problem Efficient inference for large-scale time series data.
method Combining inducing variables with Kalman filter-like recursions for linear scaling.
result General site-based approach for approximating non-Gaussian likelihoods.
Paper introduces a method to infer causal direction from limited data.
problem Inference of causal direction from limited observational data.
method Meta learning combined with causal inference to create a generative model.
result The method accurately infers causal direction across various dataset sizes.
Variational inference is a powerful tool for approximate inference. However, it mainly focuses on the evidence lower bound as variational objective and the development of other measures for variational inference is a promising area of research. This paper proposes a robust modification of evidence and a lower bound for…
Paper assesses holistic risks of inference attacks on ML models.
problem Lack of comprehensive risk assessment of inference attacks on ML models.
method Presented a threat model taxonomy for four inference attacks on five model architectures and four image datasets.
result Complexity of training dataset influences attack performance; model stealing and membership inference attacks are negatively correlated.
Stochastic variational inference (SVI), the state-of-the-art algorithm for scaling variational inference to large-datasets, is inherently serial. Moreover, it requires the parameters to fit in the memory of a single processor; this is problematic when the number of parameters is in billions. In this paper, we propose e…