Proposes SVI for covariate-shift generalization with sparse variable independence.
problem Covariate-shift generalization with limited data and unstable variables.
method Introduces sparsity constraint and combines reweighting and selection in an iterative way.
result Improves covariate-shift generalization performance on synthetic and real-world datasets.
CARMS improves gradient estimation for categorical variables.
problem Accurately backpropagating gradients through categorical variables.
method CARMS combines REINFORCE with antithetic sampling to create unbiased gradient estimators.
result CARMS outperforms competing methods on various tasks.
ARMS improves gradient estimation for binary variables using antithetic samples.
problem Estimating gradients for binary variables in discrete latent variable models.
method ARMS uses antithetic samples generated by a copula to estimate gradients more efficiently and unbiasedly.
result ARMS outperforms competing methods in training generative models and optimizing variational bounds.
Proposes a method to select variables for kernel two-sample tests.
problem Determining whether two samples have the same distribution using informative variables.
method A framework based on kernel maximum mean discrepancy (MMD) for selecting a subset of variables.
result The sample size requirements for the three kernels depend on the number of selected variables, not the data dimension.
Mean representations of VAEs are correlated but still useful for tasks.
problem Correlation between mean and sampled representations of VAEs.
method Selective posterior collapse to identify active and passive variables.
result Passive variables in mean representations are correlated but uncorrelated in sampled ones.
New Gibbs sampling method improves MCMC efficiency.
problem Improving efficiency of Gibbs sampling.
method Non-uniform random scan with selection probability optimization.
result Non-uniform scan improves mixing time of Markov chain.
The paper introduces methods to identify key variables discriminating between two datasets.
problem Identifying variables that distinguish between two datasets.
method Introduces a mathematical notion of discriminating variables and proposes two methods for their selection.
result Proposed methods improve upon existing techniques in two-sample variable selection.
A/B testing improves marketing decisions by selecting effective stratification variables.
problem Improving the sensitivity of A/B testing through stratified sampling.
method Designing an algorithm to select a subset of stratification variables for variance reduction.
result The subset selection method outperforms other variance reduction techniques in A/B testing.
Random Forest variable importance is improved by class balancing techniques.
problem Class imbalance problem in machine learning.
method Proposed a variable selection algorithm using RF variable importance and its confidence interval.
result Our algorithm efficiently selects an optimal feature set, leading to improved prediction performance.
Two statistical tasks are shown to have equivalent sample complexity.
problem Determining if a function depends on only a few variables and identifying those variables.
method Proved statistical equivalence of feature selection and junta testing through sample complexity analysis.
result Brute-force algorithm is sample-optimal for both tasks with optimal sample size.
Develops a new sampling method for gauge theories.
problem Sampling from SU(N) gauge theories. method Gauge-equivariant flows for SU(N) variables. result Constructs a class of flows respecting matrix conjugation symmetry.
Proposes a new signal model for high-dimensional, small-sample-size data.
problem Signal detection in high-dimensional, small-sample-size datasets.
method Intrinsic signal model based on dynamical system assumption.
result Taguchi method effectively detects signals in the proposed model.
We study covariance matrix estimation for the case of partially observed random vectors, where different samples contain different subsets of vector coordinates. Each observation is the product of the variable of interest with a 0−1 Bernoulli random variable. We analyze an unbiased covariance estimator under this mod…
Optimal sampling strategy improves prediction accuracy with surrogate variables under measurement constraints.
problem Measurement-constrained datasets and lack of labeled data.
method A-optimality criterion for optimal sampling, leveraging surrogate variables.
result Achieves lower asymptotic variance and reduced empirical mean squared error.
Sharp concentration bounds for i.i.d. variables.
problem Controlling the tail probabilities of independent variables.
method Extension of Sanov's theorem using large deviations and information theory.
result Matching concentration and anti-concentration bounds for i.i.d. samples of any size.
Probabilistic linear discriminant analysis (PLDA) is a method used for biometric problems like speaker or face recognition that models the variability of the samples using two latent variables, one that depends on the class of the sample and another one that is assumed independent across samples and models the within-c…
New algorithm groups variables by ancestral relationships to improve causal graph estimation accuracy.
problem Difficulty in estimating causal graphs with small sample sizes relative to variables.
method CAG algorithm groups variables based on ancestral relationships, reducing complexity and improving accuracy.
result CAG outperforms existing methods in estimation accuracy and computation time.
New algorithm learns Bayesian network structures with fewer samples.
problem Learning Bayesian network structures with limited observational data.
method Active sampling strategy to select variables for observation.
result Active algorithm finds structures close to optimal with fewer samples.
Thompson sampling is an efficient algorithm for sequential decision making, which exploits the posterior uncertainty to address the exploration-exploitation dilemma. There has been significant recent interest in integrating Bayesian neural networks into Thompson sampling. Most of these methods rely on global variable u…
Enhanced Bayesian target encoding uses sampling techniques to improve model performance.
problem Improving target encoding for better model performance in machine learning.
method Using sampling techniques in Bayesian target encoding to extract intra-category distribution information.
result Improves generalization and reduces target leakage in machine learning models.
This work considers the problem of learning the structure of multivariate linear tree models, which include a variety of directed tree graphical models with continuous, discrete, and mixed latent variables such as linear-Gaussian models, hidden Markov models, Gaussian mixture models, and Markov evolutionary trees. The …
We study parameter inference in large-scale latent variable models. We first propose an unified treatment of online inference for latent variable models from a non-canonical exponential family, and draw explicit links between several previously proposed frequentist or Bayesian methods. We then propose a novel inference…
When can reliable inference be drawn in the "Big Data" context? This paper presents a framework for answering this fundamental question in the context of correlation mining, with implications for general large scale inference. In large scale data applications like genomics, connectomics, and eco-informatics the dataset…
We adapt a Markov Random Field learning algorithm for continuous variables.
problem Learning sparse pairwise Markov Random Fields with continuous variables.
method Adapted Vuffray et al. (2019) algorithm for continuous variables and provided analysis.
result Sample complexity scales logarithmically with the number of variables.
This paper proposes a general adaptive procedure for budget-limited predictor design in high dimensions called two-stage Sampling, Prediction and Adaptive Regression via Correlation Screening (SPARCS). SPARCS can be applied to high dimensional prediction problems in experimental science, medicine, finance, and engineer…
DisARM improves gradient estimation for binary latent variables.
problem Challenges in training models with discrete latent variables.
method Uses antithetic sampling over continuous augmentation.
result DisARM consistently outperforms ARM and baseline methods in log-likelihood and variance.
Langevin autoencoders improve deep latent variable models with efficient posterior sampling.
problem Efficient posterior sampling in deep latent variable models using MCMC.
method Amortized Langevin dynamics (ALD) replaces datapoint-wise sampling with encoder updates.
result ALD is valid as an MCMC algorithm with the target posterior as a stationary distribution.
Latent variable models improve RL by facilitating efficient learning and exploration.
problem Improving sample efficiency in reinforcement learning.
method Representation view of latent variable models for state-action value functions, incorporating kernel embeddings and UCB exploration.
result Established sample complexity of the proposed approach in online and offline settings, demonstrated superior performance in benchmarks.
GEEN uses deep learning to estimate unobserved variables from observed data.
problem Estimating unobserved variables in latent variable models.
method GEEN uses deep learning with Kullback-Leibler distance to map observed measurements to latent variable realizations.
result GEEN provides a method to identify and estimate latent variables in a class of models.
Motivation: Algorithms that discover variables which are causally related to a target may inform the design of experiments. With observational gene expression data, many methods discover causal variables by measuring each variable's degree of statistical dependence with the target using dependence measures (DMs). Howev…
Improves Bayesian optimization efficiency for mixed variable spaces.
problem Boosting sample efficiency in Bayesian optimization for mixed variable spaces.
method Proposes frequency modulated (FM) kernels to model complex dependencies across different types of variables.
result BO-FM outperforms competitors in various optimization problems.
Random forest hyperparameters affect variable selection in omics studies.
problem Impact of hyperparameters on variable selection in random forests.
method Two simulation studies using theoretical and empirical data.
result Hyperparameters influence variable selection more than the splitting strategy and sample fraction.
Optimized variable orderings improve autoregressive model performance.
problem Challenges in variable ordering affect autoregressive model efficiency.
method Learn graphical model structure to inform optimal variable orderings.
result Graph-informed orderings yield higher-fidelity samples.
We introduce a new approach to variable selection, called Predictive Correlation Screening, for predictor design. Predictive Correlation Screening (PCS) implements false positive control on the selected variables, is well suited to small sample sizes, and is scalable to high dimensions. We establish asymptotic bounds f…
Extends model-x framework to handle missing data.
problem Inability to control false selections in missing data settings.
method Posterior sampled imputation, univariate imputation, joint imputation and sampling knockoffs.
result Preserves theoretical guarantees of model-x framework in missing data setting.
Standard probabilistic linear discriminant analysis (PLDA) for speaker recognition assumes that the sample's features (usually, i-vectors) are given by a sum of three terms: a term that depends on the speaker identity, a term that models the within-speaker variability and is assumed independent across samples, and a fi…
New algorithm for learning RBMs with sparse latent variables.
problem Learning RBMs with sparse latent variables efficiently.
method Algorithm with time complexity O(n^(2^s+1)) for sparse RBMs.
result Improves learning time for RBMs with sparse latent variables.
Estimating the strength of dependency between two variables is fundamental for exploratory analysis and many other applications in data mining. For example: non-linear dependencies between two continuous variables can be explored with the Maximal Information Coefficient (MIC); and categorical variables that are depende…
We study the asymptotic properties of the adaptive Lasso in cointegration regressions in the case where all covariates are weakly exogenous. We assume the number of candidate I(1) variables is sub-linear with respect to the sample size (but possibly larger) and the number of candidate I(0) variables is polynomial with …
New method controls FDR for sparse GLMs, identifying positive and negative relationships.
problem Sparse GLMs with high-dimensional data and varying sample size.
method Debiased-Lasso estimator and CLIME method for precision matrix estimation.
result Asymptotically controls directional FDR and FDV for sparse GLMs.
Enhances diffusion-based sampling for molecular systems.
problem Inefficiency and thermodynamic mode miss in diffusion-based samplers for molecular systems.
method Introduces a sequential bias along collective variables (CVs) to encourage exploration and increase temperature in the projected space.
result Improves efficiency, mode discovery, and free energy estimation; first to demonstrate reactive sampling.
New algorithms for IV regression with streaming data, avoiding matrix inversions.
problem Instrumental variable regression with streaming data.
method Viewing IV regression as a stochastic optimization problem, developing algorithms that avoid matrix inversions and mini-batches.
result Rates of convergence of order O(logT/T) and O(1/T1−ι) for linear models. Categorical variables are a natural choice for representing discrete structure in the world. However, stochastic neural networks rarely use categorical latent variables due to the inability to backpropagate through samples. In this work, we present an efficient gradient estimator that replaces the non-differentiable sa…
Many random processes can be simulated as the output of a deterministic model accepting random inputs. Such a model usually describes a complex mathematical or physical stochastic system and the randomness is introduced in the input variables of the model. When the statistics of the output event are known, these input …
Efficient Bayesian variable selection for binomial and negative binomial data.
problem Computational challenges in Bayesian variable selection for complex models.
method Tempered Gibbs Sampling and MCMC scheme.
result Demonstrated effectiveness on cancer data with thousands of covariates.
Improved hierarchical discrete VAEs for better stability and performance.
problem Training stable and efficient hierarchical discrete VAEs with numerous latent variables.
method Introducing Relaxed-Responsibility Vector-Quantisation to parameterise discrete latent variables in a hierarchical structure.
result Achieved state-of-the-art bits-per-dim results for various standard datasets.
New method for LLMs to learn reasoning by optimizing latent variables.
problem Teaching LLMs to generate logical justifications for answers.
method Formalized reasoning as latent variable model, derived FEM objective, designed sampling schemes.
result Prompt Posterior Sampling (PPS) outperforms other schemes in learning to reason.
Paper proves causal direction can be inferred from data with limited randomness.
problem Inferring causal direction from observational data with limited randomness.
method Entropy measurement and structural causal models.
result Causal direction is identifiable for most causal models with limited entropy.