We summarize our recent findings, where we proposed a framework for learning a Kolmogorov model, for a collection of binary random variables. More specifically, we derive conditions that link outcomes of specific random variables, and extract valuable relations from the data. We also propose an algorithm for computing …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “binary random variables”
We characterize and study variable importance (VIMP) and pairwise variable associations in binary regression trees. A key component involves the node mean squared error for a quantity we refer to as a maximal subtree. The theory naturally extends from single trees to ensembles of trees and applies to methods like rando…
Paper analyzes impact of PRM on binary random variables and distribution shifts.
problem Impact of performative risk minimization on binary random variables and distribution shifts.
method Formulated two measures of impact, derived explicit formulas for full information, and provided estimators for partial information.
result PRM can have amplified side effects compared to methods that do not model data shift.
BEGIN network models binary data without parametric assumptions.
problem Conditional independence in non-parametric families of binary data.
method BEGIN network models binary data using sparse linear representations and block factorizations.
result BEGIN network captures conditional independence for arbitrary binary and multinomial variables.
Recently, the binary expansion testing framework was introduced to test the independence of two continuous random variables by utilizing symmetry statistics that are complete sufficient statistics for dependence. We develop a new test based on an ensemble approach that uses the sum of squared symmetry statistics and di…
Rule-based classifiers quantify uncertainty using Bernoulli random variables.
problem Quantifying the uncertainty of precision estimates for rule-based text classifiers.
method Treat partitions of sub-strings as Bernoulli random variables, compare means using statistical tests, and combine classifiers using Dempster-Shafer theory.
result The approach can be used to combine binary classifiers into a multi-label classifier.
The Bouncy Particle Sampler is a novel rejection-free non-reversible sampler for differentiable probability distributions over continuous variables. We generalize the algorithm to piecewise differentiable distributions and apply it to generic binary distributions using a piecewise differentiable augmentation. We illust…
We propose a mixture of latent trait models with common slope parameters (MCLT) for model-based clustering of high-dimensional binary data, a data type for which few established methods exist. Recent work on clustering of binary data, based on a -dimensional Gaussian latent variable, is extended by incorporating com…
Study estimates heterogeneous principal causal effects with binary treatments and intermediate variables.
problem Estimating subgroup effects within strata defined by potential values of an intermediate variable.
method Proposes a framework for estimating and forming confidence intervals for heterogeneous principal causal effects under principal ignorability assumption. Develops several estimators with varying robustness properties.
result Established large-sample theory and analyzed bias contributions of each approach.
A note on extending Chernoff bound for unit interval random variables.
problem Extending the Chernoff bound to random variables in the unit interval.
method Proof of extension of the Chernoff bound.
result A proof provided for the extension of the Chernoff bound.
New insights into identifying mixtures of product distributions using Hadamard extensions.
problem Identifying mixtures of product distributions on binary variables.
method Analysis of Hadamard extensions of matrix products.
result Conditions for full column rank of Hadamard extensions.
DisARM improves gradient estimation for binary latent variables.
problem Challenges in training models with discrete latent variables.
method Uses antithetic sampling over continuous augmentation.
result DisARM consistently outperforms ARM and baseline methods in log-likelihood and variance.
Algorithm identifies probability distributions from noisy moments with minimal samples.
problem Identifying a probability distribution from its first noisy moments.
method Uses samples to identify a -mixture with runtime.
result Achieves optimal sample complexity and runtime for identifying -mixtures.
The restricted Boltzmann machine is a graphical model for binary random variables. Based on a complete bipartite graph separating hidden and observed variables, it is the binary analog to the factor analysis model. We study this graphical model from the perspectives of algebraic statistics and tropical geometry, starti…
To backpropagate the gradients through stochastic binary layers, we propose the augment-REINFORCE-merge (ARM) estimator that is unbiased, exhibits low variance, and has low computational complexity. Exploiting variable augmentation, REINFORCE, and reparameterization, the ARM estimator achieves adaptive variance reducti…
Sequence-to-sequence text-to-speech (TTS) is dominated by soft-attention-based methods. Recently, hard-attention-based methods have been proposed to prevent fatal alignment errors, but their sampling method of discrete alignment is poorly investigated. This research investigates various combinations of sampling methods…
Stochastic gradient Langevin dynamics (SGLD) is a computationally efficient sampler for Bayesian posterior inference given a large scale dataset. Although SGLD is designed for unbounded random variables, many practical models incorporate variables with boundaries such as non-negative ones or those in a finite interval.…
New gradient estimators for discrete variables improve model training.
problem Training models with discrete latent variables is challenging due to high gradient variance.
method Introduced novel gradient estimators based on importance sampling and statistical couplings, extending to categorical variables.
result Proposed gradient estimators outperform previous methods in systematic experiments.
We study the problem of recovering the latent ground truth labeling of a structured instance with categorical random variables in the presence of noisy observations. We present a new approximate algorithm for graphs with categorical variables that achieves low Hamming error in the presence of noisy vertex and edge obse…
We present and study models of adversarial online learning where the feedback observed by the learner is noisy, and the feedback is either full information feedback or bandit feedback. Specifically, we consider binary losses xored with the noise, which is a Bernoulli random variable. We consider both a constant noise r…
We consider analysis of relational data (a matrix), in which the rows correspond to subjects (e.g., people) and the columns correspond to attributes. The elements of the matrix may be a mix of real and categorical. Each subject and attribute is characterized by a latent binary feature vector, and an inferred matrix map…
DiffPrune: Neural Network Pruning with Deterministic Approximate Binary Gates and Regularizationstat.ML
A new method prunes neural networks efficiently without losing effectiveness.
problem Efficient pruning of neural networks without sacrificing performance.
method Deterministic approximation of binary gates and regularization.
result Pruning neural networks significantly without loss in effectiveness.
Due to the high variance of policy gradients, on-policy optimization algorithms are plagued with low sample efficiency. In this work, we propose Augment-Reinforce-Merge (ARM) policy gradient estimator as an unbiased low-variance alternative to previous baseline estimators on tasks with binary action space, inspired by …
In this paper I present an extended implementation of the Random ferns algorithm contained in the R package rFerns. It differs from the original by the ability of consuming categorical and numerical attributes instead of only binary ones. Also, instead of using simple attribute subspace ensemble it employs bagging and …
Probabilistic learning for binary classification with categorical variables.
problem Binary classification with categorical covariates.
method Probabilistic analysis and two algorithms for learning boolean functions.
result Effective learning of boolean functions from binary data.
Machine learning struggles to predict binary options movements due to randomness.
problem Predicting binary options movements using machine learning.
method Tested multiple machine learning models (RF, LR, GB, kNN) and neural networks (MLP, LSTM) on EUR/USD currency pairs.
result None of the models surpassed the ZeroR baseline accuracy, indicating randomness in binary options.
Paper proposes a QUBO formulation that reduces binary variables in Bayesian network learning.
problem Reducing the number of binary variables in QUBO formulations for Bayesian network learning.
method Proposes a new QUBO formulation that minimizes binary variables.
result Significantly reduces the number of binary variables required for Bayesian network structure learning.
Graphical models are commonly used tools for modeling multivariate random variables. While there exist many convenient multivariate distributions such as Gaussian distribution for continuous data, mixed data with the presence of discrete variables or a combination of both continuous and discrete variables poses new cha…
Two novel methods estimate multiple FDR directions for binary categorical responses.
problem Estimating multiple FDR directions for categorical responses.
method Information maximization and square loss mutual information.
result Statistical consistency of the proposed methods established.
Paper studies binary random projections with controllable sparsity patterns for computational and accuracy advantages.
problem Improving computational efficiency and accuracy in random projections.
method Proposes two sparse binary projection models with controllable sparsity patterns.
result Significant computational advantages and improved accuracies in empirical evaluations.
G-Net constructs binary neural networks with high accuracy using randomized binary embeddings.
problem Creating high-accuracy binary neural networks with theoretical guarantees.
method Proposes a novel floating-point G-Net family with randomized binary embeddings and theoretical accuracy guarantees.
result Empirically, G-Net achieves almost 30% higher accuracy on CIFAR-10 compared to prior HDC models.
New inequality for ternary variables improves on existing measures.
problem Analyzing excess losses and weighted majority votes with ternary random variables.
method Developed a split-kl inequality and its PAC-Bayes extension.
result Outperforms existing inequalities in certain regimes.
Low bit-width weights and activations are an effective way of combating the increasing need for both memory and compute power of Deep Neural Networks. In this work, we present a probabilistic training method for Neural Network with both binary weights and activations, called BLRNet. By embracing stochasticity during tr…
Background: Choosing the most performing method in terms of outcome prediction or variables selection is a recurring problem in prognosis studies, leading to many publications on methods comparison. But some aspects have received little attention. First, most comparison studies treat prediction performance and variable…
A new method identifies class-specific covariates in multi-class prediction tasks.
problem Identifying covariates specifically associated with one or more outcome classes in multi-class prediction tasks.
method Introducing multi forests (MuFs) with multi-way and binary splits to measure class-associated discriminatory ability.
result The multi-class VIM specifically ranks class-associated covariates highly, unlike conventional VIMs.
The belief propagation (BP) algorithm is widely applied to perform approximate inference on arbitrary graphical models, in part due to its excellent empirical properties and performance. However, little is known theoretically about when this algorithm will perform well. Using recent analysis of convergence and stabilit…
The most likely common causephysics.data-an
New method identifies common cause in causal insufficiency, revealing complex phase transitions.
problem Identifying common cause in causal insufficiency with observed joint probability.
method Generalized maximum likelihood method, closely related to maximum entropy principle.
result Identifies consistent common cause that aligns with the common cause principle.
We introduce a novel mechanism to tighten the local polytope relaxation for MAP inference in Markov random fields with low state space variables. We consider a surjection of the variables to a set of hyper-variables and apply the local polytope relaxation over these hyper-variables. The state space of each individual h…
To model categorical response variables given their covariates, we propose a permuted and augmented stick-breaking (paSB) construction that one-to-one maps the observed categories to randomly permuted latent sticks. This new construction transforms multinomial regression into regression analysis of stick-specific binar…
Learning Planar Ising Modelsstat.ML
Inference and learning of graphical models are both well-studied problems in statistics and machine learning that have found many applications in science and engineering. However, exact inference is intractable in general graphical models, which suggests the problem of seeking the best approximation to a collection of …
This paper describes a recursive estimation procedure for multivariate binary densities (probability distributions of vectors of Bernoulli random variables) using orthogonal expansions. For covariates, there are basis coefficients to estimate, which renders conventional approaches computationally prohibitive …
Paper extends FOFC algorithm to work with mixed data types.
problem Designing causal discovery algorithms for mixed data types.
method Proves tetrad constraint can be entailed for mixed data types and applies FOFC algorithm.
result FOFC algorithm can work on mixed data types.
We propose a new randomized ensemble technique with a provable security guarantee against black-box transfer attacks. Our proof constructs a new security problem for random binary classifiers which is easier to empirically verify and a reduction from the security of this new model to the security of the ensemble classi…
CARMS improves gradient estimation for categorical variables.
problem Accurately backpropagating gradients through categorical variables.
method CARMS combines REINFORCE with antithetic sampling to create unbiased gradient estimators.
result CARMS outperforms competing methods on various tasks.
Upcoming synoptic surveys are set to generate an unprecedented amount of data. This requires an automatic framework that can quickly and efficiently provide classification labels for several new object classification challenges. Using data describing 11 types of variable stars from the Catalina Real-Time Transient Surv…
Develops RF-GLS for binary geospatial data.
problem Challenges in extending RF to binary geospatial data.
method Proposes RF-GLS for binary data, embedding it in generalized mixed effects models.
result Establishes consistency of RF-GP for mean function and covariate effect estimation.
Learning Planar Ising Modelsstat.ML
Inference and learning of graphical models are both well-studied problems in statistics and machine learning that have found many applications in science and engineering. However, exact inference is intractable in general graphical models, which suggests the problem of seeking the best approximation to a collection of …
ARMS improves gradient estimation for binary variables using antithetic samples.
problem Estimating gradients for binary variables in discrete latent variable models.
method ARMS uses antithetic samples generated by a copula to estimate gradients more efficiently and unbiasedly.
result ARMS outperforms competing methods in training generative models and optimizing variational bounds.