arXiv research

NURD improves model performance by distilling representations independent of nuisance variables.

Machine learning in high-energy physics faces challenges from nuisance parameters, which are reviewed and techniques to mitigate their impact are discussed.

New method uses MMD estimators to enforce model invariance with missing data.

The joint PLDA model, is a generalization of PLDA where the nuisance variable is no longer considered independent across samples, but potentially shared (tied) across samples that correspond to the same nuisance condition. The original work considered a single nuisance condition, deriving the EM and scoring formulas fo…

New method for inference on strongly identified functionals even when nuisance functions are weakly identified.

We combine conditional variational autoencoders (VAE) with adversarial censoring in order to learn invariant representations that are disentangled from nuisance/sensitive variations. In this method, an adversarial network attempts to recover the nuisance variable from the representation, which the VAE is trained to pre…

New method for accurate permutation inference in CCA.

Probabilistic linear discriminant analysis (PLDA) is a method used for biometric problems like speaker or face recognition that models the variability of the samples using two latent variables, one that depends on the class of the sample and another one that is assumed independent across samples and models the within-c…

AutoBayes automates Bayesian graph exploration for robust machine learning.

DML addresses biases in machine learning by estimating nuisance functions.

Bayesian active learning tackles nuisance parameters, leading to bias and dilemmas.

We address the problem of inferring the causal effect of an exposure on an outcome across space, using observational data. The data is possibly subject to unmeasured confounding variables which, in a standard approach, must be adjusted for by estimating a nuisance function. Here we develop a method that eliminates the …

Learning representations which remain invariant to a nuisance factor has a great interest in Domain Adaptation, Transfer Learning, and Fair Machine Learning. Finding such representations becomes highly challenging in NLP tasks since the nuisance factor is entangled in a raw text. To our knowledge, a major issue is also…

We present a unified invariance framework for supervised neural networks that can induce independence to nuisance factors of data without using any nuisance annotations, but can additionally use labeled information about biasing factors to force their removal from the latent embedding for making fair predictions. Invar…

We propose the orthogonal random forest, an algorithm that combines Neyman-orthogonality to reduce sensitivity with respect to estimation error of nuisance parameters with generalized random forests (Athey et al., 2017)--a flexible non-parametric method for statistical estimation of conditional moment models using rand…

Data representations that contain all the information about target variables but are invariant to nuisance factors benefit supervised learning algorithms by preventing them from learning associations between these factors and the targets, thus reducing overfitting. We present a novel unsupervised invariance induction f…

Optimal tuning for estimating ECC in proportional asymptotics.

New method for unbinned, profiled unfolding in particle physics.

We propose a family of novel hierarchical Bayesian deep auto-encoder models capable of identifying disentangled factors of variability in data. While many recent attempts at factor disentanglement have focused on sophisticated learning objectives within the VAE framework, their choice of a standard normal as the latent…

The consistency of doubly robust estimators relies on consistent estimation of at least one of two nuisance regression parameters. In moderate to large dimensions, the use of flexible data-adaptive regression estimators may aid in achieving this consistency. However, $n^{1/2}$-consistency of doubly robust estimators is…

Optimally estimates a functional using nuisance function tuning and sample splitting.

New estimators for causal effects in DAGs with hidden variables, addressing computational and statistical challenges.

Approach for selecting features by discarding nuisance and correlated ones.

Study uses deep neural networks for inference in partially linear models with dependent data.

Proposes debiasing strategy for ill-posed regression problems.

We provide non-asymptotic excess risk guarantees for statistical learning in a setting where the population risk with respect to which we evaluate the target parameter depends on an unknown nuisance parameter that must be estimated from data. We analyze a two-stage sample splitting meta-algorithm that takes as input ar…

Supervised machine learning models often associate irrelevant nuisance factors with the prediction target, which hurts generalization. We propose a framework for training robust neural networks that induces invariance to nuisances through learning to discover and separate predictive and nuisance factors of data. We pre…

New convergence guarantees for learning with unknown nuisance parameters.

New method improves robustness of double robust estimators under complete misspecification.

New methods estimate causal effects through mediators, handling confounding without strict assumptions.

We consider estimating a low-dimensional parameter in an estimating equation involving high-dimensional nuisances that depend on the parameter. A central example is the efficient estimating equation for the (local) quantile treatment effect ((L)QTE) in causal inference, which involves as a nuisance the covariate-condit…

Bayesian method corrects bias in treatment effect estimation.

Framework for multi-task learning with semiparametric models and nuisance parameters.

ICA accurately estimates treatment effects even with confounders.

A new machine learning method handles nuisance parameters for better unfolding in particle physics.

Accuracy on in-distribution data correlates with out-of-distribution data when data is noisy or contains nuisance features.

We consider inference about a scalar parameter under a non-parametric model based on a one-step estimator computed as a plug in estimator plus the empirical mean of an estimator of the parameter's influence function. We focus on a class of parameters that have influence function which depends on two infinite dimensiona…

Paper models graph edge dependencies using latent variables for community detection.

New methods correct for time dependencies in IV regression for time series data.

Optimal first-order methods are shown to be fundamental limits in functional estimation.

New method corrects biased predictions and uncertainty estimates in classification with nuisance parameters.

Paper estimates noise covariance in correlated multi-task linear models.

New estimator optimizes black-box model errors in semiparametric estimation.

New algorithm profiles nuisance parameters for multidimensional deconvolution.

Corrects mismatch in consistency of nuisance estimators for doubly robust methods.

We review the problem of defining and inferring a "state" for a control system based on complex, high-dimensional, highly uncertain measurement streams such as videos. Such a state, or representation, should contain all and only the information needed for control, and discount nuisance variability in the data. It shoul…

Extends robust methods for causal inference, improving estimator performance.

Proposes P-learner for estimating treatment effects with proxy variables.