Study shows not all ML models are uniquely identifiable from data.
problem Identifiability issues in machine learning models.
method Investigated through a case study on gait dynamics using a bipedal-spring mass model.
result Some parameters can be identified, but others remain unidentifiable.
Estimating tree structured Gaussian Graphical Model from noisy data.
problem Recover the original independence structure from noisy observations.
method Address the unidentifiability of tree structured graphical models and provide an algorithm to find the equivalence class of trees.
result An O(n^3) algorithm to find the equivalence class of trees.
Paper tackles robust estimation of tree-structured Ising models without side information.
problem Learning tree-structured Ising models with flipped signs of variables.
method Proves unidentifiability, proposes an algorithm with logarithmic sample complexity and polynomial run-time complexity.
result Empirically demonstrates robustness of proposed algorithm in the flipped signs setting.
Develops a method to estimate policy values robustly in the presence of confounding variables.
problem Infinite-horizon reinforcement learning with unobserved confounding variables makes policy evaluation unidentifiable.
method Robust approach estimating sharp bounds on policy value using optimization over state-occupancy ratios and sensitivity model.
result Proves convergence to sharp bounds as more confounded data is collected.
The paper reviews identifiability in linear and nonlinear models, from Gaussian to non-Gaussian.
problem Identifiability issues in latent-variable and structural-equation models, especially in nonlinear cases.
method Review of identifiability theory for linear and nonlinear models, including factor analysis and structural equation models.
result Even nonparametric nonlinear models can be estimated with additional assumptions.
In this paper we investigate the geometry of the likelihood of the unknown parameters in a simple class of Bayesian directed graphs with hidden variables. This enables us, before any numerical algorithms are employed, to obtain certain insights in the nature of the unidentifiability inherent in such models, the way pos…
EiGLasso speeds up sparse Kronecker-sum covariance estimation.
problem Sparse Kronecker-sum inverse covariance estimation challenges in scalability and parameter identification.
method Newton's method combined with eigendecomposition of sample and feature graphs, approximating Hessian for speed.
result Two to three orders-of-magnitude speed-up on simulated and real-world data.
Scientists develop a model to identify treatment responders from non-responders.
problem Analyzing samples that respond to treatment in studies.
method Causal two-groups (C2G) model, empirical Bayes procedures.
result The C2G model controls false discovery rate and has near-optimal power.
We identify action representations from video data, proving their statistical benefits.
problem Identifying latent action policies from video data.
method Entropy-regularized LAPO objective, formalizing desiderata for action representations.
result Entropy-regularized LAPO identifies action representations satisfying desiderata under suitable conditions.
New method refines prediction intervals for individual treatment effects using cross-world correlation.
problem Uncertainty in individual treatment effects for high-stakes decisions.
method Introduces cross-world correlation parameter ρ to refine prediction intervals for individual treatment effects.
result Achieves more stable and accurate coverage of prediction intervals for individual treatment effects.
New issue found in value-based reinforcement learning for stochastic environments.
problem Value-based reinforcement learning struggles with stochastic state transitions.
method Demonstrated using a multiobjective Markov Decision Process (MOMDP).
result Approaches may converge to Pareto-dominated solutions instead of optimal ones.
Transfer learning improves NILM across different appliance types and domains.
problem Recovering source appliances from mains data is challenging due to unidentifiability.
method Proposed two transfer learning schemes: appliance transfer learning and cross-domain transfer learning.
result Seq2point learning can be transferred across different domains with fine tuning required only for fully connected layers.
Algorithm bounds causal queries under selection bias.
problem Selection bias affects causal analysis.
method Proposes a new algorithm to address both identifiable and unidentifiable causal queries.
result The likelihood of available data is unimodal, enabling bounds on causal queries.
Bayes classifier cannot be learned from noisy labels without knowing noise distribution.
problem Learning a Bayes classifier from noisy labels when the noise distribution is unknown.
method Demonstrates the identifiability issues and proposes a simple algorithm for learning the Bayes decision rule.
result The Bayes decision rule is generally unidentified and cannot be learned without knowing the noise distribution.
New method identifies causal relationships in presence of hidden variables.
problem Identifying causal relationships when hidden variables exist.
method Established sufficient conditions and introduced a search algorithm.
result Proved soundness and completeness of the search algorithm.
Independent component analysis (ICA) is a cornerstone of modern data analysis. Its goal is to recover a latent random vector S with independent components from samples of X=AS where A is an unknown mixing matrix. Critically, all existing methods for ICA rely on and exploit strongly the assumption that S is not Gaussian…
Criterion extends identifiability for continuous mixtures of kernels.
problem Identify continuous mixtures of kernels.
method Generating-function accessibility criterion based on moment-generating functions or Laplace transforms.
result Criterion applies to mixtures of discrete and continuous variables.
New framework estimates treatment effects based on preferences.
problem Estimating treatment effects with flexible outcomes.
method Preference-based Conditional Treatment Effect (CPTE) framework.
result CPTE provides interpretable targets and new identifiability conditions.
Analyzing the underlying structure of multiple time-sequences provides insights into the understanding of social networks and human activities. In this work, we present the \emph{Bayesian nonparametric Poisson process allocation} (BaNPPA), a latent-function model for time-sequences, which automatically infers the numbe…
Proposes a semi-supervised K-Means algorithm for better feature selection.
problem Data clustering with unknown feature quality and limited labelled data.
method Combines unsupervised sparse clustering and semi-supervised learning with labelled data.
result The algorithm identifies informative features and maintains high performance.
In this paper, we develop a parameter estimation method for factorially parametrized models such as Factorial Gaussian Mixture Model and Factorial Hidden Markov Model. Our contributions are two-fold. First, we show that the emission matrix of the standard Factorial Model is unidentifiable even if the true assignment ma…
Study on sample complexity for pure exploration in feedback graph settings.
problem Sample complexity of pure exploration in online learning with feedback graphs.
method Derive instance-specific lower bounds and present asymptotically optimal algorithm TaS-FG.
result TaS-FG is asymptotically optimal and efficient across different graph configurations.
Unified framework for measuring causality-based fairness in machine learning.
problem Challenges of measuring causality-based fairness from observational data.
method Unified definition of PC fairness and constrained optimization method.
result Correctness and effectiveness of the proposed method demonstrated on synthetic and real-world datasets.
Method estimates treatment effect bounds in sample selection models.
problem Estimating heterogeneous treatment effects in presence of sample selection.
method Debiased/double machine learning approach for non-linear and high-dimensional confounders.
result Substantially tighter effect bounds for younger users.
Estimates CATE under hidden confounding, accounting for bias and ignorance.
problem Learning CATE from high-dimensional data with unobserved confounders introduces bias and ignorance.
method Parametric interval estimator that accounts for hidden confounding and underrepresented samples.
result Estimator converges to tight bounds on CATE when there may be unobserved confounding.
Given an element of the Bloch group of a number field~F and a natural number~n, we construct an explicit unit in the field Fn=F(e2πi/n), well-defined up to $\nn$-th powers of nonzero elements of~Fn. The construction uses the cyclic quantum dilogarithm, and under the identification of the Bloch group of~$F…
Bayesian networks with hidden variables help identify causal relationships obscured by confounding.
problem Identifying causal relationships obscured by unobserved confounders.
method Use finite k-mixtures of Bayesian networks with hidden variables to recover the joint probability distribution and identify causal relationships. result First algorithm to learn mixtures of non-empty DAGs, recovering identifiable causal relationships.
New method scores DAGs by identifying unobserved confounding.
problem Unobserved confounding complicates causal discovery.
method Score-based causal discovery algorithm that accounts for unobserved confounding.
result Sparse linear Gaussian DAGs can be recovered from observed data.
Bayesian matrix factorization (BMF) is a powerful tool for producing low-rank representations of matrices and for predicting missing values and providing confidence intervals. Scaling up the posterior inference for massive-scale matrices is challenging and requires distributing both data and computation over many worke…
Bayesian network models with latent variables are widely used in statistics and machine learning. In this paper we provide a complete algebraic characterization of Bayesian network models with latent variables when the observed variables are discrete and no assumption is made about the state-space of the latent variabl…
Proposes a differentiable structure learning framework for general binary data.
problem Limitations of existing methods in discrete data structure learning.
method Formulates a differentiable optimization task for arbitrary dependencies in general discrete models.
result Establishes identifiability of complete set of compatible parameters and structures under mild assumptions.
In high-dimensional data, structured noise caused by observed and unobserved factors affecting multiple target variables simultaneously, imposes a serious challenge for modeling, by masking the often weak signal. Therefore, (1) explaining away the structured noise in multiple-output regression is of paramount importanc…
New method optimizes policies without assuming known link functions between preferences and rewards.
problem Policy alignment with unknown and unrestricted link functions.
method Formulates an f-divergence-constrained reward maximization problem, learning policies directly. result Induces a semiparametric single-index binary choice model for policy alignment.
New framework recovers reward and rationality parameters from game behavior.
problem Statistical ambiguity in identifying reward and rationality parameters in competitive games.
method Blind Inverse Game Theory (Blind-IGT) using entropy-regularized Quantal Response Equilibrium and Normalized Least Squares (NLS) estimator.
result Optimal convergence rate of O(N−1/2) for joint parameter recovery. The paper proves consistency of neural networks with regularization.
problem Overfitting in neural networks with large scale data.
method Theoretical framework of neural networks with regularization, sieves method, and minimal neural networks theory.
result The estimated neural network converges to the true underlying function as sample size increases.
A new deep learning method for energy disaggregation.
problem Energy disaggregation or non-intrusive load monitoring (NILM) to identify individual appliance power usage.
method Sequence to Point Learning based on Bidirectional Dilated Residual Network (BRDN).
result Our method outperforms state-of-the-art approaches in all appliances on REDD and UK-DALE datasets.
We describe two recently proposed machine learning approaches for discovering emerging trends in fatal accidental drug overdoses. The Gaussian Process Subset Scan enables early detection of emerging patterns in spatio-temporal data, accounting for both the non-iid nature of the data and the fact that detecting subtle p…
The paper addresses online prediction in marginally stable systems with bounded perturbations.
problem Online prediction in marginally stable linear dynamical systems with adversarial or stochastic perturbations.
method The online least-squares algorithm is used to achieve sublinear regret, with a refined regret analysis and a structural lemma.
result The online least-squares algorithm achieves sublinear regret, with polynomial dependence on the system's parameters.
New method proves neural networks can select features consistently.
problem Feature selection for deep neural networks is challenging.
method Adaptive Group Lasso selection procedure with Group Lasso as the base estimator.
result Adaptive Group Lasso is selection-consistent for a wide class of neural networks.
Paper identifies tensor ranks via prior predictive matching, solving system of equations.
problem Determining the latent dimensions (ranks) in tensor factorization models.
method Prior predictive moment matching to transform moment matching conditions into a log-linear system of equations.
result Identifies which tensor models have identifiable ranks and derives rank estimators.
Nonnegative matrix factorization (NMF) has become a workhorse for signal and data analytics, triggered by its model parsimony and interpretability. Perhaps a bit surprisingly, the understanding to its model identifiability---the major reason behind the interpretability in many applications such as topic mining and hype…
Unified framework for disentangling latent variables.
problem Unidentifiability of deep latent-variable models.
method Variational autoencoders and nonlinear ICA, with a factorized prior conditioned on an observed variable.
result Identification of true joint distribution over observed and latent variables is possible up to simple transformations.
Paper addresses quadratic feasibility problems and their sample complexity.
problem Recovering complex vectors from quadratic measurements.
method Analyzes conditions for identifiability and explores optimization landscape.
result Gradient algorithms can converge to globally optimal solutions with high probability.
A new approach to disentangled representations using structured latent priors.
problem Learning disentangled representations in unsupervised learning.
method Proposed a structured latent prior to encourage disentanglement and mitigate trade-offs.
result The structured latent prior significantly mitigates the trade-off between reconstruction loss and disentanglement.
Unified understanding of neural networks on group operations verified.
problem Understanding and verifying neural networks trained on group operations.
method Investigated one-hidden-layer neural networks trained on binary operation of finite groups, revealing structure and providing a compact proof of model performance.
result Verified explanation applies to a large fraction of networks trained on the symmetric group S5, providing a >=95% accuracy bound for 45% of models.
Paper introduces VBG for Bayesian causal structure and mechanism learning.
problem Bayesian causal structure learning with uncertainty over models.
method Variational Bayes-DAG-GFlowNet (VBG) method.
result VBG outperforms existing methods in modeling posterior over DAGs and mechanisms.
With the rapid development of high-throughput technologies, parallel acquisition of large-scale drug-informatics data provides huge opportunities to improve pharmaceutical research and development. One significant application is the purpose prediction of small molecule compounds, aiming to specify therapeutic propertie…
Develops a new causal model for path-dependent link prediction.
problem Existing causal models assume fixed node factors, but real-world links can depend on existing ones.
method Introduces causal lifting and structural pairwise embeddings for path-dependent link prediction.
result Validated on three scenarios, demonstrating improved accuracy for causal link prediction.