New approach tackles nonidentifiability in nonlinear blind source separation.
problem Nonidentifiability in nonlinear blind source separation.
method Independent mechanism analysis, incorporating causal assumptions.
result Empirical and theoretical evidence shows improved identifiability.
IMA improves representation learning even when assumptions are violated.
problem Recovering true latent codes from mixed data.
method IMA, which assumes independent causal mechanisms.
result IMA's benefits extend to violations of its assumptions.
IMA addresses non-identifiability in nonlinear ICA by assuming orthogonal Jacobian columns.
problem Non-identifiability in nonlinear ICA.
method IMA assumes orthogonal Jacobian columns and extends to manifold settings.
result IMA circumvents non-identifiability issues and can be beneficial for higher-dimensional observations.
Bounds and sensitivity analysis for causal effects with MNAR confounders.
problem Estimating causal effects with missing outcome data.
method Assumption-free bounds and sensitivity analysis for outcome-independent MNAR.
result Valid bounds and sensitivity analysis methods for causal effect estimation.
VAEs improve representation learning by inverting the data-generating process through self-consistency.
problem VAEs struggle to invert the data-generating process, yet often succeed in representation learning.
method Studied VAEs in the limit of near-deterministic decoders, proving self-consistency and showing ELBO convergence to a regularized log-likelihood.
result VAEs can perform independent mechanism analysis (IMA), recovering true latent factors under specific conditions.
Causal Component Analysis aims to recover latent variables with causal relationships.
problem Recover latent variables with causal relationships from observed mixtures.
method Introduces a likelihood-based approach using normalizing flows to estimate unmixing function and causal mechanisms.
result Demonstrates effectiveness through synthetic experiments in CauCA and ICA settings.
New bounds for KANs trained with DP-SGD, addressing correlated noise.
problem Risk bounds for Kolmogorov-Arnold Networks trained by DP-SGD with correlated noise.
method Established new optimization and population risk analysis for KANs trained with DP-SGD, addressing correlated noise.
result First optimization and population risk analysis of correlated-noise mechanisms for DP training in non-convex settings, including neural networks.
Study shows priors are crucial for accurate causal learning from unlabeled data.
problem Improving causal learning from unlabeled data.
method Investigated causal learning using Bayesian methods and analyzed the impact of priors.
result Factorized priors lead to factorized posteriors, aligning with independent causal mechanisms.
The paper tackles extrapolation in generative models by enforcing independence of mechanisms.
problem How to make generative models extrapolate to new, unseen environments?
method Developed a theoretical framework for independence of mechanisms, demonstrated on toy examples and real-world data.
result Extrapolation capabilities of generative models can be improved by enforcing independence of mechanisms explicitly during training.
In this paper, we focus on developing a novel mechanism to preserve differential privacy in deep neural networks, such that: (1) The privacy budget consumption is totally independent of the number of training steps; (2) It has the ability to adaptively inject noise into features based on the contribution of each to the…
The paper analyzes regret in bilateral trade mechanisms without prior valuations.
problem Designing efficient trade mechanisms without prior knowledge of valuations.
method Regret minimization framework over rounds of interactions with no prior knowledge of valuations.
result Characterization of regret bounds for different feedback models and valuations.
A frame independent formulation of analytical mechanics in the Newtonian space-time is presented. The differential geometry of affine values i.e., the differential geometry in which affine bundles replace vector bundles and sections of one dimensional affine bundles replace functions on manifolds, is used. Lagrangian a…
Main ideas of the differential geometry on affine bundles are presented. Affine counterparts of Lie algebroid and Poisson structures are introduced and discussed. The developed concepts are applied in a frame-independent formulation of the time-dependent and the Newtonian mechanics.
A new discrete privacy mechanism for federated learning.
problem Differentially private federated learning with communication constraints.
method Skellam mechanism based on Poisson distributions.
result Skellam mechanism provides similar privacy-accuracy trade-offs as Gaussian mechanism.
Tests whether a treatment's effect is fully mediated by observed outcomes and identifies causal mechanisms.
problem Understanding how a treatment affects an outcome through intermediate variables.
method Proposes a test to evaluate full mediation and causal mechanism identification, extending to non-randomly assigned treatments.
result A conditionally random treatment is conditionally independent of the outcome given mediators and covariates if full mediation and causal mechanism identification hold.
New framework learns disentangled causal representations from observed labels.
problem Learning meaningful disentangled causal representations from observed data.
method ICM-VAE framework using flow-based diffeomorphic functions and causal disentanglement prior.
result Induces highly disentangled causal factors and improves robustness.
New framework improves model robustness by focusing on stable relations across environments.
problem Standard supervised learning fails under data distribution shift.
method Gradient-based learning framework derived from the principle of independent causal mechanisms (ICM).
result Models generalize well to unseen scenarios, ignoring unstable relations.
We propose a method to model multi-agent behaviors with limited observation and mechanical constraints.
problem Modeling real-world multi-agent behaviors with limited observation and mechanical constraints.
method Decentralized generative models with partial observation and mechanical constraints based on hierarchical variational recurrent neural networks.
result Our method effectively models and predicts biologically plausible behaviors with minimal constraint violations.
Paper develops conformalized survival analysis method for better prediction.
problem Survival analysis models often misspecify and require strong assumptions.
method Uses conformal prediction to wrap around any survival prediction algorithm.
result Lower predictive bounds provide guaranteed coverage without strong assumptions.
PBM mechanism improves privacy and accuracy in federated learning.
problem Secure and private federated learning with limited privacy budget.
method Poisson Binomial mechanism for discrete differential privacy.
result Achieves same privacy-accuracy trade-offs as Gaussian mechanism.
Learning modular structures which reflect the dynamics of the environment can lead to better generalization and robustness to changes which only affect a few of the underlying causes. We propose Recurrent Independent Mechanisms (RIMs), a new recurrent architecture in which multiple groups of recurrent cells operate wit…
This paper tackles time series imputation by identifying and modeling different missing mechanisms.
problem Different types of missing mechanisms (MAR, MNAR) in time series data.
method Proposes a framework for time series imputation by analyzing data generation processes and modeling latent variables via variational inference and normalizing flow.
result Establishes identifiability results for latent variables under nonlinear independent component analysis, showing that latent variables are identifiable.
Statistical learning relies upon data sampled from a distribution, and we usually do not care what actually generated it in the first place. From the point of view of causal modeling, the structure of each distribution is induced by physical mechanisms that give rise to dependences between observables. Mechanisms, howe…
Distributed stochastic gradient descent is an important subroutine in distributed learning. A setting of particular interest is when the clients are mobile devices, where two important concerns are communication efficiency and the privacy of the clients. Several recent works have focused on reducing the communication c…
The postulate of independence of cause and mechanism (ICM) has recently led to several new causal discovery algorithms. The interpretation of independence and the way it is utilized, however, varies across these methods. Our aim in this paper is to propose a group theoretic framework for ICM to unify and generalize the…
Proposes counterfactual explainability for causal attribution, extending variance analysis methods.
problem Lack of mechanistic understanding in existing tools for explaining complex models.
method Extends global sensitivity analysis methods to causal explanations using directed acyclic graphs.
result Developed methods to estimate counterfactual explainability and applied to income inequality analysis.
ICA reveals deep learning's feature learning mechanisms from non-Gaussian data.
problem Understanding feature learning from non-Gaussian inputs in deep neural networks.
method Investigates ICA and SGD on synthetic and real data.
result FastICA requires n≳d4 samples for single non-Gaussian direction recovery, while SGD outperforms and optimised SGD reaches n≳d2. New principle for disentangling latent factors using sparse regularization.
problem Disentangling latent factors from complex data.
method Sparse regularization of latent mechanisms to induce disentanglement.
result Recovery of latent variables up to permutation under certain conditions.
Paper presents characteristic function of Tsallis q-Gaussian and its applications.
problem Modeling input quantities in measurement models using Tsallis q-Gaussians.
method Developed a characteristic function and proposed a numerical method for its inversion.
result Exact probability distribution of output quantities can be determined.
A new method tests independence for causal discovery on discrete data.
problem Inferring causal directions on discrete and categorical data.
method Subsampling-based method to test independence between cause and mechanism.
result Our method works for both discrete and categorical data without functional model assumptions.
New method tackles composite optimization with error feedback.
problem Challenges in distributed machine learning training and message compression.
method Combines Dual Averaging with EControl for composite optimization.
result First strong convergence analysis for composite optimization with error feedback.
We continue our study of the Cauchy problem for the homogeneous (real and complex) Monge-Ampere equation (HRMA/HCMA). In the prequel a quantum mechanical approach for solving the HCMA was developed, and was shown to coincide with the well-known Legendre transform approach in the case of the HRMA. In this article---that…
New method identifies causal structure in exchangeable data.
problem Existing causal discovery methods struggle with i.i.d. data.
method Exchangeable data provides richer conditional independence structure.
result Exchangeable data allows for unique causal structure identification.
New findings on identifying latent variables in nonlinear ICA models.
problem Identifying latent variables in nonlinear ICA models is challenging due to spurious solutions.
method Proved that conformal maps are identifiable and provided theoretical results on preventing spurious solutions.
result Conformal maps are identifiable in nonlinear ICA models, preventing spurious solutions.
We have discovered 12 independent new empirical scaling laws in foreign exchange data-series that hold for close to three orders of magnitude and across 13 currency exchange rates. Our statistical analysis crucially depends on an event-based approach that measures the relationship between different types of events. The…
In the present paper we discuss an independent on the Grothendieck-Sato isomorphism approach to the Riemann-Roch-Hirzebruch formula for an arbitrary differential operator. Instead of the Grothendieck-Sato isomorphism, we use the Topological Quantum Mechanics (more or less equivalent to the well-known constructions with…
A common assumption in causal modeling posits that the data is generated by a set of independent mechanisms, and algorithms should aim to recover this structure. Standard unsupervised learning, however, is often concerned with training a single model to capture the overall distribution or aspects thereof. Inspired by c…
Detect hidden confounding in observational data using multiple environments.
problem Detect hidden confounding in observational data.
method Theoretical framework and simulation studies to test for hidden confounding.
result The proposed procedure correctly predicts hidden confounding, especially when bias is large.
The inference of the causal relationship between a pair of observed variables is a fundamental problem in science, and most existing approaches are based on one single causal model. In practice, however, observations are often collected from multiple sources with heterogeneous causal models due to certain uncontrollabl…
Study examines cryptocurrency risk spillover effects before and after pandemic.
problem Analyzing risk propagation among cryptocurrencies during extreme events.
method Asymmetric breakpoint approach and network analysis.
result Cryptocurrency risk spillover effect increased during pandemic.
Transformers learn topic structure through embedding and attention mechanisms.
problem Understanding how transformers capture semantic structure in text.
method Combination of mathematical analysis and experiments on Wikipedia and synthetic data.
result Embedding and attention layers encode topic structure in transformers.
TCRI improves domain generalization by enforcing conditional independence constraints.
problem Limitations of existing domain generalization methods due to incomplete constraints.
method TCRI implements regularizers motivated by conditional independence constraints.
result TCRI achieves cross-domain stability and outperforms baselines in worst-domain accuracy.
The paper analyzes errors in mechanical systems with external forces.
problem Error analysis of mechanical systems with external forces.
method Analysis of variational integrators with contact order r for discrete mechanical systems. result The contact order of the integrator is the same as the contact order of the original systems.
A framework for disentangling class-related and class-independent factors in data.
problem Learning disentangled representations in variational autoencoders.
method Attention mechanism in latent space, mixture models, Bhattacharyya coefficient, semi-supervised training.
result Disentangles class-related and class-independent factors of variation.
Independent Component Analysis (ICA) - one of the basic tools in data analysis - aims to find a coordinate system in which the components of the data are independent. In this paper we present Multiple-weighted Independent Component Analysis (MWeICA) algorithm, a new ICA method which is based on approximate diagonalizat…
New mechanism for pure differential privacy on functional summaries using Laplace-like process.
problem Challenges in achieving differential privacy for complex, structured functional summaries.
method Independent Component Laplace Process (ICLP) mechanism for infinite-dimensional Hilbert space.
result Effective enhancement of utility of private summaries through oversmoothing.
We study mechanical systems subject to constraint functions that can be dependent at some points and independent at the rest. Such systems are modelled by means of generalized codistributions. We discuss how the constraint force can transmit an impulse to the motion at the points of dependence and derive an explicit fo…
The Gaussian mechanism is an essential building block used in multitude of differentially private data analysis algorithms. In this paper we revisit the Gaussian mechanism and show that the original analysis has several important limitations. Our analysis reveals that the variance formula for the original mechanism is …