Study on how deterministic dependencies affect information synergy and redundancy.
problem Understanding how deterministic dependencies impact information synergy and redundancy.
method Systematic analysis of deterministic dependencies on information decomposition.
result Identifies how negative terms can originate from deterministic dependencies and discusses implications for neural coding.
Study finds more flood risk strategies can improve outcomes in NYC.
problem Managing future flood risks with complex models.
method Used an intermediate complexity model to analyze flood risk strategies.
result More combinations of risk mitigation strategies expand the solution set and improve outcomes.
The study uncovers stock synergy networks using mutual information in Indian stock market data.
problem Detecting clusters of stocks in synergy in the Indian Stock Market.
method Analysis of high frequency data, mutual information for non-linearity, comparison with correlation method.
result Mutual information method successfully identifies effective networks compared to correlation method.
Estimates interactions between modalities for multimodal data.
problem Accurately quantifying interactions between different data types.
method Developed Lightweight Sample-wise Multimodal Interaction (LSMI) estimator using pointwise information theory.
result LSMI reveals fine-grained dynamics in multimodal data.
Study improves drug synergy prediction using ensemble learning.
problem Predicting drug synergy in complex diseases.
method Investigated different compound representations and proposed an ensemble model.
result Ensemble model outperforms baseline models.
In this work we present a review of the state of the art of information theoretic feature selection methods. The concepts of feature relevance, redundance and complementarity (synergy) are clearly defined, as well as Markov blanket. The problem of optimal feature selection is defined. A unifying theoretical framework i…
New method quantifies multivariate redundancy using maximum entropy decompositions.
problem Elusive multivariate measures of redundancy that comply with nonnegativity and axioms.
method Maximum entropy framework, rooted tree-based decompositions of mutual information.
result Quantifies different multivariate redundancy contributions.
The paper uses information theory to analyze neural processing systems.
problem Understanding how neural systems process information with contextual inputs.
method Applied a new information theory concept to decompose neural processing.
result Contextual modulation has unique information processing properties.
Unified perspective unites Bayesian optimization and active learning for efficient goal-oriented optimization.
problem Efficiently optimize expensive engineering and scientific problems with limited data.
method Unified framework linking Bayesian infill criteria and active learning criteria.
result Unified approach formalizes Bayesian infill criteria and active learning criteria.
Robotic manipulation learns synergies between pushing and grasping from scratch.
problem Discovering complex synergies between pushing and grasping for efficient robotic manipulation.
method Self-supervised deep reinforcement learning with two convolutional networks.
result System learns pushing and grasping motions that improve picking success rates and efficiency.
DL models can outperform regionalized models in hydrology by pooling diverse data.
problem Traditional wisdom in hydrology suggests regionalization improves model performance, but DL models can unify data for better performance.
method Used DL models on pooled data from different regions, showing improved performance compared to regionalized models.
result DL models can improve performance by pooling diverse data, highlighting the 'data synergy' effect.
New entropy measures reveal information flow in CNNs without approximations.
problem Understanding information flow in convolutional neural networks (CNNs).
method Developed new entropy estimators based on Renyi's α-entropy and applied PID framework.
result Validated fundamental data processing inequalities and revealed properties of CNN training.
Proposes a new method to better understand complex system interactions.
problem Current methods like Granger causality and transfer entropy fail to capture higher-order interactions.
method Introduces a generalized approach to capture multivariate causal interactions.
result The method can distinguish causal roles in synergetic interactions.
Signed pairwise interactions conflate uniqueness, redundancy, and synergy
problem Signed pairwise interactions conflate uniqueness, redundancy, and synergy
method Stochastic Hi-Fi
result Stochastic Hi-Fi recovers structure missed by scalar baselines
New theory explains how equivariant self-supervised learning improves feature extraction.
problem Contrastive learning sacrifices useful features due to invariance to data augmentations.
method Information-theoretic perspective to understand E-SSL's generalization ability.
result Equivariant self-supervised learning creates synergy between equivariant and classification tasks.
The paper proposes a method to estimate heterogeneous treatment effects using pretraining strategies.
problem Estimating conditional average treatment effects (CATE) in the presence of many covariates.
method The approach leverages prognostic factors that also predict treatment effect heterogeneity, using the R-learner framework.
result The proposed method improves estimation accuracy and power for detecting treatment effect heterogeneity.
Combining insights from neural networks and neuroscience to understand complex tasks.
problem Understanding how biological and artificial neural networks learn and solve complex tasks.
method Review of data-analysis techniques from computational neuroscience applied to DNNs, and vice versa.
result Opportunities for synergy between machine learning and neuroscience to enhance understanding of neural representations.
New metric for disentangling multivariate representations, accounting for more complex entanglements.
problem Current disentanglement metrics fail to detect entanglements involving more than two variables.
method Partial Information Decomposition framework to analyze information sharing and propose a new disentanglement metric.
result The proposed metric correctly identifies entanglements in high-dimensional spaces.
Developed Taylor series for muscle-finger system analysis.
problem Understanding the complex relationship between muscle activity and finger movement.
method Used Dendrite Net to develop Taylor series and construct relation spectrum.
result Found muscle synergy and coupling in hand movement.
Study uses information-theoretic measures to analyze neural networks and neuron importance.
problem Understanding the importance of individual neurons in neural networks.
method Cumulative ablation of neurons, analyzing entropy, mutual information, and class selectivity.
result Class selectivity is not a good indicator for classification performance, while mutual information and selectivity are positively correlated with performance.
New method calculates partial information for Gaussian systems based on dependency constraints.
problem Quantifying information sharing in multivariate Gaussian systems.
method Constructing maximum entropy models based on dependency constraints and deriving closed-form solutions.
result Closed-form solutions for Gaussian systems show differences in redundancy and synergy estimates compared to existing methods.
The problem of learning a sparse model is conceptually interpreted as the process of identifying active features/samples and then optimizing the model over them. Recently introduced safe screening allows us to identify a part of non-active features/samples. So far, safe screening has been individually studied either fo…
A novel deep learning technique combines multiple modalities, improving performance.
problem Challenges in leveraging different modalities due to noise and conflicts.
method Proposes a deep neural network that multiplicatively combines information from different modalities.
result Consistent accuracy improvements on three multimodal classification tasks.
New method uses information theory to uncover causal relationships in complex systems.
problem Discovering causal relationships in multivariate systems, especially in Bayesian networks and hypergraphs.
method Partial Information Decomposition (PID) to explicitly model higher-order interactions.
result PID components reveal direct causal neighbors and collider relationships in Bayesian networks and multi-tail hyperedges in causal hypergraphs.
Study identifies negative data externalities affecting model performance on specific groups.
problem Negative data externalities on group performance in machine learning models.
method Characterized and detected data-model inefficiencies, focusing on specific types of externalities.
result Negative data externalities can lower model performance on specific sub-groups, even with larger datasets.
Hybrid methods that utilize both content and rating information are commonly used in many recommender systems. However, most of them use either handcrafted features or the bag-of-words representation as a surrogate for the content information but they are neither effective nor natural enough. To address this problem, w…
Independent component analysis (ICA) is a powerful method for blind source separation based on the assumption that sources are statistically independent. Though ICA has proven useful and has been employed in many applications, complete statistical independence can be too restrictive an assumption in practice. Additiona…
CNF model improves molecular prediction accuracy.
problem Improving molecular prediction accuracy with limited data.
method Combining CNN and SMILES multiplicity.
result CNF model outperforms traditional descriptors on small datasets.
The purpose of the New York Workshop on Computer, Earth and Space Sciences is to bring together the New York area's finest Astronomers, Statisticians, Computer Scientists, Space and Earth Scientists to explore potential synergies between their respective fields. The 2011 edition (CESS2011) was a great success, and we w…
ClusterGraph visualizes and simplifies multidimensional data clusters for better understanding.
problem Lack of global structure information in clustering results.
method Combining clustering with Topological Data Analysis to provide global structure.
result ClusterGraph provides global layout information about clusters.
PDTS improves robustness in sequential decision-making.
problem Robust active task sampling for efficient and reliable decision-making.
method Characterizes robust active task sampling as a Markov decision process, proposes PDTS method.
result Significantly improves zero-shot and few-shot adaptation robustness.
Enhances inverse design optimization with machine learning and reduced fidelity simulations.
problem Limited compute resources in inverse design optimization.
method Synergy of multi-fidelity simulations, machine learning, and search space reduction.
result Significant computational resource savings and improved optimization performance.
New method disentangles high-order effects in feature importance.
problem Quantifying cooperative effects in feature importance.
method Adaptive Leave One Covariate Out (LOCO) method to decompose LOCO into two-body and higher-order components.
result Decomposes LOCO into two-body and higher-order components, highlighting synergistic and redundant effects.
The paper analyzes cryptocurrency trading networks using pairwise and high-order dependencies.
problem Understanding information flows and dependencies in cryptocurrency markets.
method Defined a cryptocurrency trading network using weekly log returns, analyzed using Granger causality and O-information.
result High-order dependencies reveal that stable coins play a major role in high-order effects.
Enhances optimization in multi-source settings with causal principles.
problem Optimizing functions with multiple sources of data and causal dependencies.
method Integrates Multi-Source Bayesian Optimization with Causal Bayesian Optimization principles.
result Improves optimization efficiency and reduces computational complexity.
Stock price movement reveals complex interdependencies that are simplified through linear correlation.
problem Exploring the spectral dynamics of the Indonesian capital market using structural network representations.
method Combining three dependency estimators (Pearson, MI adaptive binning, and MI-kNN) with two graph filtering schemes (MST and PMFG) and four community decoders.
result MI adaptive binning is shown to be more proportional than kNN for detecting residual information.
New approach combines PCA and t-sne for better data analysis.
problem Multiscale complexity in high-dimensional data.
method Multiscale joint characterization using PCA and t-sne.
result Joint characterization detects signals not seen by PCA or t-sne alone.
Proposes a method to learn rankings from multiple sources.
problem Learning to rank from multiple information sources.
method Generic framework for multi-view subspace learning to rank (MvSL2R) with autoencoder-like networks and end-to-end solutions.
result Enhances joint ranking with minimum view-specific ranking loss, achieving maximum global view agreements.
Combining causality, control, and reinforcement learning for system control.
problem Learning to control dynamical systems using causal, control, and reinforcement learning approaches.
method Combining causal identification, control strategies, and reinforcement learning to control dynamical systems.
result Combining different learning paradigms for effective system control.
New theory explains how noisy, high-dimensional data can still lead to robust predictions.
problem Modern machine learning models achieve high performance with noisy, high-dimensional data.
method Synthesizes principles from Information Theory, Latent Factor Models, and Psychometrics to clarify predictive robustness.
result Predictive robustness arises from data architecture and model capacity, not just data cleanliness.
Paper proposes multimodal contrastive learning for EHR data.
problem Separate treatment of structured and unstructured EHR data.
method Proposes a multimodal feature embedding generative model and a multimodal contrastive loss.
result Multimodal learning yields better feature representation than single-modality learning.
Theoretical analysis of data quality and synergies in LLMs.
problem Understanding why different training methods require different amounts of data.
method Theoretical analysis of transformers trained on a weight prediction task for linear regression.
result SFT excels on smaller datasets challenging for the pretrained model, while RL benefits from large, not overly difficult data.
This paper develops a stochastic learning-optimization model for resilient automotive supply chains.
problem Supply chain disruptions and volatile demand pose challenges to the UK automotive industry.
method Integrates Bayesian inference with inventory optimization for a two-echelon system subject to stochastic demand and disruptions.
result The integrated approach achieves significant cost reductions and improved resilience during disruptions.
This paper combines three techniques to reduce communications in distributed variational inequalities.
problem Efficiently communicating solutions in large-scale distributed variational inequalities.
method Combining similarity, compression, and local steps to reduce communication rounds and cost.
result Best theoretical guarantees of communication complexity and superior performance in adversarial learning experiments.
Sig-PCA integrates model outputs and observations to correct model biases.
problem Improving model accuracy and reliability by correcting biases and numerical approximations.
method Sig-PCA framework that combines summary statistics from model outputs with localized observations via a neural network.
result Corrects model outputs to align closely with observational data, preserving essential statistical information.
Study maps interdependence of SDGs, finds complex, dynamic linkages.
problem Identify which SDGs promote progress and how quickly.
method Used a balanced panel of 114 countries from 2000 to 2024, applying two estimators to recover directed interaction network and measure dynamic linkages.
result 84 goal linkages survive false-discovery control, showing both synergies and trade-offs, with no single goal acting as a universal accelerator.
Models to price long term loans in the securities lending business are developed. These longer horizon deals can be viewed as contracts with optionality embedded in them. This insight leads to the usage of established methods from derivatives theory to price such contracts. Numerical simulations are used to demonstrate…
X-TFC solves parametric DEs with neural networks and physics constraints.
problem Solving parametric differential equations with physics constraints.
method Combines Theory of Functional Connections and Physics-Informed Neural Networks with a single-layer Extreme Learning Machine.
result Achieves high accuracy with low computational time.