DEDACT breaks down feature importance into direct and associative components.
problem Lack of clear distinction between direct and associative feature importance.
method DEDACT framework to decompose direct and associative importance measures.
result Provides insight into sources of prediction-relevant information and feature pathways.
Mirror descent linked to information ratio via Bayesian regret bounds.
problem Understanding stability in mirror descent and its relation to information ratio.
method Developed a connection between mirror descent and information ratio using Bayesian regret bounds.
result Mirror descent with suitable estimators and distributions achieves bounds similar to information-directed sampling.
The paper proposes methods to extract and analyze individual variable information from complex dependencies.
problem Analyzing and understanding complex dependencies between multiple variables.
method Reversible normalization and iterative dependency reduction to extract individual information, and use it for direct mutual information and multi-feature Granger causality analysis.
result Decoupling of variables to analyze their individual information and direct mutual information transfers.
Obtaining complete information about the shape of an object by looking at it from a single direction is impossible in general. In this paper, we theoretically study obtaining differential geometric information of an object from orthogonal projections in a number of directions. We discuss relations between (1) a space c…
Novel GNN for signed and directed networks using magnetic signed Laplacian.
problem Efficiently modeling signed and directed networks for tasks like clustering and link prediction.
method Introduced a magnetic signed Laplacian for directed signed graphs, used it to construct a spectral GNN.
result Demonstrated effective performance on tasks involving signed and directional information.
A new framework uses directed information to efficiently select context chunks.
problem Efficiently selecting relevant context chunks for query understanding.
method Directed Information γ-covering framework, formulated as a γ-cover problem, with a greedy algorithm for context selection. result The γ-covering algorithm provides clear advantages in hard-decision regimes like context compression and single-slot prompt selection. IDS improves sparse linear bandits by balancing information and regret.
problem Sparse linear bandits in high-dimensional decision-making.
method Information-directed sampling (IDS) with Bayesian regret bounds and empirical Bayesian sparse posterior sampling.
result IDS nearly matches existing lower bounds and significantly reduces regret.
New matrix reveals cluster info in sparse directed graphs.
problem Analyzing cluster information in directed graphs.
method Proposed complex non-backtracking matrix integrating Hermitian adjacency matrix and non-backtracking matrix properties.
result The complex non-backtracking matrix holds cluster information, especially for sparse directed graphs.
Optimizes stochastic linear bandits with efficient, asymptotically optimal algorithm.
problem Optimizing stochastic linear bandits with multiple actions.
method Frequentist information-directed sampling (IDS) with a surrogate for information gain.
result Asymptotically optimal and nearly worst-case optimal in finite time.
DimeNet uses directional message passing to improve molecular predictions.
problem Lack of directional information in graph neural networks for molecules.
method Directional message passing, rotationally equivariant embeddings, spherical functions.
result DimeNet outperforms previous GNNs by 76% on MD17 and 31% on QM9.
A novel framework infers causal direction from symbolic sequences using pattern entropy.
problem Challenges in discovering causal direction from temporal symbolic data.
method Dictionary Based Pattern Entropy (DPE) framework integrating AIT and Shannon Information Theory. result Minimizing pattern level uncertainty yields a robust framework for causal discovery.
The study extracts market direction from transaction data.
problem Extracting market direction from transaction data.
method Dynamic equation with time scale selection from past transactions.
result Automatic determination of time scale for price calculation.
A framework visualizes embedding spaces of neural survival analysis models using anchor directions.
problem Visualizing complex embeddings in neural survival analysis models.
method Estimating anchor directions through clustering or user-supplied concepts, revealing relationships with raw inputs and survival times.
result Visualization strategies reveal how anchor directions relate to raw clinical features and survival time distributions.
We propose a graphical model for representing networks of stochastic processes, the minimal generative model graph. It is based on reduced factorizations of the joint distribution over time. We show that under appropriate conditions, it is unique and consistent with another type of graphical model, the directed informa…
DCGANs generate drainage networks quickly from samples.
problem High computational costs in generating large numbers of drainage networks.
method DCGANs trained with connectivity-informed directional information.
result Connectivity-informed DCGANs outperform other methods in reproducing accurate drainage networks.
Proposes a neural network model to improve predictions in biased datasets.
problem Reduces bias in predictions from datasets with selection bias.
method Integrates meta information about bias direction and quantity into a neural network model.
result Improves prediction accuracy in electoral polls with biased training data.
Summarizing large-scaled directed graphs into small-scale representations is a useful but less studied problem setting. Conventional clustering approaches, which based on "Min-Cut"-style criteria, compress both the vertices and edges of the graph into the communities, that lead to a loss of directed edge information. O…
New bounds on IDS for RL show how to balance computation and learning efficiency.
problem Understanding and optimizing information-directed sampling (IDS) for reinforcement learning.
method Developed novel information-theoretic tools to bound information ratio and cumulative information gain.
result Derived prior-free Bayesian regret bounds for IDS in tabular finite-horizon MDPs and improved computational efficiency.
Identifying the underlying directional relations from observational time series with nonlinear interactions and complex relational structures is key to a wide range of applications, yet remains a hard problem. In this work, we introduce a novel minimum predictive information regularization method to infer directional r…
It has often been taken as a working assumption that directed links in information networks are frequently formed by "short-cutting" a two-step path between the source and the destination -- a kind of implicit "link copying" analogous to the process of triadic closure in social networks. Despite the role of this assump…
New pricing algorithm learns demand curves and optimizes prices in dynamic markets.
problem Dynamic pricing in markets with incomplete demand information and shifting conditions.
method Actor-Critic Information-Directed Pricing (ACIDP) using IDS algorithms and auditing procedures.
result ACIDP outperforms UCB and TS in market environment shifts.
A new algorithm STE for model-based RL improves learning rates.
problem Sparse rewards and computational intractability of estimating information gain.
method Developed a novel algorithm based on Stein Information Directed Exploration (STE)E.
result Achieves sublinear Bayesian regret, outperforming prior approaches.
The paper extends option pricing theory for markets with informed traders.
problem Discontinuity in option pricing for markets with informed traders.
method New models for option pricing in complete markets considering informed traders' information on stock price direction and return mean.
result The discontinuity puzzle in option pricing is resolved using continuous diffusion price processes.
Extends GCNs to directed graphs for better performance.
problem Limited application of GCNs to undirected graphs.
method Uses first- and second-order proximity to extend GCNs to directed graphs.
result DGCN outperforms state-of-the-art methods on citation and co-purchase datasets.
Order-flow entropy predicts price magnitude without directionality.
problem Predicting price magnitude in financial markets.
method Real-time order-flow entropy computed from a 15-state Markov transition matrix.
result Order-flow entropy predicts the magnitude of intraday returns with high accuracy.
We present an algorithm, Decision-Directed Data Decomposition (D4), which decomposes a dataset into two components. The first contains most of the useful information for a specified supervised learning task. The second orthogonal component contains little information about the task but retains associations and informat…
Unified framework maps financial market dynamics using TE and KM, revealing directional information flow.
problem Challenges in traditional correlation analysis of financial markets, especially during crises.
method Combines Transfer Entropy (TE) and Kramers-Moyal (KM) expansion to analyze dynamic interactions among major indices.
result Increased directional information flow during crises, highlighting gold-dollar and oil-equity linkages.
We analyze the information-theoretic limits for the recovery of node labels in several network models. This includes the Stochastic Block Model, the Exponential Random Graph Model, the Latent Space Model, the Directed Preferential Attachment Model, and the Directed Small-world Model. For the Stochastic Block Model, the…
A fundamental problem in geostatistical modeling is to infer the heterogeneous geological field based on limited measurements and some prior spatial statistics. Semantic inpainting, a technique for image processing using deep generative models, has been recently applied for this purpose, demonstrating its effectiveness…
Using transfer entropy, we observed the strength and direction of information flow between stock indices. We uncovered that the biggest source of information flow is America. In contrast, the Asia/Pacific region the biggest is receives the most information. According to the minimum spanning tree, the GSPC is located at…
Privacy subsidy found in market trading with noisy direction signals.
problem Analyzing welfare and bid-ask spread in a market with privacy mechanisms.
method Closed-form derivation of bid-ask spread and welfare under flip-noise direction observation.
result Privacy subsidy of μηΔ from liquidity pool to traders, robust across models. The use of imitation learning to learn a single policy for a complex task that has multiple modes or hierarchical structure can be challenging. In fact, previous work has shown that when the modes are known, learning separate policies for each mode or sub-task can greatly improve the performance of imitation learning. …
This work improves online SGD's sample complexity for multi-index models by considering higher-order terms.
problem Suboptimal sample complexity for learning multi-index models using online SGD.
method Focus on both second- and higher-order terms to improve sample complexity.
result Online SGD achieves ildeO(dPL−1) samples for multi-index models. Unified algorithm for efficient pure exploration using dual variables.
problem Efficiently achieving a specific goal through adaptive experimentation.
method Introducing dual variables to derive optimal allocation conditions, leading to Information-Directed Selection.
result Top-two Thompson sampling attains asymptotic optimality for Gaussian best-arm identification.
Enhances multi-modular models by directing information flow between components.
problem Improving predictive performance in multi-modular models with misspecification.
method Introduces Semi-Modular Inference (SMI) with an influence parameter to control information flow between modules.
result SMI allows for tunable and directed information flow, improving prediction in some settings.
New method clusters directed and undirected graphs without losing directional information.
problem Clustering directed graphs due to asymmetry in edge connectivity.
method Generalized Dirichlet Energy (GDE) and generalized spectral clustering (GSC).
result GSC outperforms existing methods in clustering accuracy and robustness.
IDS improves reinforcement learning with contextual information.
problem Optimizing IDS for contextual reinforcement learning.
method Investigated contextual bandit problems and proposed a computationally-efficient IDS.
result Contextual IDS outperforms conditional IDS by considering future contexts.
We introduce a framework for Newton's flows in probability space with information metrics, named information Newton's flows. Here two information metrics are considered, including both the Fisher-Rao metric and the Wasserstein-2 metric. A known fact is that overdamped Langevin dynamics correspond to Wasserstein gradien…
A new framework uses information theory to detect anomalies in images without labeled data.
problem Detect anomalies in images without labeled data.
method A direct objective function using information theory to maximize the distance between normal and anomalous data.
result The proposed framework significantly outperforms state-of-the-arts on multiple benchmark datasets.
MFNets constructs efficient multifidelity surrogates from diverse information sources.
problem Creating accurate surrogates from multiple, potentially costly or inaccurate data sources.
method Directed acyclic graph of connections, gradient-based minimization of least squares objective, flexible information source structure.
result Error reduction by orders-of-magnitude, especially in low-data scenarios.
This paper considers the problem of embedding directed graphs in Euclidean space while retaining directional information. We model a directed graph as a finite set of observations from a diffusion on a manifold endowed with a vector field. This is the first generative model of its kind for directed graphs. We introduce…
New algorithms detect and estimate rank-one signals with prior directional information.
problem Detecting and estimating rank-one signals with directional prior information.
method Construct nonlinear Laplacians and examine top eigenvalues and eigenvectors.
result Nonlinear Laplacian algorithms outperform direct spectral methods for biased signals.
By establishing a connection between bi-directional Helmholtz machines and information theory, we propose a generalized Helmholtz machine. Theoretical and experimental results show that given \textit{shallow} architectures, the generalized model outperforms the previous ones substantially.
We propose a method to learn causal response representations through direct effect analysis.
problem Uncovering direct causal effects in complex, multivariate settings.
method Our method bridges conditional independence testing with causal representation learning, formulating an optimisation problem to maximise evidence against conditional independence.
result The largest eigenvalue distribution can be bounded by an F-distribution, providing testable conditional independence. This paper reviews information theory in open-world machine learning.
problem Lack of a unified theoretical foundation for open-world machine learning.
method Synthesis of information theoretic approaches.
result Established a pathway toward provable and trustworthy open world intelligence.
We investigate the strength and the direction of information transfer in the U.S. stock market between the composite stock price index of stock market and prices of individual stocks using the transfer entropy. Through the directionality of the information transfer, we find that individual stocks are influenced by the …
How many bits of information are required to PAC learn a class of hypotheses of VC dimension d? The mathematical setting we follow is that of Bassily et al. (2018), where the value of interest is the mutual information I(S;A(S)) between the input sample S and the hypothesis outputted by the learning algo…
A new method optimizes slicing directions for SW distances to improve high-dimensional probability measure comparison.
problem Challenging identification of informative slicing directions for SW distances.
method Constrained learning approach to optimize slicing directions, using continuous relaxations and gradient-based primal-dual approach.
result Demonstrated efficacy in learning more informative slicing directions on various high-dimensional data.