A new model tackles language generation issues by using discrete variational attention.
problem Information under-representation and posterior collapse in variational autoencoders.
method Proposes a discrete variational attention model with categorical distribution over attention mechanism.
result Enhances latent space for language generation and avoids posterior collapse.
The paper tackles fairness in forecasting and learning linear dynamical systems.
problem Under-representation bias in training data for multiple subgroups.
method Introducing subgroup-fair and instant-fair learning of LDS from multiple trajectories of varying lengths, using hierarchies of convexifications of non-commutative polynomial optimisation problems.
result Empirical results show both the beneficial impact of fairness considerations on statistical performance and encouraging effects of exploiting sparsity on run time.
This paper aims to determine the images of the braid group under representations afforded by the Yang Baxter equation when the solution is a nontrivial 4×4 matrix. Making the assumption that all the eigenvalues of the Yang Baxter solution are roots of unity, leads to the conclusion that all the images are fini…
New method finds balanced clusters in graphs using auxiliary information.
problem Finding balanced clusters in graphs with population-level constraints.
method Proposes individual-level balancing constraint and develops spectral clustering algorithms.
result Establishes first statistical consistency result for constrained spectral clustering.
The paper proves rigidity theorems for forms on reductive symmetric spaces.
problem Local rigidity of forms on reductive symmetric spaces under representations of discrete groups.
method General local rigidity theorem for pull-backs of homogeneous forms, reinterpretation of old results.
result Volume of closed manifolds is constant under deformation of G/H-structure. The study proposes a framework to assess sustainability of firms using fund-level classifications and portfolio holdings.
problem To capture market-based sustainability assessments of firms.
method Exploiting fund-level sustainability classifications and granular portfolio holdings to construct Market-Implied Sustainability (MIS) scores.
result MIS scores capture sustainability dimensions different from conventional ESG ratings and improve portfolio performance.
Improves fairness in machine learning by adding underrepresented group data.
problem Machine learning biases across subgroups due to under-representation or societal biases.
method Data augmentation via pairwise mixup across subgroups to balance subpopulations.
result Achieves fair outcomes with robust if not improved accuracy.
New methods for estimating treatment effects with missing data.
problem Missing outcome data complicates estimating treatment effects.
method Proposed two de-biased machine learning estimators (mDR-learner and mEP-learner) to address under-representation.
result Oracle efficiency of the proposed estimators under reasonable conditions.
Automated decision making based on big data and machine learning (ML) algorithms can result in discriminatory decisions against certain protected groups defined upon personal data like gender, race, sexual orientation etc. Such algorithms designed to discover patterns in big data might not only pick up any encoded soci…
Paper studies complex Lagrangian surfaces and their relation to SL(3,C)-representations.
problem Minimal Lagrangian surfaces in bi-complex hyperbolic space and their representations.
method Introduces bi-complex Higgs bundles and parameterizes SL(3,C)-quasi-Fuchsian representations. result Parameterization of SL(3,C)-quasi-Fuchsian representations by an open set in Teichmüller space. Recent advances in computer vision and deep learning have led to breakthroughs in the development of automated skin image analysis. In particular, skin cancer classification models have achieved performance higher than trained expert dermatologists. However, no attempt has been made to evaluate the consistency in perfo…
We simplify information measure computation using learned features.
problem Computing information measures from raw data is computationally expensive.
method Developed a separable design for computing information measures from learned feature representations.
result A variety of information measures can be computed efficiently through learned feature representations.
A new framework for information theory considers computational constraints.
problem Understanding information in complex systems with computational limitations.
method Variational extension of Shannon's information theory with computational constraints.
result Predictive V-information can be created through computation and reliably estimated from data. An asymmetric information model is introduced for the situation in which there is a small agent who is more susceptible to the flow of information in the market than the general market participant, and who tries to implement strategies based on the additional information. In this model market participants have access t…
We study a simple model of an asset market with informed and non-informed agents. In the absence of non-informed agents, the market becomes information efficient when the number of traders with different private information is large enough. Upon introducing non-informed agents, we find that the latter contribute signif…
New method quantifies redundant information using information bottleneck.
problem Quantifying redundant information among multiple sources.
method Formulated as an information bottleneck problem, termed redundancy bottleneck.
result Extracts information that best predicts the target without revealing source identity.
This paper reviews information theory in open-world machine learning.
problem Lack of a unified theoretical foundation for open-world machine learning.
method Synthesis of information theoretic approaches.
result Established a pathway toward provable and trustworthy open world intelligence.
Review of information plane analyses in neural networks, highlighting mixed results and methodological challenges.
problem Understanding the relationship between information-theoretic compression and neural network performance.
method Literature review and detailed analysis of information quantity estimation methods.
result Information plane compression is not necessarily information-theoretic but compatible with geometric compression.
Information geometry offers new tools for statistical analysis.
problem Statistical analysis of probability distributions.
method Geometric perspective on statistical manifolds.
result New applications in radar sensing, signal processing, etc.
Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an outcome's plausibility. Information measures based on Shannon's concept of entropy include realization information, Kullback-Leibler divergenc…
In this paper, we present a new approach to interpret deep learning models. By coupling mutual information with network science, we explore how information flows through feedforward networks. We show that efficiently approximating mutual information allows us to create an information measure that quantifies how much in…
Before the massive spread of computer technology, information was far from complex. The development of technology shifted the paradigm: from individuals who faced scarce and costly information to individuals who face massive amounts of information accessible at low costs. Nowadays we are living in the era of big data a…
In financial markets valuable information is rarely circulated homogeneously, because of time required for information to spread. However, advances in communication technology means that the 'lifetime' of important information is typically short. Hence, viewed as a tradable asset, information shares the characteristics…
Introduces relative information gain for improving Gaussian process regression rates.
problem Improving the sample complexity of estimating or maximizing unknown functions.
method Introduces relative information gain, interpolates between effective dimension and information gain, and proves PAC-Bayesian bounds.
result Obtains minimax-optimal rates of convergence through the relative information gain.
This paper uses information theory to improve risk modeling in big data.
problem Insufficient application of information theory in actuarial science.
method Explores information theory to uncover performance limits of insurance big data systems.
result Guidance for risk modeling and actuarial pricing systems.
In information theory, Fisher information and Shannon information (entropy) are respectively used to quantify the uncertainty associated with the distribution modeling and the uncertainty in specifying the outcome of given variables. These two quantities are complementary and are jointly applied to information behavior…
Investors pay for additional asset information based on utility maximization.
problem Determining the optimal price for additional asset information.
method Solving a stochastic control problem with partial information and utility maximization.
result Investors choose to purchase information at a deterministic time.
New method detects information leakage using approximate Bayes predictor.
problem Unintentional exposure of sensitive information via observable data.
method Statistical learning theory and information theory framework, approximating Bayes predictor's log-loss and accuracy.
result MI can be accurately estimated to detect ILs, outperforming state-of-the-art baselines.
Market strategies minimize Fisher information to minimize risk.
problem Applying minimum Fisher information principle to market dynamics.
method Analytical extension to quantum harmonic oscillator eigenstates and Gibbs distribution.
result Minimizing Fisher information reduces information and risk.
New approach quantifies overfitting in high-dimensional regression.
problem Quantifying and avoiding overfitting in large neural networks.
method Information bottleneck theory to minimize residual information while maximizing relevant bits.
result Characterized the relative information efficiency of randomized regression compared to optimal algorithms.
Unified notation simplifies information-theoretic concepts in machine learning.
problem Opaque notation for information-theoretic quantities in machine learning.
method Proposed a practical and unified notation for information-theoretic quantities.
result Unified notation facilitates new intuitions and rederivations in machine learning.
Study optimal portfolios for traders with asymmetric information and delay.
problem Optimizing portfolios for traders with delayed insider information.
method Anticipating stochastic calculus and white noise approach.
result Optimal portfolios maximize expected logarithmic utility under various financial models.
Strategic brokers exploit private information in broker-mediated markets, affecting informed traders' performance.
problem Strategic interactions and information leakage in broker-mediated markets.
method Study of strategic trading behavior and information leakage in a broker-mediated market.
result Brokers hold a strategic advantage over informed traders due to information leakage in trading flows.
Proposes a new method to enhance neural learning by maximizing information gain.
problem Improving neural learning by selecting key variables to maximize information gain.
method Adaptive Ensemble Kalman Filter to quantify uncertainty and maximize information gain.
result The proposed method enables the neural network to learn more effectively from stochastic systems.
Mirror descent linked to information ratio via Bayesian regret bounds.
problem Understanding stability in mirror descent and its relation to information ratio.
method Developed a connection between mirror descent and information ratio using Bayesian regret bounds.
result Mirror descent with suitable estimators and distributions achieves bounds similar to information-directed sampling.
Study shows changes in information sharing between Bitcoin markets during 2017 crash.
problem Understanding information dynamics in Bitcoin markets during the 2017 crash.
method Analysis of high-frequency market-microstructure observables using information theoretic measures.
result Temporal changes in information sharing across markets, including predictability, memory, and synchronous coupling.
Study on optimal information acquisition in Kyle model with entropy cost.
problem Optimal information acquisition in Kyle model with entropy cost.
method Continuous signals are optimal, and any signal with a logit posterior distribution yields the same ex-ante value.
result Posterior expected payoff becomes normally distributed as information acquisition cost increases.
Study on information evolution in interactive decision making using multi-armed bandits.
problem Understanding information dynamics in interactive decision making.
method Stochastic multi-armed bandit problem, focusing on optimal arm with a fixed margin.
result Distinct growth phases in mutual information, showing decoupling between success probability and information gain.
Study reveals mutual information is crucial for understanding algorithm performance in stochastic convex optimization.
problem Uncertainty in capturing the exceptional performance of learning algorithms using existing information-theoretic generalization bounds.
method Examined the relationship between mutual information and generalization in stochastic convex optimization.
result Mutual information is necessary for true risk minimization in stochastic convex optimization, indicating existing bounds fall short.
We create a formal framework for the design of informative securities in prediction markets. These securities allow a market organizer to infer the likelihood of events of interest as well as if he knew all of the traders' private signals. We consider the design of markets that are always informative, markets that are …
The paper proposes methods to extract and analyze individual variable information from complex dependencies.
problem Analyzing and understanding complex dependencies between multiple variables.
method Reversible normalization and iterative dependency reduction to extract individual information, and use it for direct mutual information and multi-feature Granger causality analysis.
result Decoupling of variables to analyze their individual information and direct mutual information transfers.
There are (at least) three approaches to quantifying information. The first, algorithmic information or Kolmogorov complexity, takes events as strings and, given a universal Turing machine, quantifies the information content of a string as the length of the shortest program producing it. The second, Shannon information…
We model continuous-time information flows generated by a number of information sources that switch on and off at random times. By modulating a multi-dimensional Lévy random bridge over a random point field, our framework relates the discovery of relevant new information sources to jumps in conditional expectation mart…
Identifying features that leak information about sensitive attributes is a key challenge in the design of information obfuscation mechanisms. In this paper, we propose a framework to identify information-leaking features via information density estimation. Here, features whose information densities exceed a pre-defined…
Proposes indifference pricing to estimate weak information value.
problem Estimating the value of weak information in financial models.
method Tractable framework quantifying additional information, stability analysis.
result Sharp conditions for stability with counterexamples, including replicable claims.
The background for the general mathematical link between utility and information theory investigated in this paper is a simple financial market model with two kinds of small traders: less informed traders and insiders, whose extra information is represented by an enlargement of the other agents' filtration. The expecte…
The paper introduces submodular information measures for machine learning applications.
problem Generalizing information-theoretic measures to non-random variables.
method Developing combinatorial information measures based on submodular functions.
result Submodular mutual information is submodular in one argument for certain submodular functions.
In modern recommender systems, both users and items are associated with rich side information, which can help understand users and items. Such information is typically heterogeneous and can be roughly categorized into flat and hierarchical side information. While side information has been proved to be valuable, the maj…