The paper extends physics-based information maximization to complex bandit problems.
problem Designing efficient decision-making policies for complex bandit problems.
method Information and free-energy maximization principles adapted to three distinct bandit types.
result Information maximization leads to strong performance in complex bandit problems.
Paper uses HodgeRank and information maximization for efficient crowdsourced ranking.
problem Crowdsourced ranking quality improvement with limited budget.
method Information maximization applied to HodgeRank for active sampling.
result Boosts sampling efficiency compared to traditional methods.
New bandit algorithm maximizes information gain.
problem Optimizing decision-making in uncertain environments.
method Approximates information maximization using entropy and free energy principles.
result Asymptotic optimality proven for two-armed bandit problem.
Study examines insider information's impact on arbitrage and utility maximization in financial portfolios.
problem Analyzing the relationship between insider information and arbitrage in financial portfolio optimization.
method Examines the utility maximization problem under different utility functions (logarithmic and CRRA) with and without no temporary-bankruptcy restriction, considering altered information flow.
result Insider information's value is bounded when arbitrage holds, and it does not always imply arbitrage.
Proposes a new method to enhance neural learning by maximizing information gain.
problem Improving neural learning by selecting key variables to maximize information gain.
method Adaptive Ensemble Kalman Filter to quantify uncertainty and maximize information gain.
result The proposed method enables the neural network to learn more effectively from stochastic systems.
Paper proposes InfoAE for disentangled representation learning.
problem Learning disentangled representations from unlabeled data.
method InfoAE learns disentangled representation by maximizing mutual information.
result Achieved 98.9% test accuracy on MNIST with unsupervised training.
Maximizing margins leads to lossless compression of training data.
problem Generalization in supervised learning.
method Information-theoretic interpretation of margin maximization.
result Margin maximization is a form of lossless maximal compression.
The paper analyzes generalization of noisy, iterative algorithms using maximal leakage.
problem Analyzing the generalization behavior of noisy, iterative learning algorithms.
method Information-theoretic framework with maximal leakage metric.
result Explicit upper bounds on maximal leakage for various scenarios.
Proposes a framework to maximize mutual information in VAE models for better latent code representation.
problem Lack of explicit measurement of the quality of learned representations in VAE models.
method Variational Mutual Information Maximization Framework for VAE.
result Maximizes mutual information between latent codes and observations, improving latent code representation.
Kernel methods linked to feature subspaces and maximal correlation kernels.
problem Understanding kernel methods and their relationship to feature extraction.
method Established a correspondence between feature subspaces and kernels, introduced maximal correlation kernels, and demonstrated their optimality.
result Kernel SVM on maximal correlation kernel achieves minimum prediction error.
Information-maximization clustering learns a probabilistic classifier in an unsupervised manner so that mutual information between feature vectors and cluster assignments is maximized. A notable advantage of this approach is that it only involves continuous optimization of model parameters, which is substantially easie…
Investors pay for additional asset information based on utility maximization.
problem Determining the optimal price for additional asset information.
method Solving a stochastic control problem with partial information and utility maximization.
result Investors choose to purchase information at a deterministic time.
MIC consistently estimates dependence in large datasets.
problem Estimating dependence between variable pairs in large datasets.
method Proving consistency of MIC as an estimator.
result MIC is a consistent estimator of population statistic MIC*.
Solves utility maximization for delayed informed investors.
problem Maximizing utility in a discrete time framework with delayed information.
method Utilizes theory from [4] and optimal portfolio guessing.
result Solution for exponential utility maximization in a multivariate normal setting with delay.
Optimizes information acquisition to reduce estimation risk and maximize utility.
problem Estimation risk in investor decision-making.
method Derives closed-form value functions using CARA and CRRA utility functions, employs variational methods to explore optimal acquisition.
result Acquiring information earlier is more valuable in reducing estimation risk and achieving higher utility.
Greedy policy maximizes information in unknown linear systems.
problem Exploration in unknown linear dynamical systems.
method Online greedy policy maximizing information.
result Competitive performance compared to gradient-based methods.
Study utility maximization with delayed information in continuous time Gaussian markets.
problem Maximizing utility with delayed information in continuous time Gaussian markets.
method Purely probabilistic approach based on Radon-Nikodym derivatives of Gaussian measures.
result Solution for optimal control and value in a specific Gaussian framework.
A novel method integrates feature and topology views for unsupervised graph representation learning.
problem Lack of mutual information across feature and topology views in graph representation learning.
method Proposes a multi-view representation learning module and a common representation learning module using mutual information maximization and reconstruction loss minimization.
result Demonstrates effectiveness in integrating feature and topology views, achieving comparable or better performance than supervised methods.
The paper interprets VQ-VAE loss as a form of information bottleneck.
problem Understanding the VQ-VAE loss function.
method Interpreted VQ-VAE loss as variational deterministic information bottleneck (VDIB) and variational information bottleneck (VIB).
result VQ-VAE loss can be derived from VDIB and approximated by VIB.
Semi-supervised clustering aims to introduce prior knowledge in the decision process of a clustering algorithm. In this paper, we propose a novel semi-supervised clustering algorithm based on the information-maximization principle. The proposed method is an extension of a previous unsupervised information-maximization …
TIM maximizes mutual information for few-shot learning, outperforming state-of-the-art methods.
problem Few-shot learning with limited labeled data.
method Transductive Information Maximization (TIM) with alternating-direction solver.
result Significant improvement in accuracy across various datasets and networks.
MINIMALIST maximizes mutual information for likelihood estimation from simulated data.
problem Learning model parameters from likelihood functions that cannot be computed.
method Maximizes mutual information between simulated data and model parameters using neural networks.
result Different methods aiming at the same optimal energy form can be directly benchmarked.
We review recent results about the maximal values of the Kullback-Leibler information divergence from statistical models defined by neural networks, including naive Bayes models, restricted Boltzmann machines, deep belief networks, and various classes of exponential families. We illustrate approaches to compute the max…
The paper proposes a method to learn structured representations from unlabeled data using mutual information maximization.
problem Learning structured representations from unlabeled data.
method Adversarial maximization of mutual information between a structured latent variable and a target variable.
result The proposed method outperforms current baselines in document hashing and yields highly compressed interpretable representations.
InfoMax-VAE enhances VAEs for better representation learning.
problem Lack of meaningful learned representations in VAEs.
method Combines VAEs with mutual information maximization.
result Significantly boosts quality of learned high-level representations.
InfoOT improves data alignment by maximizing mutual information.
problem Optimal transport's limitations in handling clusters, outliers, and new data.
method InfoOT extends optimal transport by maximizing mutual information while minimizing distances.
result InfoOT outperforms optimal transport in domain adaptation, cross-domain retrieval, and single-cell alignment.
IGNN improves GNNs by maximizing edge-state transform mutual information.
problem Optimizing GNNs for better relational information.
method Variational information maximization to learn optimal transform parameters.
result IGNN achieves state-of-the-art performance on molecular graph tasks.
A new framework extracts informative features from multiple data types.
problem Finding maximally correlated feature representations from multimodal data.
method Soft-HGR framework, which relaxes whitening constraints while preserving feature geometry.
result Soft-HGR learns more informative feature mappings and is more efficient to optimize.
Maximizes mutual information to improve graph neural networks performance.
problem Loss of information between nodes in GNNs aggregation and iteration schemes.
method Explores mutual information maximization in the aggregation and iteration scheme of GNNs.
result Improves state-of-the-art performance on graph tasks.
The paper tackles adversarial robustness by maximizing worst-case mutual information.
problem Training robust machine learning models against adversarial inputs is challenging.
method Develops a notion of representation vulnerability and an unsupervised learning method to maximize worst-case mutual information.
result Proves a lower bound on minimum adversarial risk and supports robustness of representations.
Estimates utility functions and information costs from YouTube comments.
problem Estimating rational inattention in Bayesian agents.
method Deep learning for clustering framing information, inverse reinforcement learning.
result Constructive estimates of utility and information costs.
Estimates user preferences from noisy paired comparisons.
problem Estimating user preferences from noisy paired comparisons.
method Greedy information maximization strategies.
result Superior preference estimation over state-of-the-art methods.
SQFA learns features maximizing Fisher-Rao distance for better classification.
problem Improving classification accuracy through feature learning.
method SQFA learns linear features maximizing Fisher-Rao distance between class-conditional distributions.
result SQFA-H features achieve the best classification accuracy.
MIRO learns robust latent spaces by maximizing mutual information with future information.
problem Robust perception in complex, unstructured environments with low sample complexity.
method MIRO maximizes mutual information in a latent space for model-based reinforcement learning.
result MIRO outperforms reconstruction objectives in cluttered scenes.
The paper examines utility maximization in markets with hidden Gaussian drift, finding restrictions on model parameters.
problem Utility maximization problems in markets with hidden Gaussian drift mean-reverting processes.
method Derives sufficient conditions for bounded maximum expected utility of terminal wealth for models with full and partial information.
result Restrictions on model parameters for bounded maximum expected utility.
QAInfomax improves reading comprehension by maximizing mutual information, achieving state-of-the-art performance.
problem Distractor sentences in question answering datasets are hard to distinguish from relevant ones.
method QAInfomax regularizes reading comprehension models to learn mutual information among passages, questions, and answers.
result QAInfomax achieves state-of-the-art performance on Adversarial-SQuAD dataset.
Stokes' theorem's boundary maximizes entropy.
problem Characterizing the boundary of a manifold using entropy.
method Maximizing entropy for codimension-1 submanifolds satisfying Stokes' theorem.
result The boundary of a manifold maximizes the entropy functional.
Maximizes coding rate difference for robust, discriminative features.
problem Learning robust, discriminative features from high-dimensional data.
method Maximal Coding Rate Reduction (MCR^2) principle.
result Significantly more robust to label corruptions in classification.
STDGI learns node representations for spatio-temporal graphs via mutual information maximization.
problem Challenges in learning node representations for spatio-temporal graphs due to structural changes over time.
method STDGI is a fully unsupervised approach based on mutual information maximization that exploits both spatial and temporal dynamics.
result STDGI's learned node representations improve spatio-temporal auto-regressive forecasting models.
We are working to develop automated intelligent agents, which can act and react as learning machines with minimal human intervention. To accomplish this, an intelligent agent is viewed as a question-asking machine, which is designed by coupling the processes of inference and inquiry to form a model-based learning unit.…
DIM learns deep representations by maximizing mutual information, outperforming unsupervised methods.
problem Learning useful representations from unlabeled data.
method Maximizing mutual information between input and output of a deep neural network encoder, incorporating prior knowledge.
result DIM outperforms other unsupervised learning methods and competes with fully-supervised learning on classification tasks.
We present a two-stage approach for learning dictionaries for object classification tasks based on the principle of information maximization. The proposed method seeks a dictionary that is compact, discriminative, and generative. In the first stage, dictionary atoms are selected from an initial dictionary by maximizing…
Develops a framework to obfuscate sensitive attributes in machine learning models.
problem Minimizing information leakage of sensitive attributes in crowdsourced data.
method Proposes a minimax optimization formulation and proves an information-theoretic lower bound.
result Adversarial learning achieves the best trade-off between attribute obfuscation and accuracy.
A new algorithm maximizes entropy or mutual information for efficient inference of nonstationary Gaussian processes.
problem Nonstationary dynamics in real-world phenomena pose challenges to accurate modeling.
method LISAL algorithm that adaptively maximizes entropy or mutual information on induced latent dynamics and marginal likelihood.
result Efficient inference of nonstationary Gaussian processes for large-scale real-world applications.
This paper concerns the recursive utility maximization problem under partial information. We first transform our problem under partial information into the one under full information. When the generator of the recursive utility is concave, we adopt the variational formulation of the recursive utility which leads to a s…
Feature selection is one of the most fundamental problems in machine learning. An extensive body of work on information-theoretic feature selection exists which is based on maximizing mutual information between subsets of features and class labels. Practical methods are forced to rely on approximations due to the diffi…
We describe Information Forests, an approach to classification that generalizes Random Forests by replacing the splitting criterion of non-leaf nodes from a discriminative one -- based on the entropy of the label distribution -- to a generative one -- based on maximizing the information divergence between the class-con…
Maximizes mutual info across views for better image representations.
problem Improving image representation learning through multiple views.
method Maximizing mutual information between features from multiple views.
result ImageNet accuracy of 68.1% using linear evaluation, significantly outperforming prior methods.