Construct quaternionic-Kähler metrics from special Kähler manifolds with specific BPS structure variations.
problem Construct quaternionic-Kähler metrics from special Kähler manifolds with mutually local variations of BPS structures.
method Construct quaternionic-Kähler metrics from a conical special Kähler manifold with a certain type of mutually-local variation of BPS structures. Provide global and local explicit formulas for the quaternionic-Kähler metric.
result Construct quaternionic-Kähler metrics that are positive-definite and deform the 1-loop corrected Ferrara-Sabharval metric.
Proposes VCLANC for attributed network clustering using node and attribute embeddings.
problem Lack of mutual affinity exploitation between nodes and attributes in graph convolution.
method Dual variational auto-encoders for node and attribute embeddings, Gaussian mixture model priors, mutual distance and clustering assignment hardening losses.
result Demonstrates effectiveness on real-world attributed network datasets.
Proposes a framework to maximize mutual information in VAE models for better latent code representation.
problem Lack of explicit measurement of the quality of learned representations in VAE models.
method Variational Mutual Information Maximization Framework for VAE.
result Maximizes mutual information between latent codes and observations, improving latent code representation.
A new method estimates mutual information using neural classifiers.
problem Estimating mutual information for high-dimensional data is challenging.
method Trains a classifier to estimate joint distribution probability.
result Demonstrates high accuracy and reduces variance compared to variational methods.
The paper proposes a method to learn structured representations from unlabeled data using mutual information maximization.
problem Learning structured representations from unlabeled data.
method Adversarial maximization of mutual information between a structured latent variable and a target variable.
result The proposed method outperforms current baselines in document hashing and yields highly compressed interpretable representations.
Improved Monte-Carlo models by constraining mutual information between latent and observable variables.
problem Training density models leads to latent variables being useless.
method Weave tighter Monte-Carlo bounds with mutual information constraints.
result Improved training of models with continuous and discrete latent variables.
We introduce the Mutual Information Machine (MIM), a novel formulation of representation learning, using a joint distribution over the observations and latent state in an encoder/decoder framework. Our key principles are symmetry and mutual information, where symmetry encourages the encoder and decoder to learn differe…
The mutual information is a core statistical quantity that has applications in all areas of machine learning, whether this is in training of density models over multiple data modalities, in maximising the efficiency of noisy transmission channels, or when learning behaviour policies for exploration by artificial agents…
Measuring mutual information from finite data is difficult. Recent work has considered variational methods maximizing a lower bound. In this paper, we prove that serious statistical limitations are inherent to any method of measuring mutual information. More specifically, we show that any distribution-free high-confide…
Mutual information bounds generalization error in variational classifiers.
problem Controlling overfitting in variational classifiers.
method Derive bounds on generalization error using mutual information.
result Mutual information bounds the generalization error in variational classifiers.
AMI framework improves text generation by optimizing mutual information between source and target.
problem Previous MI approaches ignored the backward network, leading to loose variational bounds.
method AMI is a saddle point optimization framework that iteratively promotes and demotes generated instances.
result AMI significantly outperforms baselines on various text generation tasks.
Proposes a framework to improve VAE latent codes using mutual information.
problem Lack of explicit measure for VAE latent variable quality.
method Variational Mutual Information Maximization Framework.
result Improves relationships between latent codes and observations.
Novel mutual information bound improves statistical inference rates.
problem Improving statistical inference rates in Bayesian nonparametrics.
method Introduces a novel mutual information bound.
result Improved contraction rates for fractional posteriors.
Hybrid method improves mutual information estimation from samples.
problem Estimating mutual information from joint distributions is challenging.
method Proposes a hybrid method combining discriminative and generative approaches.
result Hybrid methods yield tighter variational bounds on mutual information.
Feature selection is one of the most fundamental problems in machine learning. An extensive body of work on information-theoretic feature selection exists which is based on maximizing mutual information between subsets of features and class labels. Practical methods are forced to rely on approximations due to the diffi…
New neural network approach using mutual information.
problem Training neural networks for imbalanced datasets.
method Converts neural network classifiers to mutual information evaluators.
result New form of softmax leads to better classification accuracy, especially for imbalanced datasets.
Proposes a robust VIB approach using soft labels and mutual info estimation.
problem Improving robustness of VIB to adversarial perturbations.
method Refines categorical class information with soft labels from a reference network, relaxes Gaussian posterior assumption.
result Significantly outperforms benchmarked models on MNIST and CIFAR-10.
MSRL learns a representation maximizing mutual info with response variables.
problem Learning sufficient representations for complex, multi-dimensional data.
method Variational mutual information, deep neural networks, generalized Dudley's inequality.
result MSRL achieves consistent and accurate representation learning.
Paper describes profiles of multivariate normal distributions and novel estimators for mutual information.
problem Estimating mutual information for complex distributions.
method Analytical description of profiles, introduction of Bend and Mix Models, Monte Carlo estimation.
result Bend and Mix Models accurately estimate mutual information profiles and provide Bayesian estimates.
In this paper, we suggest a framework to make use of mutual information as a regularization criterion to train Auto-Encoders (AEs). In the proposed framework, AEs are regularized by minimization of the mutual information between input and encoding variables of AEs during the training phase. In order to estimate the ent…
A new contrastive MI estimator improves efficiency and tightness.
problem Efficient and tight mutual information estimation.
method Contrastive Fenchel-Legendre optimization.
result The FLO estimator is tight and converges under stochastic gradient descent.
Variational approaches based on neural networks are showing promise for estimating mutual information (MI) between high dimensional variables. However, they can be difficult to use in practice due to poorly understood bias/variance tradeoffs. We theoretically show that, under some conditions, estimators such as MINE ex…
New method learns diverse solutions in reinforcement learning without gradient bias.
problem Lack of diverse solutions in reinforcement learning tasks.
method Maximizes state-action-based mutual information directly, using variational lower bound.
result Successfully learns an infinite set of diverse solutions.
This work improves independence tests for high-dimensional data.
problem Detecting subtle dependencies between high-dimensional random variables with complex distributions.
method Develops two approaches to learn powerful independence tests using variational mutual information and HSIC.
result Optimized HSIC tests generally outperform other approaches on detecting structured dependence.
In the field of machine learning, it is still a critical issue to identify and supervise the learned representation without manually intervening or intuition assistance to extract useful knowledge or serve for the downstream tasks. In this work, we focus on supervising the influential factors extracted by the variation…
Collaborative filtering (CF) has been successfully employed by many modern recommender systems. Conventional CF-based methods use the user-item interaction data as the sole information source to recommend items to users. However, CF-based methods are known for suffering from cold start problems and data sparsity proble…
Study examines Indian equity mutual funds' investment style and risk-shifting.
problem Understanding how Indian equity mutual funds' investment styles affect their returns.
method Estimating size and style beta coefficients, identifying breakpoints, analyzing investment styles, and assessing risk-shifting intensity.
result Funds can enhance returns by shifting to high-return styles like Small Value and Small Blend.
Estimating and optimizing Mutual Information (MI) is core to many problems in machine learning; however, bounding MI in high dimensions is challenging. To establish tractable and scalable objectives, recent work has turned to variational bounds parameterized by neural networks, but the relationships and tradeoffs betwe…
Mathematical study of instanton corrected q-map spaces and their isometries.
problem Understanding the isometries of instanton corrected q-map spaces.
method Study of isometries of instanton corrected q-map spaces associated to PSR manifolds.
result Explicit example of instanton corrected q-map space with full SL(2,Z) acting by isometries.
Maximizes image representation dependence for self-supervised learning.
problem Learning meaningful image representations from unlabeled data.
method Maximizes Hilbert-Schmidt Independence Criterion (HSIC) between image transformations and identity.
result Matches state-of-the-art performance on ImageNet and other vision tasks.
Enhances graph modeling with hyperbolic geometry and variational inference.
problem Challenges in modeling relational data with complex dependencies.
method Semi-implicit hierarchical variational Bayes with Poincaré embedding and mutual information regularization.
result Improves graph representation quality and flexibility in edge prediction and node classification.
LSB is a new MCMC method for discrete spaces that reduces target evaluations.
problem Sampling in discrete domains with high efficiency and adaptability.
method Local self-balancing proposals, mutual information objective, self-balancing learning.
result LSB converges with fewer target evaluations compared to existing methods.
Variational Autoencoder (VAE), a simple and effective deep generative model, has led to a number of impressive empirical successes and spawned many advanced variants and theoretical investigations. However, recent studies demonstrate that, when equipped with expressive generative distributions (aka. decoders), VAE suff…
Traditional nearest points methods use all the samples in an image set to construct a single convex or affine hull model for classification. However, strong artificial features and noisy data may be generated from combinations of training samples when significant intra-class variations and/or noise occur in the image s…
Proposes a new method to measure epistemic uncertainty in Bayesian neural networks.
problem Measuring epistemic uncertainty in Bayesian neural networks for out-of-distribution detection.
method Proposes measuring disagreement between logits and their pre-softmax counterparts as an epistemic uncertainty measure.
result Proposed epistemic uncertainty scores outperform mutual information and equal predictive entropy performance.
Investigates invariant hulls of functionals on manifolds.
problem Understanding meaningful functionals on manifolds through reparameterizations.
method Uses inner-variations to transform arbitrary functionals into invariant realizations.
result Explicit computations for volume functional in N-dimensional manifolds, especially in N=2. Mutual information maximization has emerged as a powerful learning objective for unsupervised representation learning obtaining state-of-the-art performance in applications such as object recognition, speech recognition, and reinforcement learning. However, such approaches are fundamentally limited since a tight lower …
Paper proposes a method to extract style features from unlabeled data.
problem Extracting fine-grained features like styles from unlabeled data.
method Contrastive conditioned variational autoencoders with mutual information constraints.
result The method efficiently extracts style features from real-world natural image datasets.
Proposes VAE-KRnet for density estimation and variational Bayes.
problem Challenges of density estimation and posterior approximation in high dimensions.
method Combines VAE and KRnet for density estimation, incorporating mutual information maximization.
result Improves posterior approximation in high-dimensional cases.
Variational autoencoders (VAEs) have ushered in a new era of unsupervised learning methods for complex distributions. Although these techniques are elegant in their approach, they are typically not useful for representation learning. In this work, we propose a simple yet powerful class of VAEs that simultaneously resul…
New insights explain why β-VAEs fail at disentanglement.
problem Disentanglement performance of β-VAEs peaks at intermediate β and collapses as regularization increases. method Formalized information-theoretic mechanism, introduced λβ-VAE to stabilize disentanglement. result Strong regularization pressure leads to mutual information collapse in β-VAEs. Due to the advantage of achieving a better performance under weak regularization, elastic net has attracted wide attention in statistics, machine learning, bioinformatics, and other fields. In particular, a variation of the elastic net, adaptive elastic net (AEN), integrates the adaptive grouping effect. In this paper,…
Proposes CLUB for reliable MI minimization in high dimensions.
problem Estimating and minimizing mutual information in high-dimensional spaces.
method Contrastive Log-ratio Upper Bound (CLUB) for MI minimization.
result CLUB provides reliable estimation of mutual information.
Meta learning with information theory and Gaussian processes.
problem Few-shot learning problems.
method Information bottleneck, mutual information, variational approximations, Gaussian processes.
result Competitive accuracy on few-shot classification problems.
InfoQGAN uses mutual information to improve QGANs, overcoming mode collapse and feature disentanglement issues.
problem Mode collapse and lack of feature control in QGANs.
method Integrates InfoGAN principles with variational quantum circuit, classical discriminator, and MINE for mutual information optimization.
result InfoQGAN effectively mitigates mode collapse and achieves robust feature disentanglement.
Document clustering and topic modeling are two closely related tasks which can mutually benefit each other. Topic modeling can project documents into a topic space which facilitates effective document clustering. Cluster labels discovered by document clustering can be incorporated into topic models to extract local top…
IndiSeek learns disentangled representations by balancing independence and completeness.
problem Learning disentangled representations with mutual information in multi-modal data.
method Combines independence-enforcing objective with a reconstruction loss that bounds conditional mutual information.
result Demonstrates effectiveness on synthetic data, CITE-seq, and real-world multi-modal benchmarks.
A new multi-label CPC method improves mutual information estimation and representation learning.
problem Underestimation of mutual information in contrastive predictive coding.
method Introducing a multi-label classification problem to overcome the logm bound in mutual information estimation. result The new method exceeds the logm bound and leads to better mutual information estimation and improved unsupervised representation learning.