arXiv research

Proposes Mutual Regression Distance for better distribution comparison.

We propose three measures of mutual dependence between multiple random vectors. All the measures are zero if and only if the random vectors are mutually independent. The first measure generalizes distance covariance from pairwise dependence to mutual dependence, while the other two measures are sums of squared distance…

GWIB improves counterfactual regression by balancing latent distributions and reducing selection bias.

Proposes mutual information for regression without prior knowledge.

In data science, it is often required to estimate dependencies between different data sources. These dependencies are typically calculated using Pearson's correlation, distance correlation, and/or mutual information. However, none of these measures satisfy all the Granger's axioms for an "ideal measure". One such ideal…

We propose Bayesian extensions of two nonparametric regression methods which are kernel and mutual $k$-nearest neighbor regression methods. Derived based on Gaussian process models for regression, the extensions provide distributions for target value estimates and the framework to select the hyperparameters. It is show…

A new mutual information lower bound for multimodal regression active learning.

Estimating entropy and mutual information consistently is important for many machine learning applications. The Kozachenko-Leonenko (KL) estimator (Kozachenko & Leonenko, 1987) is a widely used nonparametric estimator for the entropy of multivariate continuous random variables, as well as the basis of the mutual inform…

This paper introduces GEMINI, a new mutual information metric for unsupervised neural network training.

This paper introduces GEMINI, a new metric for unsupervised neural network training that avoids the need for regularizations.

This work improves generalisation bounds using chaining and information theory.

Study entropic regularization of Gaussian measures and processes on Hilbert space.

Computing approximate nearest neighbors in high dimensional spaces is a central problem in large-scale data mining with a wide range of applications in machine learning and data science. A popular and effective technique in computing nearest neighbors approximately is the locality-sensitive hashing (LSH) scheme. In thi…

InfoOT improves data alignment by maximizing mutual information.

Cross-entropy loss linked to metric learning, outperforming complex pairwise losses.

The $k$-nearest neighbor classification method ($k$-NNC) is one of the simplest nonparametric classification methods. The mutual $k$-NN classification method (M$k$NNC) is a variant of $k$-NNC based on mutual neighborship. We propose another variant of $k$-NNC, the symmetric $k$-NN classification method (S$k$NNC) based …

Active learning method reduces labeling cost for regression models with aggregated data.

Sequence models assign probabilities to variable-length sequences such as natural language texts. The ability of sequence models to capture temporal dependence can be characterized by the temporal scaling of correlation and mutual information. In this paper, we study the mutual information of recurrent neural networks …

The setting for this brief paper is R^3. Distance between two spheres is understood as distance delta between spherical centers. For instance, a Reuleaux tetrahedron T is the intersection of four unit balls satisfying delta=1 pairwise. Volume and surface area of T are already well-known; our humble contribution is to c…

Lumbermark clusters data robustly, slicing limbs of mutual reachability trees.

DC-SIS selects features faster than mRMR for Parkinson's vocal diagnosis.

The study quantifies the impact of fund miscategorization using machine learning.

New algorithms detect outliers in high-dimensional data with arbitrary shapes.

Hierarchical Federated Learning bounds generalize using Wasserstein distance.

Bayesian active learning method improved for censored regression data.

We propose two nonlinear regression methods, named Adversarial Orthogonal Regression (AdOR) for additive noise models and Adversarial Orthogonal Structural Equation Model (AdOSE) for the general case of structural equation models. Both methods try to make the residual of regression independent from regressors while put…

Paper proposes MIM-DRCFR to learn disentangled factors for better treatment effect estimation.

Study finds exact limits for sparse regression with fewer observations than usual.

We apply both distance-based (Jin and Matteson, 2017) and kernel-based (Pfister et al., 2016) mutual dependence measures to independent component analysis (ICA), and generalize dCovICA (Matteson and Tsay, 2017) to MDMICA, minimizing empirical dependence measures as an objective function in both deflation and parallel m…

Mixture distributions arise in many parametric and non-parametric settings -- for example, in Gaussian mixture models and in non-parametric estimation. It is often necessary to compute the entropy of a mixture, but, in most cases, this quantity has no closed-form expression, making some form of approximation necessary.…

Mutual information maximization has emerged as a powerful learning objective for unsupervised representation learning obtaining state-of-the-art performance in applications such as object recognition, speech recognition, and reinforcement learning. However, such approaches are fundamentally limited since a tight lower …

Selective regression allows abstention to improve fairness criteria.

Study finds stock selection ability of Chinese mutual funds is better than asset allocation ability.

In this paper, we propose novel strategies for neutral vector variable decorrelation. Two fundamental invertible transformations, namely serial nonlinear transformation and parallel nonlinear transformation, are proposed to carry out the decorrelation. For a neutral vector variable, which is not multivariate Gaussian d…

Framework generates multimodal datasets with known MI for benchmarking.

Study on CEF discount in Bangladesh, finds size and maturity impact, turnover negative.

A novel method integrates feature and topology views for unsupervised graph representation learning.

Most work in algorithmic fairness to date has focused on discrete outcomes, such as deciding whether to grant someone a loan or not. In these classification settings, group fairness criteria such as independence, separation and sufficiency can be measured directly by comparing rates of outcomes between subpopulations. …

We propose a test of independence of two multivariate random vectors, given a sample from the underlying population. Our approach, which we call MINT, is based on the estimation of mutual information, whose decomposition into joint and marginal entropies facilitates the use of recently-developed efficient entropy estim…

A new method assesses regression models' global optimality.

This work connects Cramér distance to QR-DQN for DRL.

Spatio-temporal graphs such as traffic networks or gene regulatory systems present challenges for the existing deep learning methods due to the complexity of structural changes over time. To address these issues, we introduce Spatio-Temporal Deep Graph Infomax (STDGI)---a fully unsupervised node representation learning…

Mutual information has been successfully adopted in filter feature-selection methods to assess both the relevancy of a subset of features in predicting the target variable and the redundancy with respect to other variables. However, existing algorithms are mostly heuristic and do not offer any guarantee on the proposed…

Study models Indian stock market using hyperbolic geometry for market stability and volatility analysis.

Study of Bayes optimal learning in high-dimensional linear regression with network side information.

KSG mutual information estimator, which is based on the distances of each sample to its k-th nearest neighbor, is widely used to estimate mutual information between two continuous random variables. Existing work has analyzed the convergence rate of this estimator for random variables whose densities are bounded away fr…

In this paper, we discuss how a Gromov-Hausdorff-like distance function over the space of all isometric classes of compact $C^k$-Riemannian manifolds should be defined in the aspect of the Riemannan submanifold theory, where $k\geq 1$. The most important fact in this discussion is as follows. The Hausdorff distance fun…

Proposes a new k-NN algorithm to improve classification accuracy by removing noise and pseudo-neighbours.