Conditional Mutual Information (CMI) is a measure of conditional dependence between random variables X and Y, given another random variable Z. It can be used to quantify conditional dependence among variables in many data-driven inference problems such as graphical models, causal learning, feature selection and time-se…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “classifier-based CMI”
New method uses CDMs to improve CI testing without distributional assumptions.
problem Testing conditional independence when the conditional distribution is unknown.
method Uses conditional diffusion models (CDMs) to approximate and a classifier-based CMI estimator.
result Proposed method performs better than GAN-based CI tests and controls type I and II errors.
Unified framework for proving generalization bounds in machine learning.
problem Proving generalization bounds for machine learning algorithms.
method Conditional mutual information (CMI) framework to express and optimize bounds.
result Unified framework for proving generalization bounds in the realizable setting.
Unified CMI bounds for meta learning, bridging information-theoretic and classical learning theory.
problem Lack of unified generalization bounds for meta learning.
method Evaluated CMI (e-CMI) framework for meta learning.
result Unified generalization bounds for meta learning in terms of e-CMI.
This work analyzes generalization in federated learning using information theory.
problem Generalization performance in federated learning is less explored compared to centralized learning.
method The work applies an information-theoretic analysis via the conditional mutual information (CMI) framework to study federated learning's two-level generalization.
result The work derives multiple CMI-based bounds, including hypothesis-based CMI bounds and fast-rate evaluated CMI bounds, which improve convergence rates for specific model aggregation strategies and structured loss functions.
We provide an information-theoretic framework for studying the generalization properties of machine learning algorithms. Our framework ties together existing approaches, including uniform convergence bounds and recent methods for adaptive data analysis. Specifically, we use Conditional Mutual Information (CMI) to quant…
New bounds using samplewise evaluated CMI for deep neural networks.
problem Improving generalization bounds for deep neural networks.
method Introduced a new family of information-theoretic generalization bounds using samplewise evaluated conditional mutual information (CMI).
result The new bounds can be tighter than previous ones for deep neural networks.
Mutual Information (MI) is often used for feature selection when developing classifier models. Estimating the MI for a subset of features is often intractable. We demonstrate, that under the assumptions of conditional independence, MI between a subset of features can be expressed as the Conditional Mutual Information (…
Paper tightens CMI bounds for learning algorithms.
problem Generalization error in statistical learning.
method Stochastic projection and lossy compression.
result New CMI bounds yield generalization guarantees of .
New method tests CMI using deep neural networks for high-dimensional data.
problem Testing conditional mean independence in high-dimensional settings.
method Population CMI measure and bootstrap-based testing with deep generative neural networks.
result Strong empirical performance and versatility in various scenarios.
The paper shows that the Gauss map of minimal surfaces is open and meagre in the space of holomorphic maps.
problem Characterizing the set of minimal surfaces with a specific Gauss map.
method Analyzing the spaces of conformal minimal immersions and holomorphic maps, and using topological properties.
result The Gauss map assignment is an open map, and the set of minimal surfaces satisfying the Osserman curvature estimate is meagre.
Meta-learning bound uses conditional mutual information.
problem Bounding generalization performance in meta-learning.
method Extends CMI framework to meta-learning with a meta-supersample.
result Explicit bound involving two CMI terms.
Due to the advantage of achieving a better performance under weak regularization, elastic net has attracted wide attention in statistics, machine learning, bioinformatics, and other fields. In particular, a variation of the elastic net, adaptive elastic net (AEN), integrates the adaptive grouping effect. In this paper,…
Testing for conditional independence is a core aspect of constraint-based causal discovery. Although commonly used tests are perfect in theory, they often fail to reject independence in practice, especially when conditioning on multiple variables. We focus on discrete data and propose a new test based on the notion of …
New monitoring method detects ML risk models' performance changes in medical interventions.
problem Monitoring ML risk models in healthcare is complicated by confounding medical interventions.
method Developed a new score-based CUSUM monitoring procedure with dynamic control limits.
result Valid inference is possible if conditional exchangeability or time-constant selection bias hold.
This paper surveys a few aspects of the global theory of wave equations. This material is structured around the contents of a minicourse given by the second author during the CMI/ETH Summer School on evolution equations during the Summer of 2008.
Canonical Correlation Analysis (CCA) is a classic technique for multi-view data analysis. To overcome the deficiency of linear correlation in practical multi-view learning tasks, various CCA variants were proposed to capture nonlinear dependency. However, it is non-trivial to have an in-principle understanding of these…
Proposes a new bound on generalization error using conditional mutual information.
problem Improving the generalization error bound in machine learning.
method Combines error decomposition and conditional mutual information techniques.
result New bound is order-wise better than previous ones in a simple Gaussian setting.
Estimates conditional mutual information using a minmax formulation.
problem Estimating conditional mutual information in high dimensions.
method Uses a minmax optimization problem to train a neural network.
result Improves estimation accuracy compared to existing methods.
A new method improves model generalization by recognizing representations.
problem Generalization issues in existing Information Bottlenecks.
method Recognizable Information Bottleneck (RIB) using f-CMI and density ratio matching.
result Improves model generalization through recognizability regularization.
Feature selection aims to select the smallest feature subset that yields the minimum generalization error. In the rich literature in feature selection, information theory-based approaches seek a subset of features such that the mutual information between the selected features and the class labels is maximized. Despite …
Develops a new framework for analyzing sequential decision-making problems using information theory.
problem Lack of information-theoretic generalization bounds for sequential decision-making problems.
method Introduces a sequential supersample framework that separates learner filtration from proof-side enlargement, controlling the generalization gap by sequential CMI.
result Establishes a sequential CMI that controls the generalization gap in sequential decision-making problems.
Study shows weak homotopy equivalences for complete minimal surfaces.
problem Understanding complete minimal surfaces and their properties.
method Analyzes algebraic null immersions and conformal minimal immersions.
result Inclusion and differential mappings are weak homotopy equivalences.
Optimization of Trading Physics Models of Marketsphysics.comp-ph
We describe an end-to-end real-time S&P futures trading system. Inner-shell stochastic nonlinear dynamic models are developed, and Canonical Momenta Indicators (CMI) are derived from a fitted Lagrangian used by outer-shell trading models dependent on these indicators. Recursive and adaptive optimization using Adaptive …
Prediction models in mobility and transportation maintenance systems have been dramatically improved through using machine learning methods. This paper proposes novel machine learning models for intelligent road inspection. The traditional road inspection systems based on the pavement condition index (PCI) are often as…
Study optimizes fairness in predictive models by balancing utility and separation.
problem Balancing fairness and utility in predictive models.
method Information-theoretic approach using conditional mutual information (CMI).
result Reduces separation violations while maintaining or improving utility.
Adaptive stopping in MCMC using classifier-based dynamics
problem Sampling from complex, unnormalized probability densities
method Training state-dependent neural classifiers
result Significant reduction in average trajectory lengths
Convolutional neural networks improve image classification accuracy.
problem Improving accuracy in image classification.
method Analyzing the convergence rate of misclassification risk for image classifiers.
result A rate of convergence independent of image dimension proves the effectiveness of CNNs.
New method disentangles latent subspaces under correlation shifts.
problem Correlations between factors of variation make disentanglement models less robust.
method Enforces independence between subspaces conditioned on available attributes using adversarial CMI minimization.
result Models are disentangled and robust under correlation shifts, including in weakly supervised settings.
Topological data analysis aims to extract topological quantities from data, which tend to focus on the broader global structure of the data rather than local information. The Mapper method, specifically, generalizes clustering methods to identify significant global mathematical structures, which are out of reach of man…
Deep learning classifiers are known to be vulnerable to adversarial examples. A recent paper presented at ICML 2019 proposed a statistical test detection method based on the observation that logits of noisy adversarial examples are biased toward the true class. The method is evaluated on CIFAR-10 dataset and is shown t…
Fault detection problem for closed loop uncertain dynamical systems, is investigated in this paper, using different deep learning based methods. Traditional classifier based method does not perform well, because of the inherent difficulty of detecting system level faults for closed loop dynamical system. Specifically, …
An inductive probabilistic classification rule must generally obey the principles of Bayesian predictive inference, such that all observed and unobserved stochastic quantities are jointly modeled and the parameter uncertainty is fully acknowledged through the posterior predictive distribution. Several such rules have b…
We propose a new formulation of Multiple-Instance Learning (MIL). In typical MIL settings, a unit of data is given as a set of instances called a bag and the goal is to find a good classifier of bags based on similarity from a single or finitely many "shapelets" (or patterns), where the similarity of the bag from a sha…
This thesis explores supervised classification methods using Bayesian and exchangeability theories.
problem Assigning objects into predefined classes using training data and auxiliary information.
method Bayesian inductive theories and exchangeabilities (de Finetti and partition exchangeability).
result Optimal classifiers for different scenarios of object features and categories.
The nearest-centroid classifier is a simple linear-time classifier based on computing the centroids of the data classes in the training phase, and then assigning a new datum to the class corresponding to its nearest centroid. Thanks to its very low computational cost, the nearest-centroid classifier is still widely use…
Defines explanations for classifier outcomes using causal concepts.
problem Understanding classifier outcomes in a causal context.
method Proposes a new definition of explanation based on causality, compares it with existing notions, and evaluates it experimentally.
result Experimental evaluation shows the new definition's effectiveness on financial datasets.
Neyman-Pearson testing improves goodness of fit in detecting new physics.
problem Detecting small anomalies in data distributions.
method Employing Neyman-Pearson strategy with a rich parametrized family of models.
result Neyman-Pearson testing is more sensitive to small departures and unbiased towards specific anomalies.
In this paper, we present an algorithm for minimizing the difference between two submodular functions using a variational framework which is based on (an extension of) the concave-convex procedure [17]. Because several commonly used metrics in machine learning, like mutual information and conditional mutual information…
CI-GNN uses GNNs to diagnose psychiatric disorders by identifying causally relevant brain regions.
problem Leveraging GNNs for psychiatric diagnosis requires interpretable models to understand decision-making.
method CI-GNN integrates Granger causality into GNNs to identify causally relevant subgraphs.
result CI-GNN provides more reliable and concise explanations of psychiatric diagnoses.
We consider the binary classification problem in a setup that preserves the privacy of the original sample. We provide a privacy mechanism that is locally differentially private and then construct a classifier based on the private sample that is universally consistent in Euclidean spaces. Under stronger assumptions, we…
Although deep learning has shown great success in recent years, researchers have discovered a critical flaw where small, imperceptible changes in the input to the system can drastically change the output classification. These attacks are exploitable in nearly all of the existing deep learning classification frameworks.…
PolyGraph Discrepancy improves graph generative model evaluation.
problem Inability of existing metrics to provide an absolute performance measure and comparability across different graph descriptors.
method Approximates Jensen-Shannon distance using binary classifiers trained to distinguish between real and generated graphs.
result PGD provides a more robust and insightful evaluation compared to MMD metrics.
Spherical quadrilaterals classified based on geometric properties.
problem Classifying spherical quadrilaterals up to isometry.
method Developing map and condition of genericity.
result Space of quadrilaterals with prescribed angles consists of finitely many open curves.
32 knot projections classified based on forbidden Reidemeister moves.
problem Classifying knot projections based on allowed Reidemeister moves.
method 32 homotopy classifications based on forbidden Reidemeister moves.
result 20 non-trivial cases of knot projections are mutually different.
Classifiers based on sparse representations have recently been shown to provide excellent results in many visual recognition and classification tasks. However, the high cost of computing sparse representations at test time is a major obstacle that limits the applicability of these methods in large-scale problems, or in…
A widespread approach in machine learning to evaluate the quality of a classifier is to cross -- classify predicted and actual decision classes in a confusion matrix, also called error matrix. A classification tool which does not assume distributional parameters but only information contained in the data is based on th…
We consider the two-group classification problem and propose a kernel classifier based on the optimal scoring framework. Unlike previous approaches, we provide theoretical guarantees on the expected risk consistency of the method. We also allow for feature selection by imposing structured sparsity using weighted kernel…