Very often features come with their own vectorial descriptions which provide detailed information about their properties. We refer to these vectorial descriptions as feature side-information. In the standard learning scenario, input is represented as a vector of features and the feature side-information is most often i…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “feature information”
Identifying features that leak information about sensitive attributes is a key challenge in the design of information obfuscation mechanisms. In this paper, we propose a framework to identify information-leaking features via information density estimation. Here, features whose information densities exceed a pre-defined…
We simplify information measure computation using learned features.
problem Computing information measures from raw data is computationally expensive.
method Developed a separable design for computing information measures from learned feature representations.
result A variety of information measures can be computed efficiently through learned feature representations.
Feature selection, which searches for the most representative features in observed data, is critical for health data analysis. Unlike feature extraction, such as PCA and autoencoder based methods, feature selection preserves interpretability, meaning that the selected features provide direct information about certain h…
Active feature selection uses mutual information to choose fewer labels for better feature selection.
problem Selecting features with limited labeled data.
method Uses active feature selection with mutual information criterion, optimizing label selection for better feature quality.
result Algorithm selects features with higher mutual information using fewer labels than the data set size.
Feature selection is one of the most fundamental problems in machine learning. An extensive body of work on information-theoretic feature selection exists which is based on maximizing mutual information between subsets of features and class labels. Practical methods are forced to rely on approximations due to the diffi…
Feature selection can efficiently identify the most informative features with respect to the target feature used in training. However, state-of-the-art vector-based methods are unable to encapsulate the relationships between feature samples into the feature selection process, thus leading to significant information los…
Graph cross network improves graph classification accuracy.
problem Improving graph classification accuracy.
method Graph cross network (GXN) with vertex infomax pooling (VIPool) and feature-crossing layer.
result Improves graph classification accuracy by 2.12% and 1.15%.
A neural network approach for feature selection using mutual information.
problem Feature ranking and selection leading to sub-optimal solutions for class separability.
method Stochastic mutual information gradient estimation for dimensionality reduction.
result The network projects features onto an output space maximizing mutual information with class labels.
Maximizing Mutual Information Across Feature and Topology Views for Learning Graph Representationscs.LG
A novel method integrates feature and topology views for unsupervised graph representation learning.
problem Lack of mutual information across feature and topology views in graph representation learning.
method Proposes a multi-view representation learning module and a common representation learning module using mutual information maximization and reconstruction loss minimization.
result Demonstrates effectiveness in integrating feature and topology views, achieving comparable or better performance than supervised methods.
New causal measures improve feature selection in AI models.
problem Lack of causal interpretability in AI models.
method Introduces causal entropy and causal information gain to assess feature control.
result Demonstrates superiority of causal information gain in feature selection.
A new method for dynamic feature selection outperforms existing approaches.
problem Sequentially selecting features based on current information in machine learning.
method Greedy selection of features based on conditional mutual information, combined with a learning approach for optimization.
result The method outperforms existing feature selection methods in experiments.
Classic feature selection techniques remove those features that are either irrelevant or redundant, achieving a subset of relevant features that help to provide a better knowledge extraction. This allows the creation of compact models that are easier to interpret. Most of these techniques work over the whole dataset, b…
Dynamic acquisition of features improves predictions with limited data.
problem Limited or uncertain data requires additional relevant information for accurate assessments.
method Proposes models that dynamically acquire new features using conditional mutual information and arbitrary conditional flow.
result Demonstrates superior performance over baselines in multiple settings.
MCFNet recovers spatial detail and fuses it with semantic information for real-time segmentation.
problem Recovering spatial detail information and fusing it with semantic information in real-time.
method Proposes a new architecture (MCFNet) with feature refinement and fusion modules, and a gating unit.
result Achieves competitive performance with high speed (75.5% mIOU, 151.3 FPS on Cityscapes).
Proposes a method to select features for subgroup datasets with systematic missing data.
problem Feature selection for datasets with subgroup structure and systematic missing data.
method Develops a heterogeneous graph neural network to propagate information between feature-subgroup-target variable connections.
result Demonstrates improved feature selection performance and scalability.
DEDACT breaks down feature importance into direct and associative components.
problem Lack of clear distinction between direct and associative feature importance.
method DEDACT framework to decompose direct and associative importance measures.
result Provides insight into sources of prediction-relevant information and feature pathways.
The creation of social ties is largely determined by the entangled effects of people's similarities in terms of individual characters and friends. However, feature and structural characters of people usually appear to be correlated, making it difficult to determine which has greater responsibility in the formation of t…
Feature selection aims to select the smallest feature subset that yields the minimum generalization error. In the rich literature in feature selection, information theory-based approaches seek a subset of features such that the mutual information between the selected features and the class labels is maximized. Despite …
GOLFS selects features for clustering by combining global and local information.
problem Feature selection for high-dimensional clustering without labels.
method Combines global and local information via manifold learning and regularized self-representation.
result Improves feature selection and clustering accuracy.
Graph Information Bottleneck (GIB) optimizes graph representations for robustness against adversarial attacks.
problem Challenges in learning graph representations due to structure and feature information.
method GIB is an information-theoretic principle that balances expressiveness and robustness by maximizing mutual information between representation and target, while constraining mutual information with input data.
result GIB-based models are more robust to adversarial attacks, achieving up to 31% improvement.
Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learningcs.LG
Our work proves CSF can recover ground-truth features in RL, improving understanding of feature learning.
problem Understanding the role of representation and mutual information in reinforcement learning.
method Investigates Contrastive Successor Features (CSF) method for identifiable representation learning in reinforcement learning.
result Proves CSF can recover ground-truth features up to a linear transformation.
Mutual Information (MI) is often used for feature selection when developing classifier models. Estimating the MI for a subset of features is often intractable. We demonstrate, that under the assumptions of conditional independence, MI between a subset of features can be expressed as the Conditional Mutual Information (…
We introduce instancewise feature selection as a methodology for model interpretation. Our method is based on learning a function to extract a subset of features that are most informative for each given example. This feature selector is trained to maximize the mutual information between selected features and the respon…
We study graph matching with correlated Gaussian features and find thresholds for exact recovery.
problem Graph matching with correlated Gaussian features.
method Information-theoretic thresholds and conditions for exact and almost exact recovery.
result Contextual information introduces a richer structure, with thresholds for exact and almost exact recovery no longer coinciding.
Supervised Quadratic Feature Analysis: Information Geometry Approach for Dimensionality Reductionstat.ML
SQFA learns features maximizing Fisher-Rao distance for better classification.
problem Improving classification accuracy through feature learning.
method SQFA learns linear features maximizing Fisher-Rao distance between class-conditional distributions.
result SQFA-H features achieve the best classification accuracy.
We introduce a framework for dynamic adversarial discovery of information (DADI), motivated by a scenario where information (a feature set) is used by third parties with unknown objectives. We train a reinforcement learning agent to sequentially acquire a subset of the information while balancing accuracy and fairness …
Automates feature selection and weighting in molecular systems.
problem Optimal feature selection and alignment in molecular systems.
method Differentiable Information Imbalance (DII) method for automated feature ranking and scaling.
result Automated feature selection and scaling that preserves information content and interpretability.
Feature selection is a technique to screen out less important features. Many existing supervised feature selection algorithms use redundancy and relevancy as the main criteria to select features. However, feature interaction, potentially a key characteristic in real-world problems, has not received much attention. As a…
Proposes fwelnet to improve prediction using feature information.
problem Improving prediction accuracy in supervised learning settings.
method fwelnet uses feature-weighted elastic net to adapt feature penalties.
result fwelnet outperforms lasso in test MSE and feature selection.
DIVI clusters noisy high-dimensional data with stable feature gating.
problem Challenging clustering in high-dimensional noisy data.
method Data-informed variational clustering framework combining global feature gating and adaptive structure growth.
result DIVI performs competitively under severe feature noise and remains computationally feasible.
Feature selection has been proven a powerful preprocessing step for high-dimensional data analysis. However, most state-of-the-art methods tend to overlook the structural correlation information between pairwise samples, which may encapsulate useful information for refining the performance of feature selection. Moreove…
Introduces top- regularization for better feature selection in machine learning.
problem Limited ability of existing feature selection methods to reconcile feature representativeness and inter-correlations.
method Top- regularization, which induces a sub-architecture on the model's architecture to select informative features and model complex relationships.
result Uniform approximation error bound for top- regularization approximating high-dimensional sparse functions.
Study shows how information loss and operation loss are related in feature representations.
problem Understanding the relationship between information loss and operation loss in feature representations.
method Analyzes the interplay between weak information loss and operation loss in continuous representations.
result Specific forms of vanishing information loss imply vanishing MPE loss in classification.
Improves machine learning models by incorporating physical laws into feature maps.
problem Lack of model interpretability in classical machine learning approaches.
method Physics-informed feature maps constructed from physical laws and dimensional analysis.
result Enhanced model interpretability and potential discovery of new physical equations.
TIPRDC anonymizes data features to protect privacy while retaining useful information.
problem Privacy concerns from crowdsourced data hinder deep learning applications.
method Hybrid training method combining adversarial and mutual information estimation.
result Feature extractor hides private information while preserving original data features.
Objective: A variety of pattern analysis techniques for model training in brain interfaces exploit neural feature dimensionality reduction based on feature ranking and selection heuristics. In the light of broad evidence demonstrating the potential sub-optimality of ranking based feature selection by any criterion, we …
Proposes QNN to protect input privacy in neural networks.
problem Protecting input privacy in neural networks.
method Quaternion-valued neural network (QNN) to hide input information.
result QNN effectively protects input privacy without significant accuracy loss.
We propose a novel objective function for learning robust deep representations of data based on information theory. Data is projected into a feature-vector space such that the mutual information of all subsets of features relative to the supervising signal is maximized. This objective function gives rise to robust repr…
This paper offers a characterization of fundamental limits on the classification and reconstruction of high-dimensional signals from low-dimensional features, in the presence of side information. We consider a scenario where a decoder has access both to linear features of the signal of interest and to linear features o…
Kernel methods linked to feature subspaces and maximal correlation kernels.
problem Understanding kernel methods and their relationship to feature extraction.
method Established a correspondence between feature subspaces and kernels, introduced maximal correlation kernels, and demonstrated their optimality.
result Kernel SVM on maximal correlation kernel achieves minimum prediction error.
New method quantifies feature interactions in machine learning models.
problem Capturing high-order interactions and feature contributions in predictive models.
method Information-theoretic approach using Conditional Mutual Information (CMI) via k-NN.
result Accurately recovers feature interactions in synthetic and real-world datasets.
We propose a novel strategy for extracting features in supervised learning that can be used to construct a classifier which is more robust to small perturbations in the input space. Our method builds upon the idea of the information bottleneck by introducing an additional penalty term that encourages the Fisher informa…
Unified framework for removing unwanted information from machine learning models.
problem Removing undesirable features or data points from machine learning models while preserving utility.
method Information-theoretic regularization approach for data point and feature unlearning.
result Unified mathematical framework with provable guarantees for both data point and feature unlearning.
Cost-efficient feature selection for multi-label classification in medicine.
problem Feature selection in multi-label classification with cost constraints.
method Sequential feature selection maximizing conditional mutual information, followed by cost-free feature selection using shadow features.
result The method effectively reduces prediction costs in medical applications.
Many methods have been proposed for community detection in networks, but most of them do not take into account additional information on the nodes that is often available in practice. In this paper, we propose a new joint community detection criterion that uses both the network edge information and the node features to…
Markov blanket feature selection, while theoretically optimal, is generally challenging to implement. This is due to the shortcomings of existing approaches to conditional independence (CI) testing, which tend to struggle either with the curse of dimensionality or computational complexity. We propose a novel two-step a…
This study ranks feature-block importance in multiblock neural networks.
problem Understanding feature contributions in multiblock neural networks.
method Three methods: composite, knock-in, and knock-out strategies.
result Each strategy has its merits for specific application scenarios.