KOC+ uses privileged information to improve one-class classification performance.
problem Outlier detection and novelty detection using kernel methods.
method Kernel ridge regression with correction function for privileged information.
result KOC+ achieves better generalization performance compared to traditional methods.
A new multi-class active learning method combining informativeness and representativeness.
problem Efficiently labeling large datasets with limited resources.
method A hybrid informative and representative criterion approach for multi-class active learning.
result The proposed method outperforms state-of-the-art methods on multiple UCI datasets.
New bounds study class-specific generalization error in machine learning.
problem Existing generalization theories assume uniform class performance, but in practice, classes vary significantly.
method Developed novel information-theoretic bounds using KL divergence and CMI.
result Theoretical bounds accurately capture complex class-generalization error behavior.
Enhances anomaly detection using privileged information in one-class SVM.
problem Anomaly detection in normal state description.
method Introduces a new one-class classification algorithm that incorporates privileged information.
result Improved performance in anomaly detection, validated on synthetic and real datasets.
Average-case information complexity for learning is bounded, revealing only O(d) bits for most concepts.
problem Understanding the average-case information leakage in learning algorithms for concept classes.
method Developed a learning algorithm that reveals O(d) bits of information for most concepts in a class of VC-dimension d.
result Most concepts in the class do not require large amounts of information leakage, revealing only O(d) bits on average.
Direct sum result for information complexity in PAC learning.
problem Minimum information required for consistent and proper learning.
method Introduced a class of functions with information complexity and proved a direct sum result.
result Direct sum result for information complexity in PAC learning.
Researchers find a class/cross-class structure in deep learning spectra.
problem Understanding the structure of deep learning classifiers.
method Empirical spectral analysis of deepnet classifiers.
result The cross-class structure explains various features in deepnet spectra.
New entropies and divergences defined using group theory.
problem Defining new entropies and divergences with group-theoretical properties.
method Formal group theory and information geometry to construct new entropies and divergences.
result A method for constructing new entropies and divergences from known ones.
Study uses information-theoretic measures to analyze neural networks and neuron importance.
problem Understanding the importance of individual neurons in neural networks.
method Cumulative ablation of neurons, analyzing entropy, mutual information, and class selectivity.
result Class selectivity is not a good indicator for classification performance, while mutual information and selectivity are positively correlated with performance.
Proposed SMO algorithm for OC-SVM+ significantly outperforms non-sequential algorithms.
problem One-class SVM with privileged information
method Sequential Minimal Optimization (SMO) algorithm
result Finite-time convergence established
A new method combines topological features with graph convolutional networks for improved paper classification.
problem Classifying papers based on their content and structure.
method Combining topological features of nodes with information propagation through Graph Convolutional Networks (GCN).
result The method improves classification accuracy on CiteSeer and Cora datasets, matching or exceeding text-based classification results.
New method boosts GANs by creating new classes from clustering, improving sample quality.
problem Improving GAN sample quality with class information.
method Clustering representations learned by GAN to create new classes, enhancing conditional GANs.
result Generated samples reach state-of-the-art Inception scores for CIFAR-10 and STL-10 datasets.
Paper proposes an improved domain adaptation technique using class-based information.
problem Adapting classifiers across domains with labeled source and unlabeled target datasets.
method Adversarial discriminator approach informed by class structure in source dataset.
result State-of-the-art results achieved on benchmark datasets.
Bayesian model enhances phenotype discovery in asthma EHRs.
problem Lack of interpretability in unsupervised learning phenotyping of EHR data.
method Operationalized a Bayesian latent class framework with clinical knowledge priors.
result Identified an asthma sub-phenotype with elevated eosinophil levels and allergy markers.
This paper improves multi-class calibration methods using mutual information maximization-based binning.
problem Calibration of deep neural network predictions, especially for small prior classes.
method I-Max concept for binning, shared class-wise calibration strategy.
result Improves multi-class ranking and calibration performance using a small calibration set.
Zero-shot audio classification using class label embeddings.
problem Classifying audio without labeled data.
method Bilinear model with audio feature embeddings and class label embeddings.
result Achieved accuracy up to 39.7% for natural audio categories.
The paper measures semantic information production in generative models using information theory.
problem Measuring when semantic decisions are made during generative model training.
method Using an online formula for the optimal Bayesian classifier, the paper estimates conditional entropy and determines time intervals for highest information transfer.
result Semantic information transfer is highest in intermediate stages of diffusion, with different classes making decisions at different times.
Two new undersampling methods improve classification accuracy for imbalanced datasets.
problem Class imbalance and distributional differences in large datasets lead to biased models and poor predictive performance.
method Mutual information-based stratified simple random sampling and support points optimization.
result Empirical results show higher balanced classification accuracy compared to traditional techniques.
Solves multi-class imbalanced data problem with geometry-based sampling and synthetic data.
problem Handling imbalanced multi-class data in classification problems.
method Two novel methods: undersampling and oversampling.
result Efficacy demonstrated through comparison with state-of-the-art methods.
Paper analyzes iterative learning for concept classes and learns half-spaces.
problem Learning concept classes efficiently with iterative learners.
method Analyzes various settings of iterative learning and provides a constructive algorithm for half-spaces.
result Constructive iterative algorithm for learning half-spaces from informant.
Two new regularizers leverage class information to improve deep network performance.
problem Improving deep network performance and feature independence for classification tasks.
method Class-wise Covariance Regularizer (cw-CR) and Variance Regularizer (cw-VR) designed to manipulate statistical characteristics per class.
result Significant improvements in classification performance for 21 out of 22 tasks.
Improved GANs generate high-res images with class information.
problem Synthesizing high-resolution photorealistic images.
method Conditional GANs with label conditioning for improved image synthesis.
result 128x128 resolution samples are more discriminable and diverse than lower resolutions.
Develops a method to evaluate model consistency with data using information theory.
problem Evaluating the consistency of a model with observed data.
method Information-theoretic approach based on model's ability to generate similar data.
result The method can be used for sequential and nonlinear data, and is validated on synthetic and real data.
Feature selection is one of the most fundamental problems in machine learning. An extensive body of work on information-theoretic feature selection exists which is based on maximizing mutual information between subsets of features and class labels. Practical methods are forced to rely on approximations due to the diffi…
This paper develops, in a Brownian information setting, an approach for analyzing the preference for information, a question that motivates the stochastic differential utility (SDU) due to Duffie and Epstein [Econometrica 60 (1992) 353-394]. For a class of backward stochastic differential equations (BSDEs) including th…
This work improves metric learning models by incorporating class hierarchies.
problem Class hierarchies are often ignored in classification-based metric learning models.
method Trained softmax classifier and metric learning models with predefined class hierarchies.
result ProxyDR model performs better in hierarchical inference and hierarchy-informed performance.
New bounds explain modern machine learning algorithms' generalization.
problem Explaining generalization behavior of modern machine learning algorithms.
method Proposes a new complexity measure based on empirical Rademacher complexity of an algorithm- and data-dependent hypothesis class.
result Obtains novel bounds with finite fractal dimension, simplifies proofs, and recovers known results.
Paper tackles class-incremental learning by combining self-supervised learning to mitigate prior information loss.
problem Catastrophic forgetting and prior information loss in class-incremental learning.
method Combining self-supervised learning with class-incremental learning to mitigate prior information loss.
result Proposed method outperforms state-of-the-art methods.
A novel method learns representations from PU data without needing class-prior estimation.
problem Training classifiers from only positive and unlabeled data requires accurate class-prior probability estimation.
method Information-theoretic representation learning based on the information-maximization principle.
result Our method combined with deep neural networks achieves state-of-the-art PU classification performance.
Physics-informed neural networks solve PDEs using neural networks.
problem Solving nonlinear partial differential equations (PDEs) with neural networks.
method Physics-informed neural networks trained to solve PDEs while respecting physical laws.
result Physics-informed neural networks can infer solutions to PDEs and create differentiable surrogate models.
New measures generalize existing ones, linking information and risk.
problem Linking information measures and risk in statistical decision problems.
method Introducing new families of divergence measures and deriving an information processing equality.
result Extension of variational φ-divergence representation to multiple distributions. SQFA learns features maximizing Fisher-Rao distance for better classification.
problem Improving classification accuracy through feature learning.
method SQFA learns linear features maximizing Fisher-Rao distance between class-conditional distributions.
result SQFA-H features achieve the best classification accuracy.
Generative framework tackles zero-shot learning with adversarial domain adaptation.
problem Domain shift between seen and unseen class distributions in zero-shot learning.
method End-to-end learning of seen and unseen class distributions, adversarial domain adaptation.
result Superior accuracies compared to state-of-the-art models on various benchmark datasets.
MILDA uses unlabelled data to compute LDA projections.
problem Training LDA models with unlabelled data.
method Minimal prior information to compute LDA projection vector.
result MILDA closely matches supervised LDA performance and adapts to non-stationary data.
Learn class-invariant and symmetry-equivariant representations for multi-class data.
problem Deep neural networks learn opaque representations; we aim to make them more transparent.
method Probabilistic modelling with two separate latent variables: invariant and equivariant.
result Qualitative and quantitative performance competitive with other methods, with little tuning.
Proposes a robust VIB approach using soft labels and mutual info estimation.
problem Improving robustness of VIB to adversarial perturbations.
method Refines categorical class information with soft labels from a reference network, relaxes Gaussian posterior assumption.
result Significantly outperforms benchmarked models on MNIST and CIFAR-10.
A new hierarchy quantifies agency in systems based on information processing.
problem Lack of a measurable, universal definition for agency in intelligent systems.
method Developed a bottom-up framework based on information processing hierarchy.
result Identified three orders of information processing (I, II, III) as necessary for agency.
This paper proposes a unified framework for recognizing seen and unseen classes using visual and semantic prototypes.
problem Class overfitting and misclassification of unseen classes in zero-shot learning.
method Decomposes G-ZSL into OSR and ZSL, introduces semantic side-information for OSR, and uses a VSG-CNN framework.
result Improves recognition performance and cognitive ability for unknown classes.
SketchEmbedNet learns image representations from sketches, useful for few-shot learning.
problem Learning image representations from sketches for few-shot learning.
method Training a model to produce sketches of images, focusing on informative embeddings.
result Model produces informative embeddings of novel images, classes, and datasets.
A new method embeds visual features into semantic space for open-set recognition.
problem Learning unseen classes in open-set recognition.
method Vocabulary-informed Extreme Value Learning (ViEVL) combining EVL and ViL.
result ViEVL embeds visual features into semantic space probabilistically, solving open-set recognition.
The information-based asset-pricing framework of Brody, Hughston and Macrina (BHM) is extended to include a wider class of models for market information. In the BHM framework, each asset is associated with a collection of random cash flows. The price of the asset is the sum of the discounted conditional expectations of…
Draft proposes adapting neural networks to match naive Bayes classifiers.
problem Bridge between neural networks and naive Bayes classifiers.
method Class-conditional compression and disentanglement using variational bounds.
result Latent representations enable naive Bayes classifier performance.
ADGAN improves risk tolerance prediction by aligning cross-domain data.
problem Lack of professional knowledge and domain-specific models in risk tolerance studies.
method Asymmetric cross-Domain Generative Adversarial Network (ADGAN) for domain scale inequality.
result ADGAN better handles class imbalance and unqualified data than state-of-the-art methods.
Paper develops error rates for physics-informed learning, comparing it to data-driven methods.
problem Understanding the trade-off between soft penalties and hard constraints in PISL.
method Develops complexity-dependent error rates using the small-ball method.
result Physics-informed estimators have comparable error rates to hard constrained methods, differing only by constants.
We simplify information measure computation using learned features.
problem Computing information measures from raw data is computationally expensive.
method Developed a separable design for computing information measures from learned feature representations.
result A variety of information measures can be computed efficiently through learned feature representations.
Bayesian method optimizes causal effect estimation from observational data.
problem Estimating causal effects from observational data with selection bias.
method Bayesian nonparametric approach using KL divergence and Fano's method.
result Optimal information rate achieved by a specific class of priors and an adaptation procedure.
Exploits class similarity for better machine learning models with confidence labels and projective loss functions.
problem Poor model performance due to confusing similar classes.
method Exploits class similarity with confidence labels and projective loss functions.
result Improved model performance on noisy labels.
This paper bounds meta-generalization gap using information theory.
problem Improving sample efficiency for new tasks in meta-learning.
method Information-theoretic upper bounds on meta-generalization gap for two meta-learning classes.
result Novel ITMI bounds for noisy iterative algorithms.