Enhanced BCPNN learns hidden representations without labels.
problem Unsupervised learning of hierarchical representations in neural networks.
method Extended BCPNN architecture with local Hebbian learning mechanisms.
result Demonstrated capability for unsupervised learning of salient hidden representations.
Minimal variations guide unsupervised learning for better downstream tasks.
problem Efficiently describing raw data for various future tasks.
method Minimal variations as a guiding principle for unsupervised representation learning.
result Unveiling minimal variations as a principle behind unsupervised learning.
Paper summarizes unsupervised learning challenges for disentangled representations.
problem Unsupervised learning of disentangled representations without inductive biases.
method Theoretical and practical analysis of existing approaches.
result Unsupervised disentanglement is fundamentally impossible without inductive biases.
Unsupervised learning models can be indistinguishable without identifiability, leading to unreliable representations.
problem Unsupervised learning models may be indistinguishable without identifiability, making it impossible to recover a ground truth generative model.
method Construction based on nonlinear independent component analysis theory to illustrate potential failure cases.
result Counterexamples show that identifiability is crucial for reliable unsupervised representation learning.
Unsupervised method learns hierarchical graph representations without labels.
problem Lack of hierarchical graph representations and need for labeled data in GNNs.
method Maximizes mutual information between local and global graph representations.
result Comparable performance to supervised methods on graph classification benchmarks.
A major goal of unsupervised learning is to discover data representations that are useful for subsequent tasks, without access to supervised labels during training. Typically, this involves minimizing a surrogate objective, such as the negative log likelihood of a generative model, with the hope that representations us…
Proposes a novel framework for unsupervised domain adaptation using causal representations.
problem Transferability of deep model representations across domains is limited.
method Integrates causal inference into deep learning pipeline for domain-invariant feature learning.
result Demonstrates superior performance in unsupervised domain adaptation using causal representations.
Paper proposes nearly unsupervised hashcode learning for relation extraction.
problem Relation extraction from text, especially in biomedical domains.
method Optimized hashcode representations learned from data points without class labels, followed by supervised classification.
result Significant accuracy improvements over state-of-the-art methods.
Unsupervised learning representations generalize better than supervised learning under distribution shifts.
problem Robustness of unsupervised representations to distribution shift.
method Extensive evaluation on synthetic and realistic datasets, including controllable domain generalization datasets.
result Unsupervised representations learned from SSL and AE generalize better than supervised learning under various distribution shifts.
LORL learns object-centric representations from vision and language.
problem Learning disentangled, object-centric scene representations from vision and language.
method LORL integrates unsupervised object discovery and segmentation with language input to learn object-centric concepts.
result LORL improves unsupervised object discovery methods and aids downstream tasks.
This paper critically examines unsupervised disentangled representation learning, revealing challenges and limitations.
problem The difficulty of unsupervised learning of disentangled representations and the challenges in evaluation metrics.
method Theoretical analysis and a large-scale experimental study covering 8 datasets and 14000 models.
result Well-disentangled models cannot be identified without supervision, and different evaluation metrics disagree on what constitutes disentanglement.
A novel algorithm for unsupervised graph representation learning combining coarsening and mutual information maximization.
problem Current limitations in unsupervised graph representation learning, especially in embedding new graphs and considering both micro- and macro-structures.
method Combines coarsening with mutual information maximization to produce high-quality embeddings.
result The algorithm produces high-quality embeddings that are competitive with state-of-the-art methods.
In this work, we perform unsupervised learning of representations by maximizing mutual information between an input and the output of a deep neural network encoder. Importantly, we show that structure matters: incorporating knowledge about locality of the input to the objective can greatly influence a representation's …
XGBOD combines unsupervised and supervised methods for better outlier detection.
problem Enhanced outlier detection from normal observations in various datasets.
method Hybrid approach using unsupervised representation learning to improve a supervised classifier.
result XGBOD outperforms competing methods across seven datasets.
Unsupervised learning summarizes EHR data into a patient status vector.
problem Challenges in modeling electronic health records due to irregularities and varying procedures/diagnoses.
method Two-step unsupervised representation learning scheme using auto-encoders and forecasting tasks.
result Improved generalization performance on mortality and readmission tasks.
The goal of unsupervised representation learning is to extract a new representation of data, such that solving many different tasks becomes easier. Existing methods typically focus on vectorized data and offer little support for relational data, which additionally describe relationships among instances. In this work we…
Study on robustness of unsupervised representation learning in slightly misspecified settings.
problem Identify nonlinear representation learning in slightly misspecified settings.
method Formalize and investigate Independent Component Analysis (ICA) with observations generated by a mixing function close to a local isometry.
result Approximate identifiability of nonlinear ICA with almost isometric mixing functions.
This article reviews statistical methods for learning data representations.
problem Learning meaningful representations of data.
method Statistical perspective on unsupervised and supervised representation learning.
result Recent advances in representation learning from a statistical viewpoint.
A new unsupervised contrastive learning framework improves time series representation learning.
problem Lack of labeled data in time series data.
method Proposes an unsupervised contrastive learning framework using a novel contrastive loss and data augmentation.
result Framework outperforms other approaches on univariate and multivariate time series, and benefits transfer learning.
Proposes a new model for unsupervised clustering with latent variables.
problem The challenge of unsupervised clustering in machine learning.
method Clustered Generator Model with continuous and discrete latent variables.
result Achieves competitive unsupervised clustering accuracy and disentangled latent representations.
Learning disentangled representation from any unlabelled data is a non-trivial problem. In this paper we propose Information Maximising Autoencoder (InfoAE) where the encoder learns powerful disentangled representation through maximizing the mutual information between the representation and given information in an unsu…
New PAC-Bayesian bounds improve CURL's representation learning.
problem Lack of theoretical understanding of CURL's performance.
method Extended Arora et al.'s PAC-Bayes framework to non-iid setting.
result Derived PAC-Bayesian generalisation bounds for CURL.
Disentangled representations have recently been shown to improve fairness, data efficiency and generalisation in simple supervised and reinforcement learning tasks. To extend the benefits of disentangled representations to more complex domains and practical applications, it is important to enable hyperparameter tuning …
DiSeNE generates interpretable node embeddings without supervision.
problem Lack of interpretability in unsupervised node embeddings.
method Disentangled representation learning with novel objective functions and metrics.
result DiSeNE produces interpretable node embeddings aligned with graph structure.
In this work, we investigate unsupervised representation learning on medical time series, which bears the promise of leveraging copious amounts of existing unlabeled data in order to eventually assist clinical decision making. By evaluating on the prediction of clinically relevant outcomes, we show that in a practical …
A novel method integrates feature and topology views for unsupervised graph representation learning.
problem Lack of mutual information across feature and topology views in graph representation learning.
method Proposes a multi-view representation learning module and a common representation learning module using mutual information maximization and reconstruction loss minimization.
result Demonstrates effectiveness in integrating feature and topology views, achieving comparable or better performance than supervised methods.
Unsupervised feature learning has shown impressive results for a wide range of input modalities, in particular for object classification tasks in computer vision. Using a large amount of unlabeled data, unsupervised feature learning methods are utilized to construct high-level representations that are discriminative en…
The paper tackles adversarial robustness by maximizing worst-case mutual information.
problem Training robust machine learning models against adversarial inputs is challenging.
method Develops a notion of representation vulnerability and an unsupervised learning method to maximize worst-case mutual information.
result Proves a lower bound on minimum adversarial risk and supports robustness of representations.
UNTIE learns representations of coupled categorical data.
problem Challenges in learning from unlabeled categorical data with complex couplings.
method UNTIE approach for unsupervised representation learning of heterogeneous couplings.
result UNTIE significantly improves categorical data representations on 25 diverse datasets.
New unsupervised learning technique learns independent kernels for better machine learning tasks.
problem Improving unsupervised representation learning for machine learning tasks.
method Stacking convolutional transforms using alternating proximal minimization scheme.
result DCTL outperforms shallow version CTL on benchmark datasets.
We release a large ECG dataset for arrhythmia subtype discovery.
problem Discovering unknown subtypes of arrhythmia from continuous raw signals.
method Unsupervised representation learning task using semi-supervised evaluation.
result Qualitative evaluations show potential for representation learning in arrhythmia sub-type discovery.
IGT learns graph representations without supervision.
problem Building deep unsupervised graph representations.
method Generic complex-valued spectral graph architecture from Fourier transform generalization, greedy concave objective for discriminative and invariant features.
result IGT learns both discriminative and invariant features from graph topology.
This work proposes unsupervised learning by predicting random distances in neural networks.
problem Lack of labelled data in unsupervised learning tasks.
method Train neural networks to predict random distances in a randomly projected space, optimizing for genuine class structures.
result Learned representations outperform state-of-the-art methods in anomaly detection and clustering.
LOBSTUR-GNN adapts bootstrapping for unsupervised GNNs, improving node representation learning.
problem Hyperparameter tuning and lack of established methodologies for unsupervised GNNs.
method Adapts bootstrapping techniques for local graph dependencies and uses CCA for embedding consistency.
result 65.9% improvement in classification accuracy compared to uninformed hyperparameter selection.
In this paper we propose a strategy for semi-supervised image classification that leverages unsupervised representation learning and co-training. The strategy, that is called CURL from Co-trained Unsupervised Representation Learning, iteratively builds two classifiers on two different views of the data. The two views c…
This paper reviews nonlinear ICA for disentangled representations in unsupervised learning.
problem Finding useful disentangled representations in unsupervised deep learning.
method Review of nonlinear ICA theory and algorithms for disentanglement.
result Nonlinear ICA can be shown to estimate useful disentangled representations.
In this work we explore the generalization characteristics of unsupervised representation learning by leveraging disentangled VAE's to learn a useful latent space on a set of relational reasoning problems derived from Raven Progressive Matrices. We show that the latent representations, learned by unsupervised training …
Extends information bottleneck to multi-view unsupervised learning.
problem Identifying superfluous information in unlabeled multi-view data.
method Multi-view information bottleneck model, leveraging data augmentation.
result State-of-the-art results on Sketchy and MIR-Flickr datasets.
A good representation for arbitrarily complicated data should have the capability of semantic generation, clustering and reconstruction. Previous research has already achieved impressive performance on either one. This paper aims at learning a disentangled representation effective for all of them in an unsupervised way…
New unsupervised method selects hard negative samples for contrastive learning.
problem How to select good negative examples for contrastive learning without using true similarity information.
method Developed a new family of unsupervised sampling methods for hard negative selection.
result Improves downstream performance across multiple modalities.
Plan2Vec learns image representations without labels, improving control tasks.
problem Learning image representations without labeled data.
method Constructs a weighted graph using near-neighbor distances and extrapolates to global embedding.
result Plan2Vec achieves accurate long-term value estimates in control tasks with reduced computational and memory costs.
We investigate the effects of the unsupervised pre-training method under the perspective of information theory. If the input distribution displays multiple views of the supervision, then unsupervised pre-training allows to learn hierarchical representation which communicates these views across layers, while disentangli…
This research shows unsupervised GANs can perform object segmentation without labels.
problem Performing object segmentation without pixel or image-level labels.
method Used large-scale unsupervised GAN models to differentiate foreground from background.
result Demonstrated high-quality saliency masks and new state-of-the-art performance.
Discriminative clustering learns from both labeled and unlabeled data.
problem Clustering complex datasets with limited labeled data.
method Gradient-based stochastic training and optimal transport with entropic regularization.
result The method can learn feature representations even in fully unsupervised settings.
Proposes an unsupervised graph neural network for entire graph representation.
problem Lack of unsupervised methods for entire graph representation.
method Combines hierarchical graph neural networks and mutual information maximization.
result Improves state-of-the-art performance on multiple graph level tasks.
This paper studies learning the representations of whole graphs in both unsupervised and semi-supervised scenarios. Graph-level representations are critical in a variety of real-world applications such as predicting the properties of molecules and community analysis in social networks. Traditional graph kernel based me…
HDGI learns node representations for heterogeneous graphs.
problem Challenges in learning node representations for heterogeneous graphs.
method HDGI uses meta-path structure, graph convolution, and semantic-level attention to maximize local-global mutual information.
result HDGI outperforms state-of-the-art methods on graph-related tasks.
Proposes a novel graph representation learning framework using contrastive methods.
problem Graph representation learning for graph-structured data.
method Leverages a contrastive objective at the node level, generating two graph views by corruption and learning node representations by maximizing agreement.
result Consistently outperforms existing state-of-the-art methods on transductive and inductive learning tasks.