Paper proposes a new unsupervised clustering method using attention models.
problem Unsupervised community detection on graphs.
method Optimizes soft modularity loss on Bethe Hessian embeddings.
result Model performs competitively with classical and GNN methods.
Proposes RN for unsupervised attention in neural networks.
problem Limited, imbalanced, and non-stationary input distributions in various tasks.
method Inspired by neuronal adaptation, RN uses MDL principle and universal code length for incremental layer-wise computation.
result Outperforms existing normalization methods across diverse tasks.
A new unsupervised method removes CT metal artifacts using beta-CycleGAN and attention.
problem Metal artifact reduction in computed tomography (CT) images.
method Unsupervised learning using a beta-CycleGAN architecture with attention mechanism.
result Improved metal artifact removal that preserves image details.
R-SQAIR adds relational bias to sequential object attention models for better object interactions.
problem Traditional sequential multi-object attention models struggle with relational inferences.
method Proposes R-SQAIR, a relational extension of SQAIR with a parallel pairwise interaction module.
result Demonstrates gains in object relations and combinatorial generalization over sequential mechanisms.
Attention learns PCA on Gaussian data, proving its connection to principal component analysis.
problem Principal component analysis on Gaussian data.
method Analysis of attention mechanisms through PCA, covering finite and infinite prompt regimes.
result Attention aligns with principal eigenvectors of covariance matrices, converging to optimal solutions in the infinite-prompt limit.
The paper shows how continuous attention can lead to better object representations.
problem Learning good representations from data without labeled examples.
method Integrating unsupervised learning and reinforcement learning with intrinsic motivation.
result The proposed algorithm improves object recognition in few-shot settings.
Deep Divergence Graph Kernels learn graph representations without supervision.
problem Learning graph representations without feature engineering or labeled graphs.
method Unsupervised method using cross-graph attention networks and divergence scores.
result Learned representations achieve competitive results on graph classification tasks.
SPACE models complex scenes by decomposing objects and backgrounds.
problem Scalability and unsupervised object-oriented scene representation learning.
method Generative latent variable model combining spatial-attention and scene-mixture approaches.
result SPACE achieves factorized object representations and decomposes complex scenes.
Interprets how intrinsic motivation shapes behavior in RL agents.
problem Understanding how intrinsic motivation influences behavior in reinforcement learning agents.
method Analyzed five RL agents in procedurally generated environments using various interpretability techniques.
result Curiosity-driven agents exhibit broader and more dynamic attention than extrinsically motivated agents.
Graph attention auto-encoder reconstructs graph structure and attributes.
problem Lack of methods to reconstruct graph structure and node attributes in graph auto-encoders.
method Stacked encoder/decoder layers with self-attention mechanisms, regularized node representations to reconstruct graph structure.
result Competitive performance on node classification benchmarks, including inductive learning.
New method improves text classification without labeled target data.
problem Improving text classification under domain shift without labeled target data.
method Diversity-based generalization using multi-head attention with diversity constraints.
result Method matches state-of-the-art performance without labeled target data.
Transformers can cluster data from Gaussian mixtures without supervision.
problem Clustering data from Gaussian mixtures without labeled data.
method Theoretical analysis of attention-based layers, focusing on a simplified two-head attention layer and an identity matrix attention layer.
result Attention-based layers can align with true mixture centroids and adapt to input-specific distributions.
SuTaT creates dialogue summaries for tete-a-tetes without labeled data.
problem Lack of high-quality paired dialogue-summary data.
method Unsupervised model for tete-a-tetes, modeling customer and agent roles separately.
result SuTaT outperforms on automatic and human evaluations.
Two computational models analyze political topics in social media tweets.
problem Measuring political attention in social media is labor-intensive and restrictive.
method Two computational models: supervised classifier and unsupervised topic model.
result Models provide different benefits: supervised classifier reduces labor, unsupervised model uncovers political and non-political uses.
UGRAPHEMB embeds graphs into vectors preserving their proximity, achieving competitive results.
problem Graph-level representation learning in an unsupervised and inductive manner.
method UGRAPHEMB uses graph-graph proximity to embed graphs into a vector space. MSNA generates multi-scale node attention for graph-level embedding.
result UGRAPHEMB achieves competitive accuracy in graph classification, similarity ranking, and visualization tasks.
SANNE model generates embeddings for unseen nodes in graph networks.
problem Lack of embeddings for unseen nodes in graph networks.
method SANNE uses a transformer self-attention network to generate embeddings.
result SANNE achieves state-of-the-art results for node classification.
RTFN extracts robust temporal features for time series analysis.
problem Challenges in extracting sufficient shapelets from time series data.
method Combines temporal feature networks and attentional LSTM networks.
result RTFN outperforms in supervised and unsupervised time series analysis.
Aligns attention distributions for improved accuracy and robustness.
problem Improving the accuracy and robustness of neural networks using attention mechanisms.
method Alignment attention that encourages key and query distributions to match within each head.
result Alignment attention leads to better accuracy, uncertainty estimation, and robustness across various tasks.
Model improves information transfer from visual streams.
problem Challenges in unsupervised learning from continuous visual data.
method Inspired by physics, maximizes mutual information through temporal process.
result Focus of attention enhances information transfer from input stream.
Novel fusion of autoencoders predicts sleepiness from speech.
problem Predicting sleepiness from speech recordings.
method Attention-based and recurrent sequence to sequence autoencoders for unsupervised representation learning.
result Fusion of autoencoders' representations achieves higher correlation with sleepiness scales.
In this work, we investigate unsupervised representation learning on medical time series, which bears the promise of leveraging copious amounts of existing unlabeled data in order to eventually assist clinical decision making. By evaluating on the prediction of clinically relevant outcomes, we show that in a practical …
End-to-end training solves deep unsupervised contrastive learning problems.
problem Theoretical analysis of unsupervised contrastive learning.
method End-to-end training of deep neural networks.
result Approximate stationary solutions found for non-convex contrastive loss.
SCALOR learns scalable object representations for crowded scenes.
problem Scalability in scenes with many objects.
method Spatially-parallel attention and proposal-rejection mechanisms.
result SCALOR can handle up to a hundred objects in crowded scenes.
We introduce the Attentive Unsupervised Text (W)riter (AUTR), which is a word level generative model for natural language. It uses a recurrent neural network with a dynamic attention and canvas memory mechanism to iteratively construct sentences. By viewing the state of the memory at intermediate stages and where the m…
Slot Attention extracts object-centric representations from images.
problem Learning distributed representations that don't capture natural scene composition.
method Slot Attention module interfaces with CNN outputs to produce task-dependent abstract slots.
result Slot Attention enables generalization to unseen compositions.
Graph neural networks benefit from attention under specific conditions.
problem Understanding and improving the effectiveness of attention in graph neural networks.
method Designing controlled graph reasoning tasks, analyzing performance under various conditions, proposing weakly-supervised training.
result Attention can provide significant gains in performance under certain conditions, but its effect is often negligible or harmful.
Unsupervised learning filters tweets for emergency services during crises.
problem Challenges in filtering relevant information from social web data during disasters.
method Multi-task domain adversarial attention network for unsupervised domain adaptation.
result The multi-task model outperforms single task models in filtering relevant tweets.
Unsupervised RNNGs perform similarly to supervised ones in language modeling and grammar induction.
problem Training RNNGs requires annotated data, which is costly.
method Amortized variational inference with a neural CRF parser.
result Unsupervised RNNGs achieve comparable performance to supervised ones.
TabNet learns tabular data with attention, improving performance and interpretability.
problem Efficient learning and interpretability in tabular data.
method Sequential attention to feature selection, self-supervised learning.
result Outperforms other models on tabular datasets, interpretable feature attributions.
Proposes ACLAE-DT for unsupervised anomaly detection in multivariate time series.
problem Challenges in building anomaly detection frameworks for multivariate time series data.
method Attention-based ConvLSTM Autoencoder with Dynamic Thresholding.
result Demonstrates superior performance over state-of-the-art methods.
Meta-continual learning improves few-shot learning performance.
problem Limited applicability of deep learning solutions to real-world data.
method Unsupervised meta-continual learning with meta-optimization and self-attention.
result Meta-continual learning achieves competitive performance even without supervision.
The ability to decompose scenes in terms of abstract building blocks is crucial for general intelligence. Where those basic building blocks share meaningful properties, interactions and other regularities across scenes, such decompositions can simplify reasoning and facilitate imagination of novel scenarios. In particu…
Diffusion models enhance speech without supervision.
problem Challenges in generalizing supervised speech enhancement methods to unseen conditions.
method Unsupervised speech enhancement using diffusion-based generative models.
result Demonstrates promising results compared to supervised and unsupervised baselines.
A new method for feature selection in high-dimensional data.
problem Dealing with noise and high-dimensional data in unsupervised feature selection.
method Sparse PCA via l2,p-norm regularization, combined with an efficient optimization algorithm. result The proposed method effectively selects features from real-world data sets.
A new deep learning framework selects representative samples for unsupervised learning.
problem Selecting representative samples for unsupervised learning in non-linear data.
method DUAL framework using an encoder-decoder architecture to learn nonlinear embeddings and a selection block to choose representative samples.
result DUAL outperforms state-of-the-art methods in selecting representative samples for unsupervised learning.
In the last decade, supervised deep learning approaches have been extensively employed in visual odometry (VO) applications, which is not feasible in environments where labelled data is not abundant. On the other hand, unsupervised deep learning approaches for localization and mapping in unknown environments from unlab…
LORL learns object-centric representations from vision and language.
problem Learning disentangled, object-centric scene representations from vision and language.
method LORL integrates unsupervised object discovery and segmentation with language input to learn object-centric concepts.
result LORL improves unsupervised object discovery methods and aids downstream tasks.
Proposes a novel node embedding framework for graphs using Fisher Information.
problem Lack of theoretical understanding of attention-based GNNs.
method Uses hierarchical kernels and Fisher Information to learn node embeddings.
result Proposed method outperforms existing GNNs on node classification benchmarks.
POUF fine-tunes large models without labeled data.
problem Lack of labeled data for fine-tuning large pre-trained models.
method Prompt-oriented unsupervised fine-tuning.
result Consistent improvements across various tasks.
GraphDINO learns neuronal morphologies from unlabeled data.
problem Unsupervised learning of neuronal morphologies from unlabeled data.
method Transformer-based approach with novel attention mechanism and data augmentation.
result GraphDINO yields morphological clusterings on par with expert classification.
A new method detects concept drift without true labels.
problem Detecting concept drift in unsupervised settings.
method Student-teacher learning paradigm for drift detection.
result The method outperforms state-of-the-art approaches in experiments.
This paper introduces GEMINI, a new mutual information metric for unsupervised neural network training.
problem The mutual information (MI) as a clustering objective does not lead to satisfactory clusters.
method The authors generalised MI by changing its core distance, introducing GEMINIs that do not require regularizations and can automatically select the number of clusters.
result GEMINIs can automatically select the number of clusters without requiring a priori knowledge of the number of clusters.
TAnoGan detects anomalies in time series data using GANs.
problem Anomaly detection in time series data.
method Generative Adversarial Networks (GAN) for unsupervised anomaly detection.
result TAnoGan outperforms traditional and neural network models in anomaly detection.
MPVAA learns holistic patient representations from mixed healthcare data.
problem Learning personalized patient representations from heterogeneous healthcare data.
method Mixed Pooling Multi-View Attention Autoencoder (MPVAA) that integrates non-linear relationships among multiple data modalities.
result MPVAA generates more effective patient representations than state-of-the-art methods.
Finite mixture models have been used for unsupervised learning for some time, and their use within the semi-supervised paradigm is becoming more commonplace. Clickstream data is one of the various emerging data types that demands particular attention because there is a notable paucity of statistical learning approaches…
Discrete-AIR model identifies objects in images with interpretable latent codes.
problem Identifying objects in images without labeled data.
method Recurrent Auto-Encoder with structured latent distributions for discrete, continuous, and spatial attention.
result Discrete-AIR model uses minimal latent variables for efficient inference.
Paper proposes a new dataset for group anomaly detection in physics.
problem Group anomaly detection in physics.
method Development of a synthetic benchmark dataset and comparison of anomaly detection techniques.
result Demonstrated performance of anomaly detection techniques on the new dataset.
This work addresses encoding biases in neural networks by tailoring models with unsupervised losses.
problem Improving neural network representations and reducing the generalization gap.
method Inspired by transductive learning, the authors propose tailoring and meta-tailoring to optimize unsupervised losses during prediction time.
result Models trained with tailoring and meta-tailoring perform better on the task objective after adapting to unsupervised losses.