New similarity index avoids limitations of CCA in neural networks.
problem Limitations of existing methods in measuring neural network representation similarity.
method Introducing a similarity index based on centered kernel alignment (CKA) to measure representational similarity matrices.
result CKA reliably identifies correspondences between representations in networks trained from different initializations.
Unified understanding of neural representation similarity measures.
problem Fragmented research landscape of neural network similarity measures.
method Observation and exploration of connections between shape distances and normalized Bures similarity.
result Cosine of the Riemannian shape distance equals normalized Bures similarity.
Tile2Vec learns spatially meaningful representations without labels.
problem Lack of unsupervised methods for geospatial data.
method Unsupervised representation learning using the distributional hypothesis.
result Tile2Vec improves performance in spatial classification tasks.
Deconfounds neural network representation similarity metrics to improve consistency and accuracy.
problem Confounding by population structure in similarity metrics like RSA and CKA.
method Covariate adjustment regression to adjust for confounders.
result Improves detection of semantically similar neural networks and consistency in transfer learning.
Geometric stability measures neural network robustness, distinguishing from similarity metrics.
problem Lack of robustness in neural network representations.
method Introduces geometric stability, quantified by Shesha metric measuring self-consistency.
result Stability and similarity are uncorrelated, revealing distinct properties of neural network robustness.
The paper analyzes MAML's representation using RSA, revealing that feature reuse is not the primary reason for its success.
problem Understanding why model-agnostic meta-learning (MAML) works well in few-shot learning tasks.
method Representation similarity analysis (RSA) applied to MAML's few-shot learning instantiation.
result Feature reuse is not the primary reason for MAML's success; instead, it is the learning task itself that increases representation similarity.
Enhanced geographical features improve predictive models for colorectal cancer survival curves.
problem Predicting colorectal cancer survival curves in Iowa.
method Used neural networks to explore feature representations, comparing ABC performance.
result Spectral analysis-based representations improve predictive performance by approximately 40%.
Novel tRSA combines geometry and topology for brain and model analysis.
problem Traditional RSA overlooks topological information in neural representations.
method Topological RSA (tRSA) using nonlinear monotonic transforms.
result Robust model comparisons and novel insights into neural computation.
Bayesian correlated component analysis identifies brain process similarities across multiple stimulus views.
problem Investigating brain process similarity in responses to multiple views of a stimulus.
method Hierarchical probabilistic model that evaluates universality of spatial networks across multi-view data.
result Bayesian correlated component analysis evaluates favorably against other algorithms and identifies variability in spatial representations.
New measures link neural representation geometry to decoding ability.
problem Understanding how neural representations relate to decoding ability.
method Showed that popular similarity measures can be interpreted from a decoding perspective.
result Proved that measures like CKA and CCA quantify alignment between optimal linear readouts.
New method learns behavioral representations from mobility data.
problem Analyzing behavioral similarity of moving individuals from CDR trajectories.
method mob2vec framework combining segmentation, generalization, and unsupervised learning.
result Mob2vec generates low-dimensional vector representations preserving mobility behavior similarities.
Unified toolkit for comparing neural representations using SRTD and NTS.
problem Heuristic asymmetry and unbounded scores in existing divergences.
method Developed SRTD and NTS to address these issues.
result Unified, robust, and scale-invariant metric for comparing neural representations.
Modified RV-coefficient reveals how training affects neural network representations.
problem Understanding how training affects intermediate representations in convolutional neural networks.
method Experimented with modified RV-coefficient (RV2) to compare activation patterns in deep networks trained on varying amounts of data and layers.
result RV2 successfully recovered expected similarity patterns and provided interpretable similarity matrices.
The paper analyzes unsupervised learning using contrastive methods and introduces a theoretical framework.
problem Learning useful feature representations from unlabeled data.
method Introduces latent classes and contrasts similar vs. non-similar data points.
result Proves guarantees on the performance of learned representations on downstream tasks.
A new method identifies similar mutual funds using graph learning.
problem Identifying similar mutual funds with nuanced portfolio similarities.
method Node2Vec machine learning method applied to a weighted bipartite network of funds and assets.
result Identifies structural similarity among mutual funds' portfolios.
MSA compares neural representations' intrinsic geometry for better understanding.
problem Existing similarity measures fail to capture subtle distinctions between neural network solutions.
method Metric similarity analysis (MSA) using Riemannian geometry.
result MSA can disentangle features of neural computations and compare nonlinear dynamics.
The paper studies how neural networks evolve representations, finding a unique fixed point for nonlinear activations.
problem Understanding how neural networks transform input data across layers.
method Theoretical framework for the evolution of the kernel sequence, using mean-field regime and Hermite polynomials.
result For nonlinear activations, the kernel sequence converges globally to a unique fixed point.
New framework to test neural network representation similarity measures.
problem Disagreements among dissimilarity measures in neural networks.
method Statistical testing framework to evaluate measures based on functional behavior.
result Current metrics have different weaknesses; a classical baseline performs surprisingly well.
New findings on representation changes in transfer learning.
problem Understanding representation changes in transfer learning.
method Analysis of pre-trained and fine-tuned representations in transfer learning.
result Representation changes mainly occur in the head, but early layers are also important for cross-domain adaptation.
This paper investigates how forgetting affects neural network representations and stabilizes deeper layers.
problem Catastrophic forgetting in machine learning models trained on sequential tasks.
method Representational analysis techniques and empirical studies on CIFAR-10 and CIFAR-100 datasets.
result Deeper layers are disproportionately the source of forgetting, and methods to mitigate forgetting stabilize these layers.
dna2vec creates consistent vectors from DNA sequences, addressing sequence analysis challenges.
problem Inequivalent distances between one-hot vectors of k-mers and limitations of machine learning on long DNA sequences.
method Proposes a neural network-based approach to train distributed representations of variable-length k-mers.
result Summing dna2vec vectors is equivalent to nucleotide concatenation and correlates with sequence similarity.
Defines metrics to compare neural network representations.
problem Comparing neural network representations across different architectures and tasks.
method Developed a family of metric spaces and modified existing measures to quantify representational dissimilarity.
result Identified relationships between neural representations and anatomical features.
PRESTO maps latent representations across diverse ML models.
problem Understanding variability in latent representations across different ML models.
method Uses persistent homology to characterize latent spaces and measure their pairwise similarity.
result Preserves desirable properties and enables sensitivity analysis of latent representations.
New method compares neural network representations, revealing generalization and structure.
problem Comparing neural network representations is hard due to varying structures and evolution.
method Projection Weighted CCA (CCA) to differentiate signal and noise.
result Networks that generalize converge to similar representations, while memorizers diverge.
Representations of C∗-algebras are realized on section spaces of holomorphic homogeneous vector bundles. The corresponding section spaces are investigated by means of a new notion of reproducing kernel, suitable for dealing with involutive diffeomorphisms defined on the base spaces of the bundles. Applications of th…
SAFE detects fake news by analyzing text and image similarities.
problem Detecting fake news with less focus on text-image similarity.
method SAFE uses neural networks to extract text and visual features, then learns their relationship to predict fake news.
result SAFE effectively recognizes fake news based on text, images, or mismatches.
Proposes a method to compare neural networks using feature and gradient vectors.
problem Understanding the behavior of neural networks trained on different datasets.
method Defines a similarity index using feature and gradient vectors, and employs sketching techniques for efficient comparison.
result Demonstrates superior performance in computing similarity of neural networks trained on different datasets.
Paper analyzes how contrastive learning structures learned representations.
problem Understanding the structure of learned representations in contrastive learning.
method Kernel-based contrastive learning framework (KCL) and statistical dependency viewpoint.
result Theoretical upper bound and generalization error bound for KCL.
New findings clarify the link between distributional closeness and representational similarity.
problem When and why do different neural network representations become similar?
method Identifiability theory, focusing on model families including autoregressive language models.
result Small Kullback-Leibler divergence does not guarantee similar representations.
Graph neural networks improve with edge similarity constraints in RNA structure analysis.
problem Lack of edge similarity constraints in graph neural networks.
method Introduced a graph neural network layer that leverages prior information about edge similarities.
result Edge similarity constraints do not enhance performance in graph neural networks.
Simplifies fair PCA with fast, efficient solution.
problem Learning fair low-rank approximations of data.
method Conceptually simple approach with analytic solution.
result Faster and similar results to existing fair PCA methods.
Enhances autoencoders with kernel alignment for better data representation.
problem Lack of clear properties for autoencoders to capture in data representations.
method Aligns inner products between codes with a kernel matrix to capture topological properties.
result Effective data representations learned, preserving input data similarities.
New method learns low-dimensional representations of AI-generated treatments.
problem Representing AI-generated treatments without losing semantic meaning.
method Double kernel representation learning with alternating minimization.
result Efficiently learned representations guide generative models and facilitate adaptive online experiments.
A new method ranks autoencoder hidden nodes based on their saliency.
problem Autoencoders lack saliency measures similar to PCA.
method Supervised node saliency (SNS) method using normalized entropy difference (NED).
result SNS identifies salient hidden nodes in autoencoders.
Develops an ordinal-similarity framework for scalable and interpretable representation alignment.
problem Measuring representation similarity in large datasets.
method Triplet and Quadruplet Similarity Indices.
result Demonstrates inherent interpretability, robustness to outliers, and computational efficiency.
Proposes a new similarity learning framework for brain networks.
problem Learning similarity metrics for neuroimages, especially fMRI and DTI.
method End-to-end similarity learning framework using Higher-order Siamese GCN.
result Achieves significant AUC gains over existing methods.
COREL learns latent representations that naturally cluster, outperforming CCE.
problem Training neural networks to learn useful latent representations.
method Attractive-Repulsive Loss Framework for Clustering-Oriented Representation Learning (COREL).
result COREL variants outperform CCE in various classification tasks.
Feature normalization prevents collapse in non-contrastive learning dynamics.
problem Non-contrastive learning can collapse into a single point due to lack of repulsive force.
method Extended previous theory based on L2 loss to cosine loss, considering feature normalization.
result Cosine loss induces stable equilibrium, preventing collapse even with insufficient repulsive force.
A new diffusion kernel framework for data analysis.
problem Data analysis with manifold learning.
method Measure-based diffusion kernel for manifold learning.
result Preserves pairwise diffusion distances and is invariant to scale.
AI models aligned with human vision perform well on few data tasks.
problem Few-shot learning performance with limited data.
method Information-theoretic analysis and empirical testing of 491 models.
result Highly aligned models show better robustness to attacks and domain shifts.
Unified deep network learns shared representation and cross-media similarity metric for multimedia data.
problem Improving cross-media retrieval by capturing complex correlations among multiple media types.
method Unified Network for Cross-media Similarity Metric (UNCSM) that combines shared representation learning and distance metric calculation.
result UNCSM outperforms state-of-the-art methods on 4 cross-media datasets.
Deep Belief Network predicts lncRNA-disease associations with high accuracy.
problem Accurately identifying lncRNA-disease associations to understand lncRNA functionality and disease mechanism.
method Proposes a DBN-based model using heterogeneous networks and DBN for feature learning.
result Obtained AUC of 0.96 and AUPR of 0.967 on standard dataset.
A novel unsupervised feature selection method using subspace clustering and self-expressive model.
problem Feature selection for large datasets with minimal labeling effort.
method Subspace clustering with adaptive representation learning and regularized regression.
result The method effectively captures sample similarities and discriminative information.
DORA analyzes deep neural networks' internal representations to detect spurious correlations.
problem Detecting spurious correlations in deep neural networks' internal representations.
method DORA uses Extreme-Activation (EA) distance measure to assess representation similarities.
result Identifies internal representations capable of detecting spurious correlations.
Proposes a new method for analyzing fMRI data using gradient-based RSA and Searchlight.
problem Time-consuming and stability issues in classical RSA techniques for large-scale fMRI data.
method Gradient-based Representational Similarity Analysis (GRSA) with Searchlight.
result SSL-GRSA achieves superior performance compared to other RSA algorithms on multi-subject datasets.
New metric for disentangling multivariate representations, accounting for more complex entanglements.
problem Current disentanglement metrics fail to detect entanglements involving more than two variables.
method Partial Information Decomposition framework to analyze information sharing and propose a new disentanglement metric.
result The proposed metric correctly identifies entanglements in high-dimensional spaces.
Extract common latent factors from graphs for better representation learning.
problem Graph-level representation learning challenges due to limited labeled data and poor negative sample selection.
method Graph-wise Common Latent Factor Extraction (GCFX) using deepGCFX model.
result Improved graph-level and node-level tasks performance compared to state-of-the-art methods.
A new method for measuring document similarity using hierarchical optimal transport.
problem Inability to measure semantic similarities and scalability issues in past document similarity measures.
method Model documents as distributions over topics, topics as distributions over words, solve optimal transport problem on topics.
result Hierarchical optimal transport provides better interpretability and scalability with comparable performance.