Method measures weight similarity in neural networks using normalization and statistical inference.
problem Quantifying weight similarity in non-convex neural networks.
method Chain normalization rule and hypothesis-training-testing statistical inference.
result Weights of identical neural networks converge to similar local solutions.
Deconfounds neural network representation similarity metrics to improve consistency and accuracy.
problem Confounding by population structure in similarity metrics like RSA and CKA.
method Covariate adjustment regression to adjust for confounders.
result Improves detection of semantically similar neural networks and consistency in transfer learning.
Neural networks auto-denoise similar inputs, enabling new statistical analysis.
problem Estimating similarity of inputs for neural networks.
method Define and quantify similarity from neural network perspective, using parameter variation impact on outputs.
result Estimate sample density and quantify denoising effect without true labels.
Improves confidence calibration in neural networks by smoothing labels based on class similarity.
problem Improving confidence calibration in deep neural networks for safety-critical applications.
method Proposes a novel label smoothing technique where label values are based on similarities with the reference class, using different similarity measurements.
result Consistently outperforms state-of-the-art calibration techniques on various datasets and network architectures.
The paper analyzes the generalization of deep neural networks for metric and similarity learning.
problem Lack of rigorous understanding of generalization performance in metric and similarity learning.
method Derive explicit form of true metric, construct structured deep ReLU neural network, establish excess risk bounds.
result Explicit excess risk bounds for metric and similarity learning are derived.
Task loss matching misrepresents similarity between neural network layers.
problem Measuring similarity between neural network layers using task loss matching.
method Task loss matching vs. direct matching; comparison with CCA and CKA.
result Direct matching provides a better similarity index than task loss matching.
fMRI is a unique non-invasive approach for understanding the functional organization of the human brain, and task-based fMRI promotes identification of functionally relevant brain regions associated with a given task. Here, we use fMRI (using the Poffenberger Paradigm) data collected in mono- and dizygotic twin pairs t…
Proposes a new neural network approach to credit assignment.
problem Credit assignment problem in deep neural networks.
method Contrastive similarity matching objective function.
result Deep networks learn to match similarity between layers.
Quantum networks learn task-dependent asymmetric similarity measures.
problem Challenges of conventional distance functions in capturing meaningful similarity.
method GQSim: Quantum networks for learning task-dependent (a)symmetric similarity.
result Quantum similarity measures extract salient features and achieve theoretically guaranteed performance.
Recent work has sought to understand the behavior of neural networks by comparing representations between layers and between different trained models. We examine methods for comparing neural network representations based on canonical correlation analysis (CCA). We show that CCA belongs to a family of statistics for mea…
Proposes neural similarity for CNNs to enhance flexibility and performance.
problem Limited flexibility of inner product-based convolution in CNNs.
method Introduces neural similarity as a learnable parametric similarity measure, and proposes NSL for adaptive learning from data.
result Dynamic neural similarity improves flexibility and performance in visual recognition and few-shot learning.
PSimGNN partitions graphs into subgraphs for efficient graph similarity computation.
problem Efficiently compute graph similarity scores for large graphs.
method Graph partitioning followed by subgraph-level and node-level comparisons using a graph neural network.
result PSimGNN outperforms state-of-the-art methods in graph similarity computation tasks.
Modified RV-coefficient reveals how training affects neural network representations.
problem Understanding how training affects intermediate representations in convolutional neural networks.
method Experimented with modified RV-coefficient (RV2) to compare activation patterns in deep networks trained on varying amounts of data and layers.
result RV2 successfully recovered expected similarity patterns and provided interpretable similarity matrices.
Proposes a method to compare neural networks using feature and gradient vectors.
problem Understanding the behavior of neural networks trained on different datasets.
method Defines a similarity index using feature and gradient vectors, and employs sketching techniques for efficient comparison.
result Demonstrates superior performance in computing similarity of neural networks trained on different datasets.
Graph neural networks improve with edge similarity constraints in RNA structure analysis.
problem Lack of edge similarity constraints in graph neural networks.
method Introduced a graph neural network layer that leverages prior information about edge similarities.
result Edge similarity constraints do not enhance performance in graph neural networks.
Investigation of the market graph attracts a growing attention in market network analysis. One of the important problem connected with market graph is to identify it from observations. Traditional way for the market graph identification is to use a simple procedure based on statistical estimations of Pearson correlatio…
As a highlighting research topic in the multimedia area, cross-media retrieval aims to capture the complex correlations among multiple media types. Learning better shared representation and distance metric for multimedia data is important to boost the cross-media retrieval. Motivated by the strong ability of deep neura…
One popular generative model that has high-quality results is the Generative Adversarial Networks(GAN). This type of architecture consists of two separate networks that play against each other. The generator creates an output from the input noise that is given to it. The discriminator has the task of determining if the…
From a sequence of similarity networks, with edges representing certain similarity measures between nodes, we are interested in detecting a change-point which changes the statistical property of the networks. After the change, a subset of anomalous nodes which compares dissimilarly with the normal nodes. We study a sim…
A new method identifies similar mutual funds using graph learning.
problem Identifying similar mutual funds with nuanced portfolio similarities.
method Node2Vec machine learning method applied to a weighted bipartite network of funds and assets.
result Identifies structural similarity among mutual funds' portfolios.
We propose weighted inner product similarity (WIPS) for neural network-based graph embedding. In addition to the parameters of neural networks, we optimize the weights of the inner product by allowing positive and negative values. Despite its simplicity, WIPS can approximate arbitrary general similarities in…
funcGNN uses graph neural networks to estimate program similarity efficiently.
problem Estimating accurate program similarity for software engineering tasks.
method funcGNN trains on labeled CFG pairs to predict GED between unseen programs using effective embedding vectors.
result funcGNN achieves lower error rate (0.00194) and is 23 times faster than traditional methods.
New findings clarify the link between distributional closeness and representational similarity.
problem When and why do different neural network representations become similar?
method Identifiability theory, focusing on model families including autoregressive language models.
result Small Kullback-Leibler divergence does not guarantee similar representations.
Partial fusion combines neural networks to balance accuracy and efficiency.
problem Balancing accuracy and computational cost in neural networks.
method Extending weight aggregation methods based on neuron-level similarity, using partial optimal transport to match similar neurons.
result Achieves a flexible tradeoff between computational cost and performance.
We propose shifted inner-product similarity (SIPS), which is a novel yet very simple extension of the ordinary inner-product similarity (IPS) for neural-network based graph embedding (GE). In contrast to IPS, that is limited to approximating positive-definite (PD) similarities, SIPS goes beyond the limitation by introd…
Framework for inferring latent structure from sparse, imperfectly detected bipartite networks.
problem Recovering latent structure from sparse, imperfectly detected bipartite networks in ecology.
method Structured sparse nonnegative low-rank factorization with detection probability estimation and ADMM-based algorithm.
result Improved recovery of latent factors and structure compared to existing methods.
Paper tests similarity between networks using a bootstrap method.
problem Determining if two networks are similar or proportional.
method Parametric bootstrap approach and Frobenius norm-based test.
result The method is versatile and consistent under various models.
TLMG4Eth combines language and graph models for Ethereum fraud detection.
problem Current fraud detection methods fail to consider semantic and similarity patterns in Ethereum transactions.
method TLMG4Eth uses a transaction language model and graph-based methods to capture semantic, similarity, and structural features.
result TLMG4Eth detects anomalies in Ethereum transactions more effectively than existing methods.
Graph change-point detection method learns graph similarity from data.
problem Detect abrupt changes in dynamic networks.
method Siamese graph neural network for graph similarity learning.
result Method detects changes in diverse types of networks with minimal data history.
Network Embeddings (NEs) map the nodes of a given network into d-dimensional Euclidean space Rd. Ideally, this mapping is such that `similar' nodes are mapped onto nearby points, such that the NE can be used for purposes such as link prediction (if `similar' means being `more likely to be connected') or c…
Proposes a simple neural network model similar to gradient boosted decision trees.
problem Building a neural network equivalent to gradient boosted decision trees.
method Converts an ensemble of decision trees to a neural network, relaxes properties, and trains a simple neural network model.
result The proposed Hammock model achieves similar performance to gradient boosted decision trees.
Enhances graph neural networks by considering feature similarities in node aggregation.
problem Ignoring node feature similarities in traditional graph aggregation schemes.
method Interprets node aggregation as kernel weighting, proposing a framework that considers feature similarities.
result Proposed framework outperforms traditional GCNs in real-world applications.
New measure assesses neural network models' functional similarity.
problem Measuring functional similarity between similar-performing neural networks.
method Robust nonparametric hypothesis testing framework.
result Proposed measure assesses neural networks' functional similarity.
Graph similarity search is among the most important graph-based applications, e.g. finding the chemical compounds that are most similar to a query compound. Graph similarity computation, such as Graph Edit Distance (GED) and Maximum Common Subgraph (MCS), is the core operation of graph similarity search and many other …
Learning a similarity metric has gained much attention recently, where the goal is to learn a function that maps input patterns to a target space while preserving the semantic distance in the input space. While most related work focused on images, we focus instead on learning a similarity metric for neuroimages, such a…
We consider the representation power of siamese-style similarity functions used in neural network-based graph embedding. The inner product similarity (IPS) with feature vectors computed via neural networks is commonly used for representing the strength of association between two nodes. However, only a little work has b…
We introduce GSimCNN (Graph Similarity Computation via Convolutional Neural Networks) for predicting the similarity score between two graphs. As the core operation of graph similarity search, pairwise graph similarity computation is a challenging problem due to the NP-hard nature of computing many graph distance/simila…
Learning informative representations of data is one of the primary goals of deep learning, but there is still little understanding as to what representations a neural network actually learns. To better understand this, subspace match was recently proposed as a method for assessing the similarity of the representations …
A new kernel measures brain network similarities, improving disease classification.
problem Lack of edge weight information in existing graph kernels for brain connectivity networks.
method Ordinal pattern kernel for weighted brain connectivity networks.
result The ordinal pattern kernel achieves better classification performance than state-of-the-art graph kernels.
Proposes MGMN for end-to-end graph similarity learning.
problem Lack of cross-level interactions in graph similarity learning.
method Multi-level graph matching network (MGMN) combining node-graph matching and siamese graph neural networks.
result MGMN outperforms state-of-the-art models on graph-graph classification and regression tasks.
Study shows how task similarity affects forgetting in teacher-student setup.
problem Catastrophic forgetting in continual learning.
method Extended teacher-student setup to multiple teachers, analyzing similarity between tasks.
result Task similarity, whether at readouts or features, influences forgetting and transfer.
Convolution Neural Networks (CNN) have recently achieved state-of-the art performance on handwritten Chinese character recognition (HCCR). However, most of CNN models employ the SoftMax activation function and minimize cross entropy loss, which may cause loss of inter-class information. To cope with this problem, we pr…
New algorithm detects community labels in networks using unlabeled data.
problem Detect community labels in networks with partially labeled data.
method Proposes an algorithm using structural similarity metrics.
result Theoretical guarantees for misclassification error.
New framework to test neural network representation similarity measures.
problem Disagreements among dissimilarity measures in neural networks.
method Statistical testing framework to evaluate measures based on functional behavior.
result Current metrics have different weaknesses; a classical baseline performs surprisingly well.
Inverse depth scaling found in LLMs due to similar layers averaging error.
problem Understanding how depth affects loss in large language models.
method Analysis of LLMs and toy residual networks.
result Loss scales inversely proportional to depth in LLMs.
Deep learning improves forensic matching of casings.
problem Identifying if two casings came from the same firearm.
method Contrastive neural networks trained on casings dataset.
result Contrastive learning achieved better ROC AUC than CMC.
International trade has been increasingly organized in the form of global value chains (GVCs) where different stages of production are located in different countries. This recent phenomenon has substantial consequences for both trade policy design at the national or regional level and business decision making at the fi…
SimLoss improves classification by considering class similarities.
problem Equal punishment of all misclassifications in CCE.
method Integrates class similarities into Categorical Cross Entropy using matrices constructed from task-specific knowledge.
result Significant improvements over CCE on Age Estimation and Image Classification.