Contrastive embeddings improve neural architecture search performance.
problem Improving performance of neural architecture search algorithms.
method Contrastive learning to identify networks based on data Jacobians and produce embeddings.
result Traditional black-box optimization algorithms can reach state-of-the-art performance with contrastive embeddings.
Network embedding methodologies, which learn a distributed vector representation for each vertex in a network, have attracted considerable interest in recent years. Existing works have demonstrated that vertex representation learned through an embedding method provides superior performance in many real-world applicatio…
Develops a binarized GNN for more efficient graph embeddings.
problem Real-valued GNN parameters limit efficiency and scalability.
method Integrates binarization into GNN-based graph embedding approaches.
result Binarized graph neural network (BGN) achieves state-of-the-art performance with significant efficiency gains.
Models use embeddings and attention for better claim severity prediction.
problem Improving predictive modeling of claim severity with categorical data.
method Developed neural networks and attention-based architectures with embeddings.
result Attention-based models enhance predictive performance with contextual augmentation.
Embeds sparsity in deep neural networks, allowing exact zero parameters.
problem Learning sparse structures in deep networks.
method Embeds sparsity into neural network structure, allowing exact zero parameters during training.
result Can learn both structured and unstructured sparsity.
Combines BERT and graph CNN for improved text classification.
problem Text classification problems
method Combining BERT embedding and graph convolutional neural network
result Graph CNN model performs better than classical models combined with BERT
We embed KKT points in neural networks of different sizes.
problem Classifying data using homogeneous neural networks.
method Introducing KKT point embedding principle and proving it for different network types.
result KKT points of a smaller network can be mapped to those of a larger network via linear transformations.
Improved molecular property prediction using WL embedding in GNNs.
problem Limited performance of GNNs in predicting molecular properties.
method Explored Weisfeiler-Lehman (WL) embedding to replace GNN layers, enhancing representability and performance.
result WL embedding consistently improves GNN performance across multiple datasets.
Improved speech enhancement with larger neural networks using novel embeddings and biases.
problem Decreased robustness of speech enhancement models to real-world use cases.
method Frequency-positional embeddings, semi-supervised training, biased loss function.
result Better performance on real recordings with improved large neural network architecture.
Growing amounts of online user data motivate the need for automated processing techniques. In case of user ratings, one interesting option is to use neural networks for learning to predict ratings given an item and a user. While training for prediction, such an approach at the same time learns to map each user to a vec…
Embedded ensembles improve neural network performance efficiently.
problem Improving neural network performance with fewer resources.
method Analyzing the wide network limit of gradient descent dynamics using Neural-Tangent-Kernel.
result Embedded ensembles exhibit two regimes: independent and collective, affecting performance.
G-Net constructs binary neural networks with high accuracy using randomized binary embeddings.
problem Creating high-accuracy binary neural networks with theoretical guarantees.
method Proposes a novel floating-point G-Net family with randomized binary embeddings and theoretical accuracy guarantees.
result Empirically, G-Net achieves almost 30% higher accuracy on CIFAR-10 compared to prior HDC models.
Enhanced GNN with expanded attention window and partially random embeddings.
problem Limited expressivity of traditional GNNs in distinguishing non-isomorphic graphs.
method Graph attention network with expanding attention window and partially random initial embeddings. Head dropout for regularization.
result Improved ability to differentiate between non-isomorphic graphs.
Graph neural networks refine speaker embeddings for better session-level diarization.
problem Local speaker distinction in meeting sessions using deep embeddings.
method Graph Neural Networks (GNNs) refine speaker embeddings using session-level structural information.
result Spectral clustering on refined embeddings outperforms original embeddings significantly.
PoET-BiN reduces power consumption in neural networks on embedded devices.
problem Power inefficiency in neural network implementations on embedded platforms.
method Look-Up Table based implementation with a modified Decision Tree approach.
result Near state-of-the-art results with up to 6 orders of magnitude energy reduction.
Embedding principle explains loss landscape of deep neural networks.
problem Understanding the structure of loss landscapes in deep neural networks.
method Proposed an embedding principle that critical points of narrower DNNs can be embedded to critical points of wider DNNs.
result Wide DNNs are often attracted by highly-degenerate critical points embedded from narrower DNNs.
Paper presents a novel hyperbolic neural network for efficient data representation.
problem Efficient representation of hierarchical data in hyperbolic space.
method Develops a fully hyperbolic neural network using projections and equivariant embeddings.
result Proves the proposed embedding is isometric and equivariant under Lorentz transformations.
Optimal Transport Graph Neural Networks (OT-GNN) improves graph embeddings by using optimal transport.
problem Graph Neural Networks (GNN) often lose structural or semantic information when aggregating node embeddings.
method Combines optimal transport (OT) with parametric graph models to compute graph embeddings from Wasserstein distances between node embeddings and prototype point clouds.
result OT-GNN outperforms popular methods on molecular property prediction tasks and produces smoother graph representations.
Learning powerful data embeddings has become a center piece in machine learning, especially in natural language processing and computer vision domains. The crux of these embeddings is that they are pretrained on huge corpus of data in a unsupervised fashion, sometimes aided with transfer learning. However currently in …
New lower bounds on embedding dimensions for neural network architectures.
problem Ensuring neural networks can handle symmetries like permutations in high dimensions.
method Novel technique to prove lower bounds on embedding dimensions.
result Proves new lower bounds on embedding dimensions for Deep Sets and Janossy pooling.
Acoustic Neighbor Embeddings map speech and text to fixed dimensions for phonetic confusability.
problem Mapping speech and text to fixed dimensions for phonetic confusability.
method Adapting SNE to sequential inputs, training two encoder neural networks.
result More accurate results with low-dimensional embeddings in word recognition tasks.
We theoretically study the landscape of the training error for neural networks in overparameterized cases. We consider three basic methods for embedding a network into a wider one with more hidden units, and discuss whether a minimum point of the narrower network gives a minimum or saddle point of the wider one. Our re…
A new method for fast graph embedding using diffusion graphs.
problem Efficiently generating graph embeddings for large networks.
method Diffusion graphs for rapid vertex sequence generation.
result Improved accuracy and performance with higher edge density.
Paper proposes GrokTransfer to eliminate delayed generalization in neural networks.
problem Delayed generalization in neural networks, compromising predictability and efficiency.
method Trains a smaller, weaker model to reach a nontrivial test performance, then uses its learned input embedding to initialize the stronger model.
result GrokTransfer enables the target model to generalize directly without delay, across various tasks.
Smooth embedding space improves NAS performance.
problem Efficiently predicting good neural architectures.
method Two-sided variational graph autoencoder.
result Smooth embedding space facilitates extrapolation to unseen architectures.
Meta-learning for Koopman spectral analysis with short time-series data.
problem Lack of long time-series for training embedding functions in Koopman spectral analysis.
method Meta-learning approach using bidirectional LSTM and neural network to estimate embedding functions from short time-series.
result The proposed method achieves better performance in eigenvalue estimation and future prediction compared to existing methods.
Graph convolutional networks fail to use eigenvectors beyond the first, unlike spectral embedding.
problem Understanding when graph convolutional networks fail compared to spectral embedding.
method Presented a simple generative model to illustrate failure.
result Graph convolutional networks fail to use eigenvectors beyond the first in certain graphs.
In this work, we contribute a new multi-layer neural network architecture named ONCF to perform collaborative filtering. The idea is to use an outer product to explicitly model the pairwise correlations between the dimensions of the embedding space. In contrast to existing neural recommender models that combine user em…
Paper explores understanding of neural source code embeddings.
problem Lack of understanding of contents and characteristics of code2vec embeddings.
method Small case study using code2vec embeddings to create binary SVM classifiers and compare performance with handcrafted features.
result Code2vec embeddings perform similarly to handcrafted features and have more evenly distributed information gains.
HGNN learns augmented features for deep multi-task learning.
problem Feature learning for deep multi-task learning.
method Hierarchical Graph Neural Network (HGNN) with two levels of graph neural networks.
result Significant performance improvement in classification tasks.
The widespread availability of electronic health records (EHRs) promises to usher in the era of personalized medicine. However, the problem of extracting useful clinical representations from longitudinal EHR data remains challenging. In this paper, we explore deep neural network models with learned medical feature embe…
ALF reduces network parameters and operations by 70% and 61%, respectively, on embedded hardware.
problem Efficient deployment of deep learning models on resource-constrained hardware.
method Autoencoder-based low-rank filter-sharing technique.
result ALF achieves significant compression with minimal accuracy loss.
Randomized Geometric Algebra for Convex Neural Networks Optimizes Transfer Learning.
problem Training neural networks to global optimality via convex optimization.
method Randomized algorithms in Clifford's Geometric Algebra for hypercomplex vector spaces.
result Convex optimization and geometric algebra improve LLMs' robustness and reliability in transfer learning.
Two novel methods improve network embedding for completely-imbalanced labels.
problem Improving network embedding for networks with completely-imbalanced labels.
method Two novel semi-supervised network embedding methods: RSDNE and RECT.
result Experimental results show the superiority of the proposed methods.
CLEANN detects and mitigates neural network Trojans without labeled data.
problem Trojans in embedded neural networks that bypass detection during inference.
method Dictionary learning and sparse approximation for identifying Trojan triggers, lightweight algorithm/hardware co-design.
result Efficient real-time execution on resource-constrained platforms, competitive attack resiliency.
Proposes a block-based model for attributed network embedding.
problem Handles both assortative and disassortative networks.
method Assigns nodes to blocks based on similar linkage patterns, using neural networks to preserve attribute information.
result Consistently outperforms state-of-the-art methods on disassortative networks.
Neural Spacetimes learn DAGs by embedding nodes in a spacetime manifold.
problem Learning representations of weighted directed acyclic graphs (DAGs).
method Trainable deep learning-based geometries (Neural Spacetimes) that encode both edge weights and causality.
result Universal embedding theorem for DAGs with sub-cubic parameters and low distortion.
New embedding method in function spaces improves expressiveness.
problem Enhancing expressiveness in knowledge graph embeddings.
method Employing polynomial functions and neural networks with varying layer complexities.
result Improved expressiveness and more degrees of freedom in entity representation.
Paper presents a novel neural network method for topic detection in micro-blogs.
problem Challenging topic detection without known topic count.
method Unsupervised neural sentence embedding model with attention mechanism.
result Improved clustering algorithm (RADBSCAN) discovers topics based on dataset character.
This paper proposes grid cells encode position via a conformal isometric embedding of 2D physical space.
problem Hexagonal grid firing patterns in grid cells.
method Learning a distance-preserving position embedding in neural space using a recurrent neural network.
result The conformal isometric embedding of 2D physical space into neural space explains hexagonal grid firing patterns.
Ring-reservoir networks simplify graph embeddings efficiently.
problem Efficient graph embeddings using deep neural networks.
method Progressive simplification of Reservoir Computing models to ring topology.
result Ring-reservoir networks show consistent advantages in predictive performance.
Small neural networks embed arbitrary metric spaces into Gaussian mixtures.
problem Embedding arbitrary metric spaces into a fixed space with low distortion.
method Probabilistic transformers of small depth and width.
result Embeddings with low metric distortion for various metric spaces.
The paper monitors artificial neural networks using embeddings and multivariate control charts.
problem Ensuring the stability of artificial neural networks during deployment.
method Monitoring the latent feature representation (embedding) of ANN outputs using multivariate control charts based on data depth and ranks.
result The proposed method effectively detects nonstationary data streams in ANN outputs.
We present Graph Random Neural Features (GRNF), a novel embedding method from graph-structured data to real vectors based on a family of graph neural networks. The embedding naturally deals with graph isomorphism and preserves the metric structure of the graph domain, in probability. In addition to being an explicit em…
Proposes RNNE for dynamic network embedding.
problem Handling dynamic networks with changing node and edge counts.
method Recurrent Neural Network Embedding (RNNE) for topologically evolving graphs and temporal graphs.
result RNNE outperforms state-of-the-art algorithms in network reconstruction, classification, and link prediction.
Study shows gMPNNs struggle with OOD link prediction in larger test graphs.
problem Inductive out-of-distribution link prediction in larger test graphs.
method Theoretical analysis and development of a gMPNN with structural pairwise embeddings.
result Structural node embeddings from gMPNNs converge to random guessing as test graphs grow.
The recent success of raw audio waveform synthesis models like WaveNet motivates a new approach for music synthesis, in which the entire process --- creating audio samples from a score and instrument information --- is modeled using generative neural networks. This paper describes a neural music synthesis model with fl…
LASE learns graph embeddings by unrolling GD iterations into a neural network.
problem Learning graph embeddings from graph inputs.
method Truncating and interpreting GD iterations as layers in a GNN.
result LASE outperforms eigendecomposition routines and GNNs in link prediction and node classification.