Proposes a model for multi-agent reinforcement learning with hierarchical graph attention network.
problem Limited transferability of trained policies to new multi-agent tasks.
method Uses hierarchical graph attention network for representation learning and multi-agent actor-critic for policy learning.
result Demonstrates superior performance in mixed cooperative and competitive tasks compared to existing methods.
Paper proposes a new hierarchical attention mechanism for multi-scale data.
problem Challenges in applying neural attention to multi-scale, multi-modal data.
method Developed a mathematical framework for multi-modal, multi-scale data and derived optimal neural attention mechanics.
result Proposed hierarchical attention mechanism improves transformer performance in multi-scale, multi-modal settings.
This study compares hierarchical and non-hierarchical models for open-domain multi-turn dialog generation.
problem Which kind of models (hierarchical or non-hierarchical) is better for open-domain multi-turn dialog generation?
method Systematically compared nearly all representative hierarchical and non-hierarchical models over the same experimental settings.
result Nearly all hierarchical models are worse than non-hierarchical models in open-domain multi-turn dialog generation, except for HRAN.
HAXMLNet tackles extreme multi-label text classification with hierarchical attention.
problem Tagging each text with relevant labels from an extreme-scale label set.
method Proposes a hierarchical structure with multi-label attention for efficient and effective XMTC.
result HAXMLNet achieves competitive performance compared to state-of-the-art methods.
Gated attention improves performance by using a hierarchical mixture of experts.
problem Improving performance of self-attention mechanisms in Transformers.
method Rigorously show that gated attention can be modeled as a hierarchical mixture of experts, providing a theoretical justification for its benefits.
result Gated attention is more sample-efficient than multi-head self-attention, requiring fewer data points to achieve the same estimation error.
HAGAN uses hierarchical attention to improve cross-domain sentiment classification.
problem Cross-domain sentiment classification with domain discrepancy.
method Hierarchical attention in GANs to produce domain-indistinguishable document representations.
result HAGAN outperforms existing methods on Amazon review dataset.
Improved text summarization using neural semantic encoders with hierarchical structure.
problem Capturing long-term dependencies in text summarization.
method Proposed a novel hierarchical Neural Semantic Encoder (NSE) model augmented with lemma and PoS tags.
result Significantly outperformed state-of-the-art models in ROUGE metric.
Paper proposes HGAT for detecting fake news on HIN.
problem Viral spread of fake news causing social harm.
method Hierarchical Graph Attention Network (HGAT) for node representation learning in HIN.
result HGAT outperforms text-based and network-based models.
HANET combines LSTM and attention mechanisms for better financial forecasting.
problem Lack of distinct macroeconomic regimes in financial datasets.
method Hierarchical Cross-Attention mechanism integrating long-run macro contexts with high-frequency market dynamics.
result HANET outperforms neural forecasters, especially during turbulent periods.
FATHOM model improves sensor data analysis with attention and LSTM.
problem Scarcity of training data from multiple sensors.
method Federated multi-task hierarchical attention model (FATHOM) with attention mechanism and LSTM.
result FATHOM outperforms baselines in sensor data classification and regression.
New model improves DDI extraction with 78.5% macro F1-score.
problem Extracting Drug-Drug Interactions from text.
method Attention-Wrapped Hierarchical BLSTMs.
result Model outperforms state-of-the-art with macro F1-score of 0.785.
Memory networks are neural networks with an explicit memory component that can be both read and written to by the network. The memory is often addressed in a soft way using a softmax function, making end-to-end training with backpropagation possible. However, this is not computationally scalable for applications which …
This work improves KG embeddings by integrating hyperbolic and attention mechanisms.
problem Preserving hierarchical and logical patterns in KGs with low-dimensional embeddings.
method Combines hyperbolic reflections/rotations with attention mechanisms to capture complex relational patterns.
result Improves MRR by up to 6.1% on standard benchmarks and new state-of-the-art results in high dimensions.
A transformer model improves spell correction with hierarchical attention.
problem Improving spell correction accuracy and speed.
method Multi encoder-single decoder transformer architecture with hierarchical attention.
result Significant improvement in CER, WER, and SER error rates.
Transformer models outperform recurrent ones in modeling hierarchical data.
problem Modeling hierarchical structure in data.
method Introducing Multiresolution Transformer Networks leveraging self-attention.
result Multiresolution Transformer Networks significantly outperform state-of-the-art models on query suggestion datasets.
HTAD predicts diagnoses from EHRs using target-aware attention.
problem Lack of specificity in aggregating clinical records for diagnosis prediction.
method Hierarchical target-attentive mechanism for EHRs.
result Target-aware attention significantly improves diagnosis prediction performance.
Proposes a novel framework for multi-label text classification.
problem Lack of coherent consideration of non-consecutive and long-distance semantics and hierarchical relations among labels.
method Hierarchical taxonomy-aware and attentional graph capsule recurrent CNNs framework.
result Significantly improves multi-label text classification performance.
HKT improves sequence processing with multi-scale attention and kernel analysis.
problem Processing sequences at multiple scales with efficient attention mechanisms.
method Trainable causal downsampling and convex weights for level-specific score matrices.
result HKT achieves consistent gains over standard attention across various tasks.
Proposes a novel model for healthcare and SME credit risk prediction.
problem Lack of guidance from global view in sequence representation learning for time series modeling.
method Hierarchical Global View-guided (HGV) sequence representation learning framework with GGE and β-Attn modules. result Competitive prediction performance compared with other known baselines.
Solution for voice conversion with limited data using hierarchical seq2seq and attention models.
problem Voice conversion between speakers with limited parallel audio pairs.
method Hierarchical sequence to sequence model with attention-based decoder, trained on single speaker dataset.
result Improved voice conversion quality using mel spectrograms and wavenet vocoder.
HCRNN uses hierarchical contexts to improve recommendation models.
problem Challenges in modeling user interest transitions and drifts in recommendation systems.
method Introduces HCRNN with three hierarchical contexts (global, local, temporary) and a hierarchical context-based gate structure.
result HCRNN outperformed other models in sequential recommendation tasks.
DyHATR learns dynamic heterogeneous networks for better link prediction.
problem Learning effective representations of dynamic heterogeneous networks for link prediction.
method Hierarchical attention for heterogeneous information and temporal RNN for evolutionary patterns.
result DyHATR significantly outperforms state-of-the-art baselines on link prediction tasks.
Recent progress on deep learning has made it possible to automatically transform the screenshot of Graphic User Interface (GUI) into code by using the encoder-decoder framework. While the commonly adopted image encoder (e.g., CNN network), might be capable of extracting image features to the desired level, interpreting…
SCHA-VAE generates novel data from limited examples using hierarchical context aggregation.
problem Generating data from a novel distribution with limited examples.
method Hierarchical context aggregation with attention-based point to set-level aggregation.
result Hierarchical approach better captures intrinsic variability in small data.
Time series prediction has been studied in a variety of domains. However, it is still challenging to predict future series given historical observations and past exogenous data. Existing methods either fail to consider the interactions among different components of exogenous variables which may affect the prediction ac…
Orion-Bix combines biaxial attention and meta-learning for tabular few-shot learning.
problem Scaling and generalizing tabular models with mixed numeric and categorical fields, weak feature structure, and limited labeled data.
method Orion-Bix uses biaxial attention and meta-learned in-context reasoning to efficiently capture local and global dependencies.
result Orion-Bix outperforms gradient-boosting baselines and state-of-the-art tabular models on public benchmarks.
HATS predicts stock and market index movements using hierarchical graph attention.
problem Accurately predicting stock and market index movements using relational data.
method Hierarchical Graph Attention Network (HATS) selectively aggregates information from different relation types.
result HATS outperforms existing methods in predicting stock and market index movements.
Insurance companies must manage millions of claims per year. While most of these claims are non-fraudulent, fraud detection is core for insurance companies. The ultimate goal is a predictive model to single out the fraudulent claims and pay out the non-fraudulent ones immediately. Modern machine learning methods are we…
This paper proposes a new approach to Transformers by integrating hierarchical associative memory with MetaFormers.
problem Theoretical framework for Transformers and MLP-Mixers remains underdeveloped.
method Integrating hierarchical associative memory with MetaFormers to create a parallelized MLP-Mixer.
result Symmetry-breaking effects improve the performance of the MLP-Mixer in image recognition tasks.
HiGraphDTI learns drug and target representations from molecular graphs to predict DTIs.
problem Inaccurate drug-target interaction prediction due to insufficient chemical information extraction.
method Hierarchical graph representation learning to extract chemical information from atoms, motifs, and molecules.
result HiGraphDTI outperforms state-of-the-art methods in DTI prediction and interaction interpretation.
Transformers mimic Bayesian reasoning in controlled settings, revealing geometric mechanisms.
problem Verifying if transformers perform Bayesian reasoning rigorously in natural data.
method Constructing Bayesian wind tunnels with known posteriors and proving memorization impossibility.
result Transformers achieve 10−3-10−4 bit accuracy in Bayesian posteriors, while MLPs fail. Model detects depression from transcribed interviews using affective language.
problem Detecting depression from transcribed clinical interviews.
method Hierarchical Attention Network with affective conditioning.
result Model achieves state-of-the-art F1 scores in depression detection.
STG2Seq predicts multi-step passenger demand with graph and hierarchical structure.
problem Predicting passenger demand over multiple time horizons is challenging due to nonlinear and dynamic spatial-temporal dependencies.
method Proposes a graph-based model with a hierarchical graph convolutional structure to capture spatial and temporal correlations.
result Consistently outperforms baseline and state-of-the-art models on real-world datasets.
Enhances neural networks in hyperbolic space for better data structure capture.
problem Capturing hierarchical data structures efficiently.
method Unified hyperbolic model for neural network components.
result Superior parameter efficiency and outperformance over Euclidean methods.
Anti-phishing aims to detect phishing content/documents in a pool of textual data. This is an important problem in cybersecurity that can help to guard users from fraudulent information. Natural language processing (NLP) offers a natural solution for this problem as it is capable of analyzing the textual content to per…
Learning and memory are intertwined in our brain and their relationship is at the core of several recent neural network models. In particular, the Attention-Gated MEmory Tagging model (AuGMEnT) is a reinforcement learning network with an emphasis on biological plausibility of memory dynamics and learning. We find that …
Language recognition system is typically trained directly to optimize classification error on the target language labels, without using the external, or meta-information in the estimation of the model parameters. However labels are not independent of each other, there is a dependency enforced by, for example, the langu…
In (Yang et al. 2016), a hierarchical attention network (HAN) is created for document classification. The attention layer can be used to visualize text influential in classifying the document, thereby explaining the model's prediction. We successfully applied HAN to a sequential analysis task in the form of real-time m…
We explore solutions for automated labeling of content in bug trackers and customer support systems. In order to do that, we classify content in terms of several criteria, such as priority or product area. In the first part of the paper, we provide an overview of existing methods used for text classification. These met…
Paper uses HGNN to predict stock types from relationships and temporal data.
problem Predicting stock types from complex market data.
method Integrates stock relationships and temporal data using HGNN.
result Effective prediction of stock types with HGNN model.
ASAP improves graph pooling for hierarchical graph representations.
problem Pooling in graphs fails to effectively capture substructure or scale to large graphs.
method ASAP uses self-attention and modified GNN to capture node importance and learn sparse soft cluster assignments.
result Combining ASAP with GNN architectures leads to state-of-the-art results on graph classification benchmarks.
This paper addresses identifiability issues in HLAMs with hierarchical constraints.
problem Identifiability of HLAMs with hierarchical constraints.
method Developed sufficient and necessary identifiability conditions.
result Characterizes impacts of different attribute types in the graph on identifiability.
Paper provides first theoretical guarantees for hyperbolic space learning.
problem Learning a classifier in hyperbolic space for hierarchical data.
method Efficient algorithm for large-margin hyperplane learning in hyperbolic space.
result The low embedding dimension in hyperbolic space leads to superior classifier learning guarantees.
MolHF generates complex molecules with hierarchical flow-based model.
problem Designing novel molecular structures with desired properties.
method MolHF is a hierarchical normalizing flow model that generates molecular graphs in a coarse-to-fine manner.
result MolHF achieves state-of-the-art performance in random generation and property optimization.
HGNN learns augmented features for deep multi-task learning.
problem Feature learning for deep multi-task learning.
method Hierarchical Graph Neural Network (HGNN) with two levels of graph neural networks.
result Significant performance improvement in classification tasks.
A very brief history of relative valuation in neoclassical finance since 1973 is presented, with attention to core currency issues for emerging economies. Price formation is considered in the context of hierarchical causality, with discussion focussed on identifying mathematical modelling challenges for robust and tran…
Proposes EM-HRNN model for better hierarchical language representation.
problem Learning hierarchical representations in natural language processing.
method Introduces latent indicator layer and EM algorithm for training.
result EM-HRNN model outperforms other RNN-based models in document classification.
A new convolutional spectral kernel network learns hierarchical and local features.
problem Lack of deep learning in non-stationary spectral kernels.
method Introduces convolutional filters and deep architectures into non-stationary spectral kernels, derives generalization error bounds, and introduces regularizers.
result Validated the effectiveness of the convolutional spectral kernel network on real-world datasets.