Graph convolutional networks (GCNs) have shown the powerful ability in text structure representation and effectively facilitate the task of text classification. However, challenges still exist in adapting GCN on learning discriminative features from texts due to the main issue of graph variants incurred by the textual …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Dual Attention”
Visual Question Answering (VQA) requires AI models to comprehend data in two domains, vision and text. Current state-of-the-art models use learned attention mechanisms to extract relevant information from the input domains to answer a certain question. Thus, robust attention mechanisms are essential for powerful VQA mo…
Improves dialogue response model interpretability using attention and regularization.
problem Improving interpretability of dual encoder models for dialogue response suggestions.
method Integrates attention mechanism and novel regularization loss to emphasize important words.
result Improves model accuracy and interpretability compared to existing methods.
A framework for transformer attention layers derived from SVR.
problem Developing principled attention mechanisms for transformers.
method Mapping self-attention to SVR, deriving new attention types.
result Improved transformer performance and efficiency.
DanSmp predicts stock movement using a hybrid-relational MKG and dual attention networks.
problem Predicting stock price trends in volatile financial markets.
method Constructs a bi-typed MKG with hybrid-relations and uses DanSmp, a dual attention network, to learn momentum spillover signals.
result DanSmp improves stock prediction accuracy using the MKG.
The Nonlinear autoregressive exogenous (NARX) model, which predicts the current value of a time series based upon its previous values as well as the current and past values of multiple driving (exogenous) series, has been studied for decades. Despite the fact that various NARX models have been developed, few of them ca…
Graph convolutional networks (GCNs) have recently become one of the most powerful tools for graph analytics tasks in numerous applications, ranging from social networks and natural language processing to bioinformatics and chemoinformatics, thanks to their ability to capture the complex relationships between concepts. …
CATA++ improves scientific article recommendations by learning latent spaces with attention.
problem Natural data sparsity limits MF performance in recommender systems.
method CATA++ uses a dual autoencoder with attention to learn latent spaces from content and contextual information.
result CATA++ significantly improves MF performance on real-world datasets.
Survey on biharmonic Riemannian submersions, a dual concept of biharmonic submanifolds.
problem Understanding biharmonic Riemannian submersions as a dual concept.
method Short survey focusing on biharmonic Riemannian submersions.
result Survey provides insights into the dual concept of biharmonic submanifolds.
Hydra boosts efficiency for long-context reasoning in resource-constrained settings.
problem Quadratic complexity of transformers limits long-context reasoning in resource-constrained systems.
method Hydra uses a modular architecture with adaptive routing between sparse global attention, mixture-of-experts, and dual memories.
result Hydra achieves significant throughput and accuracy improvements for long-context reasoning.
Long-term prediction of multivariate time series is still an important but challenging problem. The key to solve this problem is to capture the spatial correlations at the same time, the spatio-temporal relationships at different times and the long-term dependence of the temporal relationships between different series.…
In recent years, there has been a surge of interest in developing deep learning methods for non-Euclidean structured data such as graphs. In this paper, we propose Dual-Primal Graph CNN, a graph convolutional architecture that alternates convolution-like operations on the graph and its dual. Our approach allows to lear…
Singular dual pairsmath.SG
We generalize the notions of dual pair and polarity introduced by S. Lie and A. Weinstein in order to accommodate very relevant situations where the application of these ideas is desirable. The new notion of polarity is designed to deal with the loss of smoothness caused by the presence of singularities that are encoun…
Hybrid model predicts flow and pressure in water systems.
problem Predicting flow and pressure in water distribution systems with complex spatial-temporal correlations.
method Hybrid dual-stage spatial-temporal attention-based recurrent neural networks (hDS-RNN).
result Our model outperformed 9 baseline models in flow and pressure series prediction.
Improved speech enhancement with MNTFA using time-frequency attention.
problem Speech enhancement with limited model size and memory.
method Designing MNTFA with self-attention modules for long sequences and joint training.
result MNTFA achieves better performance with fewer parameters than DPCRN.
DualVDT improves time-series forecasting with a novel dual reparametrized structure.
problem Time-series forecasting with improved performance and analytical rigor.
method Dual reparametrized variational mechanisms on VAE, latent score based generative model, reverse time stochastic differential equation, variational ancestral sampling, KL divergence reduction.
result Advanced performance in time-series forecasting with reduced KL divergence.
CADE learns dual node representations for better generalization.
problem Transductive graph embeddings cannot generalize to unseen nodes or across different graphs.
method CADE combines real-time neighborhoods with neighbor-attentioned representation, preserving known node memory.
result CADE outperforms state-of-the-art methods in generalization and context-awareness.
Attentive Neural Controlled Differential Equations for Time-series Classification and Forecastingcs.LG
ANCDEs improve time-series forecasting and classification using attention in NCDEs.
problem Improving time-series forecasting and classification using neural controlled differential equations.
method Integrating attention into neural controlled differential equations (ANCDEs).
result ANCDEs consistently show the best accuracy in time-series classification and forecasting.
A new method for multi-criteria recommender systems using graph attention networks.
problem Lack of nuanced relationships between users and items based on specific criteria.
method MDGAT, a multi-edge bipartite graph with dual attention networks and contrastive learning.
result MDGAT achieves higher accuracy in predicting item ratings compared to baseline methods.
Transformer models align words through attention weights, closely approximating Optimal Transport.
problem Understanding the internal mechanism of transformer models in language processing.
method Empirical evidence and theoretical analysis of attention weights and their relation to Optimal Transport.
result Transformer models can simulate gradient descent on the dual of entropy-regularized OT problem, providing a theoretical foundation for token alignment.
TLOB predicts stock prices better than existing models by adapting a simple MLP to LOB data.
problem Predicting stock prices from LOB data is challenging and complex.
method TLOB uses a transformer model with dual attention to capture spatial and temporal dependencies.
result TLOB outperforms state-of-the-art models across multiple datasets and horizons.
We consider the diffeological pseudo-bundles of exterior algebras, and the Clifford action of the corresponding Clifford algebras, associated to a given finite-dimensional and locally trivial diffeological vector pseudo-bundle, as well as the behavior of the former three constructions (exterior algebra, Clifford action…
Unified Kantorovich duality for multimarginal optimal transport on Polish spaces.
problem Optimal transport of multiple probability distributions.
method Unified Kantorovich duality theory for multimarginal optimal transport on general Polish product spaces.
result Unified duality theory for multimarginal optimal transport, extending classical two-marginal conjugacy.
Linearized attention fails to converge to NTK limit even at large widths.
problem Understanding the convergence of attention mechanisms to the kernel regime.
method Analyzes linearized attention and its relationship to the NTK limit, considering practical widths and conditions.
result Linearized attention does not converge to its NTK limit at any practical width, revealing a fundamental trade-off.
Given a compact Riemann surface and a complex reductive Lie group equipped with real structures, we define antiholomorphic involutions on the moduli space of -Higgs bundles over . We investigate how the various components of the fixed point locus match up, as one passes from to its Langlands dual $^LG…
CInA method uses attention to improve causal inference.
problem Challenges in causal inference, especially in complex tasks.
method CInA method utilizes self-supervised causal learning with multiple unlabeled datasets and transformer-type architecture.
result CInA effectively generalizes to out-of-distribution datasets and various real-world datasets.
Dual-CLVSA: a Novel Deep Learning Approach to Predict Financial Markets with Sentiment Measurementsq-fin.ST
Dual-CLVSA predicts financial markets using both trading data and sentiment measurements.
problem Predicting financial markets with complex interactions and emotional influences.
method Hybrid convolutional LSTM-based variational sequence-to-sequence model with attention.
result Dual-CLVSA effectively fuses trading data and sentiment measurements, improving prediction performance.
DAM: A Universal Dual Attention Mechanism for Multimodal Timeseries Cryptocurrency Trend Forecastingecon.GN
DAM improves cryptocurrency trend forecasting using multimodal data.
problem Simplistic merging of sentiment data in cryptocurrency trend forecasting.
method Dual Attention Mechanism (DAM) integrating financial metrics and sentiment analysis.
result DAM outperforms conventional models by up to 20% in prediction accuracy.
Proposes a fair meta-learning framework for few-shot classification.
problem Fairness in few-shot learning.
method Primal-Dual subgradient approach for fair initialization.
result Significant improvements over prior work in fairness.
PDNAS optimizes GNN architectures for diverse datasets.
problem Inadequate adaptability and combinatorial search space in GNNs.
method Dual architecture search (micro- and macro-architectures) with gradient-based optimization.
result PDNAS finds deeper GNNs with better performance on diverse datasets.
KATA improves associative recall by optimizing feature maps derived from nonnegative attention weights.
problem Linear attention's poor performance on associative recall tasks.
method Formulates attention recall as a spherical-packing problem and introduces Kernelized Linear Attention Activations (KATA).
result KATA features offer a favorable capacity-interference tradeoff, enabling efficient associative recall.
When Daan Krammer and Stephen Bigelow independently proved that braid groups are linear, they used the Lawrence-Krammer-Bigelow representation for generic values of its variables q and t. The t variable is closely connected to the traditional Garside structure of the braid group and plays a major role in Krammer's alge…
Dual-edge spatial Jacobian image graph for interpretable diabetic retinopathy grading
problem Automated diabetic retinopathy grading from color fundus photographs
method Dual-edge spatial-Jacobian image graph
result 0.8076 accuracy, 0.8312 quadratic weighted kappa, 0.5915 macro-F1, 0.9330 adjacent-grade accuracy
K-SVD algorithm has been successfully applied to image denoising tasks dozens of years but the big bottleneck in speed and accuracy still needs attention to break. For the sparse coding stage in K-SVD, which involves constraint, prevailing methods usually seek approximate solutions greedily but are less effe…
Although our main interest here is developing an appropriate analog, for diffeological vector pseudo-bundles, of a Riemannian metric, a significant portion is dedicated to continued study of the gluing operation for pseudo-bundles introduced in arXiv:1509.03023. We give more details regarding the behavior of this opera…
The financial crisis showed the importance of measuring, allocating and regulating systemic risk. Recently, the systemic risk measures that can be decomposed into an aggregation function and a scalar measure of risk, received a lot of attention. In this framework, capital allocations are added after aggregation and can…
End-to-End Models for the Analysis of System 1 and System 2 Interactions based on Eye-Tracking Dataq-bio.NC
While theories postulating a dual cognitive system take hold, quantitative confirmations are still needed to understand and identify interactions between the two systems or conflict events. Eye movements are among the most direct markers of the individual attentive load and may serve as an important proxy of informatio…
ATS2S model predicts RUL of industrial equipment using attention mechanism.
problem Accurate estimation of RUL for industrial equipment to improve maintenance schedules and reduce costs.
method ATS2S model that optimizes reconstruction and RUL prediction losses, uses attention mechanism, and integrates encoder and decoder features.
result ATS2S model achieves superior performance over 13 state-of-the-art methods on four real datasets.
Optimal transport simplifies machine learning by comparing probability measures.
problem Comparing and manipulating probability distributions in machine learning.
method Uses optimal transport to compare and manipulate probability distributions, combining statistical and geometric perspectives.
result Optimal transport provides a unified framework for various machine learning tasks.
The clustering methods have recently absorbed even-increasing attention in learning and vision. Deep clustering combines embedding and clustering together to obtain optimal embedding subspace for clustering, which can be more effective compared with conventional clustering methods. In this paper, we propose a joint lea…
A new deep learning framework selects representative samples for unsupervised learning.
problem Selecting representative samples for unsupervised learning in non-linear data.
method DUAL framework using an encoder-decoder architecture to learn nonlinear embeddings and a selection block to choose representative samples.
result DUAL outperforms state-of-the-art methods in selecting representative samples for unsupervised learning.
New method for tensor completion using nonconvex dual total variation.
problem Tensor completion from partial measurements with exponential-family noise.
method Proposed dual-TV (DTV) regularizers for tensor completion under exponential-family noise.
result Theoretical upper bounds on recovery error for tensor completion.
Two algorithms solve nonconvex minimax problems with linear constraints, achieving complexity guarantees.
problem Nonconvex minimax problems with coupled linear constraints.
method Zeroth-order primal-dual alternating projected gradient (ZO-PDAPG) and zeroth-order regularized momentum primal-dual projected gradient (ZO-RMPDPG) algorithms.
result Iteration complexity guarantees for solving nonconvex-(strongly) concave minimax problems with coupled linear constraints.
ADAVI tackles variational inference for large HBM models in neuroimaging.
problem Large, pyramidally-organized HBM models in neuroimaging studies.
method Automatic dual amortized variational inference using neural networks and attention-based hierarchical encoding.
result Significantly reduced parameterization of the variational family, maintaining expressivity.
The ability to look multiple times through a series of pose-adjusted glimpses is fundamental to human vision. This critical faculty allows us to understand highly complex visual scenes. Short term memory plays an integral role in aggregating the information obtained from these glimpses and informing our interpretation …
Study on portfolio selection and risk arbitrage in financial markets.
problem Analyzing optimal portfolios and risk arbitrage in financial markets with coherent risk measures.
method Characterization of optimal portfolios, dual representation, and interplay between EMMs and absolutely continuous measures.
result The absence of -arbitrage is linked to the interplay between EMMs and absolutely continuous measures.
We define symmetric spaces in arbitrary dimension and over arbitrary non-discrete topological fields $\K$, and we construct manifolds and symmetric spaces associated to topological continuous quasi-inverse Jordan pairs and -triple systems. This class of spaces, called smooth generalized projective geometries, generaliz…
Open problem: fixed-budget best arm identification complexity.
problem Understanding the complexity of identifying the best arm in a fixed budget setting.
method Analyzing existing results and conjectures in the fixed-confidence setting.
result Open questions remain about the fixed-budget setting.