Transformers can solve complex filtering problems for non-Gaussian signals.
problem Non-linear and non-Markovian filtering problems for conditionally Gaussian signals.
method Continuous-time transformer models called filterformers.
result Filterformers can approximate the conditional law of non-Markovian and conditionally Gaussian signal processes.
Improved action detection for multi-person videos using attention filtering.
problem Difficulty in distinguishing relevant parts of multi-person videos for action detection.
method Fovea attention filtering and generalized binary loss function.
result 20% relative improvement in mAP over baseline in AVA dataset.
KFAtt improves CTR prediction by modeling user behavior with Kalman filtering attention.
problem Improving CTR prediction in personalized e-commerce search engines.
method KFAtt combines Kalman filtering with attention mechanisms to model user behavior.
result KFAtt outperforms existing methods in CTR prediction, achieving better performance in both offline and online settings.
An attacker may use a variety of techniques to fool an automatic speaker verification system into accepting them as a genuine user. Anti-spoofing methods meanwhile aim to make the system robust against such attacks. The ASVspoof 2017 Challenge focused specifically on replay attacks, with the intention of measuring the …
A new spectrum attention mechanism improves time series classification.
problem Improving robustness and classification accuracy in time series classification.
method Proposes a spectrum attention mechanism (SAM) to filter and highlight important frequency components, using L1 regularization and a tumbling window for segmentation.
result Experimental results show that the proposed SSAM method produces better feature representations and improves classification accuracy.
NAS model improves social recommendation accuracy using neural attention.
problem Capturing and weighing friends' preferences in social recommendation systems.
method Proposes a Neural Attention mechanism (NAS) for Social collaborative filtering.
result NAS model outperforms state-of-the-art methods in publicly available datasets.
This paper predicts stock prices using LLMs and news embeddings.
problem Predicting stock prices with high accuracy and relevance.
method Integrates LLMs with stock name embeddings and attention mechanisms for news filtering.
result Reduces MAE by 7.11% compared to baseline.
Transformers can approximate Kalman Filtering in linear systems with small error.
problem Approximating Kalman Filtering using Transformers for linear dynamical systems.
method Two-step reduction: 1) Softmax self-attention block approximates Nadaraya-Watson kernel smoothing, 2) This estimator approximates Kalman Filter.
result Constructs a Transformer that implements the Kalman Filter with small additive error, uniformly bounded in time.
Paper proposes a forecasting model combining autoregressive models with spectral attention.
problem Time series forecasting across various domains.
method Combines deep autoregressive models with Spectral Attention (SA) module.
result SAAM consistently demonstrates improved forecasting accuracy compared to state-of-the-art approaches.
3D Axial-Attention improves lung nodule classification accuracy.
problem Limited 3D attention in existing methods.
method Proposes 3D Axial-Attention network with 3D positional encoding.
result 3D Axial-Attention achieves state-of-the-art performance.
This paper improves collaborative filtering by integrating user and item embeddings with attention.
problem Sparse ratings and limited robustness of Bayesian methods in collaborative filtering.
method Proposes a VAE-based Bayesian MF framework that leverages both data and embedding information.
result The proposed method enhances robustness and accuracy of collaborative filtering models.
Visual attention serves as a means of feature selection mechanism in the perceptual system. Motivated by Broadbent's leaky filter model of selective attention, we evaluate how such mechanism could be implemented and affect the learning process of deep reinforcement learning. We visualize and analyze the feature maps of…
Hybrid approach combines transformer and Bayesian filtering for robust multiple particle tracking.
problem Challenges in tracking multiple particles in noisy scenes due to combinatorial explosion of hypotheses.
method Attention-Bayesian hybrid framework using transformer for association and Bayesian filtering for pruning hypotheses.
result Improved tracking accuracy and robustness against spurious detections.
A-MMSE uses attention to learn efficient OFDM channel estimation.
problem Accurate OFDM channel estimation requires second-order statistics, which are hard to obtain in practice.
method A-MMSE is a model-based DNN framework that learns linear MMSE filters via Attention Transformer, reducing inference complexity.
result A-MMSE outperforms other methods in normalized MSE across various SNR conditions.
A new framework for efficient sequence maps using Bayesian filtering and covariance.
problem Designing efficient recurrent sequence maps from explicit memory assumptions.
method Design-model framework, exact Bayesian filtering, query-dependent readout, linear-Gaussian instantiation.
result Improved robustness and retrieval performance across various benchmarks.
This study improves state estimation for nonlinear systems using conditional normalizing flows.
problem Performance degradation of traditional filtering algorithms in nonlinear systems with non-Gaussian uncertainty.
method Uses conditional normalizing flows with MLP, transformer, or state-space models for state and parameter estimation.
result Optimal-transport-inspired kinetic loss mitigates overparameterization in flows.
A new hybrid GNN framework tackles oversmoothing in graph data.
problem Oversmoothing in graph convolutional networks limits their expressive power and generalization.
method Combines traditional GCN filters with band-pass filters defined via geometric scattering and introduces an attention framework.
result Improves expressive power and generalization of graph convolutional networks.
Convolutional neural networks (CNNs) have recently emerged as a popular building block for natural language processing (NLP). Despite their success, most existing CNN models employed in NLP share the same learned (and static) set of filters for all input sentences. In this paper, we consider an approach of using a smal…
Examines multiagent systems for complex learning tasks.
problem Achieving cohesive learning behavior in multiagent networks.
method General formulation for multiagent dynamics and conditions for learning.
result Conditions for achieving cohesive learning behavior in multiagent networks.
Paper proposes multiscale self-attentive convolutions for vision and language.
problem Improving language and vision understanding models using self-attention.
method Developed 1D and 2D Self Attentive Convolutions (SAC), multiscale SAC (MSAC).
result MSAC enhances model performance for vision and language tasks.
Develops inverse extended Kalman filter for predicting adversarial steps.
problem Predicting adversarial Kalman filter estimates from limited information.
method Proposes inverse extended Kalman filter (I-EKF) for non-linear systems with unknown inputs.
result Derives I-EKF with theoretical stability guarantees and consistency proofs.
Improved multi-modal emotion recognition using deep learning.
problem Combining acoustic and text modalities for emotion recognition.
method Proposes a deep learning-based approach to fuse text and acoustic data using SincNet for acoustic features and parallel DCNN and Bi-RNN branches for text processing with cross attention.
result Achieves 3.5% improvement in weighted accuracy compared to existing methods.
Unsupervised learning filters tweets for emergency services during crises.
problem Challenges in filtering relevant information from social web data during disasters.
method Multi-task domain adversarial attention network for unsupervised domain adaptation.
result The multi-task model outperforms single task models in filtering relevant tweets.
CATA++ improves scientific article recommendations by learning latent spaces with attention.
problem Natural data sparsity limits MF performance in recommender systems.
method CATA++ uses a dual autoencoder with attention to learn latent spaces from content and contextual information.
result CATA++ significantly improves MF performance on real-world datasets.
Confidence-based filtering reveals latent structure in diffusion models.
problem Unclear latent structure in diffusion models.
method Confidence scores from a classifier.
result Class-relevant latent structure emerges under confidence-based filtering.
SpGAT learns graph representations using spectral attention for efficiency.
problem Efficiently capturing global graph patterns with minimal parameters.
method Introduces Spectral Graph Attention Network (SpGAT) using spectral domain attention mechanisms and a fast Chebychev approximation.
result SpGAT achieves better global pattern recognition with fewer parameters compared to GAT.
Unlike images or videos data which can be easily labeled by human being, sensor data annotation is a time-consuming process. However, traditional methods of human activity recognition require a large amount of such strictly labeled data for training classifiers. In this paper, we present an attention-based convolutiona…
A new method routes EEG covariance matrices across domains using adaptive subspace selection.
problem Challenges in cross-domain EEG decoding due to distinct SPD manifold regions.
method Dynamic Stiefel routing with expert filters and cross-attention for adaptive subspace projection.
result Consistent gains across three datasets: balanced accuracy improves from 0.773 to 0.823, 0.757 to 0.809, and 0.801 to 0.839.
Introduces Spectral Attention for better long-range time series forecasting.
problem Challenges in capturing long-range dependencies in time series forecasting.
method Spectral Attention mechanism that preserves temporal correlations and long-range dependencies.
result Achieves state-of-the-art results on 11 real-world time series datasets.
As a robust nonlinear similarity measure in kernel space, correntropy has received increasing attention in domains of machine learning and signal processing. In particular, the maximum correntropy criterion (MCC) has recently been successfully applied in robust regression and filtering. The default kernel function in c…
This paper uses deep RL to optimize market quotes from LOB data.
problem Optimizing quotes for market making from complex LOB data.
method Attn-LOB neural network with convolutional filters and attention mechanism for feature extraction; hybrid reward function for continuous action space.
result The RL agent outperforms traditional methods in market making tasks.
Recent studies in the field of human vision science suggest that the human responses to the stimuli on a visual display are non-deterministic. People may attend to different locations on the same visual input at the same time. Based on this knowledge, we propose a new stochastic model of visual attention by introducing…
Enhances financial time series forecasting with a multi-period learning framework.
problem Accurate financial time series forecasting requires considering both short-term and long-term trends.
method Proposes a Multi-period Learning Framework (MLF) with three modules: Inter-period Redundancy Filtering, Learnable Weighted-average Integration, and Multi-period self-Adaptive Patching.
result Improves financial time series forecasting accuracy and efficiency.
We extend variational autoencoders (VAEs) to collaborative filtering for implicit feedback. This non-linear probabilistic model enables us to go beyond the limited modeling capacity of linear factor models which still largely dominate collaborative filtering research.We introduce a generative model with multinomial lik…
Flexible priors improve VAE-based CF models for better user preference modeling.
problem Simplistic priors in VAEs limit user preference modeling and deeper representation learning.
method Incorporated flexible priors and gating mechanisms into VAEs for collaborative filtering.
result Flexible priors and gating mechanisms significantly improve recommendation performance.
Post-shifted BN prevents filter collapse in BN networks, improving model performance.
problem Filter collapse in BN networks reduces network capacity and harms model performance.
method Post-shifted BN (psBN) to prevent filter collapse by making BN parameters trainable again.
result psBN prevents filter collapse and increases model performance in various tasks.
Forecasting multivariate time series data, such as prediction of electricity consumption, solar power production, and polyphonic piano pieces, has numerous valuable applications. However, complex and non-linear interdependencies between time steps and series complicate the task. To obtain accurate prediction, it is cru…
ASARS integrates temporal dynamics from CF into session-based RNN for better personalized recommendations.
problem Discarding long-term data across sessions in session-based RNNs.
method ASARS framework that combines attentional network and inter-session temporal dynamic model.
result ASARS improves personalized recommendation performance on four real datasets.
HAMN combines CF models to improve drug repositioning.
problem Efficient drug repositioning with cold start problem.
method Hybrid Attentional Memory Network (HAMN) integrating memory and attention mechanisms.
result HAMN outperforms other models in drug repositioning tasks.
RGCF improves collaborative filtering by refining graph convolution embeddings.
problem GCN-based recommendation models introduce noise and redundancy, limiting high-order connectivity capture.
method Developed RGCF, a new GCN-based Collaborative Filtering model with redesigned embeddings.
result RGCF significantly outperforms state-of-the-art models on public datasets.
The application of machine learning techniques to large-scale personalized recommendation problems is a challenging task. Such systems must make sense of enormous amounts of implicit feedback in order to understand user preferences across numerous product categories. This paper presents a deep learning based solution t…
A new method infers neural trajectories in real-time, improving experimental design.
problem Real-time inference of neural trajectories for immediate feedback.
method Exponential family variational Kalman filter (eVKF) for online learning.
result eVKF achieves competitive performance on synthetic and real-world data.
SPADE improves demand forecasting accuracy by 4.5% for post-promotion periods.
problem Overreacting to peak events in demand forecasting leads to biased forecasts.
method SPADE splits forecasting into two tasks: one for peak events and another for post-peak events, using masked convolution filters and a specialized Peak Attention module.
result Overall PPE improvement of 4.5%, 30% improvement for most affected forecasts after promotions and holidays, and 3.9% improvement in PE accuracy.
Proposes a deep hybrid model for better recommendation systems.
problem Limited studies on hybrid recommender systems and the need for more advanced approaches.
method Integrates deep learning with ID embeddings and auxiliary features for improved recommendation.
result Improves recommendation results over deep learning models using ID embeddings.
Spectro-Riemannian Graph Neural Networks integrate spectral and curvature signals for better graph representation learning.
problem Enhance graph representation learning by leveraging spectral and curvature signals.
method Proposes Spectro-Riemannian Graph Neural Networks (CUSP) that combines spectral and curvature insights.
result Empirical evaluation shows CUSP outperforms state-of-the-art models by up to 5.3%.
User preferences for items can be inferred from either explicit feedback, such as item ratings, or implicit feedback, such as rental histories. Research in collaborative filtering has concentrated on explicit feedback, resulting in the development of accurate and scalable models. However, since explicit feedback is oft…
This paper debiases KG embeddings by filtering sensitive attributes.
problem Social and cultural biases in KG representations.
method Exploring and filtering popularity and gender biases in KGE.
result Sensitivity of KG embeddings to sensitive attributes like gender.
Scale-equivariant CNNs handle scale changes for improved performance.
problem Translation equivariance is not sufficient for handling scale changes in CNNs.
method Developed scale-equivariant convolutional networks with steerable filters.
result Demonstrated state-of-the-art results on MNIST-scale and STL-10 datasets.