Paper tackles noisy annotations by considering workers' attention levels.
problem Noisy annotations from workers with varying expertise.
method Proposes a probabilistic model that incorporates workers' attention for accurate label quality estimation.
result Improves aggregated labels by quantifying the relationship between workers' attention and label quality.
Studying a softmax-attention model, we show that the learned query converges to the latent signal subspace spanned by the informative direction.
problem Understanding the theoretical principles of attention mechanisms in large-scale token collections.
method Deriving a population objective and analyzing the limiting ordinary differential equation of the learning dynamics.
result The learned query asymptotically recovers the latent signal up to the intrinsic sign ambiguity.
ATT-MADDPG models teammates' policies in RL for cooperative systems.
problem Modeling and exploiting changing policies of teammates in cooperative multi-agent systems.
method Extends DDPG with a centralized critic and attention mechanism to model dynamic joint policies.
result ATT-MADDPG outperforms state-of-the-art RL methods and achieves better scalability and robustness.
Neural model for dynamic entity relatedness ranking.
problem Dynamic entity relationships in real-world applications.
method A neural network-based approach leveraging collective attention as supervision.
result Our model achieves better results than competitive baselines on large-scale datasets.
Area attention allows dynamic area-based attention in memory.
problem Fixed attention granularity limits model performance.
method Area attention dynamically determines area shape and size via learning.
result Area attention improves performance on neural machine translation and image captioning.
Machine learning automates digitization of historical data.
problem Manual transcription is costly and difficult for large, detailed datasets.
method Apply machine learning techniques for unsupervised layout classification and attention-based neural networks.
result Machine learning can automate the digitization process for historical data.
The study finds that Chinese internet users have different search behaviors and attention patterns.
problem Heterogeneity in search behavior and attention among Chinese internet users.
method Data extraction technology to analyze Baidu Index keyword search volume data.
result Chinese internet users exhibit different search behaviors and attention patterns.
Study shows perceptual boost of visual attention varies with task difficulty and size.
problem Understanding how task-dependent the perceptual boost of visual attention is in natural settings.
method Designed and trained neural networks on various visual tasks, comparing results to baseline.
result Perceptual boost of attention is stronger with more difficult tasks and weaker with larger task sets.
Hybrid model predicts flow and pressure in water systems.
problem Predicting flow and pressure in water distribution systems with complex spatial-temporal correlations.
method Hybrid dual-stage spatial-temporal attention-based recurrent neural networks (hDS-RNN).
result Our model outperformed 9 baseline models in flow and pressure series prediction.
Study uses deep learning with attention to predict fetal brain gestational age.
problem Accurately predicting fetal brain gestational age for early diagnosis.
method Attention-based deep learning model combining multi-view MRI data.
result Age prediction performance with R2 = 0.94 using multi-view MRI and attention.
Modeling student behaviors and multiple predictions for early intervention.
problem Predicting student outcomes and interactions among multiple tasks.
method Proposes a variant of LSTM and soft-attention mechanism for heterogeneous behaviors, and co-attention mechanism for task interactions.
result Demonstrated effectiveness in predicting student outcomes and interactions.
FedACS uses attention to select clients with similar data for federated learning.
problem Non-IID data and data scarcity in federated learning.
method FedACS integrates an attention mechanism to prioritize clients with similar data distributions.
result FedACS improves federated learning performance by addressing non-IID data and data scarcity.
A deep learning model improves pedestrian tracking accuracy.
problem Pedestrian tracking accuracy is low, especially with inertial measurement unit.
method Deep learning model using IMU and LIDAR data, attention mechanism.
result Preliminary results show improved accuracy.
Paper presents a neural network for recognizing human activities from unlabeled sensor data.
problem Time-consuming annotation of sensor data for activity recognition.
method Attention-based convolutional neural network for weakly labeled data.
result Attention model improves accuracy in recognizing human activities.
Simple attention model outperforms complex sEMG classifiers.
problem Improving myoelectric control for robotic prosthetics.
method Attention-based model for sEMG signal classification.
result Simple model achieves benchmark results on multiple datasets.
Abstractor enhances Transformers for relational reasoning, improving sample efficiency and performance.
problem Improving sample efficiency and performance in relational tasks.
method Introduces Abstractor module with relational cross-attention to enable explicit relational reasoning.
result Dramatic improvements in sample efficiency and performance on various relational tasks.
ConvMFiT classifies client utterances in counseling dialogues.
problem Classifying client utterances for better counseling outcomes.
method Fine-tuning a pre-trained conversation model on counselor-client dialogues.
result ConvMFiT outperforms state-of-the-art models in classification.
Graph convolution model uses self-attention to predict diseases from multi-modal data.
problem Predicting diseases from diverse multi-modal data.
method Graph convolution with self-attention layer.
result Significantly outperforms state-of-the-art methods in disease prediction.
A deep learning method for XML with autoencoder and ranking loss.
problem XML with large label collections, high complexity, inter-label and feature dependencies, and noisy labels.
method Word-vector-based self-attention, ranking-based AutoEncoder architecture.
result Competitive performance on benchmark datasets.
Enhanced neural model detects misinformation by considering author context.
problem Plateaued accuracy in traditional rumor classifiers.
method Ensemble neural architecture with word-level attention and author context.
result Promising results on Twitter dataset, detecting rumors more accurately.
Deep CNN detects mind wandering from EEG data.
problem Detecting mind wandering to reorient attention.
method Channel-wise deep convolutional neural network (CNN) model.
result 91.78% accuracy in detecting mind wandering.
Gradient boosting of regression trees is a competitive procedure for learning predictive models of continuous data that fits the data with an additive non-parametric model. The classic version of gradient boosting assumes that the data is independent and identically distributed. However, relational data with interdepen…
AGML model improves indoor localization with sparse fingerprints using meta-learning and graph neural networks.
problem Maintaining high localization accuracy with extremely sparse fingerprints.
method Attentional Graph Neural Network (AGNN) and meta-learning framework with data augmentation strategies.
result AGML model consistently outperforms baseline methods across various metrics.
The paper proposes a novel model to forecast patent citations using multi-attention recurrent networks.
problem Forecasting forward citations to patents to discover emerging technologies.
method The approach employs a sequence-to-sequence model with an attention-of-attention mechanism to capture dependencies in multiple time sequences.
result The proposed model outperforms state-of-the-art models in forward citation forecasting.
CDSA uses self-attention to impute missing values in multivariate, geo-tagged time series data.
problem Missing values in multivariate, geo-tagged time series data.
method Cross-Dimensional Self-Attention (CDSA) for sequence modeling across time, location, and sensor measurements.
result CDSA outperforms state-of-the-art methods in imputation and forecasting on real-world datasets.
A new method for estimating joint value functions in multi-scene reinforcement learning.
problem High variance in samples for policy gradient computations in multi-scene environments.
method Sparse attention mechanism over multiple value function hypotheses to approximate the true joint value function.
result Significant improvements in reward scores and enhanced navigation efficiency across OpenAI ProcGen environments.
Study uses attention-based method to detect different types of online harassment.
problem Detecting different types of online harassment in social media content.
method Multi-attention based approach using Recurrent Neural Networks to address imbalanced data.
result Demonstrates effectiveness of attention-based mechanism for detecting various types of online harassment.
LNUCB-TA improves MAB performance by dynamically adjusting exploration rates and recognizing spatiotemporal patterns.
problem Suboptimal performance in environments with rapidly changing reward structures and static exploration rates.
method Hybrid model combining linear and nonlinear estimation, with adaptive k-NN for temporal attention.
result Significantly outperforms state-of-the-art algorithms in cumulative and mean reward, convergence, and robustness.
Paper proposes HGTAN for better stock trend prediction.
problem Predicting stock price trends is challenging and crucial for investors.
method Temporal-relational hypergraph tri-attention network (HGTAN).
result HGTAN outperforms existing methods in stock trend prediction.
MPSA-DenseNet improves accent classification accuracy.
problem Accurate English accent identification.
method Combines multi-task learning and PSA attention mechanism with DenseNet.
result MPSA-DenseNet outperforms other models in accent classification.
Improved sarcasm detection using hybrid neural network with a new dataset.
problem Predicting sarcasm in text remains challenging for machines.
method Proposed a hybrid Neural Network architecture with attention mechanism.
result Improves classification accuracy by ~ 5%.
Method identifies dual cognitive system interactions using eye-tracking data.
problem Quantify interactions between System 1 and System 2 cognitive processes.
method Modified Stroop test, machine learning, eye-tracking data.
result Machine learning can distinguish between different tasks and characterize attentional load.
Study shows Wikipedia views correlate with digital currency market performance.
problem Lack of systematic investigation into online attention and crypto market dynamics.
method Quantified the interplay between Wikipedia attention and digital currency market performance using edit history and view history.
result A simple trading strategy informed by Wikipedia views outperformed classic strategies.
In open set recognition (OSR), almost all existing methods are designed specially for recognizing individual instances, even these instances are collectively coming in batch. Recognizers in decision either reject or categorize them to some known class using empirically-set threshold. Thus the decision threshold plays a…
Machine learning suffers from poor design, data, and evaluation practices.
problem Ad hoc design, poor data hygiene, and lack of statistical rigor in model evaluation.
method Examines the entire machine learning process from design to evaluation, highlighting common pitfalls and providing recommendations.
result Common pitfalls in machine learning research and development are identified and actionable recommendations are provided.
RKT model improves knowledge tracing by considering exercise relations and student forget behavior.
problem Traditional KT models fail to consider both exercise relations and student forget behavior.
method RKT model uses relation-aware self-attention to incorporate exercise relations and student forget behavior.
result RKT model outperforms state-of-the-art KT methods on real-world datasets.
A-MIL improves histopathology image classification and localization.
problem Improving diagnosis of breast cancer through better interpretation of histopathology images.
method Frame image classification as multiple instance learning, use attention-based learning for localization.
result A-MIL achieves better localization without compromising classification accuracy.
RAN model recognizes multiple activities from unlabeled sensor data.
problem Handling weakly labeled multi-activity data from wearable sensors.
method Recurrent Attention Networks (RAN) for sequential multi-activity recognition and localization.
result RAN model can infer multiple activities and determine activity locations from unlabeled data.
This paper proposes a multi-head attention model for predicting RUL in IIoT environments.
problem Estimating RUL for complex industrial equipment using IIoT data.
method Multi-Head Attention Mechanism combined with LSTM for multi-dimensional time-series data.
result The proposed model outperforms state-of-the-art models on benchmark datasets.
We provide a benchmark dataset for hand gesture recognition using force myography.
problem Lack of publicly available benchmark data for force myography hand gesture recognition.
method Collected data from 20 persons covering 18 unique gestures using a commercially available sensor setup.
result Improved gesture recognition accuracy through transfer learning.
New algorithm scales NDPP learning and inference to large item collections.
problem Memory and runtime limitations in existing NDPP learning and inference algorithms.
method Introduced a new NDPP kernel decomposition for learning and a linear-complexity MAP inference algorithm.
result Our algorithms scale linearly in M, matching prior work's predictive performance. Study shows annotation instrument design affects model performance in hate speech detection.
problem Impact of annotation instrument design on model performance in hate speech detection.
method Collected annotations from five experimental conditions of an annotation instrument, fine-tuned BERT models on each dataset, evaluated performance on holdout portion.
result Significant differences in model performance and annotations across conditions.
The machine learning community has recently devoted much attention to the problem of inferring causal relationships from statistical data. Most of this work has focused on uncovering connections among scalar random variables. We generalize existing methods to apply to collections of multi-dimensional random vectors, fo…
Improved audio event recognition using audiovisual transformers.
problem Weakly labeled audio event classification with limited temporal information.
method Adapted transformer architecture that combines audio and video inputs.
result State-of-the-art performance on audiovisual data set.
Some general features of kinetic multi-agent models are reviewed, with particular attention to the relation between the agent saving propensities and the form of the equilibrium wealth distribution. The effect of a finite cutoff of the saving propensity distribution on the corresponding wealth distribution is studied. …
The recently presented idea to learn heuristics for combinatorial optimization problems is promising as it can save costly development. However, to push this idea towards practical implementation, we need better models and better ways of training. We contribute in both directions: we propose a model based on attention …
A new Hopfield model reduces inefficiency in large transformer models.
problem Inefficiency in training large transformer-based models.
method Introduces an Outlier-Efficient Hopfield Model (OutEffHop) to improve model performance.
result Achieves up to 26% reduction in model output norms across four models.
InGRA models for efficient Granger causality learning in multivariate time series.
problem Efficiently modeling Granger causality in large-scale multivariate time series data.
method Inductive GRanger causal modeling (InGRA) framework with prototypical Granger causal attention.
result InGRA detects common causal structures and infers Granger causal structures for new individuals.