New distributed EnKF method for non-sequential assimilation of large datasets.
problem Computational intensity and order dependencies in traditional EnKF.
method Distributed computing for full model error covariance matrix.
result Non-sequential assimilation outperforms sequential in performance.
This review assesses deep-learning methods for complex sequential data.
problem Lack of robustness and transparency in deep-learning frameworks for irregular sequential data.
method Systematic literature review of existing algorithms.
result Recurrent neural networks dominate in performance evaluation of deep-learning frameworks.
We present PubMed 200k RCT, a new dataset based on PubMed for sequential sentence classification. The dataset consists of approximately 200,000 abstracts of randomized controlled trials, totaling 2.3 million sentences. Each sentence of each abstract is labeled with their role in the abstract using one of the following …
Proposes using frequent sequences to improve sequential recommendation models.
problem Combining user history and recent actions for personalized recommendations.
method Uses frequent sequences to identify relevant parts of user history, embedding items based on preferences and dynamics in a unified metric model.
result Outperforms state-of-the-art methods, especially on sparse datasets.
In order to build efficient deep recurrent neural architectures, it is essential to analyze the complexityof long distance dependencies (LDDs) of the dataset being modeled. In this paper, we presentdetailed analysis of the dependency decay curve exhibited by various datasets. The datasets sampledfrom a similar process …
FMT model improves multimodal sequential learning across language, vision, and acoustic data.
problem Modeling spatio-temporal dynamics across multiple modalities.
method Factorized Multimodal Transformer (FMT) that models intramodal and intermodal dynamics in a factorized manner.
result FMT outperforms existing models on 3 datasets and 21 labels, setting new state of the art.
Teaches sequential learners with changing inner states to improve future performance.
problem Teaching sequential learners with evolving inner states.
method Introduces an optimal control approach for multi-agent learning.
result Optimal control approach improves future performance of learners.
An adversarial detector identifies anomalous sequences in sequential data.
problem Detecting anomalous sequences in one-class settings with limited data.
method Solves a minimax problem to find an optimal detector against the worst-case sequences from a generator, using marked point process model.
result Demonstrated good performance on simulations and real credit card fraud datasets.
A new dataset tracks user interactions and click responses in online marketplaces.
problem Lack of exposure data in recommender systems datasets.
method Proposes a novel dataset including slates and click responses, allowing more accurate likelihood models.
result Models using exposure data show more natural likelihood, reducing bias towards previously exposed items.
Language models trained on chess board states outperform those on moves, even with causal masking.
problem Applying causal masking to spatial data for training unimodal language models.
method Trained bidirectional and causal self-attention models on both spatial (board-based) and sequential (move-based) chess data.
result Models trained on spatial board states achieve stronger playing strength than those trained on sequential data, even with causal masking.
Characterizing temporal dependence patterns is a critical step in understanding the statistical properties of sequential data. Long Range Dependence (LRD) --- referring to long-range correlations decaying as a power law rather than exponentially w.r.t. distance --- demands a different set of tools for modeling the unde…
A new method reduces hyperparameter tuning evaluations by using sequential tests.
problem Time-consuming hyperparameter tuning in machine learning.
method Sequential Random Search (SQRS) extending regular random search.
result SQRS finds similarly well-performing parameter settings with fewer evaluations.
In this paper, we propose an AdaBoost-assisted extreme learning machine for efficient online sequential classification (AOS-ELM). In order to achieve better accuracy in online sequential learning scenarios, we utilize the cost-sensitive algorithm-AdaBoost, which diversifying the weak classifiers, and adding the forgett…
VQ-DRAW compresses images and generates realistic samples.
problem Learning compact discrete representations of images.
method Sequential discrete VAE with vector quantization.
result VQ-DRAW effectively compresses and generates images.
Stochastic variational inference for collapsed models has recently been successfully applied to large scale topic modelling. In this paper, we propose a stochastic collapsed variational inference algorithm in the sequential data setting. Our algorithm is applicable to both finite hidden Markov models and hierarchical D…
In many real-world applications, data is not collected as one batch, but sequentially over time, and often it is not possible or desirable to wait until the data is completely gathered before analyzing it. Thus, we propose a framework to sequentially update a maximum margin classifier by taking advantage of the Maximum…
SPlit optimizes dataset splitting for better model performance.
problem Improving model performance through optimal dataset splitting.
method Adapting Support Points (SP) algorithm for subsampling and categorical variables in a sequential nearest neighbor approach.
result SPlit significantly improves worst-case testing performance compared to random splitting.
Study aggregation of statistical evidence under unknown dependence using group-invariance.
problem Aggregating statistical evidence under unknown and complex dependence structures.
method Develops a framework using group-invariance and permutation-based constructions to aggregate evidence across transformed datasets.
result Shows uniform improvement in critical values for single-batch aggregation over deterministic calibrations, adapting to unknown dependence structures.
Paper tests Markov assumption in sequential decision making.
problem Testing the Markov assumption in sequential decision making.
method Forward-Backward Learning procedure to test MA without assuming parametric forms.
result The proposed test plays a crucial role in identifying optimal policies in complex decision processes.
A new model learns latent spaces for graph data.
problem Scalability and expressivity limitations in graph generative models.
method Sequential Graph Variational Autoencoder (SGVAE) that learns latent spaces directly from graph data.
result Promising results on a cycle dataset, but need for permutation relaxation.
This paper proposes an attention module augmented relational network called SARN(Sequential Attention Relational Network) that can carry out relational reasoning by extracting reference objects and making efficient pairing between objects. SARN greatly reduces the computational and memory requirements of the relational…
Study improves neural network performance in sequential learning for image classification.
problem Improving neural network performance in sequential learning for image classification.
method Evaluation of approaches for computing prequential description lengths, proposing forward-calibration and replay-streams.
result Improved description lengths for image classification datasets, outperforming previous results.
SigGPDE scales sparse Gaussian processes for sequential data.
problem Predicting and quantifying uncertainty in sequential data.
method Sparse variational inference framework for Gaussian Processes, leveraging GP signature kernel gradients as PDE solutions.
result Significant computational gains and state-of-the-art performance on large sequential datasets.
Proposes a method to accelerate safe sequential learning using offline data.
problem Limited exploration due to disconnected safe regions and slow task learning.
method Safe transfer sequential learning using Gaussian processes and offline data.
result Enhances global exploration across multiple disjoint safe regions with lower data consumption.
Understanding how a learned black box works is of crucial interest for the future of Machine Learning. In this paper, we pioneer the question of the global interpretability of learned black box models that assign numerical values to symbolic sequential data. To tackle that task, we propose a spectral algorithm for the …
RAN model recognizes multiple activities from unlabeled sensor data.
problem Handling weakly labeled multi-activity data from wearable sensors.
method Recurrent Attention Networks (RAN) for sequential multi-activity recognition and localization.
result RAN model can infer multiple activities and determine activity locations from unlabeled data.
Dynamic SBI improves SBI efficiency without rounds, reducing simulation and training costs.
problem Efficiently perform complex scientific inference with high-dimensional data.
method Adaptive dataset transformation, parallel simulation and training.
result Significant improvements in simulation and training efficiency.
Stochastic variational inference for collapsed models has recently been successfully applied to large scale topic modelling. In this paper, we propose a stochastic collapsed variational inference algorithm for hidden Markov models, in a sequential data setting. Given a collapsed hidden Markov Model, we break its long M…
Current state-of-the-art nonparametric Bayesian text clustering methods model documents through multinomial distribution on bags of words. Although these methods can effectively utilize the word burstiness representation of documents and achieve decent performance, they do not explore the sequential information of text…
A novel signature method improves sequential data prediction accuracy.
problem Improving prediction accuracy for sequential and temporal data.
method Signature method based on rough path theory, with a specific embedding called lead-lag.
result Lead-lag embedding consistently outperforms other embeddings across various datasets and algorithms.
PSMF factorizes time-varying datasets into a dictionary and time-varying coefficients.
problem Factorizing time-varying and non-stationary datasets with temporal nonlinearities.
method Probabilistic Sequential Matrix Factorization (PSMF) using nonlinear Gaussian state-space models and approximate extended Kalman filtering.
result PSMF can account for temporal nonlinearities and estimate generic subspace models.
CB-GLNs learn video data's complex dependencies via graph representation.
problem Capturing complex dependency structures in sequential data like videos.
method Represent video data as a graph, find compositional dependencies via graph-cut and message passing.
result CB-GLNs efficiently learn video data's semantic compositional structure.
Unified framework for robust A/B testing under model misspecification.
problem Improving sample efficiency in A/B testing with model misspecification.
method Unified framework for contextual bandit and dynamic settings, proving worst-case mean squared error bounds.
result Empirically validated approach using synthetic and real-world datasets.
Improves sequential recommendation with relation-aware self-attention.
problem Improving accuracy in sequential recommendation.
method Integrates Transformer's self-attention mechanism with a probabilistic model of recommendation context.
result Significant improvements over recent baseline models.
Mining frequent sequential patterns consists in extracting recurrent behaviors, modeled as patterns, in a big sequence dataset. Such patterns inform about which events are frequently observed in sequences, i.e. what does really happen. Sometimes, knowing that some specific event does not happen is more informative than…
Context-aware recommender systems (CARSs) apply sensing and analysis of user context in order to provide personalized services. Adding context to a recommendation model is challenging, since the addition of context may increases both the dimensionality and sparsity of the model. Recent research has shown that modeling …
Improved dataset distillation for images and texts boosts model accuracy.
problem Reducing dataset size for faster and more energy-efficient model training.
method Simultaneous distillation of images and soft labels, extending to text datasets.
result 2-4% increase in accuracy for image classification tasks, 20% reduction in distilled samples.
We present a large-scale empirical study of catastrophic forgetting (CF) in modern Deep Neural Network (DNN) models that perform sequential (or: incremental) learning. A new experimental protocol is proposed that enforces typical constraints encountered in application scenarios. As the investigation is empirical, we ev…
We develop a Bayesian approach to learning from sequential data by using Gaussian processes (GPs) with so-called signature kernels as covariance functions. This allows to make sequences of different length comparable and to rely on strong theoretical results from stochastic analysis. Signatures capture sequential struc…
Framework predicts melt pool geometry with AI, improving manufacturing quality.
problem Achieving consistent product quality in Metal Additive Manufacturing.
method Surprise-guided sequential learning framework integrating CTGAN for limited data.
result Enhanced predictive accuracy for melt pool dimensions.
Recent sequential pattern mining methods have used the minimum description length (MDL) principle to define an encoding scheme which describes an algorithm for mining the most compressing patterns in a database. We present a novel subsequence interleaving model based on a probabilistic model of the sequence database, w…
The paper addresses estimating long-term treatment effects with monotone missing data.
problem Estimating long-term treatment effects with missing data, especially monotone missing.
method The paper introduces the sequential missingness assumption for identification and proposes three novel estimation methods: inverse probability weighting, sequential regression imputation, and SeqMSM. It also introduces a balancing-enhanced approach, BalanceNet, to improve estimation accuracy.
result The proposed methods, including BalanceNet, effectively estimate long-term treatment effects with monotone missing data.
Postprocessing reduces Bayesian optimization steps for global optima.
problem Slow convergence in Bayesian optimization for high-dimensional problems.
method Prohibits duplicated samples in the dataset postprocessing method.
result Significantly reduces the number of sequential steps to find the global optimum.
We introduce a new approach for amortizing inference in directed graphical models by learning heuristic approximations to stochastic inverses, designed specifically for use as proposal distributions in sequential Monte Carlo methods. We describe a procedure for constructing and learning a structured neural network whic…
Stochastic gradient descent (SGD) is a well known method for regression and classification tasks. However, it is an inherently sequential algorithm at each step, the processing of the current example depends on the parameters learned from the previous examples. Prior approaches to parallelizing linear learners using SG…
SurvSurf predicts first hitting times for intermittent events without monotonic violations.
problem Predicting first hitting times for intermittent events with monotonicity guarantees.
method Partially monotonic neural network for sequential events, incorporating unobserved events.
result SurvSurf outperforms existing models in MSE and IBS metrics.
Unsupervised framework learns symmetry from time sequences.
problem Learning symmetry from time sequences without labeled data.
method Meta-sequential prediction (MSP) framework that leverages stationary properties.
result Hidden disentangled structure emerges as a by-product of training.
SLHF uses sequential game theory to optimize preferences from human feedback.
problem Optimizing preferences from human feedback in sequential settings.
method SLHF frames the problem as a sequential-move game between Leader and Follower, decomposing the optimization into refinement and adversarial optimization.
result SLHF achieves strong alignment across diverse preference datasets and scales to large models.