Develops M2 model for next-basket recommendation considering user preferences, item popularity, and transition patterns.
problem Next-basket recommendation problem considering user preferences, item popularity, and transition patterns.
method Mixed model with preferences, popularities, and transitions (M2) using ed-Trans for transition patterns among items.
result Significantly outperforms state-of-the-art methods on all datasets in all tasks, with up to 22.1% improvement.
Proving that next-token prediction makes language models generate coherent long documents.
problem Understanding why language models generate coherent documents despite focusing on next-token prediction.
method Proving the power of next-token prediction in learning longer-range structure using Recurrent Neural Networks (RNN).
result Optimizing next-token prediction in RNNs yields a model that closely approximates the training distribution, even for long-range coherence.
The study compares weather prediction algorithms and finds Nexting performs well for slowly varying signals.
problem Comparing prediction algorithms for short-term weather forecasting.
method Evaluation of neural networks, regression trees, and Nexting on historical weather data.
result Nexting performed well for slowly varying signals with sufficient training data.
Improves tree model performance by considering future node splits.
problem Improving tree model performance.
method Next-Depth Lookahead Tree (NDLT) model that evaluates future node splits.
result Enhanced tree model performance.
Which song will Smith listen to next? Which restaurant will Alice go to tomorrow? Which product will John click next? These applications have in common the prediction of user trajectories that are in a constant state of flux over a hidden network (e.g. website links, geographic location). What users are doing now may b…
Next basket recommendation improved with context-aware item representations.
problem Predicting users' next purchases based on historical transactions.
method Pre-trained context-aware item representations using transformers.
result IERT outperforms state-of-the-art methods in next basket prediction.
Model predicts next destination for users based on past trips and features.
problem Predicting the next destination in multi-destination trips.
method Used Cleora for city graph embedding and EMDE for prediction.
result Achieved 2nd place in Booking Data Challenge.
Reinforcement learning after next-token prediction aids in learning from diverse sequence lengths.
problem Learning from sequences of varying lengths and complexity.
method Introducing a framework to study reinforcement learning with autoregressive transformers, focusing on next-token prediction and mixture distributions of short and long sequences.
result Reinforcement learning after next-token prediction enables autoregressive transformers to generalize from long sequences, even when they are rare.
This paper examines how different decoding algorithms for LLMs align with various goals.
problem Consistency of decoding algorithms with different goals in LLMs.
method Analysis of greedy, lookahead, random sampling, and temperature-scaled random sampling algorithms.
result Random sampling is consistent with the true probability distribution, but other goals require optimal algorithms for specific probability distributions.
The study explores how Transformers predict the next token in a sequence.
problem Understanding the mechanism behind Transformers' autoregressive learning ability.
method Exploring the approximation ability of Transformers for next-token prediction through specific instances and a causal kernel descent method.
result Transformer models can learn context-dependent functions f f f for next-token prediction based on past and current observations. New model mimics neural next item recommendation using Hankel matrices.
problem Next item recommendation efficiency and structural knowledge capture.
method Tensor factorization with Hankel matrix representation.
result Model performs competitively with neural networks but is simpler.
A new approach predicts next observations without explicit decoding for better control.
problem High-dimensional observations and unknown dynamics in real-world control tasks.
method Proposes a novel information-theoretic LCE approach using predictive coding to develop a decoder-free model.
result The model reliably learns a controllable latent space leading to superior performance.
Law explains how LLMs learn to predict next tokens.
problem Understanding how LLMs process input data internally.
method Introduced a precise law governing token embeddings in LLMs.
result Each layer equally contributes to next-token prediction accuracy.
Linear properties are either universal or absent across language models.
problem Explaining the prevalence of linear properties in language models.
method Proved identifiability of distribution-equivalent next-token predictors and analyzed various notions of linearity.
result Linear properties either hold in all or none distribution-equivalent next-token predictors.
NEXT learns efficient paths in high dimensions using neural exploration-exploitation trees.
problem Learning efficient path planning in high-dimensional spaces.
method Neural Exploration-Exploitation Trees (NEXT) integrating neural architecture and UCB algorithm.
result NEXT achieves better sample efficiency and outperforms state-of-the-art methods.
This study tackles mutual fund portfolio prediction, focusing on novel items.
problem Predicting novel items in mutual fund portfolios is challenging and less explored.
method Created a comprehensive benchmark dataset and evaluated various recommender system models.
result Autoencoder-based approaches outperform state-of-the-art models in predicting novel items.
Next-gen reservoir computers fail to predict complex processes, highlighting need for better architectures.
problem Predicting complex, non-Markovian processes with recurrent neural networks.
method Lower bound from Fano's inequality and analysis of large probabilistic state machines.
result Next-generation reservoir computers have an error probability at least 60% higher than optimal for highly non-Markovian processes.
We are now witnessing the increasing availability of event stream data, i.e., a sequence of events with each event typically being denoted by the time it occurs and its mark information (e.g., event type). A fundamental problem is to model and predict such kind of marked temporal dynamics, i.e., when the next event wil…
iPrescribe offers fast online offer recommendations using deep learning.
problem Online offer recommendation in real-time.
method Ensemble of deep learning and machine learning algorithms, optimized streaming technology stack, and efficient LSTM deployment.
result 90th percentile recommendation latency of 38 milliseconds.
SANST uses self-attentive networks with spatial and temporal embeddings for better POI recommendations.
problem Next point-of-interest (POI) recommendation for users based on their history.
method SANST incorporates spatio-temporal patterns into self-attentive networks.
result SANST outperforms state-of-the-art models by up to 13.65% in nDCG@10.
Optuna introduces a new hyperparameter optimization framework.
problem Efficient hyperparameter optimization for machine learning models.
method Define-by-run API, efficient implementation, versatile architecture.
result Optuna is the first software meeting next-generation hyperparameter optimization criteria.
Research improves federated text models for next word prediction.
problem Training models on distributed devices efficiently and effectively.
method Employ transfer learning in federated learning for next word prediction.
result Enhancements to current baselines with pretrained embeddings and whole model pretraining.
Let K ~ \widetilde{K} K be a two-periodic braid and let K K K be its quotient. In this paper we show there is a spectral sequence from the next-to-top winding number grading of the sutured annular Khovanov homology of the closure of K ~ \widetilde{K} K to the next-to-top winding number grading of the sutured annular Khovanov hom…
MPE models traffic trajectory data to predict next locations.
problem Predicting next locations from traffic trajectory data.
method Mobility pattern embedding model MPE.
result MPE significantly outperforms state-of-the-art methods in next location prediction.
We consider the problem of generating automatic code given sample input-output pairs. We train a neural network to map from the current state and the outputs to the program's next statement. The neural network optimizes multiple tasks concurrently: the next operation out of a set of high level commands, the operands of…
Bayesian framework optimizes 3D view selection for specific tasks.
problem Optimizing 3D view selection for specific tasks in reconstruction.
method Bayesian decision theory, prior and posterior distributions, stochastic surface reconstruction.
result Framework achieves superior performance with fewer views.
Predicts user next location using CDR data.
problem Predicting user's next location using CDR data.
method Data processing framework with deep neural network model.
result Error of prediction decreased from 74% to 55%.
This paper explores how Transformers predict next tokens in autoregressive tasks.
problem Understanding the success of Transformers in autoregressive learning.
method Trained a Transformer on a next-token prediction task, focusing on commuting orthogonal matrices.
result Trained Transformers can be seen as implementing gradient descent for a specific objective function.
New bifurcation found in perturbations of non-generic closed self-shrinkers.
problem Understanding the behavior of perturbations in non-generic closed self-shrinkers.
method Analyzing the mean curvature flow singularity transitions.
result Different types of singularity transitions based on perturbation direction.
Market forecasts converge to true values if some agents are correct.
problem Convergence of market forecasts in dynamic prediction markets.
method Dynamic model of prediction market with agents making forecasts.
result Aggregated market forecasts converge to conditional expectations.
Improves event prediction in complex processes using Petri nets and deep learning.
problem Predicting the next event in complex processes given a state.
method Enhanced Petri net model with time decay functions and deep learning.
result Significant performance improvements over state-of-the-art methods.
Gradient descent biases linear models in next-token prediction towards data entropy.
problem Optimization bias in next-token prediction models.
method Analysis of gradient descent on linear models with sparse conditional distributions.
result Gradient descent selects parameters that equate token logits differences to log-odds in the data subspace.
Predicts next item in sequential bundles using Transformers.
problem Predicting next item in sequentially consumed bundles.
method Used custom Transformers, GPT-3, LSTM, reinforcement learning, Markov models.
result Custom Transformer with decoder-only architecture most accurate.
Language models help text classification tasks by predicting next words.
problem Lack of theoretical understanding of why language models perform well on downstream tasks.
method Mathematical study of the connection between next word prediction and text classification, formalizing it and quantifying the benefit.
result Language models that are ε-optimal in cross-entropy learn features that can solve classification tasks with linear approximation.
Predictive business process monitoring methods exploit logs of completed cases of a process in order to make predictions about running cases thereof. Existing methods in this space are tailor-made for specific prediction tasks. Moreover, their relative accuracy is highly sensitive to the dataset at hand, thus requiring…
A new method improves language model extrapolation without changing training.
problem Improving language model performance on sequences not seen during training.
method Perturbation-based training procedure for language models.
result The method consistently improves out-of-support prediction.
Interdisciplinary comparison of sequence modeling methods for next-element prediction.
problem Comparing sequence modeling methods across different fields.
method Experimental evaluation of four real-life sequence datasets using machine learning, process mining, and grammar inference techniques.
result Machine learning techniques outperform interpretability-focused methods in next-element prediction accuracy.
Paper proposes a GAN-based method for better next event prediction in business processes.
problem Insufficient training data and sub-optimal network configuration limit deep learning approaches to next event prediction.
method Adversarial training framework using Generative Adversarial Networks (GANs) for sequential temporal data.
result The proposed approach achieves at least as good accuracy as non-adversarial methods and outperforms them in accuracy and prediction earliness.
Next-gen reservoir computing models dynamical systems from time-series data.
problem Modeling dynamical systems from time-series data.
method Pseudorandom nonlinear projection of time-delay embedded inputs.
result Models remain stable over long rollouts and generalize beyond training data.
The paper analyzes the training dynamics of a transformer for next-token prediction.
problem Understanding the non-asymptotic performance of transformers in next-token prediction.
method Characterizes training dataset properties, designs a two-stage training algorithm, and analyzes attention gradient properties.
result Trained transformers converge sub-linearly to max-margin solutions and exhibit linear convergence in cross-entropy loss.
The unique third-order invariant variational equation in three-dimensional (pseudo)Euclidean space is derived.
Census of 10-tetrahedra hyperbolic 3-manifolds with 150,730 new examples.
problem Census of orientable cusped hyperbolic 3-manifolds up to 10 tetrahedra.
method Complete enumeration of minimal ideal triangulations.
result 439,898 new exceptional Dehn fillings and simplest knot exteriors.
Predicts traffic flow using reinforcement learning and sensor data.
problem Accurately predict expanding and evolving long-term streaming traffic networks.
method Formulates the problem as a continuous reinforcement learning task, where the agent predicts future traffic based on sensor data.
result The approach improves accuracy in predicting traffic flow by updating the agent's state representation over time.
We devise a USDCHF trading strategy using the dynamics of gold as a filter. Our strategy involves modelling both USDCHF and gold using a coupled hidden Markov model (CHMM). The observations will be indicators, RSI and CCI, which will be used as triggers for our trading signals. Upon decoding the model in each iteration…
Paper uses AI methods to forecast Bitcoin prices.
problem Inaccurate Bitcoin price predictions in previous studies.
method Combines EEMD and LSTM for next-day price forecast.
result Improves Bitcoin price prediction accuracy.
This paper shows ARMs and EBMs are equivalent, revealing ARM lookahead capabilities.
problem Understanding the lookahead capabilities of next-token prediction models.
method Unified view of ARMs and EBMs, establishing a bijection and deriving equivalence.
result ARMs and EBMs are equivalent, revealing ARM lookahead capabilities.
VAIOM models financial returns using continuous input and categorical output.
problem Modeling continuous, noisy, and heterogeneous financial data.
method VAIOM is a decoder-only Transformer that separates input representation from output likelihood.
result VAIOM models outperform fixed single-bar LightGBM baseline in both Test halves.
Mathematical framework for language models processes text and predicts next tokens.
problem Understanding and optimizing the performance of large language models.
method Describes encoding, prediction models, learning from data, and deployment of LLMs.
result Demonstrates remarkable empirical successes and provides a platform for further research.