Combines experimental and historical data for robust policy evaluation.
problem Policy evaluation with mixed data sources, especially experimental vs historical.
method Linear integration of estimators from experimental and historical data, optimized for MSE minimization.
result Proposed estimators outperform traditional methods in ridesharing company data.
Industry datasets used for text classification are rarely created for that purpose. In most cases, the data and target predictions are a by-product of accumulated historical data, typically fraught with noise, present in both the text-based document, as well as in the targeted labels. In this work, we address the quest…
KaoKore dataset extracts faces from pre-modern Japanese art for machine learning.
problem Lack of relevant datasets for historical Japanese artworks in machine learning.
method Extracted faces from pre-modern Japanese artwork to create a new dataset.
result Demonstrated the dataset's value for image classification and creative applications.
Data-driven method for option pricing using historical asset prices.
problem Tackling the gap between historical asset prices and risk-neutral option pricing.
method Identifying a pricing kernel process, solving utility maximization and functional optimization problems using deep learning.
result Demonstrated the efficiency of the data-driven option pricing methodology.
How do we learn from biased data? Historical datasets often reflect historical prejudices; sensitive or protected attributes may affect the observed treatments and outcomes. Classification algorithms tasked with predicting outcomes accurately from these datasets tend to replicate these biases. We advocate a causal mode…
Improves RL from historical data by stitching trajectories.
problem Lack of high-quality data for offline RL.
method Trajectory Stitching (TS) to augment historical data with synthetic actions.
result Improves RL policy performance over baseline.
We consider the stochastic multi-armed bandit problem and the contextual bandit problem with historical observations and pre-clustered arms. The historical observations can contain any number of instances for each arm, and the pre-clustering information is a fixed clustering of arms provided as part of the input. We de…
A new GNN model predicts stock trends by learning historical and future correlations.
problem Limited improvement in stock trend prediction models due to ignoring future patterns.
method DishFT-GNN framework that trains a teacher and student model to capture historical and future data correlations.
result State-of-the-art performance on real-world datasets.
Novel method for nowcasting implied volatility using neural operators.
problem Dynamic and spatially changing option prices in financial markets.
method Operator Deep Smoothing using graph neural operators.
result Highly accurate implied volatility smoothing on ten years of S&P 500 options data.
Recommender systems often face heterogeneous datasets containing highly personalized historical data of users, where no single model could give the best recommendation for every user. We observe this ubiquitous phenomenon on both public and private datasets and address the model selection problem in pursuit of optimizi…
Wittgenstein's Rule Following evolves datasets by extrapolating structural descriptors.
problem Generating meaningful continuations of evolving datasets.
method Wittgenstein's Rule Following (WRF) uses structural descriptors to extrapolate trajectories and average historical descriptors.
result WRF can generate meaningful continuations of evolving datasets.
New algorithm combines new and historical data with different input dimensions for linear regression.
problem Combining new and historical data with different input dimensions for improved accuracy.
method Proposes a transfer learning algorithm with rigorous theoretical robustness analysis.
result Achieves state-of-the-art performance on 9 real-life datasets.
The Analog Ensemble (AnEn) method tries to estimate the probability distribution of the future state of the atmosphere with a set of past observations that correspond to the best analogs of a deterministic Numerical Weather Prediction (NWP). This model post-processing method has been successfully used to improve the fo…
Study shows survivorship bias inflates returns in India's small-cap index.
problem Survivorship bias in emerging market small-cap indices.
method Reconstructing historical index composition through market capitalization ranking and comparing equal-weight portfolios of current constituents versus all historical members.
result Survivor-only backtesting overstates returns by 4.94 percentage points and Sharpe ratios by 0.097.
New method detects and mitigates historical bias in data.
problem Detecting and explaining historical bias in data.
method Developed a sample bias criterion and algorithms to measure and counter sample bias.
result Derived bias score provides sample-level attribution and explanation of historical bias.
EHNA learns node embeddings from historical network neighborhoods.
problem Capturing temporal information in evolving networks.
method Temporal random walk and deep learning model with attention mechanism.
result EHNA outperforms existing methods in network reconstruction and link prediction tasks.
Study on newsvendor problem with censored data, showing how much information is lost.
problem Minimizing costs in a newsvendor problem with limited historical demand data.
method Distributionally robust optimization framework, evaluating policies based on worst-case regret.
result Characterization of information loss due to demand censoring and development of a robust algorithm.
Historical documents present many challenges for offline handwriting recognition systems, among them, the segmentation and labeling steps. Carefully annotated textlines are needed to train an HTR system. In some scenarios, transcripts are only available at the paragraph level with no text-line information. In this work…
Machine learning techniques are presented for automatic recognition of the historical letters (XI-XVIII centuries) carved on the stoned walls of St.Sophia cathedral in Kyiv (Ukraine). A new image dataset of these carved Glagolitic and Cyrillic letters (CGCL) was assembled and pre-processed for recognition and predictio…
Paper proposes AI for stock market forecasting using external knowledge.
problem Forecasting stock prices influenced by external factors.
method Learning from historical data and external temporal knowledge graphs modeled as Hawkes processes.
result Dynamic representations effectively rank stocks based on returns.
It is well known that the historical logs are used for evaluating and learning policies in interactive systems, e.g. recommendation, search, and online advertising. Since direct online policy learning usually harms user experiences, it is more crucial to apply off-policy learning in real-world applications instead. Tho…
Unified HS and related methods with explicit modeling assumptions.
problem Lack of clear assumptions in HS methods for Value-at-Risk.
method Explicitly defined parametric model for asset returns and extraction of innovation process.
result HS and related methods require more assumptions than commonly acknowledged.
Machine learning (ML) can automate decision-making by learning to predict decisions from historical data. However, these predictors may inherit discriminatory policies from past decisions and reproduce unfair decisions. In this paper, we propose two algorithms that adjust fitted ML predictors to make them fair. We focu…
This paper presents a Convolutional Neural Network (CNN) based page segmentation method for handwritten historical document images. We consider page segmentation as a pixel labeling problem, i.e., each pixel is classified as one of the predefined classes. Traditional methods in this area rely on carefully hand-crafted …
Bridges uplift modeling and sequential decision-making with online budget allocation.
problem Treatment allocation under budget constraints in digital advertising.
method Budget-Constrained Causal Bandits (BCCB) integrates learning, exploration, and budget pacing.
result Data-efficiency crossover: BCCB operates effectively from the first user, 3-5x lower performance variance.
The link between different psychophysiological measures during emotion episodes is not well understood. To analyse the functional relationship between electroencephalography (EEG) and facial electromyography (EMG), we apply historical function-on-function regression models to EEG and EMG data that were simultaneously r…
New trading strategy uses deep neural networks for future stock price predictions.
problem Traditional backtesting of trading strategies is unreliable for future trades.
method Developed a deep neural network to predict stock prices and select optimal trading strategies.
result Neural network predictions improve trading performance metrics.
Estimates long-term effects of new treatments using historical and short-term data.
problem Estimating long-term effects of novel treatments with limited historical data.
method Surrogate indices, dynamic treatment effect estimation, and double machine learning combined in a unified pipeline.
result Consistent and asymptotically normal estimates of long-term effects under Markovian assumption.
We consider a new approach to portfolio selection in presence of transaction costs which allows to map the problem into one without costs. The proposed approach connects all the quantities of interest to exit times and probabilities to reach barriers. This leads to analytic results in the Wiener case and to directly me…
Click-through rate (CTR) prediction is an essential task in industrial applications such as video recommendation. Recently, deep learning models have been proposed to learn the representation of users' overall interests, while ignoring the fact that interests may dynamically change over time. We argue that it is necess…
ArtificialReplay improves data efficiency in bandits using historical data.
problem Data inefficiency in warm-starting bandit algorithms.
method ArtificialReplay, a meta-algorithm for incorporating historical data into any bandit algorithm.
result ArtificialReplay uses only a fraction of historical data compared to a full warm-start approach, achieving identical regret.
ADR helps LLMs find and use historical analogies for foresight analysis.
problem LLMs struggle to find relevant historical analogies due to surface-level matching.
method Proposes CANA framework with mechanism alignment and cross-analogy confirmation.
result CANA improves historical analogy generation by up to 10%.
Machine learning automates digitization of historical data.
problem Manual transcription is costly and difficult for large, detailed datasets.
method Apply machine learning techniques for unsupervised layout classification and attention-based neural networks.
result Machine learning can automate the digitization process for historical data.
Proposes dynamic borrowing method for historical data in clinical trials.
problem Insufficient statistical power in rare and pediatric disease clinical trials.
method Dynamic borrowing method based on frequentist approach using similarity measures.
result Demonstrates usefulness of dynamic borrowing in reanalyzing clinical trial data.
Automatic recognition of the historical letters (XI-XVIII centuries) carved on the stoned walls of St.Sophia cathedral in Kyiv (Ukraine) was demonstrated by means of capsule deep learning neural network. It was applied to the image dataset of the carved Glagolitic and Cyrillic letters (CGCL), which was assembled and pr…
Data describing historical economic growth are analysed. Included in the analysis is the world and regional economic growth. The analysis demonstrates that historical economic growth had a natural tendency to follow hyperbolic distributions. Parameters describing hyperbolic distributions have been determined. A search …
New method interprets machine learning forecasts as historical analogies.
problem Interpreting machine learning predictions as a sum of predictor contributions.
method Expressing predictions as a linear combination of in-sample values with weights based on pairwise proximity scores.
result The approach provides sparser interpretations in settings with many regressors and little training data.
This study reviews techniques to estimate volatility and price Variance Swaps.
problem Estimating historical volatility and pricing Variance Swaps.
method Review of existing techniques.
result Discussion of various methods to estimate volatility and price Variance Swaps.
ARED introduces a new dataset for Argentina's real estate market.
problem Lack of mixed modality datasets for Argentina's real estate market.
method Developed a comprehensive real estate price prediction dataset series for Argentina.
result Dataset captures time-dependent phenomena on a market level.
This paper tackles RL issues with robust policies using historical data.
problem Limited data and mismatch between training and testing environments.
method Distributionally robust offline RL with linear function approximation.
result Achieved error bounds for sample complexity in RL.
Study efficient sequential evaluation of large language models using historical data.
problem Sequentially evaluate a new large language model (LLM) on a fixed question set.
method Construct a confidence sequence (CS) and design active querying rules to shrink CS width.
result Simple uniform sampling can sometimes outperform adaptive querying rules.
Catastrophic forgetting can be a significant problem for institutions that must delete historic data for privacy reasons. For example, hospitals might not be able to retain patient data permanently. But neural networks trained on recent data alone will tend to forget lessons learned on old data. We present a differenti…
Predict stock price movements using financial data and news articles with LLMs.
problem Predicting stock price movements using financial data and news articles.
method Combining financial data and news articles, employing pre-trained LLMs, and using retrieval augmentation techniques.
result Predicted stock price movements with a weighted F1-score of 58.5% and 59.1%.
Deep learning framework predicts streamflow and flood probabilities in Australian catchments.
problem Large-scale flooding prediction challenges due to model calibration and missing data.
method Ensemble quantile-based deep learning framework using quantile regression and CAMELS dataset.
result Notable efficacy and uncertainties in streamflow forecasts with varied catchment properties.
Improved Bayesian inference using power priors with historical data.
problem Improving Bayesian inference with historical data.
method Generalized power priors that adapt to the α parameter of Amari's α-divergence. result Improved performance through appropriate choices of the α parameter. CDRE estimates density ratios in streaming data without historical samples.
problem Online learning with shifting data distributions.
method Iterative estimation of density ratios between initial and current distributions.
result CDRE outperforms standard DRE in estimating divergences between distributions.
Real-time fuel leakage detection framework MOCPD improves accuracy.
problem Early detection of fuel leakage to prevent hazards and losses.
method Memory-based Online Change Point Detection (MOCPD) framework.
result MOCPD outperforms baseline methods in detection accuracy.
End-to-end solution for recognizing handwritten numerals, avoiding traditional preprocessing steps.
problem Handwritten numeral string recognition with traditional preprocessing steps.
method YoLo-based model for automatic detection and recognition, avoiding heuristic-based preprocessing and segmentation.
result Proposed method reduces complexity and is a feasible end-to-end solution for numeral string recognition.