Proposes a new method to learn representations directly optimized for a task.
problem Learning representations optimized for unrelated tasks.
method Jointly learns representation and prediction function for a specific task.
result Learned representations outperform pre-trained ones and are more sample-efficient.
The study analyzes neural network predictions of knot invariants and finds that braid representations work best.
problem Understanding and predicting knot invariants using neural networks.
method Investigated different knot representations and invariants, proposed a cosine similarity score.
result Braid representations are best for predicting knot invariants, and some invariants are easier to learn than others.
Self-supervised learning improves by predicting known information, reducing labeled data needs.
problem Efficiently learn useful semantic representations without labeled data.
method Develops a mechanism exploiting statistical connections between pretext tasks to learn representations that solve downstream tasks.
result Proves linear layer yields small approximation error and drastically reduces labeled sample complexity.
Deep learning reveals ubiquitous predictability in high-frequency returns.
problem Predicting returns in order book markets at high frequencies.
method Volume representation of the order book, deep learning models, model confidence sets.
result Predictability in mid-price returns is ubiquitous at high frequencies.
Enhances RL agents with predictive internal representations.
problem Improving model-free reinforcement learning agents.
method Introduces Deep InfoMax (DIM) objective to train predictive internal representations.
result Successfully learned predictive representations in synthetic settings.
Unsupervised representation learning has succeeded with excellent results in many applications. It is an especially powerful tool to learn a good representation of environments with partial or noisy observations. In partially observable domains it is important for the representation to encode a belief state, a sufficie…
Deep predictive coding networks are neuroscience-inspired unsupervised learning models that learn to predict future sensory states. We build upon the PredNet implementation by Lotter, Kreiman, and Cox (2016) to investigate if predictive coding representations are useful to predict brain activity in the visual cortex. W…
Neural networks are capable of learning rich, nonlinear feature representations shown to be beneficial in many predictive tasks. In this work, we use such models to explore different geographical feature representations in the context of predicting colorectal cancer survival curves for patients in the state of Iowa, sp…
The brain optimizes memory by forgetting what's predictable, improving generalization.
problem Memory consolidation struggles with representational drift, semanticisation, and offline replay.
method Proposes predictive forgetting as a mechanism to optimize generalization by reducing complexity.
result Predictive forgetting improves information-theoretic generalization bounds on stored representations.
Click-through rate (CTR) prediction is a critical task in online advertising systems. Most existing methods mainly model the feature-CTR relationship and suffer from the data sparsity issue. In this paper, we propose DeepMCP, which models other types of relationships in order to learn more informative and statistically…
New algorithms for hierarchical classification using conformal prediction.
problem Valid prediction sets in hierarchical classification tasks.
method Extended split conformal prediction framework with two inference algorithms.
result Empirical evaluations show effectiveness in achieving nominal coverage.
Study investigates predictive coding models for phonemic learning.
problem Understanding how predictive coding models generalize to different languages and dataset sizes.
method Investigated Autoregressive Predictive Coding and Contrastive Predictive Coding models in phoneme discrimination tasks for two languages with varying dataset sizes.
result Contrastive Predictive Coding model converges rapidly and outperforms Autoregressive Predictive Coding on both languages.
We study the strong predictable representation property in filtrations initially enlarged with a random variable L. We prove that the strong predictable representation property can always be transferred to the enlarged filtration as long as the classical density hypothesis of Jacod (1985) holds. This generalizes the ex…
Study compares atom representations in graph neural networks for molecular properties.
problem Incorrect attribution of results in molecular property prediction due to varying atom features.
method Evaluated multiple atom representations on free energy, solubility, and metabolic stability predictions.
result Different atom representations can lead to varying predictive performance in graph neural networks.
This paper explores how combining quantitative factors and news from LLMs improves stock return prediction.
problem Improving stock return prediction using quantitative factors and news.
method Introduces a fusion learning framework to learn unified representations from factors and LLM-generated newsflow, comparing combination, summation, and attentive methods. Explores mixture models and decoupled training approaches.
result Effective multimodal modeling of factors and news improves stock return prediction and selection.
In this paper, we advocate for representation learning as the key to mitigating unfair prediction outcomes downstream. Motivated by a scenario where learned representations are used by third parties with unknown objectives, we propose and explore adversarial representation learning as a natural method of ensuring those…
Object-based factorizations provide a useful level of abstraction for interacting with the world. Building explicit object representations, however, often requires supervisory signals that are difficult to obtain in practice. We present a paradigm for learning object-centric representations for physical scene understan…
We study the problem of learning permutation invariant representations that can capture "flexible" notions of containment. We formalize this problem via a measure theoretic definition of multisets, and obtain a theoretically-motivated learning model. We propose training this model on a novel task: predicting the size o…
Improved AutoDML estimator for causal inference using outcome-adapted shared covariate representation.
problem Efficiency in estimating treatment or policy effects in causal inference.
method Outcome-adapted AutoDML estimator that uses a shared covariate representation that is predictive of the outcome but not the Riesz representer.
result Outcome-adapted AutoDML estimator is asymptotically more efficient than baseline AutoDML.
This thesis investigates unsupervised time series representation learning for sequence prediction problems, i.e. generating nice-looking input samples given a previous history, for high dimensional input sequences by decoupling the static input representation from the recurrent sequence representation. We introduce thr…
This paper fine-tunes LLMs for stock return prediction using financial news.
problem Improving stock return forecasting accuracy using LLMs.
method Fine-tuning LLMs with text and forecasting modules, comparing encoder-only and decoder-only models, and integrating token-level representations.
result LLMs' aggregated token-level embeddings enhance return predictions for long-only and long-short portfolios.
Self-Predictive Representations improves data-efficient reinforcement learning from limited interaction.
problem Efficient reinforcement learning from limited data.
method Train agents to predict future latent state representations using self-supervised objectives.
result Achieves a median human-normalized score of 0.415 on Atari with 100k steps of interaction, 55% improvement over previous state-of-the-art.
Model predicts travel time under rare conditions using a vector-space model.
problem Predicting travel time under rare temporal conditions (e.g., holidays, school vacations) is challenging due to limited historical data and other temporal changes.
method Presented a vector-space model for encoding rare temporal conditions, allowing coherent representation learning across different conditions.
result Increased performance for travel time prediction over different baselines when using the vector-space encoding for representing the temporal setting.
New framework improves reliability of learned representations by modeling uncertainty and structural constraints.
problem Uncertainty in learned representations treated as deterministic, leading to unreliable models.
method Proposes a principled framework for reliable representation learning with uncertainty-aware regularization and structural constraints.
result Improves stability, calibration, and robustness of learned representations.
While supervised learning has enabled great progress in many applications, unsupervised learning has not seen such widespread adoption, and remains an important and challenging endeavor for artificial intelligence. In this work, we propose a universal unsupervised learning approach to extract useful representations fro…
Model uses LLM features to predict stock returns effectively.
problem Predicting stock returns from text data.
method Structured Event Representation (SER) model with attention mechanisms.
result SER-based model outperforms existing models in stock return prediction.
We examine two fundamental tasks associated with graph representation learning: link prediction and semi-supervised node classification. We present a novel autoencoder architecture capable of learning a joint representation of both local graph structure and available node features for the multi-task learning of link pr…
Develops a hybrid deep learning model for stock price prediction.
problem Predicting daily stock prices in the stock market.
method Representation learning with Stock2Vec embedding and temporal convolutional layers.
result Achieves better performance on stock price prediction than benchmarks.
Clinical notes are a rich source of information about patient state. However, using them to predict clinical events with machine learning models is challenging. They are very high dimensional, sparse and have complex structure. Furthermore, training data is often scarce because it is expensive to obtain reliable labels…
Neural network feature optimization for causal inference.
problem Estimating heterogeneous treatment effects from data.
method Genetic algorithm optimization of intermediate neural network layers for feature representations.
result Retains useful features for outcome prediction even if related to treatment assignment.
This work explains how linear representations in large language models arise from training objectives and gradient descent.
problem Understanding the origins of linear representations in large language models.
method A latent variable model to abstract and formalize concept dynamics, combined with analysis of the softmax cross-entropy objective and gradient descent.
result Linear representations emerge when learning from data matching the latent variable model, and this simple structure suffices to yield linear representations.
Deep learning with Convolutional Neural Networks has shown great promise in various areas of image-based classification and enhancement but is often unsuitable for predictive modeling involving non-image based features or features without spatial correlations. We present a novel approach for representation of high dime…
Statistical methods protecting sensitive information or the identity of the data owner have become critical to ensure privacy of individuals as well as of organizations. This paper investigates anonymization methods based on representation learning and deep neural networks, and motivated by novel information theoretica…
Develops UKP for comparing feature representations in multitask learning.
problem Comparing feature representations learned by different models without access to test data.
method Uniform Kernel Prober (UKP) for comparing representations in kernel ridge regression tasks.
result UKP provides a uniform measure of prediction error on test data without access to test data.
KINet learns object interactions without supervision for robotic pushing.
problem Lack of supervised data for object-centric forward prediction.
method End-to-end unsupervised framework using keypoint representation and contrastive estimation.
result Automatically generalizes to unseen scenarios and accurately predicts future states.
GRAPE uses graph representation to handle missing data in feature imputation and label prediction.
problem Handling missing data in machine learning tasks.
method GRAPE uses a bipartite graph where observations and features are nodes, and observed feature values are edges. It formulates feature imputation as edge-level prediction and label prediction as node-level prediction, solving these with Graph Neural Networks.
result GRAPE achieves 20% lower mean absolute error for imputation and 10% lower for label prediction compared to state-of-the-art methods.
Paper proposes a new method for learning business process representations.
problem Challenges in capturing all useful information in business process data.
method Combines Gramian Angular Fields and Convolutional Neural Networks for representation learning.
result Demonstrates effectiveness of the approach through visualization and multiple process prediction tasks.
Proposes Infomax and Domain-Independent Representations for robust causal inference.
problem Handling treatment selection bias and domain imbalance in causal inference with real-world data.
method Utilizes mutual information to learn domain-invariant representations that maximize predictive common information.
result Achieves state-of-the-art performance on causal effect inference across various data distributions.
A generalized bridge is the law of a stochastic process that is conditioned on N linear functionals of its path. We consider two types of representations of such bridges: orthogonal and canonical. The orthogonal representation is constructed from the entire path of the underlying process. Thus, future knowledge of the …
This paper learns graph node representations using global context prediction.
problem Efficiently learning useful node representations from unlabeled graph data.
method Randomly selects node pairs, trains a neural net to predict contextual positions.
result Our approach outperforms many unsupervised methods and sometimes supervised ones.
Self-Distilled Disentanglement improves counterfactual predictions by separating variables.
problem Improving counterfactual predictions in the presence of confounders and unobserved variables.
method Self-Distilled Disentanglement framework based on information theory.
result Effective counterfactual inference in synthetic and real-world datasets.
End-to-end learnable network for safer self-driving with interpretable intermediate representations.
problem Safe motion planning for self-driving vehicles.
method Differentiable semantic occupancy representation for cost calculation in motion planning.
result Significantly outperforms state-of-the-art planners in imitating human behaviors and producing safer trajectories.
The paper tackles stock prediction models by improving their generalizability to out-of-sample domains using causal representation learning.
problem Low signal-to-noise ratio and nonstationary nature of financial markets lead to poor performance of stock prediction models.
method The paper investigates Domain Generalization techniques, focusing on causal representation learning to improve model generalizability. It introduces a novel error bound and a causal discovery technique to mitigate spurious correlations.
result The proposed approach enhances the generalizability of stock prediction models, as demonstrated by numerical results.
Proposes Decodable Information Bottleneck for optimal representation learning.
problem Finding optimal representations for supervised learning.
method Integrates information retention and compression with the desired predictive family.
result Optimal representations lead to better expected test performance and can be estimated with guarantees.
Proposes a novel model for healthcare and SME credit risk prediction.
problem Lack of guidance from global view in sequence representation learning for time series modeling.
method Hierarchical Global View-guided (HGV) sequence representation learning framework with GGE and β-Attn modules. result Competitive prediction performance compared with other known baselines.
Machine learning techniques have recently been adopted in various applications in medicine, biology, chemistry, and material engineering. An important task is to predict the properties of molecules, which serves as the main subroutine in many downstream applications such as virtual screening and drug design. Despite th…
Interpretable representations improve explainable AI by translating complex data into understandable concepts.
problem Many explainers use interpretable representations but overlook their full potential and assumptions.
method An in-depth analysis of interpretable representations for tabular, image, and text data, identifying strengths, weaknesses, and desiderata.
result Linear model quantifies interpretable concepts' influence on black-box predictions, revealing their explanatory properties and manipulability.
Autoencoder learns group representations from actions, improving future prediction accuracy.
problem Learning internal models of interactions with the real world.
method Homomorphism autoencoder with group representation trained on equivariance-derived loss.
result Agents can predict future actions with improved accuracy.