Proposes a new method to learn representations directly optimized for a task.
problem Learning representations optimized for unrelated tasks.
method Jointly learns representation and prediction function for a specific task.
result Learned representations outperform pre-trained ones and are more sample-efficient.
DeepMCP improves CTR prediction by learning better feature representations.
problem Data sparsity in CTR prediction models.
method DeepMCP models user-ad, ad-ad, and feature-CTR relationships through three subnets.
result DeepMCP outperforms state-of-the-art models in CTR prediction.
The strong predictable representation property is proven for filtrations with a random variable under a density hypothesis.
problem Proving the strong predictable representation property in filtrations with a random variable.
method Using the density hypothesis of Jacod (1985), the strong predictable representation property is transferred to the enlarged filtration.
result The strong predictable representation property can always be transferred to the enlarged filtration under the density hypothesis.
Neural architectures learn belief representations for partially observable environments.
problem Learning belief states in partially observable domains.
method One-step frame prediction and contrastive predictive coding (CPC) as objective functions.
result Neural architectures can learn belief representations, encoding both state information and uncertainty.
The study analyzes neural network predictions of knot invariants and finds that braid representations work best.
problem Understanding and predicting knot invariants using neural networks.
method Investigated different knot representations and invariants, proposed a cosine similarity score.
result Braid representations are best for predicting knot invariants, and some invariants are easier to learn than others.
Autoencoder learns graph representations for link prediction and node classification.
problem Link prediction and semi-supervised node classification on graphs.
method A novel autoencoder architecture for joint local graph structure and node feature learning.
result Significant improvement over related methods for graph representation learning.
A model learns object representations for physical scene understanding without direct supervision.
problem Learning object-centric representations without direct supervision of object properties.
method Object-Oriented Prediction and Planning (O2P2) model that learns perception, physics interaction, and rendering functions.
result The model can predict physical interactions and build block towers more complex than those seen during training.
Advances fair representation learning for unknown third-party uses.
problem Mitigating unfair prediction outcomes when representations are used by third parties with unknown objectives.
method Adversarial representation learning to ensure fairness.
result Demonstrated fair transfer learning and maintained utility.
Self-supervised learning improves by predicting known information, reducing labeled data needs.
problem Efficiently learn useful semantic representations without labeled data.
method Develops a mechanism exploiting statistical connections between pretext tasks to learn representations that solve downstream tasks.
result Proves linear layer yields small approximation error and drastically reduces labeled sample complexity.
Deep learning reveals ubiquitous predictability in high-frequency returns.
problem Predicting returns in order book markets at high frequencies.
method Volume representation of the order book, deep learning models, model confidence sets.
result Predictability in mid-price returns is ubiquitous at high frequencies.
Enhances RL agents with predictive internal representations.
problem Improving model-free reinforcement learning agents.
method Introduces Deep InfoMax (DIM) objective to train predictive internal representations.
result Successfully learned predictive representations in synthetic settings.
New model learns multisets to predict containment and sizes of differences.
problem Learning permutation invariant representations for flexible containment.
method Formalize multisets, propose training on predicting symmetric difference sizes.
result Model outperforms DeepSets on predicting containment and sizes of symmetric differences.
Neural networks improve clinical note representations for predictive modeling.
problem Challenges in using clinical notes for machine learning due to high dimensionality, sparsity, and scarcity of labeled data.
method Used neural networks and transfer learning to learn representations of clinical notes.
result Neural network representations significantly outperformed baseline representations in predictive models.
Unsupervised neural models predict brain activity better than supervised methods.
problem Understanding how the brain represents visual information without direct supervision.
method Built upon PredNet, used RSA to compare PredNet representations to fMRI and MEG data.
result Unsupervised models trained to predict video frames outperform supervised image classification models in predicting brain activity.
Generative models learn useful representations for complex sequential data.
problem Sequence prediction for high-dimensional input sequences.
method Three models based on Generative Stochastic Networks (GSN) for unsupervised sequence learning.
result GSNs provide evidence as a viable framework for complex sequential data.
The brain optimizes memory by forgetting what's predictable, improving generalization.
problem Memory consolidation struggles with representational drift, semanticisation, and offline replay.
method Proposes predictive forgetting as a mechanism to optimize generalization by reducing complexity.
result Predictive forgetting improves information-theoretic generalization bounds on stored representations.
The paper explores how disentangled representations can improve fairness in prediction tasks.
problem Improving fairness in prediction tasks using disentangled representations.
method Investigates different notions of disentanglement and analyzes representations of state-of-the-art models.
result Disentanglement scores are correlated with increased fairness in prediction tasks.
Enhanced geographical features improve predictive models for colorectal cancer survival curves.
problem Predicting colorectal cancer survival curves in Iowa.
method Used neural networks to explore feature representations, comparing ABC performance.
result Spectral analysis-based representations improve predictive performance by approximately 40%.
A new autoencoder learns graph representations for link prediction and node classification.
problem Link prediction and node classification on graph data.
method Multi-task graph autoencoder architecture.
result Significant improvement over three baselines on five graph datasets.
A new unsupervised learning method for high-dimensional data.
problem Challenges in unsupervised learning for AI.
method Contrastive Predictive Coding using autoregressive models and probabilistic contrastive loss.
result Strong performance across four domains: speech, images, text, and reinforcement learning.
A novel feature representation method for non-image based features.
problem Inability of Convolutional Neural Networks for non-image based features or features without spatial correlations.
method REFINED: Representation of Features as Images with Neighborhood Dependencies.
result Higher prediction accuracy compared to existing methodologies.
New algorithms for hierarchical classification using conformal prediction.
problem Valid prediction sets in hierarchical classification tasks.
method Extended split conformal prediction framework with two inference algorithms.
result Empirical evaluations show effectiveness in achieving nominal coverage.
Study investigates predictive coding models for phonemic learning.
problem Understanding how predictive coding models generalize to different languages and dataset sizes.
method Investigated Autoregressive Predictive Coding and Contrastive Predictive Coding models in phoneme discrimination tasks for two languages with varying dataset sizes.
result Contrastive Predictive Coding model converges rapidly and outperforms Autoregressive Predictive Coding on both languages.
Study compares atom representations in graph neural networks for molecular properties.
problem Incorrect attribution of results in molecular property prediction due to varying atom features.
method Evaluated multiple atom representations on free energy, solubility, and metabolic stability predictions.
result Different atom representations can lead to varying predictive performance in graph neural networks.
This paper explores how combining quantitative factors and news from LLMs improves stock return prediction.
problem Improving stock return prediction using quantitative factors and news.
method Introduces a fusion learning framework to learn unified representations from factors and LLM-generated newsflow, comparing combination, summation, and attentive methods. Explores mixture models and decoupled training approaches.
result Effective multimodal modeling of factors and news improves stock return prediction and selection.
Estimates prediction uncertainty in neural networks using density estimation in representation space.
problem Incorrect predictions with high confidence from models trained on limited data.
method Estimates training data density in representation space and uses it to predict model uncertainty.
result Detects out-of-distribution data without prior exposure, improving model reliability.
Improved AutoDML estimator for causal inference using outcome-adapted shared covariate representation.
problem Efficiency in estimating treatment or policy effects in causal inference.
method Outcome-adapted AutoDML estimator that uses a shared covariate representation that is predictive of the outcome but not the Riesz representer.
result Outcome-adapted AutoDML estimator is asymptotically more efficient than baseline AutoDML.
The study categorizes knowledge graph relations and explains their embedding representations.
problem Understanding how knowledge graph relation representations capture semantic information.
method Categorizing knowledge graph relations into three types and deriving explicit requirements for their representations.
result Empirical properties of relation representations and the performance of methods are justified by the analysis.
This paper fine-tunes LLMs for stock return prediction using financial news.
problem Improving stock return forecasting accuracy using LLMs.
method Fine-tuning LLMs with text and forecasting modules, comparing encoder-only and decoder-only models, and integrating token-level representations.
result LLMs' aggregated token-level embeddings enhance return predictions for long-only and long-short portfolios.
Self-Predictive Representations improves data-efficient reinforcement learning from limited interaction.
problem Efficient reinforcement learning from limited data.
method Train agents to predict future latent state representations using self-supervised objectives.
result Achieves a median human-normalized score of 0.415 on Atari with 100k steps of interaction, 55% improvement over previous state-of-the-art.
Model learns disentangled frames from video, enabling future frame prediction.
problem Learning disentangled representations from video sequences.
method Temporal coherence and adversarial loss for disentanglement.
result Model can predict future frames coherently.
Model predicts travel time under rare conditions using a vector-space model.
problem Predicting travel time under rare temporal conditions (e.g., holidays, school vacations) is challenging due to limited historical data and other temporal changes.
method Presented a vector-space model for encoding rare temporal conditions, allowing coherent representation learning across different conditions.
result Increased performance for travel time prediction over different baselines when using the vector-space encoding for representing the temporal setting.
New framework improves reliability of learned representations by modeling uncertainty and structural constraints.
problem Uncertainty in learned representations treated as deterministic, leading to unreliable models.
method Proposes a principled framework for reliable representation learning with uncertainty-aware regularization and structural constraints.
result Improves stability, calibration, and robustness of learned representations.
A framework for learning disentangled representations of symmetric environments.
problem Discovering and modelling the underlying structure of environments.
method Group representation theory for disentangled representations of dynamical environments.
result Our method enables accurate long-horizon predictions and correlates with disentanglement quality.
Improved differentially private drug sensitivity prediction using compact representations.
problem Challenges in differentially private machine learning with genomic data.
method Representation learning using variational autoencoders, PCA, and random projection.
result Variational autoencoders provide the most accurate predictions for differentially private drug sensitivity prediction.
Model uses LLM features to predict stock returns effectively.
problem Predicting stock returns from text data.
method Structured Event Representation (SER) model with attention mechanisms.
result SER-based model outperforms existing models in stock return prediction.
Language models improve clinical prediction models using EHR data.
problem Limited patient data for training clinical prediction models.
method Using patient representation schemes from natural language processing.
result 3.5% mean improvement in AUROC on five prediction tasks.
Develops a hybrid deep learning model for stock price prediction.
problem Predicting daily stock prices in the stock market.
method Representation learning with Stock2Vec embedding and temporal convolutional layers.
result Achieves better performance on stock price prediction than benchmarks.
Neural network feature optimization for causal inference.
problem Estimating heterogeneous treatment effects from data.
method Genetic algorithm optimization of intermediate neural network layers for feature representations.
result Retains useful features for outcome prediction even if related to treatment assignment.
This work explains how linear representations in large language models arise from training objectives and gradient descent.
problem Understanding the origins of linear representations in large language models.
method A latent variable model to abstract and formalize concept dynamics, combined with analysis of the softmax cross-entropy objective and gradient descent.
result Linear representations emerge when learning from data matching the latent variable model, and this simple structure suffices to yield linear representations.
Develops UKP for comparing feature representations in multitask learning.
problem Comparing feature representations learned by different models without access to test data.
method Uniform Kernel Prober (UKP) for comparing representations in kernel ridge regression tasks.
result UKP provides a uniform measure of prediction error on test data without access to test data.
DySAT learns dynamic graph node representations capturing structural and temporal patterns.
problem Learning latent representations of nodes in dynamic graphs.
method Dynamic Self-Attention Network (DySAT) that combines self-attention layers for structural and temporal dimensions.
result DySAT outperforms state-of-the-art baselines in link prediction on dynamic graphs.
KINet learns object interactions without supervision for robotic pushing.
problem Lack of supervised data for object-centric forward prediction.
method End-to-end unsupervised framework using keypoint representation and contrastive estimation.
result Automatically generalizes to unseen scenarios and accurately predicts future states.
GRAPE uses graph representation to handle missing data in feature imputation and label prediction.
problem Handling missing data in machine learning tasks.
method GRAPE uses a bipartite graph where observations and features are nodes, and observed feature values are edges. It formulates feature imputation as edge-level prediction and label prediction as node-level prediction, solving these with Graph Neural Networks.
result GRAPE achieves 20% lower mean absolute error for imputation and 10% lower for label prediction compared to state-of-the-art methods.
RPSP networks combine PSRs and RNNs for reinforcement learning in POE.
problem Learning in partially observable environments.
method Recurrent filter with PSR, reactive policy, gradient descent.
result RPSP networks outperform memory-preserving models.
Paper proposes a new method for learning business process representations.
problem Challenges in capturing all useful information in business process data.
method Combines Gramian Angular Fields and Convolutional Neural Networks for representation learning.
result Demonstrates effectiveness of the approach through visualization and multiple process prediction tasks.
This work proposes multiple node representations for graphs, improving link prediction and community analysis.
problem Can nodes be best described by a single vector representation?
method A principled decomposition of the ego-network to learn multiple node representations.
result Improved link prediction accuracy by up to 90% and effective community analysis.
Proposes Infomax and Domain-Independent Representations for robust causal inference.
problem Handling treatment selection bias and domain imbalance in causal inference with real-world data.
method Utilizes mutual information to learn domain-invariant representations that maximize predictive common information.
result Achieves state-of-the-art performance on causal effect inference across various data distributions.