Bayesian VFLMSP improves multimodal survival prediction with privacy.
problem Privacy and reliability in multimodal time-to-event prediction.
method Bayesian Vertical Federated Learning (VFL) with differential privacy.
result Consistent improvements in C-index compared to existing methods.
Unified approach for multimodal data prediction using synthetic data generation.
problem Challenges in integrating heterogeneous data types for accurate predictive performance.
method Generative Distribution Prediction (GDP) framework that uses multimodal synthetic data generation.
result Empirical validation across four tasks demonstrates versatility and effectiveness of GDP.
Study improves stock movement prediction using multimodal data.
problem Inaccurate stock movement prediction due to incomplete multimodal data integration.
method Introduces MSGCA framework for robust multimodal fusion.
result MSGCA framework outperforms existing methods by 21.7% on multimodal datasets.
Deep learning predicts mental disorders from audio and text samples.
problem Predicting mental disorders from speech samples.
method Multimodal deep learning structure using various pre-trained models for audio and text embeddings, transfer learning, and auxiliary corpora.
result Acceptable accuracy in predicting mental disorders through multimodal analysis.
This paper proposes a model to learn multimodal representations robust to missing data.
problem Learning multimodal representations from heterogeneous sources of information.
method Optimizes a joint generative-discriminative objective across multimodal data and labels, factorizing representations into multimodal discriminative and modality-specific generative factors.
result The proposed model achieves state-of-the-art performance on six multimodal datasets and can reconstruct missing modalities without significant performance drop.
RiskLabs uses LLMs to predict financial risks from multimodal data.
problem Financial risk prediction using AI techniques.
method Integrates multimodal financial data (textual, vocal, time series, news) into LLMs for prediction.
result Empirical results show effectiveness in forecasting market volatility and variance.
New method for multi-modal depth prediction challenges.
problem Challenges in applying unimodal coreset selection to multi-modal data.
method Adapted state-of-the-art coreset selection technique for multimodal data.
result Challenges in extending unimodal algorithms to multi-modal scenarios.
Model predicts Bitcoin's future movements using multimodal pattern matching.
problem Challenges in predicting Bitcoin's volatile future movements.
method Ranking similar past chart patterns given current chart information.
result Improves directional prediction of Bitcoin's future movements.
Multimodal deep learning improves toxicity prediction accuracy.
problem Improving prediction accuracy of chemical compound toxicity.
method Combining multiple neural network types and data representations.
result Significantly better accuracy on a toxicity benchmark.
CoverNet predicts urban driving trajectories using diverse sets of possible actions.
problem Multimodal probabilistic trajectory prediction for urban driving.
method Frame trajectory prediction as classification over a diverse set of trajectories; dynamically generate sets based on current state.
result Outperforms state-of-the-art methods on real-world self-driving datasets.
This paper improves stock price prediction using multimodal data.
problem Accurate stock price prediction with diverse data integration.
method Combining financial metrics, tweets, and news articles through multimodal machine learning.
result Significant performance improvement in stock price prediction by up to 5%.
Proposes a novel network for CTR prediction by learning modality-specific and modality-invariant representations.
problem Learning good representation of items from multimodal features in E-commerce is challenging due to redundant information across modalities.
method Introduces a Multimodal Adversarial Representation Network (MARN) that calculates modality-specific weights and learns modality-invariant representations.
result Consistently achieves remarkable improvements over state-of-the-art methods in CTR prediction.
DECAT framework evaluates multimodal models for shared biology, detecting confounders and false positives.
problem Determining if multimodal models learn shared biology or just confounders.
method DECAT framework classifies multimodal representations into four diagnostic scenarios using null-referenced metrics.
result DECAT detects confounders and false positives in multimodal models, improving with larger cohorts and stronger representations.
Enhances stock movement prediction using Higher Order Transformers for multimodal time-series data.
problem Predicting stock movements in financial markets with complex dynamics.
method Introduced Higher Order Transformers, extending self-attention and transformer architecture to capture complex market dynamics. Employed low-rank tensor decomposition and kernel attention to manage computational complexity. Integrated technical and fundamental analysis from historical prices and tweets.
result Demonstrated effectiveness of the method on the Stocknet dataset, improving stock movement prediction.
This study improves stock price prediction using multimodal data.
problem Improving financial asset price forecasting accuracy.
method Combining candlestick time series and textual news flow data using LSTM and pre-trained models.
result Textual modality reduces MAPE by 55%.
Survey of multimodal deep generative models for diverse data types.
problem Inference of shared representations and cross-modal generation from heterogeneous multimodal data.
method Variational autoencoders and other deep generative models.
result A comprehensive survey of multimodal deep generative models.
The study develops multimodal models to predict 1-year mortality risk from large clinical datasets.
problem Limited datasets in biomedical studies that do not generalize over large heterogeneous datasets.
method Develops multimodal models using a massive clinical dataset of 25 million videos and 2.9 million ECG traces.
result Extremely low-parameter models with optimized feature selection achieve AUC of 0.89.
Study predicts traffic congestion based on population mobility data.
problem Predicting traffic congestion in multimodal transport networks.
method Machine learning methods applied to population mobility data.
result Likely prediction of congestion based on population movements.
DMAN embeds social images with deep multimodal attention networks.
problem Sub-optimal social image representation for social media data.
method Deep Multimodal Attention Networks (DMAN) that jointly embed multimodal contents and link information.
result DMAN achieves significant improvement in multi-label classification and cross-modal search compared to state-of-the-art image embeddings.
Paper presents a fast method to generate multimodal embeddings.
problem Integrating visual and linguistic information into a single representation.
method Learning a language-to-vision mapping to build multimodal embeddings.
result Mapped vectors outperform unimodal and multimodal baselines, especially in zero-shot settings.
DAIF learns fusion structure from data to improve multimodal supervised learning.
problem Tackles the challenge of determining optimal fusion granularity across heterogeneous data sources.
method Combines random matrix theory and non-parametric dependence measures to learn fusion structure directly from data.
result DAIF outperforms state-of-the-art techniques in predicting T-cell differentiation marker protein expression and patient survival.
Study proposes a multimodal model for cardiovascular risk prediction using EHRs.
problem Lack of comprehensive risk prediction from EHRs due to unstructured text.
method Proposes a multimodal BiLSTM model integrating structured and unstructured EHR data.
result Proposed BiLSTM model outperforms other DNN architectures in cardiovascular risk prediction.
GGMPs improve non-Gaussian conditional density estimation.
problem Multimodality, heteroscedasticity, and strong non-Gaussianity in conditional density estimation.
method GGMP combines local Gaussian mixture fitting, cross-input component alignment, and per-component heteroscedastic GP training.
result GGMPs improve distributional approximation on synthetic and real-world datasets.
MKBE embeds multimodal data for knowledge base completion.
problem Missing multimodal data in knowledge bases.
method Multimodal encoders and decoders for text, images, and numerical values.
result State-of-the-art link prediction with 5-7% improvement.
Simple baselines improve multimodal utterance learning.
problem Learning rich multimodal utterance representations.
method Conditional factorization of utterances into unimodal factors; extending to bimodal and trimodal factors.
result Optimal embeddings can be derived in closed form.
Unified framework explains few-shot multimodal medical imaging performance.
problem Limited labeled data in rare diseases and low-resource settings.
method PAC learning, VC theory, PAC Bayesian analysis, information gain, Chain of Thought reasoning.
result Unified theoretical framework for few-shot multimodal medical imaging.
The paper introduces multimodal generative models to improve data marginal likelihood.
problem Improving data marginal likelihood in multimodal settings.
method Derives variational bounds on the evidence for multimodal deep generative models, generalizes objectives for different model types, and benchmarks across various datasets.
result Multimodal VAEs excel in image, label, and text datasets with and without weak supervision.
DeepSIP predicts network failures' impact using CNN from syslog and traffic data.
problem Predicting service impact from network failures.
method Temporal multimodal CNN for predicting time to recovery and traffic loss.
result DeepSIP reduced prediction error by approximately 50%.
Study uses multimodal machine learning to predict ICD-10 codes.
problem Improving accuracy and interpretability of ICD-10 code predictions.
method Developed separate models for text and tabular data, integrated using an ensemble method.
result Best-performing model achieved micro-F1 of 0.7633 and micro-AUC of 0.9541.
A framework for uncertainty-aware multimodal learning using conformal Shapley intervals.
problem Uncertainty and modality level importance in multimodal learning.
method Introduces conformal Shapley intervals to quantify modality level importance and uncertainty.
result Demonstrates meaningful uncertainty quantification and strong predictive performance.
Improved multimodal learning with Gated Multimodal Units.
problem Finding an intermediate representation from multiple data sources.
method Gated neural networks for multimodal fusion.
result GMU outperformed single-modality approaches and other fusion strategies.
New model uses financial filings to predict bankruptcy, even without MDA sections.
problem Lack of complete MDA data limits traditional bankruptcy prediction models.
method Conditional Multimodal Discriminative (CMMD) model learns from accounting, market, and textual data.
result Empirical results show superior classification performance compared to traditional models.
Study improves stock price prediction by integrating international markets using deep learning.
problem Complex cross-correlation between international stock markets.
method Multimodal deep learning to forecast stock prices.
result Early and intermediate fusion models outperform late fusion and single modality models.
Generative Score Inference improves uncertainty quantification for multimodal data.
problem Accurate uncertainty quantification in multimodal learning tasks.
method Generative Score Inference (GSI) uses synthetic samples to approximate conditional score distributions.
result GSI achieves state-of-the-art performance in hallucination detection and image captioning uncertainty estimation.
Study creates a multimodal learning framework for CVD risk prediction.
problem Predicting cardiovascular disease risk in diverse populations.
method Combines cross modal transformers, graph neural networks, and causal representation learning.
result Model predicts personalized CVD risk with causal invariance across subpopulations.
PLS-SVD struggles with missing data in multimodal datasets, showing a phase transition in performance.
problem Missing data in PLS-SVD for multimodal datasets.
method Replica-symmetric analysis of spiked rectangular random matrices with missing entries.
result PLS-SVD performance transitions from uninformative to informative singular vectors at a critical signal-to-noise threshold.
New method disentangles shared and private latent factors in multimodal data.
problem Challenges in disentangling shared and private latent factors in multimodal data.
method Proposes a modification to existing multimodal Variational Autoencoders (MMVAE) to better handle modality-specific variation.
result Demonstrates improved robustness of modified MMVAE to modality-specific variation.
Paper learns multimodal transition dynamics for RL using conditional VI.
problem Learning stochastic, multimodal transition dynamics in RL.
method Conditional variational inference (VI) for complex stochasticity.
result VI successfully predicts multimodal outcomes but ignores deterministic parts.
AdapDISCOM tackles high-dimensional multimodal data with missingness and errors, improving prediction and biomarker selection.
problem High-dimensional multimodal data with block-wise missingness and measurement errors.
method AdapDISCOM introduces modality-specific weighting schemes to address heterogeneity and error magnitudes.
result AdapDISCOM consistently outperforms existing methods under heterogeneous contamination and heavy-tailed distributions.
Unified model predicts stock and systemic risks from diverse financial data.
problem Isolating financial tasks leads to missed cross-scale dependencies.
method Shared Transformer backbone with modular task heads for cross-modal attention and multi-task optimization.
result Uni-FinLLM significantly outperforms baselines in stock forecasting, credit-risk assessment, and systemic-risk detection.
MSB framework improves survival prediction in immunotherapy patients with missing data.
problem High dimensionality and blockwise missingness in multimodal clinical data.
method MSB is a late-fusion framework that independently models modality-specific features before aggregating predictions via cross-validated stacking.
result MSB outperformed baseline algorithms in predicting progression-free survival in lung cancer patients.
HCL learns shared and modality-specific latent representations for multimodal data.
problem Binary shared-private decomposition inadequately represents shared information across subsets of modalities.
method Hierarchical Contrastive Learning framework combining latent-variable formulation, structural sparsity, and contrastive objective.
result HCL accurately recovers hierarchical structure and improves predictive performance on multimodal data.
AACE learns treatment policies from EHRs using annotations to improve accuracy.
problem Learning treatment policies from multimodal EHRs with bias and inefficiency.
method Annotation-assisted coarsened effects (AACE) method.
result AACE outperforms existing methods in predicting treatment benefit from multimodal EHRs.
Deep LSTM models predict weight objectives from sleep and step data.
problem Predicting weight objectives from multimodal time-series data.
method Design and optimise cross-modal LSTM architectures, including X-LSTM.
result X-LSTM improves prediction accuracy with fewer parameters.
CQNPs enhance predictive performance and distribution modeling using quantile regression.
problem Limited predictive likelihood of Gaussian models for complex distributions.
method Introducing Conditional Quantile Neural Processes (CQNPs) that focus on estimating informative quantiles.
result Significant improvements in predictive performance and better modeling of multimodal distributions.
iStructTab improves multimodal learning by optimizing feature sequencing.
problem Redundancy, dispersion, and generalization issues in multimodal learning of images and tabular data.
method Graph-Enhanced Descriptor Sequencing (GEDS) algorithm that refines statistical descriptors through similarity graph-based computations.
result iStructTab effectively minimizes feature dispersion, improving predictive performance and robustness.
New method handles missing data in multimodal brain imaging.
problem Missing data in multimodal brain imaging.
method Full Information Linked ICA (FI-LICA) algorithm.
result FI-LICA outperforms current practices in classification and prediction.
Bayesian neural networks reveal multimodal predictive distributions.
problem Uncertainty quantification and interpretability in neural networks.
method Discretized prior for inner layer weights, Gaussian mixture approximation of posterior predictive distribution.
result Distinct parameter realizations can produce the same training error but different posterior predictive distributions.