PyTorch Frame simplifies multi-modal tabular learning with modular data and model handling.
problem Handling complex multi-modal tabular data in deep learning.
method A PyTorch-based framework that provides a data structure, model abstraction, and integration with external models.
result Demonstrated the effectiveness of PyTorch Frame in implementing and applying diverse tabular models to complex multi-modal tabular data.
Framework for handling long-tailed multi-modal data.
problem Class imbalance and long-tailed distributions in multi-modal data.
method Multi-expert architecture with modality-specific networks and dynamic fusion weights.
result Framework outperforms existing methods in long-tailed, class-imbalanced scenarios.
Deep learning enhances art market valuation by incorporating visual data.
problem Improving valuation accuracy in the art market, especially for first-time sales.
method Benchmarked classical and modern deep learning models using a large auction dataset.
result Visual embeddings add distinct economic value for first-time art sales.
FinTMMBench benchmarks RAG systems for finance tasks across multiple data types and time periods.
problem Evaluating temporal-aware multi-modal retrieval augmented generation in finance.
method TMMHybridRAG method that converts and integrates data from various modalities and temporal information.
result Demonstrated effectiveness of TMMHybridRAG in diverse financial analysis tasks.
The paper tackles interpreting DCM with image data by addressing data isomorphism.
problem Interpreting DCM with image data due to isomorphic information.
method Proposes and benchmarks two methodologies: architectural adjustments and data source mitigation.
result Direct data source mitigation is more effective for maintaining DCM's interpretability.
The paper analyzes and proposes an algorithm for multi-modal nonlinear embeddings with theoretical performance bounds.
problem Generalizability of multi-modal nonlinear embeddings to unseen data.
method Theoretical analysis and a multi-modal nonlinear representation learning algorithm motivated by performance bounds.
result The proposed algorithm yields promising performance in multi-modal image classification and cross-modal image-text retrieval applications.
Gradient-based methods improve understanding of deep learning survival models.
problem Limited interpretability of deep learning survival models hinders their adoption.
method Gradient-based explanation methods tailored to survival neural networks.
result Gradient-based methods capture feature effects and temporal dynamics.
Contrastive learning adapts to data intrinsic dimensions, learning low-dimensional representations.
problem Learning high-dimensional representations from multi-modal data.
method Multi-modal contrastive learning with temperature optimization.
result Contrastive learning adapts to intrinsic dimensions of data, not specified dimensions.
METEOR learns efficient representations from multi-modal data streams.
problem Efficiently interpreting multi-modal information in complex environments.
method METEOR learns compact representations by sharing parameters within semantically meaningful groups and preserving domain-agnostic semantics.
result METEOR reduces memory usage by around 80% compared to conventional methods.
New framework shows cross-attention improves multi-modal in-context learning.
problem Understanding multi-modal in-context learning in neural networks.
method Mathematical framework and linearized cross-attention mechanism.
result Cross-attention mechanism is provably optimal for multi-modal in-context learning.
MoCA uses a novel autoencoder to analyze multi-modal health data.
problem Challenges in analyzing continuous multi-modal health data from wearable devices.
method Proposes MoCA, a self-supervised learning framework combining transformer and masked autoencoder methods.
result Demonstrates strong performance boosts across reconstruction and classification tasks.
Adaptive anchor methods improve multi-modal learning by balancing intra-modal and inter-modal information.
problem Fixed anchor methods limit multi-modal learning by over-reliance on a single modality and inadequate cross-modal correlation.
method Adaptive anchor methods using centroid-based anchors from all modalities.
result Adaptive anchor methods like CentroBind consistently outperform fixed anchor methods across various datasets.
Develops multi-modal neural network models for improved prediction and uncertainty quantification.
problem Improving prediction accuracy and uncertainty quantification for multi-modal data.
method Multi-modal Bayesian neural network models with conjugate last-layer estimation using SVI.
result Improved prediction accuracy and uncertainty quantification compared to uni-modal models.
This paper reviews deep learning for multi-modality medical image segmentation.
problem Improving segmentation accuracy in medical images using multiple modalities.
method Overview of deep learning and multi-modal medical image segmentation, analysis of different network architectures and fusion strategies.
result Later fusion of modalities can lead to more accurate segmentation results.
Deep tabular models outperform GBDT in medical diagnosis tasks.
problem Transfer learning for tabular data in medical diagnosis.
method Proposes a pseudo-feature method for transfer learning between different feature sets.
result Tabular neural networks outperform GBDT in medical diagnosis tasks.
MET learns tabular data representations without data augmentations.
problem Lack of effective self-supervised learning methods for tabular data.
method Reconstruction-based approach using masked encoding, with separate representations for each coordinate and adversarial reconstruction loss.
result MET achieves state-of-the-art performance on five diverse tabular datasets, improving up to 9% over current methods.
This paper learns multi-modal embeddings from text, audio, and video views/modes of data in order to improve upon down-stream sentiment classification. The experimental framework also allows investigation of the relative contributions of the individual views in the final multi-modal embedding. Individual features deriv…
This work improves multi-modal generative models by using permutation-invariant neural networks.
problem Improving multi-modal generative models with tighter variational objectives.
method Developed more flexible aggregation schemes based on permutation-invariant neural networks.
result Our variational objective and flexible aggregation models can better approximate the true joint distribution.
This study optimizes multi-modal learning thresholds and algorithms in high dimensions.
problem Optimizing multi-modal learning performance in high-dimensional data.
method Analytical quantification and derivation of AMP algorithm with state evolution analysis.
result Bayes-optimal performance and recovery thresholds derived for multi-modal data.
Study reveals efficient recovery of multi-modal signals via Bayesian methods and sequential learning.
problem Recovering multiple high-dimensional signals from correlated modalities.
method Bayesian Approximate Message Passing and Sequential Curriculum Learning.
result Sequential learning strategy optimally recovers weak signals in multi-modal settings.
Orion-Bix combines biaxial attention and meta-learning for tabular few-shot learning.
problem Scaling and generalizing tabular models with mixed numeric and categorical fields, weak feature structure, and limited labeled data.
method Orion-Bix uses biaxial attention and meta-learned in-context reasoning to efficiently capture local and global dependencies.
result Orion-Bix outperforms gradient-boosting baselines and state-of-the-art tabular models on public benchmarks.
AF improves sampling from high-dimensional, multi-modal distributions.
problem Sampling from high-dimensional, multi-modal distributions is challenging.
method Annealing Flow (AF) using Continuous Normalizing Flow (CNF) with dynamic Optimal Transport (OT) objective and annealing procedures.
result AF significantly improves training efficiency and stability, outperforming state-of-the-art methods.
Human behavior expression and experience are inherently multi-modal, and characterized by vast individual and contextual heterogeneity. To achieve meaningful human-computer and human-robot interactions, multi-modal models of the users states (e.g., engagement) are therefore needed. Most of the existing works that try t…
BiSHop tackles tabular data challenges with sparse Hopfield layers.
problem Non-rotationally invariant data structure and feature sparsity in tabular data.
method Sequential column-wise and row-wise processing through interconnected directional learning modules with generalized sparse modern Hopfield layers.
result BiSHop surpasses current SOTA methods with significantly less hyperparameter tuning.
A new method for learning to defer decisions with expert advice improves over standard methods.
problem Learning to defer decisions with expert advice in systems where expert information can be modified after selection.
method An augmented surrogate that operates on the composite expert-advice action space, providing consistency guarantees and excess-risk bounds.
result The method improves over standard Learning-to-Defer and adapts its advice acquisition behavior to the cost regime.
Improves tabular data augmentation for contrastive learning.
problem Ineffective augmentation techniques for tabular data.
method Class-conditioned and feature-correlation based augmentation.
result Consistently outperforms conventional corruption methods.
Paper introduces topological eigenvalue theorems for tensor analysis in multi-modal data.
problem Lack of deep understanding of tensor structures in multi-modal data fusion.
method Introduces topological perspective to tensor eigenvalue analysis, linking eigenvalues to topological features.
result Establishes new theorems that enhance understanding of tensor structures in data fusion.
This paper proposes a method to gain extra supervision via multi-task learning for multi-modal video question answering. Multi-modal video question answering is an important task that aims at the joint understanding of vision and language. However, establishing large scale dataset for multi-modal video question answeri…
Predict stock price movements using financial data and news articles with LLMs.
problem Predicting stock price movements using financial data and news articles.
method Combining financial data and news articles, employing pre-trained LLMs, and using retrieval augmentation techniques.
result Predicted stock price movements with a weighted F1-score of 58.5% and 59.1%.
SAINT improves neural networks for tabular data with row attention and contrastive pre-training.
problem Tabular data challenges in machine learning applications.
method SAINT combines row and column attention with contrastive self-supervised pre-training.
result SAINT outperforms previous deep learning methods and even gradient boosting methods on benchmark tasks.
AutoDiff combines auto-encoder and diffusion model for realistic tabular data synthesis.
problem Generating realistic synthetic tabular data with heterogeneous features.
method Employing auto-encoder architecture to handle tabular data's complexity.
result Synthetic tables from AutoDiff show good statistical fidelity and perform well in machine learning tasks.
A new algorithm flattens multi-modal distributions for better deep learning.
problem Bayesian learning in big data with multi-modal distributions.
method Contour Stochastic Gradient Langevin Dynamics (CSGLD) algorithm.
result The CSGLD algorithm avoids local traps in deep neural networks.
Triple-GAIL learns from multiple sources to improve imitation learning for complex behaviors.
problem Limited scalability of GAIL in real-world scenarios like autonomous vehicles.
method Integrates expert demonstrations and generated experiences with an auxiliary skill selector.
result Triple-GAIL outperforms state-of-the-art methods in learning complex behaviors.
Imputation-free method learns tabular data with missing values using transformer.
problem Machine learning on tabular data with missing values often leads to unreliable outcomes due to synthetic imputation.
method Incremental attention learning (IFIAL) using transformer with attention masks.
result IFIAL outperforms state-of-the-art methods in 17 diverse tabular data sets.
Method predicts NBA players' multi-modal movement trajectories.
problem Understanding NBA players' decision-making during games.
method LSTM-based architecture with multi-modal loss function.
result Method outperforms state-of-the-art in predicting realistic trajectories.
Clinical diagnostic decision making and population-based studies often rely on multi-modal data which is noisy and incomplete. Recently, several works proposed geometric deep learning approaches to solve disease classification, by modeling patients as nodes in a graph, along with graph signal processing of multi-modal …
SubTab turns tabular data into a multi-view problem for better representation learning.
problem Lack of structure in tabular data makes it hard to apply effective self-supervised learning methods.
method Divides tabular features into subsets and uses autoencoder-like reconstruction for collaborative inference.
result SubTab achieves state-of-the-art performance on tabular datasets, matching or surpassing CNN-based methods.
This work improves manifold learning for multi-modal data.
problem Distortions and modeling errors in multi-modal data.
method Isometrizing learned Riemannian structure and balancing regularity and expressivity.
result The synergy of proposed approaches enhances manifold learning.
Diffusion models learn multi-modal distributions with optimal efficiency.
problem Learning high-dimensional distributions with low-dimensional multi-modal structures.
method Score-based diffusion models, focusing on subgaussian distributions within subspaces.
result Diffusion models require O ~ ( ε − k ∨ 2 ) \widetilde{O}(\varepsilon^{-k \vee 2}) O ( ε − k ∨ 2 ) samples for 1-Wasserstein ε \varepsilon ε error, improving over prior guarantees. Framework for causal discovery using multi-modal data.
problem Failure of representation learning in causal tasks.
method Statistical and computational framework combining representation learning and causal inference.
result Effective use of observational and perturbational data for causal discovery.
We propose a novel high-performance and interpretable canonical deep tabular data learning architecture, TabNet. TabNet uses sequential attention to choose which features to reason from at each decision step, enabling interpretability and more efficient learning as the learning capacity is used for the most salient fea…
TabNAS improves neural architecture search for tabular datasets by rejecting suboptimal architectures.
problem Finding optimal neural architectures for tabular datasets with resource constraints.
method Develops a reinforcement learning controller motivated by rejection sampling to handle resource constraints.
result TabNAS finds better models that obey resource constraints compared to previous methods.
GAMformer bridges tabular models and interpretability, offering a single-pass approach.
problem Lack of interpretability in tabular foundation models like TabPFN.
method In-context learning for GAM shape functions, training on synthetic data.
result GAMformer performs comparably to other leading GAMs across various classification benchmarks.
New divergences improve score-based methods for multi-modal distributions.
problem Blindness problem in score-based divergences for multi-modal distributions.
method Proposed a new family of divergences to mitigate blindness.
result Improved performance in density estimation compared to traditional approaches.
Recent work on imitation learning has generated policies that reproduce expert behavior from multi-modal data. However, past approaches have focused only on recreating a small number of distinct, expert maneuvers, or have relied on supervised learning techniques that produce unstable policies. This work extends InfoGAI…
Paper analyzes deep learning models for credit rating prediction using text and numerical data.
problem Improving credit rating prediction using multi-modal deep learning.
method Testing different deep learning models and fusion strategies for structured and unstructured datasets.
result CNN-based multi-modal model with two fusion strategies outperformed other models.
Fed-TDA augments federated tabular data to improve performance and privacy.
problem Non-IID data challenges in federated learning.
method Synthesizes tabular data using simple statistics for augmentation.
result Fed-TDA improves test performance and communication efficiency.
A new memory-based fusion layer improves multi-modal deep learning performance.
problem Improving performance of multi-modal deep learning by addressing long-term dependencies.
method Introducing a Memory based Attentive Fusion (MBAF) layer that incorporates both current and long-term dependencies.
result The MBAF layer enhances fusion and improves performance across different modalities and networks.