Study presents a dataset and evaluation framework for representation learning in complex multimodal systems.
problem Lack of large-scale standard datasets for representation learning in complex multimodal systems.
method Implemented and compared several approaches to representation learning on a large-scale dataset for landing an airplane.
result Representations can be used for various applications including anomaly detection and optimal control.
A new HGNN framework for complex data representation learning.
problem Learning representation for complex data in hypergraph structures.
method Designing hyperedge convolution operations for hypergraph learning.
result HGNN outperforms state-of-the-art methods in citation network and visual object recognition tasks.
Better data representations can simplify learning tasks by aligning model distributions with true data distributions.
problem Learning complexity influenced by the alignment of model distributions with true data distributions.
method Analyzed the effect of data representations on learning complexity using a task complexity score and information coding length.
result Better representations can simplify learning tasks by aligning model distributions with true data distributions, improving learning outcomes.
New algorithms extract low-dimensional representations from sequential data, revealing insights into complex processes.
problem Challenges in extracting low-dimensional representations from sequential, high-dimensional, sparse, and noisy data.
method Developed new clustering algorithms based on Block Markov Chains theory, validated on real-world data.
result These algorithms can successfully extract low-dimensional representations from real-world sequential data, revealing insights into complex processes.
New methods evaluate data representations by complexity of low-loss predictor learning.
problem Evaluating quality of data representations for downstream tasks.
method Surplus Description Length (SDL) and ε Sample Complexity (εSC) methods.
result Methods measure the information needed to approximate optimal predictor up to specified tolerance.
IsUMap improves data visualization of complex geometries.
problem Accurately representing complex, locally distorted metric spaces.
method Integrates UMAP and Isomap with Vietoris-Rips filtrations.
result Significant improvements in data representation quality.
Improves NF for complex data distributions with multiple modes.
problem Difficulty in handling data distributions with multiple isolated modes.
method Proposes a new framework using variational latent representation to improve NF.
result Significantly more powerful for generating data distributions with multiple modes.
We consider a set of probabilistic functions of some input variables as a representation of the inputs. We present bounds on how informative a representation is about input data. We extend these bounds to hierarchical representations so that we can quantify the contribution of each layer towards capturing the informati…
This thesis investigates unsupervised time series representation learning for sequence prediction problems, i.e. generating nice-looking input samples given a previous history, for high dimensional input sequences by decoupling the static input representation from the recurrent sequence representation. We introduce thr…
UNTIE learns representations of coupled categorical data.
problem Challenges in learning from unlabeled categorical data with complex couplings.
method UNTIE approach for unsupervised representation learning of heterogeneous couplings.
result UNTIE significantly improves categorical data representations on 25 diverse datasets.
Unified theory for representation learning using learnable functions.
problem Insufficient theoretical understanding of unsupervised and self-supervised learning.
method Discriminative theoretical framework for analyzing sample complexity.
result Learnable regularization functions can reduce the amount of labeled data needed.
Tactile information is important for gripping, stable grasp, and in-hand manipulation, yet the complexity of tactile data prevents widespread use of such sensors. We make use of an unsupervised learning algorithm that transforms the complex tactile data into a compact, latent representation without the need to record g…
Combining insights from neural networks and neuroscience to understand complex tasks.
problem Understanding how biological and artificial neural networks learn and solve complex tasks.
method Review of data-analysis techniques from computational neuroscience applied to DNNs, and vice versa.
result Opportunities for synergy between machine learning and neuroscience to enhance understanding of neural representations.
Characterizes representations for complex projective structures with specific branch data.
problem Understanding representations of surface groups as holonomy of complex projective structures.
method Computing holonomies for spherical metrics and affine structures with prescribed conical angles.
result Computed holonomies for spherical metrics and affine structures with specific conical angles.
FairMixRep learns fair representations from mixed data types.
problem Representation learning in mixed numerical and categorical data with fairness constraints.
method Efficient encoder-decoder framework + fairness constraints.
result Excellent performance in preserving information and fairness in mixed data representations.
This paper improves few-shot learning by reducing sample complexity using representation learning.
problem Reducing sample complexity for target tasks with limited data.
method Representation learning to pool all source task samples for target task learning.
result Representation learning can achieve substantial sample size reduction, bypassing the $Ω( rac{1}{T})$ barrier.
Interpretable representations improve explainable AI by translating complex data into understandable concepts.
problem Many explainers use interpretable representations but overlook their full potential and assumptions.
method An in-depth analysis of interpretable representations for tabular, image, and text data, identifying strengths, weaknesses, and desiderata.
result Linear model quantifies interpretable concepts' influence on black-box predictions, revealing their explanatory properties and manipulability.
MMD-B-Fair learns fair representations by minimizing MMD test power.
problem Learning fair representations of data while preserving target attributes.
method Kernel two-sample testing and block testing schemes.
result Minimizing MMD test power allows hiding sensitive attribute information.
Novel algorithm learns sparse signal representations over topological spaces.
problem Sparse representation of signals over combinatorial topological spaces.
method Leveraging Hodge theory, the paper embeds topology into a dictionary structure via concatenated sub-dictionaries, each as a polynomial of Hodge Laplacians, and optimizes the dictionary coefficients and sparse signal representation via iterative alternating algorithms.
result Efficiently learned sparse representations and underlying relational structure of topological signals.
This paper proposes a new method for learning covers of geometric datasets to improve topological inference and visualization.
problem Improving topological inference and visualization of large-scale geometric datasets.
method Proposes a method for learning topologically-faithful covers of geometric datasets using optimization.
result Simplicial complexes obtained from learned covers outperform standard methods in terms of size and representation of large-scale topology.
Paper proposes a method to learn linear regression models using multiple pre-trained models.
problem Learning a linear regression model with limited target data.
method Representation transfer learning method using multiple pre-trained models.
result The method achieves better sample complexity compared to baseline methods.
Neural networks benefit from intermediate representations, reducing sample complexity.
problem Understanding how neural networks leverage intermediate representations for hierarchical learning.
method Fixed, randomly initialized neural network as a representation function, compared with raw inputs and other trainable networks.
result Neural representations can achieve improved sample complexities compared to raw inputs, especially for low-rank polynomials.
AUC-spec optimizes graph-based SSL for complex label distributions.
problem Training accurate models with scarce labeled data and abundant unlabeled data.
method Computes a low-dimensional representation that maximizes class separation via AUC optimization.
result AUC-spec achieves competitive results on synthetic and real-world datasets.
SOLAR learns efficient representations for RL in complex image domains.
problem Efficient model-based reinforcement learning in domains with complex observations like images.
method Optimizes structured representations for inferring simple dynamics and cost models from data.
result Substantially better final performance than other model-based RL methods, more efficient than model-free RL.
Graph representation converts complex networks into vectors for easier analysis.
problem Efficient analysis of large network data.
method Introduces graph representation and network embedding models.
result Efficiently converts graph data into low-dimensional vectors.
Proposes a method to learn representations of higher-dimensional simplicial complexes.
problem Lack of methods for representing entire simplicial complexes.
method Geometric message passing schemes for end-to-end learning of simplicial complex representations.
result First method for learning representations of entire simplicial complexes.
VAE learns latent representations for bank customers' creditworthiness.
problem Improving marketing, CRM, and credit risk assessment in retail banks.
method Adopted Variational Autoencoder (VAE) to learn latent representations.
result VAE latent representations capture customers' creditworthiness and generalize to new data.
New statistical guarantees for transfer learning with diverse tasks.
problem Statistical guarantees for transfer learning across different tasks.
method Formal analysis of t+1 tasks with shared representation. result Sample complexity reduction for new tasks using shared representation.
The goal of unsupervised representation learning is to extract a new representation of data, such that solving many different tasks becomes easier. Existing methods typically focus on vectorized data and offer little support for relational data, which additionally describe relationships among instances. In this work we…
We review the problem of defining and inferring a "state" for a control system based on complex, high-dimensional, highly uncertain measurement streams such as videos. Such a state, or representation, should contain all and only the information needed for control, and discount nuisance variability in the data. It shoul…
Adversarial training leads to clean data generalization with significant robust overfitting gap.
problem Significant robust generalization gap in adversarial training.
method Two theoretical views: representation complexity and training dynamics.
result ReLU nets with O(ND) extra parameters can achieve CGRO. This paper proposes a model to learn multimodal representations robust to missing data.
problem Learning multimodal representations from heterogeneous sources of information.
method Optimizes a joint generative-discriminative objective across multimodal data and labels, factorizing representations into multimodal discriminative and modality-specific generative factors.
result The proposed model achieves state-of-the-art performance on six multimodal datasets and can reconstruct missing modalities without significant performance drop.
This paper critically examines unsupervised disentangled representation learning, revealing challenges and limitations.
problem The difficulty of unsupervised learning of disentangled representations and the challenges in evaluation metrics.
method Theoretical analysis and a large-scale experimental study covering 8 datasets and 14000 models.
result Well-disentangled models cannot be identified without supervision, and different evaluation metrics disagree on what constitutes disentanglement.
Discriminative clustering learns from both labeled and unlabeled data.
problem Clustering complex datasets with limited labeled data.
method Gradient-based stochastic training and optimal transport with entropic regularization.
result The method can learn feature representations even in fully unsupervised settings.
New framework minimizes model complexity for improved few-shot learning.
problem Empirical benefits of pre-training scale with data size but lack theoretical explanation.
method Complexity Minimization framework for meta-representation learning.
result Theoretical analysis shows error rate improves with more meta-training data.
New methods compare neural network models using geometric and topological summaries.
problem Comparing deep representations of complex networks in models and brains.
method Develops inference methods based on topological data analysis (TDA) and graph-based techniques.
result New statistical methods enable better model comparison and inference.
Novel tRSA combines geometry and topology for brain and model analysis.
problem Traditional RSA overlooks topological information in neural representations.
method Topological RSA (tRSA) using nonlinear monotonic transforms.
result Robust model comparisons and novel insights into neural computation.
FibeRed reduces complex data dimensions while preserving topology.
problem Hard embedding of topologically complex datasets in low-dimensional Euclidean space.
method Modeling datasets with vector bundles, reducing fibers while preserving topology.
result FibeRed learns topologically faithful embeddings in lower dimensions than existing methods.
Paper studies complex Lagrangian surfaces and their relation to SL(3,C)-representations.
problem Minimal Lagrangian surfaces in bi-complex hyperbolic space and their representations.
method Introduces bi-complex Higgs bundles and parameterizes SL(3,C)-quasi-Fuchsian representations. result Parameterization of SL(3,C)-quasi-Fuchsian representations by an open set in Teichmüller space. SSL theory improves representation learning from raw data.
problem Challenges in SSL, including instability and collapse.
method Precise analysis of generalization performance with a theory-friendly setup.
result Insights for SSL practitioners on data augmentation, network architecture, and training algorithm.
The paper challenges the unsupervised learning of disentangled representations, showing it's fundamentally impossible without biases.
problem The unsupervised learning of disentangled representations is fundamentally impossible without inductive biases.
method Theoretical analysis and a large-scale experimental study on seven different datasets.
result Well-disentangled models cannot be identified without supervision, and increased disentanglement does not decrease sample complexity.
Proposes a deep learning method for effective data representation.
problem Constructing effective data representations for prediction.
method A deep dimension reduction approach to learning representations with sufficiency, low dimensionality, and disentanglement.
result The proposed deep nonparametric representation is consistent and performs better than existing methods.
A grand challenge in representation learning is to learn the different explanatory factors of variation behind the high dimen- sional data. Encoder models are often determined to optimize performance on training data when the real objective is to generalize well to unseen data. Although there is enough numerical eviden…
The paper provides guarantees for learning nonlinear representations from multiple non-identically distributed data sources.
problem Learning from non-identically distributed and dependent data.
method Established statistical guarantees for learning general nonlinear representations from multiple data sources.
result The excess risk of the estimated function decays as a function of the sample complexity and task diversity.
Review of mathematical representations for biomolecular data.
problem Complexity and high dimensionality of biomolecular datasets hinder ML applications.
method Developed low-dimensional and scalable mathematical representations using algebraic topology, differential geometry, and graph theory.
result Mathematical representations improve protein-ligand binding predictions and other biomolecular applications.
New method computes optimal fairness-performance trade-off without complex models.
problem Intrinsic trade-off between fairness and classifier performance.
method Computes optimal Pareto front without training complex models.
result Optimal fair representations have useful structural properties enabling efficient computation.
Learning discrete representations of data is a central machine learning task because of the compactness of the representations and ease of interpretation. The task includes clustering and hash learning as special cases. Deep neural networks are promising to be used because they can model the non-linearity of data and s…
Representation learning has become an invaluable approach for learning from symbolic data such as text and graphs. However, while complex symbolic datasets often exhibit a latent hierarchical structure, state-of-the-art methods typically learn embeddings in Euclidean vector spaces, which do not account for this propert…