The paper examines the reliability of limit order book representations in the face of data perturbation.
problem The reliability of limit order book representations under data perturbation.
method Experimental analysis of existing representations and guidelines for future research.
result Existing representations of limit order book data are vulnerable to data perturbation.
This work characterizes how data augmentation shapes neural representations.
problem Understanding the impact of data augmentation on neural network representations.
method Embedding neural network hidden representations into a metric space invariant to transformations, analyzing shape-space trajectories.
result Increasing data augmentation strength leads to well-behaved trajectories in the embedded space, and different augmentation types steer representations in distinct directions.
Unified data representation learning improves non-parametric two-sample testing.
problem Improving non-parametric two-sample testing accuracy.
method Proposes RL-TST framework combining IRs and DRs for better test power.
result RL-TST outperforms existing methods by leveraging both IRs and DRs.
New research shows fair data representations are impossible for different tasks.
problem Achieving fairness in machine learning models trained on various tasks.
method Analyzing the limits of fair data representations.
result No representation can guarantee fairness for different tasks trained on it.
A method to prevent image representation collapse through data-dependent augmentation.
problem Representation collapse due to image augmentations that damage information.
method Formalizing a stochastic encoding process with a tug-of-war between corruption and preserved information, using infoMax objective.
result Learning a data-dependent distribution of augmentations to avoid representation collapse.
UNTIE learns representations of coupled categorical data.
problem Challenges in learning from unlabeled categorical data with complex couplings.
method UNTIE approach for unsupervised representation learning of heterogeneous couplings.
result UNTIE significantly improves categorical data representations on 25 diverse datasets.
Proposes a new method for medical diagnosis using network-based representation learning.
problem Improving medical diagnosis accuracy through better data representation.
method Heterogeneous network-based model and modified metapath2vec algorithm for learning latent node representations.
result Significant performance boost in symptom/disease classification and disease prediction tasks.
A novel method for clustering multi-view data using dual representations.
problem Clustering multi-view data with consistent and unique information.
method One-step multi-view clustering method exploiting dual representations.
result The proposed method improves clustering performance on benchmark datasets.
Neural nets learn robust geometric data representations.
problem Ensuring neural networks are robust to adversarial attacks.
method Topological Data Analysis via persistence diagrams, Lipschitz stability.
result Certified ε-robustness on ORBIT5K dataset. Unified framework for representation and causal structure learning using exchangeable data.
problem Identifying latent representations or causal structures in non-i.i.d. data.
method Identifiable Exchangeable Mechanisms (IEM) framework for representation and structure learning.
result New insights and identifiability results for causal structure and representation learning.
This paper investigates how data augmentation improves linear separation of manifold data.
problem Understanding how data augmentation enhances linear separation of manifold data.
method Investigates the conditions under which self-supervised representations can linearly separate multi-manifold data.
result Self-supervised learning can linearly separate manifolds with a smaller distance than unsupervised learning.
We identify action representations from video data, proving their statistical benefits.
problem Identifying latent action policies from video data.
method Entropy-regularized LAPO objective, formalizing desiderata for action representations.
result Entropy-regularized LAPO identifies action representations satisfying desiderata under suitable conditions.
Representations learnt through deep neural networks tend to be highly informative, but opaque in terms of what information they learn to encode. We introduce an approach to probabilistic modelling that learns to represent data with two separate deep representations: an invariant representation that encodes the informat…
Proposes a deep learning method for effective data representation.
problem Constructing effective data representations for prediction.
method A deep dimension reduction approach to learning representations with sufficiency, low dimensionality, and disentanglement.
result The proposed deep nonparametric representation is consistent and performs better than existing methods.
Proposes a new method to learn data representations by modeling sample relations.
problem Lack of rich latent structural information in DAEs.
method Explicitly models and leverages sample relations as supervision for representation learning.
result Significantly improves clustering performance on benchmark datasets.
We consider a set of probabilistic functions of some input variables as a representation of the inputs. We present bounds on how informative a representation is about input data. We extend these bounds to hierarchical representations so that we can quantify the contribution of each layer towards capturing the informati…
Since about 100 years ago, to learn the intrinsic structure of data, many representation learning approaches have been proposed, including both linear ones and nonlinear ones, supervised ones and unsupervised ones. Particularly, deep architectures are widely applied for representation learning in recent years, and have…
Interpretable representations improve explainable AI by translating complex data into understandable concepts.
problem Many explainers use interpretable representations but overlook their full potential and assumptions.
method An in-depth analysis of interpretable representations for tabular, image, and text data, identifying strengths, weaknesses, and desiderata.
result Linear model quantifies interpretable concepts' influence on black-box predictions, revealing their explanatory properties and manipulability.
Paper improves deep learning models for limit order book data.
problem Deep learning models' performance depends on robust input data representation.
method Identified and modified flaws in existing representations.
result Proposed modifications lead to state-of-the-art performance.
Learning expressive low-dimensional representations of ultrahigh-dimensional data, e.g., data with thousands/millions of features, has been a major way to enable learning methods to address the curse of dimensionality. However, existing unsupervised representation learning methods mainly focus on preserving the data re…
We present a discriminative clustering approach in which the feature representation can be learned from data and moreover leverage labeled data. Representation learning can give a similarity-based clustering method the ability to automatically adapt to an underlying, yet hidden, geometric structure of the data. The pro…
FairMixRep learns fair representations from mixed data types.
problem Representation learning in mixed numerical and categorical data with fairness constraints.
method Efficient encoder-decoder framework + fairness constraints.
result Excellent performance in preserving information and fairness in mixed data representations.
We examine the influence of input data representations on learning complexity. For learning, we posit that each model implicitly uses a candidate model distribution for unexplained variations in the data, its noise model. If the model distribution is not well aligned to the true distribution, then even relevant variati…
i-Mix improves contrastive learning across domains without domain-specific augmentations.
problem Improving contrastive representation learning for unlabeled data across diverse domains.
method i-Mix treats contrastive learning as a non-parametric classifier problem, mixing data in input and virtual label spaces.
result i-Mix consistently improves representation quality across image, speech, and tabular data domains.
New method certifies individual fairness in representations.
problem Ensuring fairness in data representations without sacrificing utility.
method Mapping similar individuals to close latent representations to certify individual fairness.
result Certifies individual fairness for existing and new data points.
DORA analyzes deep neural networks' internal representations to detect spurious correlations.
problem Detecting spurious correlations in deep neural networks' internal representations.
method DORA uses Extreme-Activation (EA) distance measure to assess representation similarities.
result Identifies internal representations capable of detecting spurious correlations.
Learning discrete representations of data is a central machine learning task because of the compactness of the representations and ease of interpretation. The task includes clustering and hash learning as special cases. Deep neural networks are promising to be used because they can model the non-linearity of data and s…
The paper tackles fair representation learning by smoothing feature mappings.
problem Legal liability for discriminatory use of data by organizations.
method Mapping features to a fair representation space, certifying fairness through chi-squared mutual information.
result Smoothing representation distribution provides generalization guarantees of fairness and maintains accuracy for downstream tasks.
A new geometric metric identifies true data changes from parametrization artifacts in high-dimensional representations.
problem Quantifying representation drift in high-dimensional data using Euclidean or cosine distances can misattribute changes due to arbitrary parametrizations.
method Introducing the Fubini Study metric to identify representations that differ only by gauge transformations.
result The Fubini Study metric isolates intrinsic evolution by remaining invariant under gauge-induced fluctuations, providing a diagnostic for meaningful structural changes.
The goal of unsupervised representation learning is to extract a new representation of data, such that solving many different tasks becomes easier. Existing methods typically focus on vectorized data and offer little support for relational data, which additionally describe relationships among instances. In this work we…
A new framework for semi-supervised learning using pseudo-representation labeling.
problem Improving deep learning models with limited labeled data.
method Pseudo-representation labeling framework integrating pseudo-labeling and self-supervised representation learning.
result Outperforms state-of-the-art semi-supervised learning methods in industrial classification problems.
GGAN improves audio representation learning with fewer labels.
problem Learning representations for specific tasks from unlabelled data.
method Guided Generative Adversarial Neural Network (GGAN).
result GGAN learns better representations with fewer labelled data.
POLAR learns efficient data acquisition policies using pretrained belief representations.
problem Challenges in learning effective policies for adaptive data acquisition.
method POLAR decouples representation learning from policy learning by leveraging pretrained predictive foundation models as belief-state encoders.
result POLAR outperforms state-of-the-art methods across diverse tasks while requiring fewer training samples.
This research shows how to learn shared representations from unpaired data.
problem Learning shared representations from unpaired data.
method Spectral embeddings of random walk matrices from each unimodal representation.
result Shared representations can be learned almost exclusively from unpaired data.
A new unsupervised contrastive learning framework improves time series representation learning.
problem Lack of labeled data in time series data.
method Proposes an unsupervised contrastive learning framework using a novel contrastive loss and data augmentation.
result Framework outperforms other approaches on univariate and multivariate time series, and benefits transfer learning.
The paper formalizes criteria for non-spurious and disentangled representations using causal methods.
problem Formalizing criteria for non-spurious and disentangled representations in representation learning.
method Causal perspective, counterfactual quantities, observable consequences of causal assertions.
result Computable metrics for assessing representation learning based on observed data.
Improves NF for complex data distributions with multiple modes.
problem Difficulty in handling data distributions with multiple isolated modes.
method Proposes a new framework using variational latent representation to improve NF.
result Significantly more powerful for generating data distributions with multiple modes.
New approach to abstract neural network representations using renormalization group.
problem Developing truly abstract representations in neural networks.
method Renormalization group approach to expand representations to encompass broader data sets.
result Representations in neural networks become more abstract as data breadth increases and depth increases.
DIVE learns video representations even with missing data.
problem Missing data in video sequences.
method Disentangled Imputed Video autoEncoder (DIVE) with missingness latent variable.
result DIVE outperforms state-of-the-art baselines in imputing and predicting missing video frames.
Improves clustering performance by mixing latent representations.
problem Finding well-defined clusters in data representations.
method Mixing Consistent Deep Clustering method that encourages realistic interpolations and semantic consistency.
result Improved clustering performance across various models and datasets.
Learning interpretable representations of data remains a central challenge in deep learning. When training a deep generative model, the observed data are often associated with certain categorical labels, and, in parallel with learning to regenerate data and simulate new data, learning an interpretable representation of…
Learning data representations that reflect the customers' creditworthiness can improve marketing campaigns, customer relationship management, data and process management or the credit risk assessment in retail banks. In this research, we adopt the Variational Autoencoder (VAE), which has the ability to learn latent rep…
T-JEPA learns tabular data representations without augmentations, outperforming traditional methods.
problem Challenges in self-supervised learning for tabular data due to lack of data augmentations.
method T-JEPA uses a Joint Embedding Predictive Architecture (JEPA) to predict latent representations of different subsets of features within the same sample.
result Significant improvement in classification and regression tasks, outperforming traditional methods.
New method learns behavioral representations from mobility data.
problem Analyzing behavioral similarity of moving individuals from CDR trajectories.
method mob2vec framework combining segmentation, generalization, and unsupervised learning.
result Mob2vec generates low-dimensional vector representations preserving mobility behavior similarities.
Enhances multi-tag classification using low-dimensional vector representations and virtual data.
problem Improving the performance of multi-tag classifiers.
method Embedding raw data into a low-dimensional feature space, then generating virtual data from linear operations on these vectors, to train multi-tag classifiers.
result Significant improvement in F1 scores (up to 224%) compared to training directly with raw data.
The key to success in machine learning (ML) is the use of effective data representations. Traditionally, data representations were hand-crafted. Recently it has been demonstrated that, given sufficient data, deep neural networks can learn effective implicit representations from simple input representations. However, fo…
Contrastive learning adapts to data intrinsic dimensions, learning low-dimensional representations.
problem Learning high-dimensional representations from multi-modal data.
method Multi-modal contrastive learning with temperature optimization.
result Contrastive learning adapts to intrinsic dimensions of data, not specified dimensions.
Framework for causal discovery using multi-modal data.
problem Failure of representation learning in causal tasks.
method Statistical and computational framework combining representation learning and causal inference.
result Effective use of observational and perturbational data for causal discovery.