BCRL learns a Bellman complete representation for offline RL policy evaluation.
problem Learning a Q-function efficiently from offline data.
method BCRL learns a linear Bellman complete representation directly from data, enabling efficient OPE.
result BCRL achieves competitive OPE error and outperforms FQE in certain scenarios.
HEAR benchmark evaluates audio representations for diverse tasks.
problem Developing a general-purpose audio representation for various tasks.
method Evaluated 29 models across 19 tasks using 16 datasets.
result No single audio representation performs holistically.
Proposes clustering as a new evaluation method for clinical knowledge embedding.
problem Traditional Link Prediction evaluation protocol loses information and harms model accuracy.
method Proposes Clustering Evaluation Protocol as an alternative.
result Experimental results show the proposed protocol can potentially replace Link Prediction.
State representation learning aims at learning compact representations from raw observations in robotics and control applications. Approaches used for this objective are auto-encoders, learning forward models, inverse dynamics or learning using generic priors on the state characteristics. However, the diversity in appl…
New methods evaluate data representations by complexity of low-loss predictor learning.
problem Evaluating quality of data representations for downstream tasks.
method Surplus Description Length (SDL) and ε Sample Complexity (εSC) methods.
result Methods measure the information needed to approximate optimal predictor up to specified tolerance.
This paper evaluates various representations for robotics tasks, improving performance in lifting, stacking, and pushing.
problem Improving data-efficiency in reinforcement learning for robotics with limited data.
method Systematic evaluation of common representations in three robotics tasks: lifting, stacking, and pushing.
result Some representations can perform as well as simulator states as agent inputs, challenging common intuitions.
A new method evaluates invariant performance of IRM-based representations.
problem Impact of data changes on machine learning model performance.
method Proposes a novel method to evaluate invariant performance of IRM-based representations.
result Establishes a robust criterion to assess invariant performance of various representation techniques.
The topology of SU(3)-representation varieties of the fundamental groups of planar webs so that the meridians are sent to matrices with trace equal to −1 are explored, and compared to data coming from spider evaluation of the webs. Corresponding to an evaluation of a web as a spider is a rooted tree. We associate t…
State representation learning, or the ability to capture latent generative factors of an environment, is crucial for building intelligent agents that can perform a wide variety of tasks. Learning such representations without supervision from rewards is a challenging open problem. We introduce a method that learns state…
This paper starts a systematic description of colored knot polynomials, beginning from the first non-(anti)symmetric representation R=[2,1]. The project involves several steps: (i) parametrization of big families of knots a la arXiv:1506.00339, (ii) evaluating Racah/mixing matrices for various numbers of strands in var…
New measures quantify diversity of latent representations using metric space magnitude.
problem Evaluating the diversity of latent representations in machine learning models.
method Developed magnitude-based measures for latent representations, stable under data perturbations.
result Demonstrated superior performance across various domains and tasks.
This paper critically examines unsupervised disentangled representation learning, revealing challenges and limitations.
problem The difficulty of unsupervised learning of disentangled representations and the challenges in evaluation metrics.
method Theoretical analysis and a large-scale experimental study covering 8 datasets and 14000 models.
result Well-disentangled models cannot be identified without supervision, and different evaluation metrics disagree on what constitutes disentanglement.
Introduction 1. The two-eigenvalue problem 2. Hecke algebra representations of braid groups 3. Duality of Jones-Wenzl representations 4. Closed images of Jones-Wenzl sectors 5. Distribution of evaluations of Jones polynomials 6. Fibonacci representations
LEEP measures transferability of learned representations efficiently.
problem Evaluating the transferability of learned representations in machine learning.
method LEEP: Log Expected Empirical Prediction, a simple measure requiring one pass through the target data set.
result LEEP predicts transfer and meta-transfer learning performance and convergence speed, outperforming existing measures.
New insights into state representations in RL help design better learning rules.
problem Lack of automatic feature learning in RL for large or continuous state spaces.
method Bootstrapping methods and theoretical analysis of temporal difference learning.
result State representations differ from other auxiliary-task-based approaches.
Deep RL approach improves MIS for complex environments.
problem Improving off-policy evaluation for complex environments.
method Uses successor representation from deep RL to decouple reward and dynamics.
result Empirically stable and applicable to high-dimensional domains.
We release the largest public ECG dataset of continuous raw signals for representation learning containing 11 thousand patients and 2 billion labelled beats. Our goal is to enable semi-supervised ECG models to be made as well as to discover unknown subtypes of arrhythmia and anomalous ECG signal events. To this end, we…
Paper defines and solves a problem in representation learning to ensure fairness with high confidence.
problem Learning fair representations with high confidence guarantees for all downstream tasks.
method Formally defines the problem, introduces FRG framework, proves high probability fairness, and demonstrates effectiveness empirically.
result FRG framework provides high-confidence guarantees for limiting unfairness across all downstream models and tasks.
Obtaining colored HOMFLY-PT polynomials for knots from 3-strand braid carrying arbitrary SU(N) representation is still tedious. For a class of rank r symmetric representations, [r]-colored HOMFLY-PT H[r] evaluation becomes simpler. Recently it was shown that H[r], for such knots from 3-strand braid, can…
This paper evaluates fairness in deep metric learning and proposes a method to reduce subgroup performance gaps.
problem The negative impact of deep metric learning representations on minority subgroup performance in downstream tasks.
method Definition of fairness in DML through inter-class, intra-class, and uniformity properties; finDML benchmark; Partial Attribute De-correlation (PARADE) method.
result Bias in DML representations propagates to downstream tasks, even with balanced training data.
Study reveals limitations of fair representation learning methods and cautions against their use in performance-sensitive tasks.
problem Limitations of fair representation learning methods in performance-sensitive tasks.
method Using causal reasoning, the study defines and formalizes different sources of dataset bias and examines the performance of fair representation learning under distribution shifts.
result Fundamental limitations on fair representation learning when evaluation data is drawn from the same distribution as training data.
New method uses contrastively trained GNNs for more reliable graph model evaluation.
problem Need effective methods to evaluate Graph Generative Models.
method Use representations from contrastively trained Graph Neural Networks (GNNs) for evaluation.
result Contrastively trained GNNs provide more reliable evaluation metrics than traditional or GNN-based approaches.
With the rising interest in graph representation learning, a variety of approaches have been proposed to effectively capture a graph's properties. While these approaches have improved performance in graph machine learning tasks compared to traditional graph techniques, they are still perceived as techniques with limite…
Derives integral representations for a Lévy process and its extremum, hitting time, with fast evaluation.
problem Efficiently evaluating the joint probability density function of a Lévy process, its supremum, and hitting time.
method Integral representations, Laplace-Fourier transforms, summation by parts, conformal deformation, trapezoid rules, Gaver-Wynn-Rho algorithm.
result Explicit calculations and fast evaluation of the joint cpdf for Lévy processes.
Replicated and validated Rank-N-Contrast for robust regression.
problem Deep regression models struggle with continuous sample orders.
method Contrastive learning of continuous representations by ranking samples.
result Improved performance and robustness of RNC framework.
Study evaluates scalability and real-world impact of disentangled representations.
problem Scalability and real-world impact of disentangled representations.
method New high-resolution dataset and architectures for disentangled representation learning.
result Disentanglement predicts out-of-distribution task performance.
A commonly used evaluation metric for text-to-image synthesis is the Inception score (IS) \cite{inceptionscore}, which has been shown to be a quality metric that correlates well with human judgment. However, IS does not reveal properties of the generated images indicating the ability of a text-to-image synthesis method…
Study evaluates margin parameter effects on knowledge embedding quality.
problem Understanding margin parameter's impact on embedding quality.
method Examined margin parameter values for multi-relational categorized data.
result Lower margin values are insufficient, while larger values cause noise.
LCIT tests conditional independence using latent representations.
problem Detecting conditional independencies in statistical and machine learning tasks.
method Generative framework for learning latent representations of target variables X and Y, then testing for remaining dependencies.
result LCIT outperforms state-of-the-art baselines consistently under different metrics and settings.
DECAT framework evaluates multimodal models for shared biology, detecting confounders and false positives.
problem Determining if multimodal models learn shared biology or just confounders.
method DECAT framework classifies multimodal representations into four diagnostic scenarios using null-referenced metrics.
result DECAT detects confounders and false positives in multimodal models, improving with larger cohorts and stronger representations.
Variational autoencoders (VAEs) learn representations of data by jointly training a probabilistic encoder and decoder network. Typically these models encode all features of the data into a single variable. Here we are interested in learning disentangled representations that encode distinct aspects of the data into sepa…
Learning disentangled representation from any unlabelled data is a non-trivial problem. In this paper we propose Information Maximising Autoencoder (InfoAE) where the encoder learns powerful disentangled representation through maximizing the mutual information between the representation and given information in an unsu…
The study uncovers latent capabilities of language models via causal representation learning.
problem Rigorous causal evaluations of language model capabilities are challenging due to confounding effects and computational costs.
method Proposes a causal representation learning framework to identify latent capability factors as causally interrelated after controlling for a common confounder (base model).
result Identifies a three-node linear causal structure explaining performance variations across 1500 models and six benchmarks.
Self-supervised methods, wherein an agent learns representations solely by observing the results of its actions, become crucial in environments which do not provide a dense reward signal or have labels. In most cases, such methods are used for pretraining or auxiliary tasks for "downstream" tasks, such as control, expl…
Deep-embedding methods aim to discover representations of a domain that make explicit the domain's class structure and thereby support few-shot learning. Disentangling methods aim to make explicit compositional or factorial structure. We combine these two active but independent lines of research and propose a new parad…
Bayesian algorithm improves word representations using semantic taxonomy.
problem Improving word representations in semantic taxonomy.
method Bayesian Hierarchical Words Representation (BHWR) learning algorithm combining Variational Bayes and semantic taxonomy modeling.
result BHWR produces better representations for rare words.
We make two theoretical contributions to disentanglement learning by (a) defining precise semantics of disentangled representations, and (b) establishing robust metrics for evaluation. First, we characterize the concept "disentangled representations" used in supervised and unsupervised methods along three dimensions-in…
We analyze disentangled representations under a causal generative process, proposing new metrics and datasets.
problem Addressing fairness and interpretability through disentangled representations with a causal perspective.
method Work under a causal generative process, proposing new metrics and datasets to study disentanglement.
result Proposed metrics capture the desiderata of disentangled causal process.
Self-supervised learning improves ECG classification performance.
problem Label scarcity in clinical 12-lead ECG data.
method Adapted self-supervised methods to ECG domain, focusing on contrastive representations and latent forecasting.
result Contrastive predictive coding adaptation yields linear evaluation performance only 0.5% below supervised performance.
Pix2Shape learns 3D scene representations from single images without supervision.
problem Learning 3D scene information from a single image without supervision.
method Pix2Shape uses an encoder, decoder, and critic network to generate 2.5D surfel-based reconstructions.
result Pix2Shape can generate complex 3D scenes from a single image, scaling with on-screen resolution.
URL benchmark evaluates uncertainty quantification in pretrained models.
problem Need for reliable uncertainty estimates in transferable pretrained models.
method Proposes URL benchmark to measure transferability of representations and uncertainty estimates.
result Transferable uncertainty quantification remains challenging but not contradictory to traditional goals.
Recently several researchers have investigated techniques for using data to learn Bayesian networks containing compact representations for the conditional probability distributions (CPDs) stored at each node. The majority of this work has concentrated on using decision-tree representations for the CPDs. In addition, re…
Researchers propose a new SSL risk decomposition method to evaluate and improve self-supervised learning models.
problem Self-supervised learning evaluation is limited to a single metric, providing little insight into model performance and improvement.
method Proposes an SSL risk decomposition that considers four error components: approximation, representation usability, probe generalization, and encoder generalization.
result Analysis of 169 SSL vision models reveals the main sources of error and provides insights for improving SSL models in specific settings.
Dual representations for robust risk measures and uncertainty sets.
problem Characterizing continuity of robust risk measures and their uncertainty sets.
method Develop dual representations for robust risk measures and uncertainty sets based on distinct geometric assumptions.
result Two dual frameworks for consolidated uncertainty sets are complementary, not interchangeable.
This paper evaluates a method to improve representations using incomplete external evidence across tasks.
problem Increasing labelled data quality and quantity is challenging due to manual labelling errors and noise.
method Evidence Transfer method using incomplete categorical external evidence.
result Evidence Transfer proves effective and robust against different levels of incompleteness.
Proposes GM Score to evaluate GANs considering diversity, disentanglement, and discriminability.
problem Evaluation of GANs for sample quality and diversity.
method Integrates various factors including intra-class and inter-class diversity, disentanglement, and discriminability metrics.
result Demonstrates improved evaluation of GANs on MNIST dataset.
This paper presents a novel clustering concept that is based on jointly learned nonlinear transforms (NTs) with priors on the information loss and the discrimination. We introduce a clustering principle that is based on evaluation of a parametric min-max measure for the discriminative prior. The decomposition of the pr…
We study the problem of off-policy policy evaluation (OPPE) in RL. In contrast to prior work, we consider how to estimate both the individual policy value and average policy value accurately. We draw inspiration from recent work in causal reasoning, and propose a new finite sample generalization error bound for value e…