Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

165331496661 · Jun 202019922001200920172026
48 results for Representation Evaluation

BCRL learns a Bellman complete representation for offline RL policy evaluation.

problem Learning a Q-function efficiently from offline data.
method BCRL learns a linear Bellman complete representation directly from data, enabling efficient OPE.
result BCRL achieves competitive OPE error and outperforms FQE in certain scenarios.

New methods evaluate data representations by complexity of low-loss predictor learning.

problem Evaluating quality of data representations for downstream tasks.
method Surplus Description Length (SDL) and ε Sample Complexity (εSC) methods.
result Methods measure the information needed to approximate optimal predictor up to specified tolerance.

This paper evaluates various representations for robotics tasks, improving performance in lifting, stacking, and pushing.

problem Improving data-efficiency in reinforcement learning for robotics with limited data.
method Systematic evaluation of common representations in three robotics tasks: lifting, stacking, and pushing.
result Some representations can perform as well as simulator states as agent inputs, challenging common intuitions.

A new method evaluates invariant performance of IRM-based representations.

problem Impact of data changes on machine learning model performance.
method Proposes a novel method to evaluate invariant performance of IRM-based representations.
result Establishes a robust criterion to assess invariant performance of various representation techniques.

The topology of SU(3)SU(3)-representation varieties of the fundamental groups of planar webs so that the meridians are sent to matrices with trace equal to 1-1 are explored, and compared to data coming from spider evaluation of the webs. Corresponding to an evaluation of a web as a spider is a rooted tree. We associate t…

2017-05-16abs ↗pdf ↗

State representation learning, or the ability to capture latent generative factors of an environment, is crucial for building intelligent agents that can perform a wide variety of tasks. Learning such representations without supervision from rewards is a challenging open problem. We introduce a method that learns state…

2019-06-19abs ↗pdf ↗

This paper starts a systematic description of colored knot polynomials, beginning from the first non-(anti)symmetric representation R=[2,1]. The project involves several steps: (i) parametrization of big families of knots a la arXiv:1506.00339, (ii) evaluating Racah/mixing matrices for various numbers of strands in var…

2015-08-12abs ↗pdf ↗

New measures quantify diversity of latent representations using metric space magnitude.

problem Evaluating the diversity of latent representations in machine learning models.
method Developed magnitude-based measures for latent representations, stable under data perturbations.
result Demonstrated superior performance across various domains and tasks.

This paper critically examines unsupervised disentangled representation learning, revealing challenges and limitations.

problem The difficulty of unsupervised learning of disentangled representations and the challenges in evaluation metrics.
method Theoretical analysis and a large-scale experimental study covering 8 datasets and 14000 models.
result Well-disentangled models cannot be identified without supervision, and different evaluation metrics disagree on what constitutes disentanglement.

New insights into state representations in RL help design better learning rules.

problem Lack of automatic feature learning in RL for large or continuous state spaces.
method Bootstrapping methods and theoretical analysis of temporal difference learning.
result State representations differ from other auxiliary-task-based approaches.

Paper defines and solves a problem in representation learning to ensure fairness with high confidence.

problem Learning fair representations with high confidence guarantees for all downstream tasks.
method Formally defines the problem, introduces FRG framework, proves high probability fairness, and demonstrates effectiveness empirically.
result FRG framework provides high-confidence guarantees for limiting unfairness across all downstream models and tasks.

This paper evaluates fairness in deep metric learning and proposes a method to reduce subgroup performance gaps.

problem The negative impact of deep metric learning representations on minority subgroup performance in downstream tasks.
method Definition of fairness in DML through inter-class, intra-class, and uniformity properties; finDML benchmark; Partial Attribute De-correlation (PARADE) method.
result Bias in DML representations propagates to downstream tasks, even with balanced training data.

Study reveals limitations of fair representation learning methods and cautions against their use in performance-sensitive tasks.

problem Limitations of fair representation learning methods in performance-sensitive tasks.
method Using causal reasoning, the study defines and formalizes different sources of dataset bias and examines the performance of fair representation learning under distribution shifts.
result Fundamental limitations on fair representation learning when evaluation data is drawn from the same distribution as training data.

New method uses contrastively trained GNNs for more reliable graph model evaluation.

problem Need effective methods to evaluate Graph Generative Models.
method Use representations from contrastively trained Graph Neural Networks (GNNs) for evaluation.
result Contrastively trained GNNs provide more reliable evaluation metrics than traditional or GNN-based approaches.

Derives integral representations for a Lévy process and its extremum, hitting time, with fast evaluation.

problem Efficiently evaluating the joint probability density function of a Lévy process, its supremum, and hitting time.
method Integral representations, Laplace-Fourier transforms, summation by parts, conformal deformation, trapezoid rules, Gaver-Wynn-Rho algorithm.
result Explicit calculations and fast evaluation of the joint cpdf for Lévy processes.

Study evaluates scalability and real-world impact of disentangled representations.

problem Scalability and real-world impact of disentangled representations.
method New high-resolution dataset and architectures for disentangled representation learning.
result Disentanglement predicts out-of-distribution task performance.

A commonly used evaluation metric for text-to-image synthesis is the Inception score (IS) \cite{inceptionscore}, which has been shown to be a quality metric that correlates well with human judgment. However, IS does not reveal properties of the generated images indicating the ability of a text-to-image synthesis method…

2019-10-31abs ↗pdf ↗

LCIT tests conditional independence using latent representations.

problem Detecting conditional independencies in statistical and machine learning tasks.
method Generative framework for learning latent representations of target variables X and Y, then testing for remaining dependencies.
result LCIT outperforms state-of-the-art baselines consistently under different metrics and settings.

DECAT framework evaluates multimodal models for shared biology, detecting confounders and false positives.

problem Determining if multimodal models learn shared biology or just confounders.
method DECAT framework classifies multimodal representations into four diagnostic scenarios using null-referenced metrics.
result DECAT detects confounders and false positives in multimodal models, improving with larger cohorts and stronger representations.

The study uncovers latent capabilities of language models via causal representation learning.

problem Rigorous causal evaluations of language model capabilities are challenging due to confounding effects and computational costs.
method Proposes a causal representation learning framework to identify latent capability factors as causally interrelated after controlling for a common confounder (base model).
result Identifies a three-node linear causal structure explaining performance variations across 1500 models and six benchmarks.

Deep-embedding methods aim to discover representations of a domain that make explicit the domain's class structure and thereby support few-shot learning. Disentangling methods aim to make explicit compositional or factorial structure. We combine these two active but independent lines of research and propose a new parad…

2018-02-14abs ↗pdf ↗

We analyze disentangled representations under a causal generative process, proposing new metrics and datasets.

problem Addressing fairness and interpretability through disentangled representations with a causal perspective.
method Work under a causal generative process, proposing new metrics and datasets to study disentanglement.
result Proposed metrics capture the desiderata of disentangled causal process.

Self-supervised learning improves ECG classification performance.

problem Label scarcity in clinical 12-lead ECG data.
method Adapted self-supervised methods to ECG domain, focusing on contrastive representations and latent forecasting.
result Contrastive predictive coding adaptation yields linear evaluation performance only 0.5% below supervised performance.

Pix2Shape learns 3D scene representations from single images without supervision.

problem Learning 3D scene information from a single image without supervision.
method Pix2Shape uses an encoder, decoder, and critic network to generate 2.5D surfel-based reconstructions.
result Pix2Shape can generate complex 3D scenes from a single image, scaling with on-screen resolution.

URL benchmark evaluates uncertainty quantification in pretrained models.

problem Need for reliable uncertainty estimates in transferable pretrained models.
method Proposes URL benchmark to measure transferability of representations and uncertainty estimates.
result Transferable uncertainty quantification remains challenging but not contradictory to traditional goals.

Recently several researchers have investigated techniques for using data to learn Bayesian networks containing compact representations for the conditional probability distributions (CPDs) stored at each node. The majority of this work has concentrated on using decision-tree representations for the CPDs. In addition, re…

2013-02-06abs ↗pdf ↗

Researchers propose a new SSL risk decomposition method to evaluate and improve self-supervised learning models.

problem Self-supervised learning evaluation is limited to a single metric, providing little insight into model performance and improvement.
method Proposes an SSL risk decomposition that considers four error components: approximation, representation usability, probe generalization, and encoder generalization.
result Analysis of 169 SSL vision models reveals the main sources of error and provides insights for improving SSL models in specific settings.

Dual representations for robust risk measures and uncertainty sets.

problem Characterizing continuity of robust risk measures and their uncertainty sets.
method Develop dual representations for robust risk measures and uncertainty sets based on distinct geometric assumptions.
result Two dual frameworks for consolidated uncertainty sets are complementary, not interchangeable.

Proposes GM Score to evaluate GANs considering diversity, disentanglement, and discriminability.

problem Evaluation of GANs for sample quality and diversity.
method Integrates various factors including intra-class and inter-class diversity, disentanglement, and discriminability metrics.
result Demonstrates improved evaluation of GANs on MNIST dataset.

Learning knowledge representation is an increasingly important technology that supports a variety of machine learning related applications. However, the choice of hyperparameters is seldom justified and usually relies on exhaustive search. Understanding the effect of hyperparameter combinations on embedding quality is …

2019-12-21abs ↗pdf ↗

We study the problem of off-policy policy evaluation (OPPE) in RL. In contrast to prior work, we consider how to estimate both the individual policy value and average policy value accurately. We draw inspiration from recent work in causal reasoning, and propose a new finite sample generalization error bound for value e…

2018-05-23abs ↗pdf ↗

Unsupervised learning representations generalize better than supervised learning under distribution shifts.

problem Robustness of unsupervised representations to distribution shift.
method Extensive evaluation on synthetic and realistic datasets, including controllable domain generalization datasets.
result Unsupervised representations learned from SSL and AE generalize better than supervised learning under various distribution shifts.