Many machine learning algorithms are trained and evaluated by splitting data from a single source into training and test sets. While such focus on in-distribution learning scenarios has led to interesting advancement, it has not been able to tell if models are relying on dataset biases as shortcuts for successful predi…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “representation biases”
Study identifies and measures biases in legal case data.
problem Addressing representation biases and sentencing disparities in legal case data.
method Utilizes two regression models: a baseline and a fair judge model.
result Quantifies biases across demographic groups in criminal data from Cook County (Illinois).
Integrates inductive biases into VAEs using intermediary latent variables.
problem Ineffective mechanisms for incorporating inductive biases into VAEs.
method InteL-VAEs use an intermediary latent space to control encoding, with a parametric function to enforce desired properties.
result InteL-VAEs lead to better generative models and representations.
New method improves model robustness to biased data.
problem Learning unbiased models from biased datasets.
method Developed epsilon-SupInfoNCE and FairKL losses.
result Improved performance on biased datasets.
Paper summarizes unsupervised learning challenges for disentangled representations.
problem Unsupervised learning of disentangled representations without inductive biases.
method Theoretical and practical analysis of existing approaches.
result Unsupervised disentanglement is fundamentally impossible without inductive biases.
The paper explores how equivariant models' biases affect latent representations for better performance.
problem The impact of inductive biases on latent representations in equivariant models.
method Demonstrates the importance of accounting for inductive biases in latent representations of equivariant models.
result Effective invariant projections can be used to retain information in latent representations, improving downstream tasks.
Extract symbolic models from deep learning with inductive biases.
problem Interpreting and discovering physical principles from deep neural networks.
method Introduce strong inductive biases in GNNs, encourage sparse latent representations, apply symbolic regression.
result Extracted symbolic equations from neural networks, including known force laws and new analytic formulas.
Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Trainingcs.LG
New method reduces bias in NLI models using ensemble adversarial training.
problem Spurious correlations between hypotheses and entailment classes in NLI datasets.
method Adversarial training with an ensemble of classifiers to reduce bias in sentence representations.
result Ensemble adversarial training produces more robust NLI models, outperforming previous methods.
Regularized training of an autoencoder typically results in hidden unit biases that take on large negative values. We show that negative biases are a natural result of using a hidden layer whose responsibility is to both represent the input data and act as a selection mechanism that ensures sparsity of the representati…
The paper addresses bias in visual recognition models by reweighting observations.
problem Bias in deep neural networks trained on biased image databases.
method Reweighting observations based on known biasing mechanisms to form a nearly debiased estimator.
result The approach can remedy representativeness issues in visual recognition systems.
Randomly initialized transformers show extreme token preferences.
problem Structural biases in randomly initialized transformers.
method Dissection of transformer architecture at initialization.
result Initialization-induced biases persist throughout training.
Novel approach uses inductive biases for semiconductor etching.
problem Significant violations of physics in etching process predictions.
method Introduced deep learning model with inductive biases.
result Fits measurements faster and follows physical behavior.
The paper presents a new method to represent directed graphs using pseudo-Riemannian manifolds.
problem Representing directed graphs in a compact and meaningful way.
method Combines pseudo-Riemannian metric structure, non-trivial global topology, and a unique likelihood function.
result Low-dimensional cylindrical Minkowski and anti-de Sitter spacetimes produce equal or better graph representations than curved Riemannian manifolds.
Social bias in machine learning has drawn significant attention, with work ranging from demonstrations of bias in a multitude of applications, curating definitions of fairness for different contexts, to developing algorithms to mitigate bias. In natural language processing, gender bias has been shown to exist in contex…
Network embedding algorithms are able to learn latent feature representations of nodes, transforming networks into lower dimensional vector representations. Typical key applications, which have effectively been addressed using network embeddings, include link prediction, multilabel classification and community detectio…
A self-supervised debiasing method using rank regularization mitigates spurious correlations in neural networks.
problem Spurious correlations cause biases in deep neural networks, affecting generalization.
method Spectral analysis of latent representations, rank regularization, self-supervised pretraining, debiasing of downstream tasks.
result The proposed framework significantly improves generalization performance and outperforms supervised debiasing approaches.
WeLa-VAE learns interpretable disentangled representations with weak supervision.
problem Learning disentangled representations without strong supervision.
method Variational inference framework with shared latent variables and modified variational lower bound.
result WeLa-VAE learns alternative disentangled representations (polar) from weak labels (distance and angle) without refined supervision.
Learning disentangled representations is considered a cornerstone problem in representation learning. Recently, Locatello et al. (2019) demonstrated that unsupervised disentanglement learning without inductive biases is theoretically impossible and that existing inductive biases and unsupervised methods do not allow to…
FairDrop improves fairness in graph representation learning by counteracting homophily.
problem Ensuring fairness in graph representation learning, especially in scenarios with protected attributes.
method Proposes a biased edge dropout algorithm (FairDrop) to counteract homophily and improve fairness.
result Successfully improves fairness in all models up to a small or negligible drop in accuracy.
Adv-SSL learns unbiased representations from unlabeled data with theoretical guarantees.
problem Learning unbiased representations from unlabeled data.
method Adv-SSL, a novel adversarial self-supervised learning approach.
result Adv-SSL achieves strong classification performance with limited downstream labels.
New meta-learning method handles nonlinear tasks for faster convergence.
problem Meta-learning with nonlinear representations and tasks.
method Derive theoretical guarantees with regularization in RKHS.
result Improved rates scaling with the number of tasks.
GQML uses symmetries from representation theory to improve quantum machine learning.
problem Creating quantum models with symmetries to improve performance.
method Introduction to representation theory for quantum learning, focusing on group actions and symmetries.
result Effective implementation of GQML requires knowledge of group representation theory.
Automatic Debiased Machine Learning for Dynamic Treatment Effects and General Nested Functionalsecon.EM
New method for estimating treatment effects without complex propensity models.
problem Estimating treatment effects in dynamic treatment regimes.
method Recursive Riesz representer estimation for de-biasing corrections.
result Directly estimates de-biasing corrections without auxiliary models.
Understanding Fairness and Prediction Error through Subspace Decomposition and Influence Analysisstat.ML
The paper proposes a method to balance fairness and prediction accuracy by adjusting data representations.
problem Machine learning models can inherit and amplify historical biases, leading to unfair outcomes.
method The paper uses subspace decomposition and influence analysis to control the fairness-utility trade-off.
result The method effectively improves fairness while preserving predictive performance.
The key idea behind the unsupervised learning of disentangled representations is that real-world data is generated by a few explanatory factors of variation which can be recovered by unsupervised learning algorithms. In this paper, we provide a sober look at recent progress in the field and challenge some common assump…
Convolutional Neural Networks (CNNs) are commonly thought to recognise objects by learning increasingly complex representations of object shapes. Some recent studies suggest a more important role of image textures. We here put these conflicting hypotheses to a quantitative test by evaluating CNNs and human observers on…
This paper critically examines unsupervised disentangled representation learning, revealing challenges and limitations.
problem The difficulty of unsupervised learning of disentangled representations and the challenges in evaluation metrics.
method Theoretical analysis and a large-scale experimental study covering 8 datasets and 14000 models.
result Well-disentangled models cannot be identified without supervision, and different evaluation metrics disagree on what constitutes disentanglement.
We present a unified invariance framework for supervised neural networks that can induce independence to nuisance factors of data without using any nuisance annotations, but can additionally use labeled information about biasing factors to force their removal from the latent embedding for making fair predictions. Invar…
Study shows bottlenecks improve image segmentation quality.
problem Robust object discovery in real-world images remains challenging.
method Empirical investigation of reconstruction bottlenecks in GENESIS model.
result Reconstruction bottlenecks determine reconstruction and segmentation quality.
This paper reviews and proposes a unified framework for contrastive learning.
problem The origins and development of contrastive learning across various fields.
method A comprehensive literature review and a general Contrastive Representation Learning framework.
result A unified framework simplifies and unifies contrastive learning methods.
Paper proposes using unlabeled data for fair decision-making.
problem Bias in decision-making algorithms due to biased labels and selective labeling.
method Variational autoencoder for learning unbiased data representations from both labeled and unlabeled data.
result Method learns fair and stable decision policies with high utility.
Theoretical study on how model architecture affects contrastive learning performance.
problem Understanding the role of model architecture in self-supervised learning.
method Theoretical analysis of contrastive learning, focusing on model capacity and clustering structures.
result Contrastive representations have lower dimensionality than the number of clusters in the data distribution.
Purpose: In the present work we describe the correction of diffusion-weighted MRI for site and scanner biases using a novel method based on invariant representation. Theory and Methods: Pooled imaging data from multiple sources are subject to variation between the sources. Correcting for these biases has become very im…
Societal bias towards certain communities is a big problem that affects a lot of machine learning systems. This work aims at addressing the racial bias present in many modern gender recognition systems. We learn race invariant representations of human faces with an adversarially trained autoencoder model. We show that …
This work addresses encoding biases in neural networks by tailoring models with unsupervised losses.
problem Improving neural network representations and reducing the generalization gap.
method Inspired by transductive learning, the authors propose tailoring and meta-tailoring to optimize unsupervised losses during prediction time.
result Models trained with tailoring and meta-tailoring perform better on the task objective after adapting to unsupervised losses.
Paper debiases multiple word embedding biases simultaneously.
problem Reduction of multiple biases in word embeddings.
method Joint multiclass debiasing approach using Word Embeddings Association Test (WEAT).
result Demonstrates reduction or complete elimination of bias in word embeddings.
Probably the most important problem in machine learning is the preliminary biasing of a learner's hypothesis space so that it is small enough to ensure good generalisation from reasonable training sets, yet large enough that it contains a good solution to the problem being learnt. In this paper a mechanism for {\em aut…
Improves fairness in machine learning by adding underrepresented group data.
problem Machine learning biases across subgroups due to under-representation or societal biases.
method Data augmentation via pairwise mixup across subgroups to balance subpopulations.
result Achieves fair outcomes with robust if not improved accuracy.
Decision making is a process that is extremely prone to different biases. In this paper we consider learning fair representations that aim at removing nuisance (sensitive) information from the decision process. For this purpose, we propose to use deep generative modeling and adapt a hierarchical Variational Auto-Encode…
Representations of data that are invariant to changes in specified factors are useful for a wide range of problems: removing potential biases in prediction problems, controlling the effects of covariates, and disentangling meaningful factors of variation. Unfortunately, learning representations that exhibit invariance …
New clustering method considers causal fairness to avoid bias.
problem Clustering algorithms can unintentionally propagate unfair disparities.
method Integrates causal fairness metrics into clustering algorithms.
result Demonstrates efficacy on datasets with known unfair biases.
Algorithm improves recommendation subset selection in the presence of biases.
problem Maximizing submodular functions for recommendation in the presence of social biases.
method Algorithm for submodular maximization with fairness constraints.
result Algorithm provably outputs subsets with near-optimal utility and proportional representation.
Paper debiases text embeddings using context injection.
problem Text embeddings capture and perpetuate biases.
method Review 19 embedding models, quantify biases, and test context injection.
result Higher performing models are more prone to biases but better at incorporating context.
Graph neural networks help AI agents learn more complex language.
problem Understanding how AI agents learn and use language.
method Developed graph referential games to compare different AI models.
result Graph neural networks enable AI to learn more complex, compositional language.
Deep Fair Learning: A Unified Framework for Fine-tuning Representations with Sufficient Networksstat.ML
Unified framework for fair representation learning in machine learning.
problem Ensuring fairness in machine learning models, especially when biased data representations lead to unfair predictions.
method Integrates nonlinear sufficient dimension reduction with deep learning to construct fair and informative representations, introducing a penalty term to enforce conditional independence between sensitive attributes and learned representations.
result Achieves a superior balance between fairness and utility, significantly outperforming state-of-the-art baselines on various data structures.
NeuralRBMLE tackles explore-exploit trade-offs in contextual bandits with neural networks.
problem Stochastic contextual bandit problem with general bounded reward functions.
method Reward-biased maximum likelihood estimation with neural networks to enforce exploration.
result Both NeuralRBMLE variants achieve regret.
This paper corrects climate model biases using a factor model approach.
problem Systematic biases in GCM outputs due to unobserved confounders.
method Factor model approach to learn latent confounders from historical data and apply them to enhance bias correction.
result Significant improvements in the accuracy of precipitation outputs.
New graph embedding method improves link prediction and node classification.
problem Improving graph embedding methods for better node representation.
method Spectral-biased random walks with neighborhood similarity bias.
result Significantly improves link prediction and node classification.