Proposes an extended disentanglement framework with new metrics.
problem Improving disentangled representations in representation learning.
method Connects DCI framework to identifiability, introduces new measures.
result Establishes a formal link between disentanglement and independent component analysis.
ProSMIN improves representation quality through probabilistic self-supervised learning.
problem Improving representation quality in self-supervised learning.
method ProSMIN uses two neural networks, online and target, to learn diverse representations through knowledge distillation and a modified scoring rule loss function.
result ProSMIN achieves superior accuracy and calibration on various downstream tasks.
New method uses random convex polytopes to measure representation quality.
problem Measuring the quality of deep learning representations.
method Random Polytope Descriptor method based on random convex polytopes.
result Regularization in autoencoders can degrade out-of-distribution detection.
CDSSL improves representation quality by integrating linear and nonlinear dependencies.
problem Scarcity of labeled data and neglect of nonlinear dependencies in SSL.
method CDSSL combines linear correlations and nonlinear dependencies using HSIC in RKHS.
result CDSSL enhances representation quality on diverse benchmarks.
ProductNet is a collection of high-quality product datasets for better product understanding. Motivated by ImageNet, ProductNet aims at supporting product representation learning by curating product datasets of high quality with properly chosen taxonomy. In this paper, the two goals of building high-quality product dat…
Learning knowledge representation is an increasingly important technology that supports a variety of machine learning related applications. However, the choice of hyperparameters is seldom justified and usually relies on exhaustive search. Understanding the effect of hyperparameter combinations on embedding quality is …
Semantic representations of words have been successfully extracted from unlabeled corpuses using neural network models like word2vec. These representations are generally high quality and are computationally inexpensive to train, making them popular. However, these approaches generally fail to approximate out of vocabul…
The paper uses thermodynamics to improve machine learning representation quality.
problem Improving the quality of learned representations for transfer learning.
method Formal connection with thermodynamics, iso-classification process, traversing the equilibrium surface.
result Demonstrates how to transfer representations while keeping classification loss constant.
C-VAE improves VAE by resolving prior issues and generating better samples.
problem Low-quality samples from VAE due to prior issues.
method Formulates VAE as OT, allows flexible priors, and uses OT formulations.
result C-VAE generates higher quality samples and latent representations.
New autoencoder learns structured representations without regularization.
problem Learning structured representations without relying on regularization.
method Proposes a novel autoencoder architecture that learns a hierarchy of latent variables.
result Improves results in generation, disentanglement, and extrapolation tasks.
Generative model improves time series prediction quality.
problem Lack of effective generative models for time series data.
method Proposed XIRP for time series, trained with WGAN-GP.
result XIRP significantly outperforms RNN-based models in predictive ability.
Partial soft-matching distance improves neural representation comparison by allowing some neurons to remain unmatched.
problem Neural representations are noisy and contain outliers, making traditional matching methods unreliable.
method Extends soft-matching distance to a partial optimal transport setting, allowing some neurons to remain unmatched.
result Partial soft-matching provides robust correspondences that are more reliable under noise and outliers.
Learning the right graph representation from noisy, multisource data has garnered significant interest in recent years. A central tenet of this problem is relational learning. Here the objective is to incorporate the partial information each data source gives us in a way that captures the true underlying relationships.…
Pre-training text representations have led to significant improvements in many areas of natural language processing. The quality of these models benefits greatly from the size of the pretraining corpora as long as its quality is preserved. In this paper, we describe an automatic pipeline to extract massive high-quality…
This work improves disentanglement in latent space models without sacrificing generation quality.
problem Trade-off between disentanglement and generation quality in latent space models.
method Manifold optimization with a sum of autoencoder and PCA reconstruction errors, on the Stiefel manifold.
result Improves disentanglement without sacrificing generation quality.
The recently introduced continuous Skip-gram model is an efficient method for learning high-quality distributed vector representations that capture a large number of precise syntactic and semantic word relationships. In this paper we present several extensions that improve both the quality of the vectors and the traini…
Contrastive learning struggles with class collapse and feature suppression, revealing bias towards simpler solutions.
problem Contrastive learning struggles with class collapse and feature suppression, especially in supervised and unsupervised settings.
method Unified theoretical framework to determine which features are learnt by CL, revealing bias towards simpler solutions.
result Bias towards simpler solutions is a key factor in class collapse and feature suppression.
Enhances DIM to match learned representations to a specific distribution.
problem Learning representations conforming to a specific distribution.
method Injecting noise into normalized outputs of the encoder while keeping the InfoMax training objective.
result Learning uniformly and normally distributed representations, as well as representations of other absolutely continuous distributions.
Improved texture synthesis using wavelet-based statistics with rectifier non-linearity.
problem Improving texture synthesis quality using wavelet representations.
method Proposes a family of statistics based on non-linear wavelet representations with a generalized rectifier non-linearity.
result Significantly improves visual quality of texture synthesis compared to classical wavelet-based models.
NeuCrowd creates high-quality samples from crowdsourced labels to improve representation learning.
problem Limited and inconsistent crowdsourced labels for representation learning.
method Unified framework that generates high-quality n-tuplet samples and learns a neural sampling network.
result NeuCrowd outperforms state-of-the-art baselines in prediction accuracy and AUC.
New method learns high-quality Laplacian representations for reinforcement learning.
problem Lack of accurate Laplacian representations in large or continuous state spaces.
method Reformulated spectral graph drawing objective to have eigenvectors as unique global minimizer.
result Learned Laplacian representations more faithfully approximate the ground truth.
Bayesian SHMM models speech units from unannotated speech.
problem Discover latent speech units from unannotated data.
method Bayesian Subspace Hidden Markov Model (SHMM).
result System performs well on synthesis quality, lower bitrate.
GGAN improves audio representation learning with fewer labels.
problem Learning representations for specific tasks from unlabelled data.
method Guided Generative Adversarial Neural Network (GGAN).
result GGAN learns better representations with fewer labelled data.
Bayesian algorithm improves word representations using semantic taxonomy.
problem Improving word representations in semantic taxonomy.
method Bayesian Hierarchical Words Representation (BHWR) learning algorithm combining Variational Bayes and semantic taxonomy modeling.
result BHWR produces better representations for rare words.
Proposes GM Score to evaluate GANs considering diversity, disentanglement, and discriminability.
problem Evaluation of GANs for sample quality and diversity.
method Integrates various factors including intra-class and inter-class diversity, disentanglement, and discriminability metrics.
result Demonstrates improved evaluation of GANs on MNIST dataset.
The paper examines the reliability of limit order book representations in the face of data perturbation.
problem The reliability of limit order book representations under data perturbation.
method Experimental analysis of existing representations and guidelines for future research.
result Existing representations of limit order book data are vulnerable to data perturbation.
This paper addresses two crucial problems of learning disentangled image representations, namely controlling the degree of disentanglement during image editing, and balancing the disentanglement strength and the reconstruction quality. To encourage disentanglement, we devise a distance covariance based decorrelation re…
In this work, we address the problem of musical timbre transfer, where the goal is to manipulate the timbre of a sound sample from one instrument to match another instrument while preserving other musical content, such as pitch, rhythm, and loudness. In principle, one could apply image-based style transfer techniques t…
Introduces TT-NF for more compact neural field representations.
problem Finding more compact and easy-to-fit neural field representations.
method Tensor Train parameterization trained with backpropagation.
result Low-rank compression improves downstream task quality metrics.
Generates high-quality images using sparse DCT representations.
problem Challenges in generating images due to high dimensionality.
method Transformers trained on sparse DCT block sequences.
result Competitive image generation quality with state-of-the-art methods.
In many situations, we need to build and deploy separate models in related environments with different data qualities. For example, an environment with strong observation equipments (e.g., intensive care units) often provides high-quality multi-modal data, which are acquired from multiple sensory devices and have rich-…
Improves shared encoder representations for better multi-task learning performance.
problem Improving quality of shared encoder representations in multi-task learning.
method Dummy Gradient norm Regularization (DGR) to decrease gradient norm of dummy task-specific predictors.
result DGR improves multi-task prediction performances and superior performance compared to existing methods.
We explore the question of whether the representations learned by classifiers can be used to enhance the quality of generative models. Our conjecture is that labels correspond to characteristics of natural data which are most salient to humans: identity in faces, objects in images, and utterances in speech. We propose …
In order to efficiently transmit and store speech signals, speech codecs create a minimally redundant representation of the input signal which is then decoded at the receiver with the best possible perceptual quality. In this work we demonstrate that a neural network architecture based on VQ-VAE with a WaveNet decoder …
Improves model classification accuracy in black-box settings.
problem Difficulty in inferring model properties due to limited query access.
method Introduces discriminative factorization to distinguish high-quality queries.
result Probability of chance-level classification decreases exponentially with query budget.
Air quality forecasting has been regarded as the key problem of air pollution early warning and control management. In this paper, we propose a novel deep learning model for air quality (mainly PM2.5) forecasting, which learns the spatial-temporal correlation features and interdependence of multivariate air quality rel…
Conditional GANs are at the forefront of natural image synthesis. The main drawback of such models is the necessity for labeled data. In this work we exploit two popular unsupervised learning techniques, adversarial training and self-supervision, and take a step towards bridging the gap between conditional and uncondit…
Improves disentangled representation learning with multi-stage modeling.
problem Trade-off between disentanglement and reconstruction quality in autoencoders.
method Penalty-based disentanglement learning followed by detail information modeling.
result Higher reconstruction quality than state-of-the-art methods with equivalent disentanglement.
SoftCLT improves time series representation learning by soft contrastive loss.
problem Ignoring inherent correlations in time series leads to poor representation quality.
method SoftCLT introduces instance-wise and temporal contrastive loss with soft assignments.
result SoftCLT consistently improves various downstream tasks in time series learning.
New methods improve feature extraction and representation quality in supervised and unsupervised DR.
problem Statistical dependence, data diversity, contrast, and interpretability in conventional DR methods.
method Combines linear and nonlinear formulations for three new independence criteria.
result Significant improvements in contrast, accuracy, and interpretability over baselines.
DDMI generates high-quality INRs by adapting positional embeddings.
problem Existing INR generative models fail to produce high-quality representations.
method DDMI uses adaptive positional embeddings and a D2C-VAE to enhance expressive power.
result DDMI outperforms existing models across multiple modalities and datasets.
In this paper, we build an organization of high-dimensional datasets that cannot be cleanly embedded into a low-dimensional representation due to missing entries and a subset of the features being irrelevant to modeling functions of interest. Our algorithm begins by defining coarse neighborhoods of the points and defin…
Hierarchical graph clustering is a common technique to reveal the multi-scale structure of complex networks. We propose a novel metric for assessing the quality of a hierarchical clustering. This metric reflects the ability to reconstruct the graph from the dendrogram, which encodes the hierarchy. The optimal represent…
VCAE improves autoencoder quality on MNIST and CelebA.
problem Overfitting and poor generative/reconstruction quality in autoencoders.
method Proposes variance-constrained autoencoder (VCAE) to enforce variance constraint on latent distribution.
result VCAE outperforms Wasserstein Autoencoder and Variational Autoencoder in quality.
Neural networks outperform kernels by learning features better.
problem Current theories of feature learning do not adequately assess feature quality.
method Introduced feature quality metric and examined existing theories empirically.
result Current theories of feature learning do not provide a sufficient foundation for neural network generalization.
Graph InfoClust learns node representations by capturing cluster-level information, improving graph mining tasks.
problem Leveraging cluster-level node information for unsupervised graph representation learning.
method Graph InfoClust (GIC) uses a differentiable K-means method to compute clusters and jointly optimizes mutual information between nodes of the same cluster.
result GIC outperforms state-of-the-art methods in various downstream tasks with a 0.9% to 6.1% gain.
Representation learning becomes especially important for complex systems with multimodal data sources such as cameras or sensors. Recent advances in reinforcement learning and optimal control make it possible to design control algorithms on these latent representations, but the field still lacks a large-scale standard …
Survey on concept factorization methods for better feature learning.
problem Understanding the quality of learned features in representation learning.
method Categorizing and summarizing recent advances in concept factorization methods.
result Provides insights into theoretical basis and current developments in concept factorization.