The paper evaluates different vector space models for text similarity.
problem Measuring semantic text similarity in natural language processing.
method Comparison of TFIDF, topic models, and neural models for patent-to-patent similarity.
result TFIDF performs well for longer, technical texts or finer distinctions.
We discuss memory models which are based on tensor decompositions using latent representations of entities and events. We show how episodic memory and semantic memory can be realized and discuss how new memory traces can be generated from sensory input: Existing memories are the basis for perception and new memories ar…
The paper updates SVD of evolving matrices using projection techniques.
problem Updating the rank-k truncated SVD of evolving matrices.
method Projection viewpoint, building subspaces to approximate singular vectors.
result The proposed algorithm leads to higher accuracy, especially for large singular values.
This work provides uncertainty intervals for semantic latent variables in disentangled latent spaces.
problem Challenges in providing meaningful uncertainty quantification for semantic information in disentangled latent spaces.
method Uses quantile regression to output heuristic uncertainty intervals, calibrates these intervals to contain true latent values, and propagates them through the generator.
result Reliably communicates semantically meaningful, principled, and instance-adaptive uncertainty in image super-resolution and image completion.
Semantic TrueLearn uses semantic graphs to improve educational recommendation systems.
problem Challenges in handling semantic and hierarchical structure in knowledge areas.
method Introduces a novel learner model that exploits semantic relatedness between knowledge components using a Wikipedia link graph.
result Achieves statistically significant improvements in predictive performance for educational engagement.
Paper proposes structured semantic perturbations to improve adversarial attacks.
problem Vulnerability of deep neural networks to adversarial attacks.
method Manipulates semantic attributes via disentangled latent codes.
result Demonstrates the effectiveness of structured semantic perturbations.
Probabilistic Latent Semantic Analysis is a novel statistical technique for the analysis of two-mode and co-occurrence data, which has applications in information retrieval and filtering, natural language processing, machine learning from text, and in related areas. Compared to standard Latent Semantic Analysis which s…
This work prevents variational autoencoders from collapsing by adding an auxiliary decoder.
problem Variational autoencoders can collapse into autodecoders, losing semantic information.
method Adding an auxiliary decoder to regularize the latent space.
result Auxiliary decoders increase semantic information in the latent space and reconstructions.
New method neutralizes gender bias in word embeddings without losing semantic information.
problem Gender biases in word embeddings trained on human-generated corpora.
method Latent Disentanglement and Counterfactual Generation with siamese auto-encoder and gradient reversal layer.
result Our method outperforms existing debiasing methods in preserving semantic information and neutralizing gender biases.
In this tutorial, I will discuss the details about how Probabilistic Latent Semantic Analysis (PLSA) is formalized and how different learning algorithms are proposed to learn the model.
Enhances word embedding by transferring external knowledge.
problem Low-frequency words in semantic space.
method Latent Semantic Imputation (LSI) integrating graph theory and spectral embeddings.
result LSI generates reliable embedding vectors for low-frequency words.
Method interprets GAN latent space via latent variable correlation analysis.
problem Understanding the inner workings of GANs.
method Analyzing correlation between latent variables and semantic contents in generated images.
result A method for controllable semantic content generation in GANs.
New framework aligns latent representations over-the-air using intelligent metasurfaces.
problem Heterogeneous transmitter-receiver models produce misaligned latent representations in semantic communication.
method Intelligent metasurfaces (SIM) emulate supervised and zero-shot semantic aligners directly in the wave domain.
result SIMs achieve up to 90% task accuracy in high SNR regimes, robust to low SNR.
Top2Vec finds topic vectors from documents and words without needing stop words or custom settings.
problem Topic modeling weaknesses, including needing known topics, stop words, and custom settings.
method Joint document and word semantic embedding to find topic vectors automatically.
result Top2Vec finds more informative and representative topics than probabilistic models.
SEMASIA provides a large dataset of latent representations for model comparison.
problem Difficulty in comparing semantic structures across different neural network models.
method Collection of latent representations from 1700 pretrained models across various benchmarks.
result Consistent semantic organization across models and datasets.
The paper predicts financial markets using news text and semantic network analysis.
problem Predicting financial markets with news data.
method Semantic network analysis of news text to assess economic keywords' importance.
result The index captures financial market phases and predicts returns and volatilities.
In this paper, we propose TopicRNN, a recurrent neural network (RNN)-based language model designed to directly capture the global semantic meaning relating words in a document via latent topics. Because of their sequential nature, RNNs are good at capturing the local structure of a word sequence - both semantic and syn…
The ability of the Generative Adversarial Networks (GANs) framework to learn generative models mapping from simple latent distributions to arbitrarily complex data distributions has been demonstrated empirically, with compelling results showing that the latent space of such generators captures semantic variation in the…
This paper uses natural language processing to create the first machine-coded democracy index, which I call Automated Democracy Scores (ADS). The ADS are based on 42 million news articles from 6,043 different sources and cover all independent countries in the 1993-2012 period. Unlike the democracy indices we have today…
Unified framework for disentangled VAEs improves latent space interpretability.
problem Challenges in evaluating and interpreting latent representations, especially for diverse data types.
method Unified bfVAE framework, FVH-LT, DBSR-LS, GAS, LSSI.
result bfVAE provides more favorable trade-off between disentanglement and reconstruction.
Improved generalization with semantic perturbations using normalizing flows.
problem Overfitting in deep neural networks training.
method Use normalizing flows for generating semantically meaningful perturbations in latent space.
result Achieved 96.6% test accuracy on CIFAR-10 with ResNet-18, outperforming existing methods.
Introduces PCG for better counterfactual explanations in vision models.
problem Ambiguity in latent-space optimization methods for counterfactual explanations.
method Constructs counterfactuals by tracing geodesics under a perceptually Riemannian metric.
result PCG outperforms baselines and reveals hidden failure modes.
Unsupervised method discovers interpretable directions in GAN latent space.
problem Discovering interpretable directions in GAN latent space without supervision.
method Model-agnostic procedure to identify directions corresponding to semantic manipulations.
result Findings include directions for background removal and competitive saliency detection performance.
CADD improves generative quality by augmenting discrete diffusion with continuous latent space.
problem Loss of semantic information between denoising steps in discrete diffusion models.
method Introduces a framework that augments discrete state space with a continuous latent space, allowing for graded, informative masked tokens.
result CADD improves generative quality across text generation, image synthesis, and code modeling.
The paper tests deep music embeddings for semantic consistency.
problem Ensuring deep music embeddings capture meaningful musical semantics.
method Proposes a systematic method to test deep music representations for semantic consistency, considering both input audio space and latent deep space.
result Distance consistency between related points is maintained in both input audio space and latent deep space.
Unsupervised framework learns latent codes for controllable generation.
problem Challenging to achieve controllable generation with GANs.
method Self-training iterative feedback from discriminator to generator.
result Better disentanglement and semantic meaningful latent codes.
The problem of searching for experts in a given academic field is hugely important in both industry and academia. We study exactly this issue with respect to a database of authors and their publications. The idea is to use Latent Semantic Indexing (LSI) and Latent Dirichlet Allocation (LDA) to perform topic modelling i…
We would like to learn a representation of the data which decomposes an observation into factors of variation which we can independently control. Specifically, we want to use minimal supervision to learn a latent representation that reflects the semantics behind a specific grouping of the data, where within a group the…
We propose a method to infer domain-specific models such as classifiers for unseen domains, from which no data are given in the training phase, without domain semantic descriptors. When training and test distributions are different, standard supervised learning methods perform poorly. Zero-shot domain adaptation attemp…
An increasing number of people are using online social networking services (SNSs), and a significant amount of information related to experiences in consumption is shared in this new media form. Text mining is an emerging technique for mining useful information from the web. We aim at discovering in particular tweets s…
Linear Discriminant Analysis (LDA) is a well-known method for dimensionality reduction and classification. Previous studies have also extended the binary-class case into multi-classes. However, many applications, such as object detection and keyframe extraction cannot provide consistent instance-label pairs, while LDA …
New hashing method improves document retrieval precision.
problem Efficiently retrieving similar documents from large text databases.
method Pairwise supervised hashing with Bernoulli VAE and unbiased gradient estimator.
result Superior performance compared to existing methods.
Low-dimensional vectors improve semantic understanding of music and language.
problem Noise in shared semantics due to individual brain biases.
method Jointly model multiple brains to learn low-dimensional vector embeddings.
result These embeddings outperform high-dimensional fMRI data in music and language classification.
Online reviews provided by consumers are a valuable asset for e-Commerce platforms, influencing potential consumers in making purchasing decisions. However, these reviews are of varying quality, with the useful ones buried deep within a heap of non-informative reviews. In this work, we attempt to automatically identify…
Improves GANs by sampling meaningful points from latent manifold.
problem Lack of semantic information in GANs' prior distribution.
method Local Coordinate Coding (LCC) for sampling and improved generator approximation.
result Improves GANs' performance in generating realistic data.
The study categorizes knowledge graph relations and explains their embedding representations.
problem Understanding how knowledge graph relation representations capture semantic information.
method Categorizing knowledge graph relations into three types and deriving explicit requirements for their representations.
result Empirical properties of relation representations and the performance of methods are justified by the analysis.
New method uses cycle consistency to enforce invariance in latent space.
problem Learning meaningful and independent factors of variation in datasets.
method Two separate latent subspaces, cycle consistency constraints, deep information bottleneck.
result Identifies more meaningful factors leading to sparser and interpretable models.
Based on the Aristotelian concept of potentiality vs. actuality allowing for the study of energy and dynamics in language, we propose a field approach to lexical analysis. Falling back on the distributional hypothesis to statistically model word meaning, we used evolving fields as a metaphor to express time-dependent c…
Deep Mixtures of Unigrams improve clustering of high-dimensional sparse text data.
problem Challenging unsupervised classification of very short documents with many terms.
method Developed a deep version of mixtures of Unigrams in a Bayesian framework.
result Deep Mixtures of Unigrams outperform traditional methods in classification accuracy.
Paper proposes using semantic spaces and traditional classifiers for sarcasm detection.
problem Sarcasm detection in text, especially on social media.
method Apply classical machine learning algorithms to texts represented in a Latent Semantic space.
result Established reference datasets and baselines for sarcasm detection.
Proposes priors for interpretable word embeddings.
problem Lack of interpretability and unsupervised nature of word embeddings.
method Use of informative priors to create interpretable probabilistic word embeddings.
result Sensible priors capture latent semantic concepts better than current methods.
In the current context of data explosion, online techniques that do not require storing all data in memory are indispensable to routinely perform tasks like principal component analysis (PCA). Recursive algorithms that update the PCA with each new observation have been studied in various fields of research and found wi…
This study analyzes data science vocabulary changes over 13 years.
problem Understanding evolution of data science terms over time.
method Exploratory Data Analysis, Latent Semantic Analysis, Latent Dirichlet Analysis, N-grams Analysis.
result Identified new vocabulary and its incorporation into scientific literature.
The abstract explains how word and relation representations capture semantic meaning.
problem Understanding how word and relation representations capture semantic meaning.
method Theoretical justification and extension of geometric relationships between word embeddings and knowledge graph representations.
result The geometric relationships between word embeddings correspond to semantic relations between words and entities in knowledge graphs.
New memory allocation scheme improves image generation performance.
problem Improving episodic and semantic memory representation in neural networks.
method Developed a hierarchical latent variable model with differentiable, locally block allocated latent memory.
result Improved conditional likelihood values on various datasets.
Efficient BNNs learn latent distributions for robust uncertainty quantification.
problem Improving robustness and uncertainty of deep neural networks.
method LP-BNN uses VAEs to learn latent distributions of BNN parameters, enabling efficient ensembles.
result LP-BNN achieves competitive results in image classification, semantic segmentation, and out-of-distribution detection.
New diffusion models improve counterfactual image generation with semantic control.
problem Challenges in preserving identity, maintaining quality, and ensuring causal model faithfulness in counterfactual image generation.
method Integrates semantic representations into diffusion models through Pearlian causality, introducing spatial, semantic, and dynamic abduction.
result Demonstrates high-level semantic identity preservation and principled trade-offs between faithful causal control and identity preservation.
We develop a method for user-controllable semantic image inpainting: Given an arbitrary set of observed pixels, the unobserved pixels can be imputed in a user-controllable range of possibilities, each of which is semantically coherent and locally consistent with the observed pixels. We achieve this using a deep generat…