Analyzes how word meaning is captured by co-occurrence features.
problem Understanding the theoretical basis of word representation by co-occurrences.
method Theoretical analysis of word representation methods using co-occurrences.
result Using multiple context features improves word prediction scores.
Study proposes an alternative method to measure societal biases using smoothed co-occurrence relations.
problem Measuring societal biases using word embeddings can introduce irrelevant concepts.
method Proposes an alternative approach using smoothed first-order co-occurrence relations.
result First-order approach shows higher correlations with actual gender bias statistics.
A new method for topic detection using hierarchical latent tree models.
problem Hierarchical topic detection in document collections.
method Graphical models (HLTMs) with binary variables at different levels representing word co-occurrence patterns and document clusters.
result Captures both general and specific topics at different levels of a hierarchical structure.
LLMs learn new tasks from unstructured data, but it depends on word co-occurrence and positional information.
problem Understanding how LLMs can learn new tasks from unstructured data without explicit training.
method Examined the capabilities of LLMs trained on unstructured data, focusing on sequence model requirements and training data structure.
result Many ICL capabilities can emerge from word co-occurrence in unstructured data, but positional information is crucial for certain tasks.
The paper introduces tensor factorization for word embeddings.
problem Creating embeddings for words with multiple meanings.
method Tensor factorization of higher-order co-occurrence arrays.
result Tensor-based embeddings can discern polysemous words' meanings.
A new method selects anchor words for better topic discovery in text corpora.
problem Selecting anchor words for improved topic modeling in text corpora.
method Proposes a new greedy method to find a minimum edge-weight anchor clique in a word similarity graph.
result The proposed method outperforms existing methods on topic quality and is faster.
The paper models and predicts co-occurrence counts using Gamma regression.
problem Predicting relevance between items or users from high-dimensional sparse co-occurrence count data.
method Shared parameter alternating zero-inflated Gamma regression models (SA-ZIG) with Fisher scoring and learning rate adjustment.
result SA-ZIG with learning rate adjustment performs satisfactorily in predicting relevance.
Analyzes word vectors and co-occurrence statistics in NLP models.
problem Understanding biases in NLP models through co-occurrence statistics.
method Developed an analytic model of statistics learned by Word2Vec and GloVe, derived the first solution to Word2Vec's algorithm, and analyzed independence in co-occurrence models.
result Demonstrated a universal property of word vectors that can reveal biases in data before they are absorbed by DL models.
Latent topic models have been successfully applied as an unsupervised topic discovery technique in large document collections. With the proliferation of hypertext document collection such as the Internet, there has also been great interest in extending these approaches to hypertext [6, 9]. These approaches typically mo…
Paper improves relation extraction in clinical texts with limited data.
problem Relation extraction in narrow knowledge domains with scarce annotated data.
method Introduces a bag-of-concepts (BoC) model and compares it with window-bounded co-occurrence (WBC).
result BoC model outperforms baseline and other complex methods on small dataset.
This paper justifies and improves lexicon-based classification without labeled data.
problem Lack of justification for lexicon-based classification and its lower accuracy compared to supervised methods.
method Derives probabilistic justification and learns weights from co-occurrence statistics.
result Lexicon-based classification can be improved without labeled data, offering higher accuracy.
Develops efficient algorithm for topic discovery with novel geometric insights.
problem Discovering topics from documents with shared latent factors.
method Geometric insights from normalized word co-occurrence matrix and isotropic random projections.
result Provably efficient algorithm with polynomial computation and sample complexity bounds.
Generative model learns from multiple sources to embed words and relationships.
problem Lack of diverse training data in specialized domains.
method Integrates evidence from diverse data sources using affine transformations on semantic vector spaces.
result Outperforms recent models on link prediction tasks and partially observed data.
New algorithm learns HMM from pairwise co-occurrences, improving topic modeling.
problem Identifying hidden Markov models from limited pairwise co-occurrence data.
method Uses pairwise co-occurrence data to uniquely identify HMMs, even if higher-order probabilities are unknown.
result Shows improved topic modeling quality with HMMs compared to bag-of-words models.
Word embeddings in hyperbolic space outperform Euclidean ones.
problem Improving word embeddings for better performance.
method Learning word embeddings in hyperbolic space using skip-gram architecture and hyperbolic distance objective function.
result Hyperbolic word embeddings show potential, especially in low dimensions, but not clear superiority over Euclidean embeddings.
Robot learns word meanings from perception-action tasks.
problem Language acquisition for robots.
method Affordance network with temporal co-occurrence of speech and actions.
result Robot forms useful word-to-meaning associations.
Paper proposes a method to align word embedding models in a joint latent space.
problem Challenges in aligning variations of word embedding models.
method Generative process using synthetic data points based on linguistic relationships.
result Substantial improvements in recovering embeddings of local neighborhoods.
Paper learns identity-sensitive word embeddings from text corpora.
problem Lack of context-aware word embeddings.
method Constructs a heterogeneous network of words and identities, then embeds into a low-dimensional space.
result Identity-sensitive word embeddings capture different meanings of words.
Semantic word embeddings represent the meaning of a word via a vector, and are created by diverse methods. Many use nonlinear operations on co-occurrence statistics, and have hand-tuned hyperparameters and reweighting methods. This paper proposes a new generative model, a dynamic version of the log-linear topic model o…
Advances word embedding methods using Markov processes.
problem Improving semantic understanding of word representations.
method Ground embeddings in cognitive literature, unify and generalize metric recovery methods.
result New algorithms for metric recovery and embedding on graphs and manifolds.
New methods for unsupervised learning of word and entity representations.
problem Learning distributed representations of words and entities from text and knowledge bases.
method MVLSA for words and NVSE for entities, both unsupervised learning methods.
result MVLSA and NVSE outperform state-of-the-art models in word and entity representation learning.
This paper improves topic modeling by embedding words and topics together.
problem Topic models struggle with short documents and approximate inference.
method Model each document as a mixture of word embeddings and each topic as a mixture of topic embeddings.
result The method optimizes topic embeddings to minimize semantic differences between words and topics.
The abstract explains how word and relation representations capture semantic meaning.
problem Understanding how word and relation representations capture semantic meaning.
method Theoretical justification and extension of geometric relationships between word embeddings and knowledge graph representations.
result The geometric relationships between word embeddings correspond to semantic relations between words and entities in knowledge graphs.
EigenNoise provides a competitive word vector initialization scheme without pre-training data.
problem Improving word vector initialization without pre-training data.
method EigenNoise uses a dense, independent co-occurrence model to initialize word vectors.
result EigenNoise can approach GloVe performance without pre-training data.
Develops a measure-theoretic framework for complex co-occurrence data.
problem Modeling and interpreting complex co-occurrences in high-dimensional data.
method Introduces measure-theoretic probability and conditional probability, investigates E-integrals.
result Establishes a rigorous measure-theoretic foundation for co-occurrence modeling.
A new method for higher-order co-occurrences in hypergraphs.
problem Computing higher-order co-occurrences in hypergraphs.
method Face-splitting product or transpose Khatri-Rao product for higher order tuple co-occurrences.
result Demonstrates the utility of the higher order co-occurrence tensor in NLP and hypergraph models.
GEMRank embeds users and items using co-occurrence relations for better collaborative filtering.
problem Lack of textual data for entity embedding in recommender systems.
method Uses profile co-occurrence for entity relations and factorization for embedding. Feeds embeddings into a neural network for predictions.
result Significantly outperforms baseline algorithms in various data sets.
A new model generates summaries by conditioning on input text and latent topics.
problem Improving abstractive summarization quality.
method Conditioning decoder output on both input text and latent topics identified by LDA.
result Strongly improved ROUGE scores on CNN/Daily Mail and WikiHow datasets.
Matrix factorization simplifies user-item co-occurrence analysis.
problem Understanding the meaning of low-dimensional matrices in matrix factorization.
method Showed matrix factorization equals calculating eigenvectors of co-occurrence matrices, using RMT insights.
result Low-dimension matrices represent a reduced noise user and item co-occurrence space.
New method tests weighted networks without thresholding, improving accuracy.
problem Testing and anomaly detection on weighted network data.
method Hierarchical Bayesian hypothesis testing framework for weighted networks.
result Method shows lower Type I error and higher statistical power compared to alternatives.
Advances citation and subject label recommendation using multi-modal adversarial autoencoders.
problem Improving recommendation systems for citations and subject labels.
method Multi-modal adversarial autoencoders with adversarial regularization, sparsity, and input modality analysis.
result Adversarial regularization consistently improves recommendation performance.
New algorithm improves topic modeling for short texts.
problem Inference of latent topics from short texts is challenging.
method Developed a stochastic divergence minimization algorithm for Biterm Topic Model (BTM).
result The new algorithm estimates latent topics more accurately and efficiently.
Transformers learn topic structure through embedding and attention mechanisms.
problem Understanding how transformers capture semantic structure in text.
method Combination of mathematical analysis and experiments on Wikipedia and synthetic data.
result Embedding and attention layers encode topic structure in transformers.
Matrix Chernoff bound for Markov chains applied to co-occurrence matrices.
problem Analyzing the behavior of co-occurrence statistics in sequential data.
method Proved a matrix Chernoff-type bound for sums of matrix-valued random variables sampled via a regular Markov chain.
result Achieved exponentially fast convergence rate and sample complexity analysis for co-occurrence matrices.
BBM models short texts using biterms to improve coherence.
problem Challenges in analyzing short texts from social media.
method Bag of Biterms (BoB) for document representation and simple statistical models.
result BBM enhances coherence and performance over traditional models.
Proposes a method for two-sided clustering of co-occurrence data.
problem Efficient clustering of co-occurrence data in multi-view settings.
method Information-theoretic multi-view co-clustering (MV-ITCC).
result Demonstrates superior performance on text and image datasets.
Paper proposes a new method for accurate data labeling using pairwise co-occurrences.
problem Accurate data labeling via crowdsourcing with limited data.
method Pairwise co-occurrences framework and algebraic/identifiability-enhanced algorithms.
result The approach can identify the Dawid-Skene model under realistic conditions.
Chromatic Learning reduces feature dimensions for sparse datasets.
problem Sparse, high-dimensional data challenges traditional learning methods.
method Graph coloring over co-occurrence graph to create dense feature representation.
result Compresses sparse datasets significantly while maintaining model accuracy.
Person re-identification (re-id), an emerging problem in visual surveillance, deals with maintaining entities of individuals whilst they traverse various locations surveilled by a camera network. From a visual perspective re-id is challenging due to significant changes in visual appearance of individuals in cameras wit…
Study on learning distributions from multiple data providers with restricted samples.
problem Learning an unknown distribution from conditional samples given a fixed family of queryable sets.
method Studied a stylized model of distribution learning from restricted conditional samples, focusing on the co-occurrence graph associated with the queryable sets.
result The optimal sample complexity of PAC learning ranges from nearly linear to quadratic, depending on the structure of the queryable sets.
A fast kernel-based measure for sparse linguistic expressions.
problem Efficiently measuring co-occurrence in sparse linguistic data.
method Derives PHSIC from HSIC, estimates it linearly, and uses various kernels.
result Empirically, PHSIC outperforms PMI in accuracy and learning speed.
Probabilistic Latent Semantic Analysis is a novel statistical technique for the analysis of two-mode and co-occurrence data, which has applications in information retrieval and filtering, natural language processing, machine learning from text, and in related areas. Compared to standard Latent Semantic Analysis which s…
An important problem in multi-label classification is to capture label patterns or underlying structures that have an impact on such patterns. This paper addresses one such problem, namely how to exploit hierarchical structures over labels. We present a novel method to learn vector representations of a label space give…
New DTMs model text evolution with Gaussian processes and scalable inference.
problem Challenges in modeling text evolution with continuous stochastic processes.
method Extended tractable priors to Gaussian processes and developed scalable inference methods.
result Found interesting patterns in large-scale datasets not accessible before.
Study examines how cluster number affects short-text clustering, introducing a stability metric.
problem Challenges in finding meaningful clusters in short-text data.
method Introduces a stability metric to determine cluster robustness and visualizes cluster subdivisions.
result Choosing a cluster number involves balancing informativeness and complexity, not seeking a single 'optimal' solution.
Deep learning solves jigsaw puzzles by classifying fragment positions.
problem Automated reconstruction of archaeological fragments from jigsaw puzzles.
method Classifies relative positions of fragments using deep neural networks and local feature co-occurrences.
result Our method outperforms state-of-the-art by 25%.
DenseHMM improves HMMs by learning dense representations that enable gradient-based optimization.
problem Learning dense representations for hidden states and observables in HMMs.
method DenseHMM uses kernelized transition probabilities and two optimization schemes.
result DenseHMM achieves superior performance and expressiveness compared to standard HMMs.
A new test statistic counts tree co-occurrences to detect edge correlation between networks.
problem Detecting edge correlation between networks using latent vertex correspondence.
method The test statistic is based on counting co-occurrences of signed trees for a family of non-isomorphic trees.
result The test runs in n2+o(1) time and succeeds with high probability for large n.