New algorithm learns HMM from pairwise co-occurrences, improving topic modeling.
problem Identifying hidden Markov models from limited pairwise co-occurrence data.
method Uses pairwise co-occurrence data to uniquely identify HMMs, even if higher-order probabilities are unknown.
result Shows improved topic modeling quality with HMMs compared to bag-of-words models.
A new method for higher-order co-occurrences in hypergraphs.
problem Computing higher-order co-occurrences in hypergraphs.
method Face-splitting product or transpose Khatri-Rao product for higher order tuple co-occurrences.
result Demonstrates the utility of the higher order co-occurrence tensor in NLP and hypergraph models.
Paper proposes a new method for accurate data labeling using pairwise co-occurrences.
problem Accurate data labeling via crowdsourcing with limited data.
method Pairwise co-occurrences framework and algebraic/identifiability-enhanced algorithms.
result The approach can identify the Dawid-Skene model under realistic conditions.
Study on learning distributions from multiple data providers with restricted samples.
problem Learning an unknown distribution from conditional samples given a fixed family of queryable sets.
method Studied a stylized model of distribution learning from restricted conditional samples, focusing on the co-occurrence graph associated with the queryable sets.
result The optimal sample complexity of PAC learning ranges from nearly linear to quadratic, depending on the structure of the queryable sets.
Develops a measure-theoretic framework for complex co-occurrence data.
problem Modeling and interpreting complex co-occurrences in high-dimensional data.
method Introduces measure-theoretic probability and conditional probability, investigates E-integrals.
result Establishes a rigorous measure-theoretic foundation for co-occurrence modeling.
Matrix factorization simplifies user-item co-occurrence analysis.
problem Understanding the meaning of low-dimensional matrices in matrix factorization.
method Showed matrix factorization equals calculating eigenvectors of co-occurrence matrices, using RMT insights.
result Low-dimension matrices represent a reduced noise user and item co-occurrence space.
Analyzes how word meaning is captured by co-occurrence features.
problem Understanding the theoretical basis of word representation by co-occurrences.
method Theoretical analysis of word representation methods using co-occurrences.
result Using multiple context features improves word prediction scores.
Advances citation and subject label recommendation using multi-modal adversarial autoencoders.
problem Improving recommendation systems for citations and subject labels.
method Multi-modal adversarial autoencoders with adversarial regularization, sparsity, and input modality analysis.
result Adversarial regularization consistently improves recommendation performance.
Matrix Chernoff bound for Markov chains applied to co-occurrence matrices.
problem Analyzing the behavior of co-occurrence statistics in sequential data.
method Proved a matrix Chernoff-type bound for sums of matrix-valued random variables sampled via a regular Markov chain.
result Achieved exponentially fast convergence rate and sample complexity analysis for co-occurrence matrices.
The paper models and predicts co-occurrence counts using Gamma regression.
problem Predicting relevance between items or users from high-dimensional sparse co-occurrence count data.
method Shared parameter alternating zero-inflated Gamma regression models (SA-ZIG) with Fisher scoring and learning rate adjustment.
result SA-ZIG with learning rate adjustment performs satisfactorily in predicting relevance.
Proposes a method for two-sided clustering of co-occurrence data.
problem Efficient clustering of co-occurrence data in multi-view settings.
method Information-theoretic multi-view co-clustering (MV-ITCC).
result Demonstrates superior performance on text and image datasets.
Anomaly detection plays an important role in modern data-driven security applications, such as detecting suspicious access to a socket from a process. In many cases, such events can be described as a collection of categorical values that are considered as entities of different types, which we call heterogeneous categor…
Study proposes an alternative method to measure societal biases using smoothed co-occurrence relations.
problem Measuring societal biases using word embeddings can introduce irrelevant concepts.
method Proposes an alternative approach using smoothed first-order co-occurrence relations.
result First-order approach shows higher correlations with actual gender bias statistics.
Analyzes word vectors and co-occurrence statistics in NLP models.
problem Understanding biases in NLP models through co-occurrence statistics.
method Developed an analytic model of statistics learned by Word2Vec and GloVe, derived the first solution to Word2Vec's algorithm, and analyzed independence in co-occurrence models.
result Demonstrated a universal property of word vectors that can reveal biases in data before they are absorbed by DL models.
Framework uncovers symmetric and asymmetric species associations from data.
problem Retrieving bidirectional species associations from co-occurrence data.
method Machine learning framework modeling latent embeddings and joint generative model.
result Framework successfully recovers known symmetric and asymmetric associations.
We present a novel method for hierarchical topic detection where topics are obtained by clustering documents in multiple ways. Specifically, we model document collections using a class of graphical models called hierarchical latent tree models (HLTMs). The variables at the bottom level of an HLTM are observed binary va…
Chromatic Learning reduces feature dimensions for sparse datasets.
problem Sparse, high-dimensional data challenges traditional learning methods.
method Graph coloring over co-occurrence graph to create dense feature representation.
result Compresses sparse datasets significantly while maintaining model accuracy.
Paper improves relation extraction in clinical texts with limited data.
problem Relation extraction in narrow knowledge domains with scarce annotated data.
method Introduces a bag-of-concepts (BoC) model and compares it with window-bounded co-occurrence (WBC).
result BoC model outperforms baseline and other complex methods on small dataset.
Person re-identification (re-id), an emerging problem in visual surveillance, deals with maintaining entities of individuals whilst they traverse various locations surveilled by a camera network. From a visual perspective re-id is challenging due to significant changes in visual appearance of individuals in cameras wit…
LLMs learn new tasks from unstructured data, but it depends on word co-occurrence and positional information.
problem Understanding how LLMs can learn new tasks from unstructured data without explicit training.
method Examined the capabilities of LLMs trained on unstructured data, focusing on sequence model requirements and training data structure.
result Many ICL capabilities can emerge from word co-occurrence in unstructured data, but positional information is crucial for certain tasks.
A fast kernel-based measure for sparse linguistic expressions.
problem Efficiently measuring co-occurrence in sparse linguistic data.
method Derives PHSIC from HSIC, estimates it linearly, and uses various kernels.
result Empirically, PHSIC outperforms PMI in accuracy and learning speed.
Probabilistic Latent Semantic Analysis is a novel statistical technique for the analysis of two-mode and co-occurrence data, which has applications in information retrieval and filtering, natural language processing, machine learning from text, and in related areas. Compared to standard Latent Semantic Analysis which s…
An important problem in multi-label classification is to capture label patterns or underlying structures that have an impact on such patterns. This paper addresses one such problem, namely how to exploit hierarchical structures over labels. We present a novel method to learn vector representations of a label space give…
Transformers learn topic structure through embedding and attention mechanisms.
problem Understanding how transformers capture semantic structure in text.
method Combination of mathematical analysis and experiments on Wikipedia and synthetic data.
result Embedding and attention layers encode topic structure in transformers.
Deep learning solves jigsaw puzzles by classifying fragment positions.
problem Automated reconstruction of archaeological fragments from jigsaw puzzles.
method Classifies relative positions of fragments using deep neural networks and local feature co-occurrences.
result Our method outperforms state-of-the-art by 25%.
DenseHMM improves HMMs by learning dense representations that enable gradient-based optimization.
problem Learning dense representations for hidden states and observables in HMMs.
method DenseHMM uses kernelized transition probabilities and two optimization schemes.
result DenseHMM achieves superior performance and expressiveness compared to standard HMMs.
A new test statistic counts tree co-occurrences to detect edge correlation between networks.
problem Detecting edge correlation between networks using latent vertex correspondence.
method The test statistic is based on counting co-occurrences of signed trees for a family of non-isomorphic trees.
result The test runs in n2+o(1) time and succeeds with high probability for large n. Latent topic models have been successfully applied as an unsupervised topic discovery technique in large document collections. With the proliferation of hypertext document collection such as the Internet, there has also been great interest in extending these approaches to hypertext [6, 9]. These approaches typically mo…
BGNN improves GNN by modeling interactions between neighbor nodes.
problem Existing GNN models fail to capture interactions between neighbor nodes, leading to suboptimal performance.
method Proposes a new graph convolution operator that augments the weighted sum with pairwise interactions of neighbor nodes.
result Empirical results show BGNN models outperform traditional GNN models in node classification accuracy.
This paper analyzes DeepWalk and node2vec for community detection in large networks.
problem Community detection in large, sparse networks.
method Low-dimensional network embedding algorithms (DeepWalk and node2vec) applied to random walk segments.
result The performance of DeepWalk and node2vec in recovering communities depends on the length of random walk segments and sparsity of the network.
We develop necessary and sufficient conditions and a novel provably consistent and efficient algorithm for discovering topics (latent factors) from observations (documents) that are realized from a probabilistic mixture of shared latent factors that have certain properties. Our focus is on the class of topic models in …
Unified framework for comparing clusterings from information-theoretic and pair-counting perspectives.
problem Divergent evaluations of unsupervised models due to different clustering similarity measures.
method Developed an analytical framework that unifies pair-counting and information-theoretic clustering similarity measures.
result Unified framework clarifies when and why the two regimes diverge and provides a principled basis for selecting and interpreting clustering similarity measures.
Develops a non-parametric Dirichlet process method for probabilistic biclustering.
problem Challenges in finding biclusters with strong co-occurrence in rows and columns.
method Dual Dirichlet process mixture models for row and column clustering, with cluster number determined by data.
result Improves bicluster extraction in text mining and gene expression analysis.
A new method selects anchor words for better topic discovery in text corpora.
problem Selecting anchor words for improved topic modeling in text corpora.
method Proposes a new greedy method to find a minimum edge-weight anchor clique in a word similarity graph.
result The proposed method outperforms existing methods on topic quality and is faster.
The paper classifies trades into types based on proximity and measures their impact on stock prices.
problem Understanding the impact of high-frequency trades on stock prices and their predictability.
method Classifies trades into five types based on proximity, measures conditional order imbalance (COI), and develops trading strategies.
result Strong positive correlations between contemporaneous returns and COIs, and positive associations with future returns for isolated trades.
GEMRank embeds users and items using co-occurrence relations for better collaborative filtering.
problem Lack of textual data for entity embedding in recommender systems.
method Uses profile co-occurrence for entity relations and factorization for embedding. Feeds embeddings into a neural network for predictions.
result Significantly outperforms baseline algorithms in various data sets.
New topic modeling method scales to large datasets.
problem Large-scale topic modeling with high co-occurrence data.
method Introduced Full Dependence Mixture (FDM) model for direct topic learning.
result FDM model performs comparably or better than benchmarks on large datasets.
To understand the relationship between news sentiment and company stock price movements, and to better understand connectivity among companies, we define an algorithm for measuring sentiment-based network risk. The algorithm ranks companies in networks of co-occurrences, and measures sentiment-based risk, by calculatin…
New model reduces bias in crowdsourced pairwise comparisons.
problem Crowdsourced pairwise comparisons are biased due to perceptual factors.
method factorBT model accounts for irrelevant factors affecting worker answers.
result factorBT produces more accurate rankings than previous models.
Most popular word embedding techniques involve implicit or explicit factorization of a word co-occurrence based matrix into low rank factors. In this paper, we aim to generalize this trend by using numerical methods to factor higher-order word co-occurrence based arrays, or \textit{tensors}. We present four word embedd…
IMPACC improves consensus clustering for bioinformatics data.
problem Consensus clustering's inefficiency and lack of interpretability for large-scale data.
method Ensemble minipatch co-occurrences, adaptive sampling of observations and features.
result Significantly improved accuracy and interpretability with substantial computational savings.
Paper introduces differential pairwise privacy for secure metric learning.
problem Securely measuring similarities of individuals given sensitive pairwise data.
method Develops differential pairwise privacy (DPP) to protect sensitive pairwise data.
result Achieves pairwise data privacy without significant performance loss.
In this study, a pairwise comparison matrix is generalized to the case when coefficients create Lie group G, non necessarily abelian. A necessary and sufficient criterion for pairwise comparisons matrices to be consistent is provided. Basic criteria for finding a nearest consistent pairwise comparisons matrix (extend…
Paper studies SGD stability and optimization error in pairwise learning.
problem Stability and optimization error of SGD for pairwise learning.
method Established stability and optimization error trade-offs for SGD in convex, strongly convex, and non-convex settings.
result Lower bounds for SGD optimization error and excess expected risk.
Cross-entropy loss linked to metric learning, outperforming complex pairwise losses.
problem Improving metric learning performance without complex optimization schemes.
method Theoretical analysis linking cross-entropy to pairwise losses, showing cross-entropy as an upper bound and equivalent to mutual information maximization.
result Minimizing cross-entropy is equivalent to maximizing mutual information, leading to state-of-the-art performance.
ROVAE uses noisy pairwise comparisons to disentangle factors in VAEs.
problem Disentangling factors in VAEs requires an inductive bias.
method Robust Ordinal VAE (ROVAE) incorporates noisy pairwise ordinal comparisons to disentangle factors.
result ROVAE outperforms existing methods and is more robust to noisy comparisons.
Develops a statistical framework to measure uncertainty in model rankings based on human preferences.
problem Uncertainty in model rankings based on human preferences due to mismatch between human and model preferences.
method Statistical framework using pairwise comparisons by humans and models to provide rank-sets for each model.
result Rank-sets constructed using only pairwise comparisons by strong models often do not cover the true ranking of human preferences.
Improves labeling quality in machine learning with pairwise feedback.
problem Scalability and quality of labeled datasets in machine learning.
method Incorporates pairwise feedback into the programmatic creation of labeled datasets.
result Even a small number of pairwise feedback sources can substantially improve label quality.