Paper proposes a new method for sentence embeddings using weighted word vectors.
problem Improving sentence embeddings for natural language processing tasks.
method A simple sentence embedding method using weighted average of word vectors followed by soft projection.
result Demonstrates effectiveness on clinical semantic textual similarity task.
Paper finds better words for topic models by reranking top words.
problem Top words in topic models are not always representative.
method Reranking words by considering marginal probability over every topic.
result Reranked top words are more representative of topics.
PCA whitening weighted by Zipfian word frequencies improves task performance.
problem Skewed word embedding spaces in neural models.
method PCA whitening weighted by empirical word frequencies following Zipf's law.
result Significantly improves task performance, surpassing baselines.
A new method selects anchor words for better topic discovery in text corpora.
problem Selecting anchor words for improved topic modeling in text corpora.
method Proposes a new greedy method to find a minimum edge-weight anchor clique in a word similarity graph.
result The proposed method outperforms existing methods on topic quality and is faster.
This paper justifies and improves lexicon-based classification without labeled data.
problem Lack of justification for lexicon-based classification and its lower accuracy compared to supervised methods.
method Derives probabilistic justification and learns weights from co-occurrence statistics.
result Lexicon-based classification can be improved without labeled data, offering higher accuracy.
W-RNN improves text classification by extracting serialized text semantics.
problem Semantic constraint in sparse representation classification methods.
method Weighted RNN using word vectors and recurrent neural networks.
result W-RNN outperforms other methods in precision, recall, F1, and loss values.
FAN improves attention weights for better relation emphasis.
problem Learning attention weights for better relation emphasis.
method Introduced a novel center-mass cross entropy loss and a focused attention backbone.
result Focused supervision leads to improved attention distribution and enhanced representation.
The paper shows context vectors are half the dimensions of word vectors.
problem Understanding the relationship between word and context embeddings.
method Starting from probabilistic assumptions, the paper shows context vectors are reflections of word vectors in half the dimensions.
result Context vectors are reflections of word vectors in approximately half the dimensions.
VALC provides concept-level interpretations of FLMs, overcoming word-level limitations.
problem Lack of higher-level structure interpretation in FLMs' attention weights.
method Formal definition of conceptual interpretation, variational Bayesian framework (VALC).
result VALC finds optimal language concepts for FLM predictions, providing concept-level interpretations.
ROTS improves sentence similarity by incorporating structural information.
problem Measuring sentence similarity with theoretical insights and structural awareness.
method Recursive Optimal Transport (ROT) framework to incorporate structural information.
result ROTS outperforms weakly supervised approaches in sentence similarity tasks.
We give a combinatorial description of closed curves on oriented surfaces in terms of certain permutations, called charts. We describe automorphisms of curves in terms of charts and compute the total number of curves counted with appropriate weights. We also discuss relations between curves, Grothendieck dessins d'enfa…
New method uses word subspaces and term-frequency to improve text classification.
problem Lack of semantic meaning in bag-of-words features.
method Proposes word subspaces and term-frequency weighted word subspaces for text classification.
result Improved text classification performance compared to state-of-the-art algorithms.
Automatically classifying the tissues types of Region of Interest (ROI) in medical imaging has been an important application in Computer-Aided Diagnosis (CAD), such as classification of breast parenchymal tissue in the mammogram, classify lung disease patterns in High-Resolution Computed Tomography (HRCT) etc. Recently…
A new neural network for text classification reduces parameters with improved accuracy.
problem Reducing the number of parameters in text classification models.
method Compositional coding, capsule network, k-means routing algorithm.
result The proposed method achieves competitive accuracy with significantly fewer parameters.
ETM identifies field-specific keywords in text classification.
problem Unsupervised text classification with field-specific keywords.
method Weighted Lasso penalty and pairwise Kullback-Leibler divergence penalty for topic separation.
result ETM improves topic coherence by 22% and 10% compared to LDA.
Model learns image-word associations from captions using contrastive learning.
problem Phrase grounding, associating image regions to caption words.
method Optimizing word-region attention to maximize mutual information, using language model guided word substitutions for negatives.
result Model achieves 76.7% accuracy on Flickr30K Entities benchmark, a 5.7% gain from weak supervision.
Transformer models align words through attention weights, closely approximating Optimal Transport.
problem Understanding the internal mechanism of transformer models in language processing.
method Empirical evidence and theoretical analysis of attention weights and their relation to Optimal Transport.
result Transformer models can simulate gradient descent on the dual of entropy-regularized OT problem, providing a theoretical foundation for token alignment.
DSA improves sentence embedding by dynamically attending to words.
problem Efficiently capturing the importance of words in sentences for embedding.
method DSA modifies dynamic routing from capsule networks for self-attention in sentences.
result DSA achieves state-of-the-art results in SNLI with fewer parameters.
Adaptively sparse Transformers improve interpretability and diversity in NLP.
problem Standard Transformers use dense attention, limiting interpretability and diversity.
method Introduces adaptively sparse Transformers using α-entmax for context-dependent sparsity. result Improves interpretability and diversity in NLP tasks without sacrificing accuracy.
Word2vec skip-gram with negative sampling is explained as a weighted logistic PCA.
problem Understanding word2vec skip-gram with negative sampling.
method Equivalence to weighted logistic PCA, comparison to other methods, extension to higher dimensions.
result Word2vec skip-gram with negative sampling is a weighted logistic PCA.
Federated learning improves wake word detection in smart home devices.
problem Out-of-domain issues with continuously running speech-based models.
method Federated learning with adaptive averaging strategy.
result Reduces communication rounds and upstream communication costs.
Improves document summarization by combining word embeddings and n-grams.
problem Exact word matching fails to measure semantic similarity between sentences.
method Uses deep embedding features and tf-idf features to improve sentence similarity measure; builds an improved sentence similarity graph; employs a submodular objective function; develops a Transformer-based compression model.
result Outperforms tf-idf based approach and achieves state-of-the-art performance on DUC04 dataset.
BlackOut speeds up RNNLMs with large vocabularies.
problem Training RNNLMs with large vocabularies is computationally expensive.
method BlackOut uses a discriminative loss and weighted sampling to efficiently train RNNLMs with million-word vocabularies.
result BlackOut achieves the lowest perplexity scores on the one billion word benchmark.
Improved POS tagging for Twitter data with limited annotations.
problem Low-quality user-generated text for POS tagging.
method Domain adaptation using neural networks with feature embeddings and pre-trained embeddings.
result 90% tagging accuracy on German Tweets.
The paper develops a decision support system for hierarchical text classification of conference proceedings.
problem Classifying documents with a fixed hierarchical structure of topics.
method Developed a weighted hierarchical similarity function to calculate topic relevance, using entropy of words to estimate weights.
result The weighted hierarchical similarity function improves ranking accuracy compared to other methods.
Improved cross-lingual document retrieval using regularized Wasserstein distance.
problem Cross-lingual document retrieval (CLDR) challenges.
method Extending Word Mover's Distance with term-weighting schemes and entropic regularization.
result Remarkable improvements in Mean Reciprocal Rank (MRR) compared to baselines.
Enhanced neural model detects misinformation by considering author context.
problem Plateaued accuracy in traditional rumor classifiers.
method Ensemble neural architecture with word-level attention and author context.
result Promising results on Twitter dataset, detecting rumors more accurately.
A new algorithm optimizes softmax units in large language models.
problem Efficiently computing gradients for large-scale language models.
method Zero-th Order method for approximating gradients.
result The algorithm converges and efficiently computes gradients.
Most existing word embedding methods can be categorized into Neural Embedding Models and Matrix Factorization (MF)-based methods. However some models are opaque to probabilistic interpretation, and MF-based methods, typically solved using Singular Value Decomposition (SVD), may incur loss of corpus information. In addi…
Bidirectional attention is shown to be equivalent to a continuous bag of words model with mixture-of-experts.
problem Understanding the statistical underpinnings of bidirectional attention.
method Exploring bidirectional attention as a mixture-of-experts model and reparameterizing it.
result Bidirectional attention can be viewed as a continuous bag of words model with mixture-of-experts weights.
Algorithm extracts deterministic PDFA from probabilistic models with improved performance.
problem Learning deterministic models from probabilistic ones with noise.
method Adapted L* algorithm for probabilistic settings, using conditional probabilities and local tolerance.
result Achieves better performance on WER and NDCG than spectral extraction of WFAs.
Improved ALS algorithm for tensor decomposition with guaranteed recovery.
problem ALS algorithm often converges to poor local optima, especially with non-uniform weights.
method Orthogonalized ALS approach, provably recovers true factors with random initialization.
result Significantly superior to traditional ALS in practical tasks.
WEST compresses word embeddings and softmax layers for memory efficiency.
problem Memory constraints in large vocabulary models.
method WEST encodes words with sequences of sub-units, improving compression without performance loss.
result WEST achieves significant compression without sacrificing performance.
New method tests weighted networks without thresholding, improving accuracy.
problem Testing and anomaly detection on weighted network data.
method Hierarchical Bayesian hypothesis testing framework for weighted networks.
result Method shows lower Type I error and higher statistical power compared to alternatives.
Generative model improves image realism with word phrase attention.
problem Natural language often involves complex foreground objects and variable background.
method Introduced region-phrase attention between true-grid regions and word phrases.
result Generated more realistic images compared to state-of-the-art algorithms.
NGSLL combines DNN accuracy with linear model interpretability.
problem Combining high accuracy of DNNs with interpretability of linear models.
method Neural generators of sparse local linear models (NGSLL) using DNNs to approximate non-linear functions.
result Effective in real-world datasets, achieving high predictive performance and interpretability.
In this paper, we classify the class of constant weighted curvature curves in the plane with a log-linear density, or in other words, classify all traveling curved fronts with a constant forcing term in R2. The classification gives some interesting phenomena and consequences including: the family of curves conv…
Submodular functions have many applications. Matchings have many applications. The bitext word alignment problem can be modeled as the problem of maximizing a nonnegative, monotone, submodular function constrained to matchings in a complete bipartite graph where each vertex corresponds to a word in the two input senten…
We propose an algorithm for deciding whether a given braid is pseudo-Anosov, reducible, or periodic. The algorithm is based on Garside's weighted decomposition and is polynomial-time in the word-length of an input braid. Moreover, a reduction system of circles can be found completely if the input is a certain type of r…
The paper models and predicts co-occurrence counts using Gamma regression.
problem Predicting relevance between items or users from high-dimensional sparse co-occurrence count data.
method Shared parameter alternating zero-inflated Gamma regression models (SA-ZIG) with Fisher scoring and learning rate adjustment.
result SA-ZIG with learning rate adjustment performs satisfactorily in predicting relevance.
PAC-Bayes analysis explains sentence vector learning from unlabeled data.
problem Understanding and improving sentence vector learning from unlabeled data.
method PAC-Bayes bound analysis for transfer learning.
result Simple heuristics and new algorithms derived from PAC-Bayes analysis.
Proposes a novel framework for multi-label text classification.
problem Lack of coherent consideration of non-consecutive and long-distance semantics and hierarchical relations among labels.
method Hierarchical taxonomy-aware and attentional graph capsule recurrent CNNs framework.
result Significantly improves multi-label text classification performance.
New techniques reuse subword embeddings in neural models, reducing size and improving performance.
problem Improving performance and reducing model size in subword-aware neural language models.
method Reusing subword embeddings and other weights in multi-layer input embedding models, tying layers consecutively bottom-up.
result Best morpheme-aware model with reused weights outperforms competitive word-level model by a large margin.
We prove the existence and the uniqueness of a conformally equivariant symbol calculus and quantization on any conformally flat pseudo-Riemannian manifold $(M,\rg)$. In other words, we establish a canonical isomorphism between the spaces of polynomials on T∗M and of differential operators on tensor densities over $M…
Paper speeds up neural language model inference by 20x for top-k word prediction.
problem Slow inference speed of neural language models on mobile devices.
method Introduced a screening model using Gumbel softmax to approximate softmax layer.
result Achieved 20.4x speedup with 98.9% precision@1 and 99.3% precision@5 for German to English translation.
New algorithm trains binary-activation, multi-level RNNs for noise-resilient, ADC-/DAC-free PIM inference.
problem Training noise-resilient, ADC-/DAC-free neural networks.
method Binary activations and multi-level weights for eNVM-based processing-in-memory circuits.
result Higher accuracy and noise resilience for recurrent networks compared to existing methods.
Trellis networks improve sequence modeling performance.
problem Sequence modeling challenges.
method Temporal convolutional network with weight tying and direct input injection.
result Trellis networks outperform state-of-the-art methods on benchmarks.
Model analyzes cooccurrence data for recommender systems and item relevance.
problem High-dimensional cooccurrence data from online platforms.
method Shared parameter Alternating Tweedie (SA-Tweedie) model with Fisher scoring and learning rate adjustment.
result SA-Tweedie model outperforms other methods in optimizing parameters.