Develops S-EFE for analyzing grouped data, improving word usage interpretation.
problem Analyzing how words are used differently across related groups of data.
method Structured exponential family embeddings (S-EFE) with hierarchical modeling and amortization.
result S-EFE enables group-specific interpretation of word usage and outperforms EFE.
Analyzes word usage on Reddit to reveal patterns in human behavior.
problem Understanding the broader heterogeneity in human behavior.
method Dimension reduction on Reddit text data to analyze word usage and topics.
result Pronouns can characterize two dimensions capturing differences in word usage.
Large speech dataset for commercial use with 9.98% word error rate.
problem Creating a diverse speech recognition dataset for commercial purposes.
method Internet search for licensed audio data with transcriptions, training model on the dataset.
result Model trained on dataset achieves 9.98% word error rate on Librispeech's test-clean test set.
Proposes spherical text embedding for better directional similarity.
problem Directional similarity is more effective but unsupervised text embeddings are typically learned in Euclidean space.
method Develops a spherical generative model and an efficient optimization algorithm for unsupervised word and paragraph embeddings.
result Achieves state-of-the-art performances on various text embedding tasks.
Embedding words in a vector space has gained a lot of attention in recent years. While state-of-the-art methods provide efficient computation of word similarities via a low-dimensional matrix embedding, their motivation is often left unclear. In this paper, we argue that word embedding can be naturally viewed as a rank…
Community detection is a fundamental task in social network analysis. In this paper, first we develop an endorsement filtered user connectivity network by utilizing Heider's structural balance theory and certain Twitter triad patterns. Next, we develop three Nonnegative Matrix Factorization frameworks to investigate th…
Survey on embedding techniques in source code.
problem Applying word embedding techniques to source code.
method Collection and categorization of articles from related work and scholarly searches.
result Word embedding has been successfully applied to various granularities of source code.
New method transfers word embeddings from large to small datasets efficiently.
problem Challenges in learning word embeddings from new domains with limited data.
method Group-sparse matrix factorization for transfer learning.
result Efficiently learns domain-specific word embeddings with less data.
Efficient DSP features reduce speech recognition memory usage.
problem Limited memory on DSPs for speech recognition.
method Developed efficient bottleneck features (BNFs) for DSPs.
result Reduced speech recognition memory usage by 10x with minimal accuracy loss.
Sequence-to-Sequence (seq2seq) modeling has rapidly become an important general-purpose NLP tool that has proven effective for many text-generation and sequence-labeling tasks. Seq2seq builds on deep neural language modeling and inherits its remarkable accuracy in estimating local, next-word distributions. In this work…
Streaming ASR with transformer achieves low WER.
problem Real-time ASR with speech recognition.
method Time-restricted self-attention and triggered attention mechanisms.
result 2.8% and 7.2% WER for LibriSpeech test data.
We are concerned with modeling the strength of links in networks by taking into account how often those links are used. Link usage is a strong indicator of how closely two nodes are related, but existing network models in Bayesian Statistics and Machine Learning are able to predict only wether a link exists at all. As …
Count-sketches reduce memory usage for deep learning models without sacrificing performance.
problem Memory constraints in deep learning models as they grow larger.
method Maintain a linear sketch to compress auxiliary variables in optimization methods.
result Compressed optimization methods maintain the same performance as full-sized baselines while using significantly less memory.
Language models help text classification tasks by predicting next words.
problem Lack of theoretical understanding of why language models perform well on downstream tasks.
method Mathematical study of the connection between next word prediction and text classification, formalizing it and quantifying the benefit.
result Language models that are ε-optimal in cross-entropy learn features that can solve classification tasks with linear approximation.
Recently, a technique called Layer-wise Relevance Propagation (LRP) was shown to deliver insightful explanations in the form of input space relevances for understanding feed-forward neural network classification decisions. In the present work, we extend the usage of LRP to recurrent neural networks. We propose a specif…
Study quantifies gender bias in language models across 7 languages.
problem Measuring gender bias in language models across multiple languages.
method Curated dataset of politicians, multilingual language models, probing language models.
result Larger language models do not show significant gender bias compared to smaller ones.
Paper tackles zero-shot activity recognition using video features and text embeddings.
problem Zero-shot activity recognition with videos.
method Auto-encoder model for multimodal joint embedding, 3D convolutional action recognition for visual features, GloVe word embeddings for textual features.
result Improved zero-shot recognition results with top-n accuracy and mean Nearest Neighbor Overlap.
New techniques enforce sparseness in recurrent models, reducing memory usage.
problem Reducing memory usage in recurrent sequence models for NLP.
method Enforcing sparseness upfront in recurrent layers for language modeling and sequence labeling.
result Predefined sparseness leads to similar performance with fewer parameters.
What is the role of social interactions in the creation of price bubbles? Answering this question requires obtaining collective behavioural traces generated by the activity of a large number of actors. Digital currencies offer a unique possibility to measure socio-economic signals from such digital traces. Here, we foc…
Model predicts cognitive health risks based on smartphone usage patterns.
problem Identifying cognitive health risks through smartphone usage.
method Structured models of smartphone interactions analyzed over 12 weeks.
result AUROC of 0.79 in discriminating between healthy and symptomatic subjects.
Chatbot uses BERT to handle financial investment questions, improving accuracy and decision-making.
problem Improving accuracy and decision-making in financial investment customer service.
method Deep Bidirectional Transformer (BERT) model, uncertainty measure comparison, mixed-integer programming, automatic spelling correction.
result Chatbot can recognize 381 intents and decide when to escalate questions.
UR-FUNNY dataset aids in understanding multimodal humor.
problem Understanding humor in a multimodal context is understudied.
method Developed a multimodal dataset (UR-FUNNY) for humor detection.
result UR-FUNNY opens the door to multimodal humor detection research.
Method learns software resource usage from snapshots.
problem Challenges in learning time-varying, correlated resource usage.
method Graph structured Schrödinger bridge problem for nonparametric learning.
result Predicts most-likely resource distributions.
This study analyzes how weather impacts bike sharing usage in Washington D.C.
problem Understanding how weather affects bike sharing usage patterns.
method Gathered bike usage and weather data, used k-means clustering algorithm to identify clusters.
result Weather significantly impacts bike usage, with temperature and precipitation being the most influential factors.
EmTract extracts emotions from financial social media text.
problem Understanding investor emotions in financial markets.
method Annotated data, DistilBERT model, embedding space augmentation.
result EmTract outperforms existing emotion classifiers.
Structured pruning reduces large language model size and latency.
problem Reducing the size and latency of large language models.
method Parameterizing weights with low-rank factorization and adaptively removing rank-1 components.
result Structured pruning outperforms other methods at various compression levels and speeds up training and inference.
Paper presents AETN for efficient user modeling from mobile app usage.
problem Efficient user modeling from mobile app usage with reduced manual effort.
method AutoEncoder-coupled Transformer Network (AETN).
result AETN achieves effective user embeddings with reduced manual effort.
DKN uses knowledge graphs to improve news recommendation.
problem Limited personalized news recommendations due to lack of external knowledge.
method Integrates knowledge graph representation into news recommendation using a deep knowledge-aware network (DKN).
result DKN achieves substantial gains over state-of-the-art models in click-through rate prediction.
Neural networks predict EV charging station usage from network layout.
problem Designing optimal EV charging station networks.
method Used neural networks to predict usage from station layout.
result Quickly estimates average usage statistics from proposed station placements.
Paper proposes transparent reporting of algorithmic energy usage to promote environmental sustainability.
problem Need for transparent reporting of algorithmic energy usage for environmental sustainability.
method Developed a Python package to make analyses of energy usage accessible to individual researchers, localized to specific power grids, and compared with global benchmarks.
result Demonstrated the use of automatically-generated Energy Usage Reports in model-choice for machine learning.
Proposes AtCoR for predicting bike station usage, improving station network reconfiguration.
problem Challenges in predicting new bike stations due to lack of historical data.
method AtCoR algorithm that predicts both existing and new bike stations using station-centered heatmaps and historical correlations.
result AtCoR outperforms existing models in predicting bike station usage.
Graphical Lasso algorithm segments occupants' energy usage behaviors in HC-CP systems.
problem Improving sustainability and energy efficiency in HC-CP systems.
method Introduced a gamification framework and applied Graphical Lasso for energy usage segmentation.
result Characterized different energy usage behaviors in HC-CP systems.
Transforms web content for better visibility in AI-driven search engines.
problem Disruption of traditional SEO by generative AI search engines.
method Fine-tunes a BART-base transformer on synthetically generated training data.
result Significant improvements in ROUGE-L and BLEU scores, and substantial visibility gains in generative search responses.
Investigations have been performed into using clustering methods in data mining time-series data from smart meters. The problem is to identify patterns and trends in energy usage profiles of commercial and industrial customers over 24-hour periods, and group similar profiles. We tested our method on energy usage data p…
DBMs model Fitbit usage patterns revealing two distinct weekly usage habits.
problem Challenges in modeling activity tracker data due to unlabeled data.
method Deep Boltzmann Machines (DBMs) for unsupervised learning of weekly usage patterns.
result Two distinct weekly usage patterns identified: frequent Monday-Tuesday use and consistent weekly use.
This study uses LSTM and SARIMA models to forecast CPU usage in cloud computing.
problem Predicting future CPU usage in cloud computing centers.
method Used LSTM and SARIMA models to forecast CPU usage over short and long time scales.
result SARIMA model performed better for long-term predictions but LSTM was more robust.
A new method selects anchor words for better topic discovery in text corpora.
problem Selecting anchor words for improved topic modeling in text corpora.
method Proposes a new greedy method to find a minimum edge-weight anchor clique in a word similarity graph.
result The proposed method outperforms existing methods on topic quality and is faster.
Most existing word embedding approaches do not distinguish the same words in different contexts, therefore ignoring their contextual meanings. As a result, the learned embeddings of these words are usually a mixture of multiple meanings. In this paper, we acknowledge multiple identities of the same word in different co…
There are certain families of words and word sequences (words in the generators of a two-generator group) that arise frequently in the Teichm{ü}ller theory of hyperbolic three-manifolds and Kleinian and Fuchsian groups and in the discreteness problem for two generator matrix groups. We survey some of the families of su…
Probabilistic FastText captures multiple word senses and sub-word structures.
problem Capturing multiple word senses and sub-word structures in word embeddings.
method Probabilistic FastText uses Gaussian mixture densities to represent words, sharing statistical strength across sub-word structures and capturing different word senses.
result Probabilistic FastText outperforms existing models on word-similarity benchmarks and discerning different meanings.
Method predicts hardware resource usage by control software with guaranteed linear convergence.
problem Predicting time-varying hardware resource availability in control software.
method Path structured multimarginal Schrödinger bridge (MSBP) for learning stochastic resource usage.
result Guaranteed linear convergence to accurate prediction of hardware resource utilization.
AppsPred predicts smartphone app usage based on context.
problem Predicting personalized usage behavior of smartphone apps based on contexts.
method Random Forest machine learning technique considering multi-dimensional contexts.
result AppsPred significantly outperforms other machine learning approaches in predicting smartphone apps.
Crowdsourcing generated useful hypotheses for predicting residential energy usage.
problem Predicting residential electric energy usage using crowdsourced data.
method Crowdsourced questions and answers were used to build a predictive model of monthly electric energy consumption.
result The crowd can generate useful hypotheses for predicting energy usage, even with sparse data.
FRAGE learns word embeddings without frequency bias, improving performance across NLP tasks.
problem Word embeddings are biased towards word frequency, affecting performance for rare words.
method Adversarial training to learn Frequency-Agnostic word Embedding (FRAGE).
result FRAGE achieves higher performance than baselines in all four NLP tasks.
A new model improves homogeneity in burn patient reimbursement.
problem Incomplete homogeneity checks for burn patients using LOS as a proxy.
method Cost-sensitive decision tree model considering patient-level cost and severity of burn.
result Identified groups with increased homogeneity compared to current HRG groups.
Geometrically transforms word embeddings into a common space for better comparison.
problem Comparing embeddings from different sources is challenging.
method Applies orthogonal rotations and Mahalanobis scaling to transform embeddings into a shared latent space.
result The method improves word similarity and analogy tasks.
We discuss a topological approach to words introduced by the author. Words on an arbitrary alphabet are approximated by Gauss words and then studied up to natural modifications inspired by the Reidemeister moves on knot diagrams. This leads us to a notion of homotopy for words. We introduce several homotopy invariants …
RIn-Close_CVC2 improves biclustering efficiency by reducing memory usage.
problem Mining maximal biclusters in numerical datasets efficiently and without redundancy.
method Proposes RIn-Close_CVC2, a new version of RIn-Close_CVC that eliminates redundant biclusters without a symbol table.
result RIn-Close_CVC2 reduces memory usage and improves runtime compared to RIn-Close_CVC.