Convolutional neural networks (CNNs) have recently emerged as a popular building block for natural language processing (NLP). Despite their success, most existing CNN models employed in NLP share the same learned (and static) set of filters for all input sentences. In this paper, we consider an approach of using a smal…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
21 results for “context-sensitive”
GOAT learns multiple node representations from graph structure alone.
problem Context-free graph representation learning limits model performance.
method Inspired by gossip and mutual attention, GOAT learns multiple node representations.
result GOAT outperforms 12 SOTA baselines on link prediction and clustering tasks.
Study finds phase transition in context-sensitive language model with short-range interactions.
problem Understanding phase transitions in language models with short-range interactions.
method Constructed a random language model with short-range interactions and investigated its statistical properties.
result Phase transition occurs in context-sensitive language models with constant context length.
New model shows natural language exhibits phase transition similar to physics.
problem Understanding critical properties in natural language models.
method Created a context-sensitive random language model.
result Demonstrated a Berezinskii--Kosterlitz--Thouless phase transition.
We present a novel language adaptable spell checking system which detects spelling errors and suggests context sensitive corrections in real-time. We show that our system can be extended to new languages with minimal language-specific processing. Available literature majorly discusses spell checkers for English but the…
Network representation learning (NRL) is a powerful technique for learning low-dimensional vector representation of high-dimensional and sparse graphs. Most studies explore the structure and metadata associated with the graph using random walks and employ an unsupervised or semi-supervised learning schemes. Learning in…
In this paper we examine a possible reason for the LSTM outperforming the GRU on language modeling and more specifically machine translation. We hypothesize that this has to do with counting. This is a consistent theme across the literature of long term dependence, counting, and language modeling for RNNs. Using the si…
We propose a principle for exploring context in machine learning models. Starting with a simple assumption that each observation may or may not depend on its context, a conditional probability distribution is decomposed into two parts: context-free and context-sensitive. Then by employing the log-linear word production…
As global greenhouse gas emissions continue to rise, the use of stratospheric aerosol injection (SAI), a form of solar geoengineering, is increasingly considered in order to artificially mitigate climate change effects. However, initial research in simulation suggests that naive SAI can have catastrophic regional conse…
Complex architectures of biological neural circuits, such as parallel processing pathways, has been behaviorally implicated in many cognitive studies. However, the theoretical consequences of circuit complexity on neural computation have only been explored in limited cases. Here, we introduce a mechanism by which direc…
Existing machine reading comprehension (MRC) models do not scale effectively to real-world applications like web-level information retrieval and question answering (QA). We argue that this stems from the nature of MRC datasets: most of these are static environments wherein the supporting documents and all necessary inf…
New framework enforces demographic parity on distribution tails.
problem Enforcing demographic parity on entire distribution can degrade accuracy.
method Optimal transport theory, focusing on distribution tails.
result More nuanced and context-sensitive fairness interventions.
The quality of data representation in deep learning methods is directly related to the prior model imposed on the representations; however, generally used fixed priors are not capable of adjusting to the context in the data. To address this issue, we propose deep predictive coding networks, a hierarchical generative mo…
A framework uses preprocessing to improve psychiatric questionnaire predictions while maintaining interpretability.
problem Weak predictive accuracy and limited interpretability of psychiatric questionnaires.
method Two-stage method: stable preprocessing followed by a linear mapping.
result REFINE outperforms other interpretable approaches in psychiatric and non-psychiatric prediction tasks.
Loops in surfaces, chord diagrams, interlace graphs: operad factorisations and generating grammarsmath.GT
Loops in surfaces and chord diagrams are studied with graph factorizations and grammars.
problem Understanding loops in surfaces and their properties.
method Factorization of filoops into spheric and toric sums, and grammars generating chordiagraphs.
result Minimal genus of filoops and stability properties under factorizations.
We consider Markov models of stochastic processes where the next-step conditional distribution is defined by a kernel density estimator (KDE), similar to Markov forecast densities and certain time-series bootstrap schemes. The KDE Markov models (KDE-MMs) we discuss are nonlinear, nonparametric, fully probabilistic repr…
Biological and artificial neural systems are composed of many local processors, and their capabilities depend upon the transfer function that relates each local processor's outputs to its inputs. This paper uses a recent advance in the foundations of information theory to study the properties of local processors that u…
Study reveals AI skin cancer classifiers underperform for darker skin phototypes, advocating for fairness auditing.
problem AI bias in dermatology, particularly for darker skin phototypes.
method Predictive Representativity (PR) framework, evaluating classifiers on HAM10000 and BOSQUE Test sets.
result Substantial performance disparities by skin phototype, highlighting AI bias.
Communication through e-mails remains to be highly formalized, conventional and indispensable method for the exchange of information over the Internet. An ever-increasing ratio and adversary nature of spam e-mails have posed a great many challenges such as uneven class distribution, unequal error cost, frequent change …
A new method for efficient neural network fine-tuning using queryable low-rank update atoms.
problem Rigidity of static low-rank adaptation methods when input and depth-wise computation vary.
method A shared queryable memory of low-rank update atoms, allowing dynamic and context-sensitive adaptation.
result Improves final test performance and training stability compared to standard low-rank adaptation.
FinLlama uses a fine-tuned Llama 2 model for financial sentiment analysis.
problem Accurate financial sentiment analysis for better trading decisions.
method Fine-tuning Llama 2 7B model on financial sentiment data, using a generator-classifier scheme.
result FinLlama provides nuanced insights into financial news articles, enhancing portfolio management.