Paper explores zero-shot cross-lingual reading comprehension using pre-trained multi-lingual model.
problem Lack of training data for every language in reading comprehension tasks.
method Systematic exploration of zero-shot cross-lingual transfer learning with a multi-lingual language representation model.
result Zero-shot cross-lingual transfer learning is feasible and translating source data into target language is not necessary.
Neural networks improve language comprehension by testing and refining hypotheses.
problem Improving language comprehension models through more sophisticated reasoning.
method Memory augmented neural networks with a hypothesis testing loop.
result Achieved state-of-the-art results on language comprehension benchmarks.
New metrics predict human sentence comprehension across languages.
problem Predicting human sentence comprehension using computational models.
method Developed sentence-level metrics using multilingual large language models.
result Achieved high accuracy in predicting human sentence reading speeds.
Bayesian EnKF improves sentence comprehension uncertainty modeling.
problem Uncertainty in human language comprehension, especially with ambiguous inputs.
method Bayesian framework using ensemble Kalman filter (EnKF) for uncertainty quantification.
result Enhanced model's ability to approximate human cognitive processing with linguistic ambiguities.
Study explores various methods for detecting offensive language.
problem Detecting offensive language in text.
method Examined custom and off-the-shelf architectures, feature vector generation approaches.
result These methods perform well on small, noisy datasets.
Improved language models with talking-heads attention.
problem Language model perplexity and quality issues.
method Added linear projections in multi-head attention.
result Better perplexities and quality in language tasks.
TreeCaps improves code comprehension for software developers.
problem Processing code efficiently for software developers.
method Tree-based capsule networks for capturing code syntactical structures and dependencies.
result TreeCaps outperforms other approaches in classifying program functionalities.
Vision and language tasks often fail to test AI comprehensively.
problem Current vision and language tasks are flawed due to dataset and evaluation issues.
method Review of current state and proposal for improvement.
result State-of-the-art systems perform well due to dataset and evaluation flaws.
Improved financial sentiment analysis using simple instruction tuning of LLMs.
problem Lack of accurate financial sentiment analysis by large language models.
method Instruction tuning of general-purpose LLMs with a small portion of financial sentiment data.
result Significant improvement in financial sentiment analysis, especially in complex scenarios.
Deep learning models outperform classical methods in text classification.
problem Improving text classification accuracy using deep learning.
method Comprehensive review of deep learning models and datasets for text classification.
result Deep learning models outperform classical methods on various text classification tasks.
Survey of deep learning methods for image captioning.
problem Generating accurate and complex image descriptions.
method Comprehensive review of deep learning techniques for image captioning.
result Analysis of strengths, limitations, and popular datasets in deep learning image captioning.
Paper augments neural nets with logic for improved performance.
problem Training neural networks with declarative knowledge without extra parameters.
method Systematically compiles logical statements into computation graphs that augment neural networks.
result Knowledge-augmented networks significantly improve performance, especially in low-data scenarios.
LORL learns object-centric representations from vision and language.
problem Learning disentangled, object-centric scene representations from vision and language.
method LORL integrates unsupervised object discovery and segmentation with language input to learn object-centric concepts.
result LORL improves unsupervised object discovery methods and aids downstream tasks.
QAInfomax improves reading comprehension by maximizing mutual information, achieving state-of-the-art performance.
problem Distractor sentences in question answering datasets are hard to distinguish from relevant ones.
method QAInfomax regularizes reading comprehension models to learn mutual information among passages, questions, and answers.
result QAInfomax achieves state-of-the-art performance on Adversarial-SQuAD dataset.
BART pretrains sequence-to-sequence models by corrupting text and reconstructing it.
problem Improving natural language generation, translation, and comprehension.
method BART uses a denoising autoencoder trained on a Transformer architecture with various noising techniques.
result BART achieves state-of-the-art performance on various NLP tasks with minimal training resources.
Stan models are compiled to generative languages and extended with new features.
problem Lack of direct support for variational inference and deep models in Stan.
method Comprehensive compilation scheme to convert Stan models to generative languages, and extension of Stan with new features.
result NumPyro backend yields a 2.3x speedup compared to Stan in geometric mean over 26 benchmarks.
LLMs are compared to Markov chains for natural language processing.
problem Theoretical analysis of LLMs' generalization capabilities.
method Equivalence between LLMs and Markov chains, studying multi-step inference.
result Derives generalization bounds for LLMs, capturing their behavior in practice.
Deep neural network predicts semantic labels for source code.
problem Difficulty in labeling and understanding new programming languages and functionalities.
method Language-agnostic deep convolutional neural network trained on Stack Overflow code snippets.
result Mean area under ROC of 0.957 and top-1 accuracy of 86.6% on Github code documents.
Survey examines distillation methods for large language models.
problem Efficiently compress large language models while preserving their capabilities.
method Knowledge Distillation and Dataset Distillation techniques.
result Integrating KD and DD can produce more effective and scalable compression strategies.
Large language models improve futures market factor models in China.
problem Designing effective factor models for Chinese futures markets.
method Used large language models (GPT) to generate 40 factors for single and multi-factor portfolios.
result GPT-generated factors outperform benchmarks with high Sharpe ratios and alphas.
ZeroSCROLLS benchmarks zero-shot natural language understanding over long texts.
problem Evaluate natural language understanding models over long texts without training data.
method Adapt six tasks from SCROLLS benchmark and add four new datasets, including novel aggregation tasks.
result Claude outperforms ChatGPT, and GPT-4 achieves highest average score.
DLM-One speeds up language generation by 500x with continuous models.
problem Efficiently generating text sequences in natural language processing.
method Score-distillation of continuous diffusion language models.
result Achieves up to 500x speedup in inference time with competitive performance.
Python library for causal discovery from observational data.
problem Revealing causal relations from observational data.
method Comprehensive collection of causal discovery methods in Python.
result Ease of use for non-specialists and modular building blocks for developers.
Chronologically consistent models maintain accuracy with time-restricted data.
problem Training data introduces lookahead bias and training leakage in large language models.
method ChronoBERT and ChronoGPT trained with only available data at each time point.
result Models achieve strong performance and competitive with larger models, mitigating lookahead bias.
Agent learns to interpret 3D instructions, generalizing from experience.
problem Grounded language learning in artificial agents.
method Simulated 3D environment, reinforcement and unsupervised learning.
result Agent learns to relate language to physical surroundings and actions.
New method to measure compositional generalization on realistic data.
problem Limited compositional generalization in machine learning.
method Maximizing compound divergence while ensuring small atom divergence.
result Machine learning architectures fail to generalize compositionally.
Paper shows pretrained models can categorize Thai social media content effectively.
problem Lack of labeled data for Thai social media content classification.
method Pretrained language models on a large noisy Thai social media corpus, fine-tuned for downstream tasks.
result State-of-the-art results achieved on Thai social text categorization tasks.
Systematic review of electronic health record phenotyping approaches.
problem Detecting patient cohorts using electronic health records.
method Comprehensive literature review of preprocessing and modeling approaches.
result Natural language processing shows promise for electronic phenotyping.
Shai is a 10B model for asset management tasks, outperforming baselines.
problem Improving performance in asset management tasks.
method Continuous pre-training and fine-tuning on asset management-specific data.
result Shai outperforms baseline models in asset management tasks.
Survey of financial LLMs in finance.
problem Limited research on Financial LLMs (FinLLMs).
method Chronological overview of PLMs, comparison of techniques, performance evaluations, and advanced tasks.
result Compilation of accessible datasets and benchmarks for AI research in finance.
Geo2DR learns graph representations using substructure patterns.
problem Learning distributed representations of graphs efficiently.
method Unsupervised learning with discrete substructure patterns and neural language models.
result Geo2DR achieves high reproducibility and interoperability in graph classification.
Study evaluates LLMs for predicting Chinese stock movements using financial news sentiments.
problem Evaluating LLMs' ability to predict stock price movements using financial news sentiments.
method Standardized experimental procedure with three LLMs, each with unique performance enhancement methods.
result Developed quantitative trading strategies and conducted back-tests to assess LLMs' performance.
Survey examines LLMs in financial trading.
problem Using LLMs to outperform professional traders in finance.
method Comprehensive review of current research on LLMs in financial trading.
result LLMs can potentially outperform professional traders in backtesting.
FairLangProc simplifies fairness in NLP models for Python users.
problem Addressing bias in NLP models for decision-making contexts.
method Develops a Python package for implementing fairness metrics and algorithms.
result Promotes the use of bias mitigation techniques in NLP.
German FinBERT improves financial text analysis performance.
problem Capturing domain-specific nuances in financial text.
method German FinBERT is a pre-trained German language model trained on a large corpus of financial data.
result German FinBERT outperforms standard models on finance-specific tasks.
Survey of financial foundation models for diverse applications.
problem Challenges in applying general-purpose FMs to financial tasks.
method Review of financial foundation models (FFMs) in three modalities.
result Emergence of FFMs designed specifically for finance.
The paper teaches machines to ask clarifying questions by applying NLI.
problem Teaching machines to ask clarifying questions for better comprehension.
method The approach is to use NLI to determine if a clarification question is about a missing piece of information in a post.
result The best model improved precision@1 by 40% and 60% over state-of-the-art baselines.
Study detects SLI in children from spontaneous narrative transcripts.
problem Detecting Specific Language Impairment (SLI) in children.
method Three-stage pipeline: feature extraction, dimensionality reduction, and classification.
result 97.13% accuracy in identifying SLI from transcripts.
This review compares extractive and abstractive summarization methods.
problem Improving abstractive summarization in natural language processing.
method Compared various approaches including supervised and unsupervised methods, deep learning, and NLP.
result Current research uses combinations of approaches, but abstractive summarization remains unsolved.
This paper studies activation sparsity in large language models, finding key trends and implications.
problem Activation sparsity in large language models (LLMs) can be improved for efficiency and interpretability.
method Proposes PPL-p% sparsity, analyzes trends with training data, width-depth ratio, and parameter scale. result ReLU is more efficient for sparsity than SiLU, and deeper architectures can improve sparsity.
This paper benchmarks FinGPT for financial datasets using open-source large language models.
problem Challenges in integrating GPT-based models with financial datasets.
method Instruction Tuning paradigm for open-source large language models adapted for financial contexts.
result Demonstrates the effectiveness and adaptability of FinGPT in financial tasks.
New algorithms make softmax optimization unbiased and scalable.
problem Efficiently computing softmax distributions with large categories.
method Proposed unbiased algorithms for maximizing softmax likelihood.
result Comprehensive outperformance on seven real-world datasets.
Unified QuesNet learns comprehensive representations for diverse test questions.
problem Lack of labeled data for test questions in online learning systems.
method Unified framework and two-level hierarchical pre-training algorithm for unsupervised learning of heterogeneous question representations.
result QuesNet effectively learns comprehensive question representations and outperforms existing methods.
Paper fine-tunes LLMs for financial tasks using data fusion.
problem Improving LLMs for financial analysis tasks.
method Fine-tuned Llama3-8B and Mistral-7B using PEFT and LoRA, combined datasets for data fusion.
result Enhanced model performance across financial tasks.
Study compares N-gram and RNN models for chord prediction on large datasets.
problem Improving chord prediction accuracy and adaptability in music.
method Compared N-gram models to various RNN architectures on multiple datasets.
result RNN models adapt to songs at test time, improving chord prediction.
Enhanced QANet tackles unanswerable questions in MRC.
problem Improving performance on unanswerable questions in MRC.
method Extends QANet architecture to handle unanswerable questions through multi-task learning.
result EQuANt achieves close to 2x better performance on SQuAD 2 compared to baseline.
A new sliced IGW distance for Gromov-Wasserstein alignment.
problem Scalability issues in Gromov-Wasserstein alignment for high-dimensional problems.
method Proposed a sliced IGW distance with rotational invariance.
result Natural rotational invariance of the sliced IGW distance.
Enhances word embedding by transferring external knowledge.
problem Low-frequency words in semantic space.
method Latent Semantic Imputation (LSI) integrating graph theory and spectral embeddings.
result LSI generates reliable embedding vectors for low-frequency words.