xVal tokenizes numbers continuously for better scientific model training.
problem Lack of continuous numerical tokenization for scientific datasets in LLMs.
method xVal: Continuous numerical tokenization strategy.
result xVal outperforms other numerical tokenization methods on scientific datasets.
Galactica learns from scientific literature to help researchers.
problem Information overload in scientific literature makes it hard to find useful insights.
method Trained on a large corpus of scientific papers, reference material, and knowledge bases.
result Outperforms existing models on various scientific tasks, including LaTeX equations and mathematical reasoning.
EduQG generates better educational questions by pre-training on scientific text.
problem Improving the quality of educational questions for scalable self-assessment.
method Adapting a large language model for educational question generation, pre-trained on scientific text.
result EduQG produces superior educational questions compared to baseline models.
The paper argues for prioritizing identifying structure over complex models for scientific discovery.
problem Underdetermination of mechanisms in high-dimensional data, leading to unreliable explanations.
method Proposes concrete standards for 'mechanistic ML' to avoid collapsing explanations.
result Large language models (LLMs) can collapse large equivalence classes of explanations, making it hard to distinguish between mechanisms.
Paper outlines a new mathematical language for experiments.
problem Formalizing the scientific process for automation.
method Formulates the scientific process in precise mathematical language.
result Novel contributions in data processing, bias variance, and deficiency.
iPrompt uses LLMs to generate natural-language explanations of data patterns.
problem Finding and explaining patterns in data using natural language.
method Interpretable autoprompting (iPrompt) that generates natural-language explanations based on LLMs.
result iPrompt can accurately find and explain data patterns, improving upon human-written prompts.
CausalEvolve improves efficiency and discovery in open-ended scientific tasks.
problem Lack of targeted guidance and knowledge organization in evolve-based agents.
method Causal scratchpad that identifies and reasons about guiding factors for evolution.
result Effective improvement in evolutionary efficiency and better solutions.
Numeracy is the ability to understand and work with numbers. It is a necessary skill for composing and understanding documents in clinical, scientific, and other technical domains. In this paper, we explore different strategies for modelling numerals with language models, such as memorisation and digit-by-digit composi…
ParaMonte simplifies Monte Carlo simulations for various scientific fields.
problem Efficiently performing Monte Carlo simulations for complex models.
method Unified, high-performance, parallelized library for C, C++, Fortran.
result Automates and streamlines Monte Carlo sampling for arbitrary-dimensional functions.
Genomic models learn DNA sequences to predict functions.
problem Understanding complex genetic interactions.
method Training LLMs on DNA sequences to predict functions.
result gLMs can predict functions of DNA elements.
New framework reveals thermodynamic principles for LLM training.
problem Understanding the training dynamics of large language models.
method Introducing Neural Thermodynamic Laws (NTL) under river-valley loss landscape assumptions.
result Key thermodynamic quantities and principles naturally emerge in LLM training.
New method extracts biological concepts from cell microscopy images.
problem Extracting meaningful concepts from vision foundation models trained on cell microscopy images.
method Sparse dictionary learning (DL) combined with PCA whitening pre-processing.
result Successfully retrieved biologically meaningful concepts like cell types and genetic perturbations.
ChemCrow enhances LLMs for chemistry tasks, automating complex chemical processes.
problem Limited access to computational chemistry tools for large-language models.
method Integrating 18 expert-designed chemistry tools into an LLM (ChemCrow).
result ChemCrow autonomously plans and executes chemical syntheses and discoveries.
Natural language processing often involves computations with semantic or syntactic graphs to facilitate sophisticated reasoning based on structural relationships. While convolution kernels provide a powerful tool for comparing graph structure based on node (word) level relationships, they are difficult to customize and…
Generates biomedical abstracts from titles, years, and keywords.
problem Difficulties in understanding biomedical research papers due to specialized language.
method Conditional transformer-based language model with metadata conditioning.
result Generated abstracts are more relevant and non-trivial than GPT-2.
Fine-tunes language models with captions for differential equation solving.
problem Overreliance on function data in operator learning.
method Integrates human knowledge through captions and fine-tunes language models.
result Significantly enhanced performance and reduced function data requirements.
The scientific literature is a rich source of information for data mining with conceptual knowledge graphs; the open science movement has enriched this literature with complementary source code that implements scientific models. To exploit this new resource, we construct a knowledge graph using unsupervised learning me…
Model learns code representations from comments for data analysis tasks.
problem Lack of descriptive labels for analyzing large code corpora.
method Weakly supervised transformer architecture for joint code and comment representation.
result Model achieves 38% accuracy increase over expert-supplied heuristics.
Enhances drug discovery models by understanding human language.
problem Low predictive quality of activity prediction models in drug discovery.
method Proposes a novel architecture with separate chemical and natural language input modules and a contrastive pre-training objective.
result Improves predictive performance on few-shot and zero-shot learning benchmarks.
Probabilistic programming languages (PPLs) are receiving widespread attention for performing Bayesian inference in complex generative models. However, applications to science remain limited because of the impracticability of rewriting complex scientific simulators in a PPL, the computational cost of inference, and the …
Many research fields codify their findings in standard formats, often by reporting correlations between quantities of interest. But the space of all testable correlates is far larger than scientific resources can currently address, so the ability to accurately predict correlations would be useful to plan research and a…
Framework evaluates AI proposals for drug discovery, finds no LLM advantage.
problem No principled framework exists for evaluating AI-guided scientific selection under budget constraints.
method Formally verified metric (BSDS/DQS) penalizes false discoveries and excessive abstention.
result LLMs provide no marginal value over existing classifiers in drug discovery.
The semantic map calibrates uncertainty from language model probabilities.
problem Uncertainty in language model probabilities for professional decisions.
method Prespecified semantic map linking probabilities of verbal responses to probabilities of declared states.
result Language-derived probabilities outperform printed numerical probabilities and recover valid uncertainty coverage.
LLMs fail to match statistical ground truth despite stable run-to-run performance.
problem LLMs lack validation against statistical ground truth in automated scientific workflows.
method Introduced a behavioral evaluation framework for LLMs, separating four decision-making dimensions.
result LLMs can exhibit near-perfect stability but diverge from statistical ground truth.
ParaDRAM automates parallel MCMC simulations across languages.
problem Efficiently sampling complex mathematical functions.
method Automated, parallel, high-performance MCMC with adaptive rejection.
result Significant memory reduction for complex simulations.
This paper surveys UQ methods for deep learning.
problem Overconfident predictions in deep learning models.
method Categorizes UQ methods by uncertainty sources.
result Identifies strengths and limitations of each category.
Study uses machine learning to predict predator-prey dynamics without prior knowledge.
problem Predicting predator-prey interactions without prior knowledge of the system.
method Applied Neural Ordinary Differential Equations (Neural ODEs) and Universal Differential Equations (UDEs) to the Lotka-Volterra model.
result UDEs outperform Neural ODEs in predicting predator-prey dynamics, especially in noisy data.
There has been a lot of recent interest in adopting machine learning methods for scientific and engineering applications. This has in large part been inspired by recent successes and advances in the domains of Natural Language Processing (NLP) and Image Classification (IC). However, scientific and engineering problems …
Leveraging new data sources is a key step in accelerating the pace of materials design and discovery. To complement the strides in synthesis planning driven by historical, experimental, and computed data, we present an automated method for connecting scientific literature to synthesis insights. Starting from natural la…
The study uncovers latent capabilities of language models via causal representation learning.
problem Rigorous causal evaluations of language model capabilities are challenging due to confounding effects and computational costs.
method Proposes a causal representation learning framework to identify latent capability factors as causally interrelated after controlling for a common confounder (base model).
result Identifies a three-node linear causal structure explaining performance variations across 1500 models and six benchmarks.
Scientific Computing relies on executing computer algorithms coded in some programming languages. Given a particular available hardware, algorithms speed is a crucial factor. There are many scientific computing environments used to code such algorithms. Matlab is one of the most tremendously successful and widespread s…
CausalGame benchmarks LLM agents' causal thinking in games.
problem Evaluating causal thinking in AI Scientists with LLMs.
method Interactive games with 14 scenarios incorporating selection bias, measurement error, and hidden confounders.
result None of the 30 LLM agents demonstrated reliable causal thinking, with the best model achieving only 68.0% survival.
New method uses LLMs to generate detailed scientific hypotheses.
problem Generating detailed, actionable scientific hypotheses from coarse initial directions.
method Hierarchical search method that incrementally adds details to hypotheses.
result Hierarchical search method consistently outperforms strong baselines on expert-annotated hypotheses.
This paper introduces Sigma, a domain-specific computational representation for collaboration in large-scale for the field of economics. A computational representation is not a programming language or a software platform. A computational representation is a domain-specific representation system based on three specific …
Due to recent technical and scientific advances, we have a wealth of information hidden in unstructured text data such as offline/online narratives, research articles, and clinical reports. To mine these data properly, attributable to their innate ambiguity, a Word Sense Disambiguation (WSD) algorithm can avoid numbers…
Remasking improves the quality of discrete diffusion models for natural language and image generation.
problem Limited iterative refinement in masked discrete diffusion models.
method Introducing ReMDM sampler that allows remasking during inference.
result Remasking enables better quality outputs with increased sampling steps.
Celeste is a procedure for inferring astronomical catalogs that attains state-of-the-art scientific results. To date, Celeste has been scaled to at most hundreds of megabytes of astronomical images: Bayesian posterior inference is notoriously demanding computationally. In this paper, we report on a scalable, parallel v…
Data mining revealed a cluster of economic, psychological, social and cultural indicators that in combination predicted corruption and wealth of European nations. This prosperity syndrome of self-reliant citizens, efficient division of labor, a sophisticated scientific community, and respect for the law, was clearly di…
Planetary exploration missions with Mars rovers are complicated, which generally require elaborated task planning by human experts, from the path to take to the images to capture. NASA has been using this process to acquire over 22 million images from the planet Mars. In order to improve the degree of automation and th…
Study bandwidth-limited training and inference of language models.
problem Training and inference of language models on scattered data with limited bandwidth.
method Analyzed two protocols: Federated Probe-Logit Distillation (FPLD) for training and Federated Conformal RAG (FC-RAG) for inference.
result Explicit high-probability KL-consistency rate and distribution-free marginal-coverage bound for Federated Conformal RAG.
The study examines how much data is needed for generative and vision-language models to make reliable predictions.
problem Ensuring reliable predictions with low data for models used in medical decision support.
method Analyzes uniform convergence bounds for VLM-induced classifiers under low-dimensional semantic representations.
result Finite-sample uniform convergence bounds for accuracy and calibration functionals of VLM-induced classifiers.
Paper proposes a method to estimate scientific parameters in hybrid models without relying on model architecture.
problem Estimating unknown parameters in hybrid models combining machine learning and scientific models.
method Sharpness-aware minimization adapted for hybrid modeling, focusing on model simplicity.
result Demonstrates effectiveness of SAM-based hybrid model learning for scientific parameter estimation.
Data science models, although successful in a number of commercial domains, have had limited applicability in scientific problems involving complex physical phenomena. Theory-guided data science (TGDS) is an emerging paradigm that aims to leverage the wealth of scientific knowledge for improving the effectiveness of da…
Paper proposes a new framework for hypothesis testing in imaging.
problem Challenges in hypothesis testing for imaging data.
method Combines self-supervised imaging, vision-language models, and non-parametric hypothesis testing.
result Demonstrates improved power and robust error control in image-based phenotyping.
Accelerometer measurements are the prime type of sensor information most think of when seeking to measure physical activity. On the market, there are many fitness measuring devices which aim to track calories burned and steps counted through the use of accelerometers. These measurements, though good enough for the aver…
New benchmarks show LLMs struggle with causal discovery.
problem Leveraging LLMs for causal discovery is unreliable due to dataset leakage.
method Developing science-grounded benchmarks and hybrid methods combining LLM predictions with statistical analysis.
result LLMs perform poorly on novel, real-world scientific studies compared to classical methods.
Symbolic regression is a powerful technique that can discover analytical equations that describe data, which can lead to explainable models and generalizability outside of the training data set. In contrast, neural networks have achieved amazing levels of accuracy on image recognition and natural language processing ta…
We describe a new method for visualizing topics, the distributions over terms that are automatically extracted from large text corpora using latent variable models. Our method finds significant n-grams related to a topic, which are then used to help understand and interpret the underlying distribution. Compared with …