SGNNs use simulations to train neural networks, improving scientific forecasting and interpretability.
problem Combining precise theory and machine learning for robust scientific modeling.
method Pretraining neural networks on diverse mechanistic simulations as training data.
result SGNNs outperform data-driven and physics-constrained models in forecasting and interpretability.
Computer simulations are invaluable tools for scientific discovery. However, accurate simulations are often slow to execute, which limits their applicability to extensive parameter exploration, large-scale data analysis, and uncertainty quantification. A promising route to accelerate simulations by building fast emulat…
New framework compresses and recovers scientific data efficiently.
problem Efficiently managing and recovering from large scientific datasets.
method Grounded in learning exponential families, preserves uncertainty and supports trade-offs.
result Preserves physical features and quantities of interest in compressed representations.
ParaMonte simplifies Monte Carlo simulations for various scientific fields.
problem Efficiently performing Monte Carlo simulations for complex models.
method Unified, high-performance, parallelized library for C, C++, Fortran.
result Automates and streamlines Monte Carlo sampling for arbitrary-dimensional functions.
Instrumented data enables causal scientific machine learning
problem Insufficient data for causal scientific machine learning
method Instrumented data with explicit model, uncertainty, and counterfactuals
result Supports causal interventions through Pearl's do-operator
ξ-torch simplifies physics-informed learning by providing differentiable functionals.
problem Training physics-informed deep neural networks requires differentiable physical simulations.
method ξ-torch offers a library of differentiable functionals for scientific simulations.
result Improves numerical stability and reduces memory requirements for higher order derivatives.
Probabilistic programming languages (PPLs) are receiving widespread attention for performing Bayesian inference in complex generative models. However, applications to science remain limited because of the impracticability of rewriting complex scientific simulators in a PPL, the computational cost of inference, and the …
SBI uses neural networks to infer model parameters from simulators.
problem Computational infeasibility of Bayesian inference for complex models.
method Training neural networks on simulator-generated data.
result Efficient Bayesian inference without likelihood evaluations.
New sampling strategy preserves relationships in multivariate scientific data.
problem Reducing storage and enabling efficient multivariate analyses on large scientific data.
method Uses principal component analysis for multivariate data and combines with existing univariate sampling algorithms.
result Efficacy demonstrated on real-world data sets, showing data reduction and multivariate analysis ease.
Machine learning methods have been remarkably successful for a wide range of application areas in the extraction of essential information from data. An exciting and relatively recent development is the uptake of machine learning in the natural sciences, where the major goal is to obtain novel scientific insights and di…
Deep neural networks provide meaningful uncertainty estimates for large-scale simulations.
problem Uncertainty estimates for deep neural network predictions from large-scale simulations.
method General variational inference approach to calibrate Bayesian uncertainties.
result Calibrated Bayesian uncertainties preserved physics-correlations in predicted quantities.
AutoSciDACT detects scientific anomalies in noisy data.
problem Detecting anomalies in large, noisy scientific datasets.
method Contrastive pre-training for low-dimensional data representations, two-sample test using NPLM.
result Strong sensitivity to small anomalies across various scientific domains.
New approach uses low-fidelity data to train ML models efficiently.
problem Training ML models with scarce high-fidelity data leads to high variance and poor generalization.
method Multifidelity linear regression using approximate control variates.
result Multifidelity training achieves similar accuracy with reduced high-fidelity data.
Machine learning algorithms such as linear regression, SVM and neural network have played an increasingly important role in the process of scientific discovery. However, none of them is both interpretable and accurate on nonlinear datasets. Here we present contextual regression, a method that joins these two desirable …
Develops a Bayesian framework for symbolic regression of scientific expressions.
problem Lack of principled uncertainty quantification and interpretability in existing symbolic regression methods.
method Hierarchical Bayesian framework with tree-structured symbolic expressions and Markov chain Monte Carlo inference.
result Robust performance on various datasets, including single-atom catalysis.
PRISM infers model structures and parameters from simulations, controlling complexity at test time.
problem Choosing among large model families for scientific discovery.
method Simulation-based encoder-decoder that infers model structures and parameters, with test-time complexity control.
result PRISM scales to large model families and performs model selection in biophysical diffusion MRI.
The paper argues for prioritizing identifying structure over complex models for scientific discovery.
problem Underdetermination of mechanisms in high-dimensional data, leading to unreliable explanations.
method Proposes concrete standards for 'mechanistic ML' to avoid collapsing explanations.
result Large language models (LLMs) can collapse large equivalence classes of explanations, making it hard to distinguish between mechanisms.
The thesis tackles overconfident approximations in simulation-based inference.
problem Overconfident conclusions from machine learning approximations in statistical analyses.
method Introduces balancing and Bayesian neural networks to reduce overconfidence.
result Balancing and Bayesian neural networks lead to less overconfident approximations.
This paper introduces a framework for combining scientific knowledge of physics-based models with neural networks to advance scientific discovery. This framework, termed physics-guided neural networks (PGNN), leverages the output of physics-based model simulations along with observational features in a hybrid modeling …
This paper analyzes machine learning workflows in climate modeling.
problem Challenges in integrating machine learning with climate modeling.
method Analysis of case studies focusing on design patterns and workflow structure.
result Synthesis of workflow design patterns across diverse projects in ML-enabled climate modeling.
Paper develops an attention mechanism for long-term scientific impact prediction.
problem Predicting the long-term impact of scientific papers based on citation records.
method Develops an attention mechanism to predict long-term scientific impact.
result Emphasizing the limited attention can better stand on the shoulders of giants.
ACE improves GBI for simulators by approximating cost functions, making inference more efficient.
problem Inference for misspecified simulators is overly restrictive.
method Amortized cost estimation (ACE) for Generalized Bayesian Inference (GBI).
result ACE provides accurate cost predictions and more efficient inference.
Study improves interpretability in generative models by disentangling latent variables in scientific datasets.
problem Extracting generative factors from complex, high-dimensional datasets in unsupervised or semi-supervised settings.
method Introducing Aux-VAE, a novel architecture within the VAE framework, which disentangles latent variables by guiding them with auxiliary variables.
result Aux-VAE achieves disentanglement with minimal modifications to the standard VAE loss function, validated on multiple datasets.
TCR simplifies complex models into interpretable causal factors.
problem Understanding complex phenomena in high-dimensional models.
method Information theoretic objective for learning TCR from interventional data.
result TCR generates interpretable high-level explanations from complex models.
SCaSML improves PDE solvers by correcting errors efficiently.
problem Reliable and error-free high-dimensional PDE solutions.
method Defect correction method to derive a Structural-preserving Law of Defect.
result SCaSML achieves faster convergence and reduced errors in high-dimensional PDEs.
ParaDRAM automates parallel MCMC simulations across languages.
problem Efficiently sampling complex mathematical functions.
method Automated, parallel, high-performance MCMC with adaptive rejection.
result Significant memory reduction for complex simulations.
Modern investigation in economics and in other sciences requires the ability to store, share, and replicate results and methods of experiments that are often multidisciplinary and yield a massive amount of data. Given the increasing complexity and growing interaction across diverse bodies of knowledge it is becoming im…
Enhances neural operators with physics knowledge for more accurate simulations.
problem Improving accuracy and generalization of neural operators for physical systems.
method Jointly learns from original PDEs and simplified forms, incorporating fundamental physics.
result Significant improvement in nRMSE across various PDE problems.
A new method for designing accurate emulators using deep learning with interval calibration.
problem Designing accurate emulators for scientific processes with modern machine learning methods.
method Learn-by-Calibrating (LbC) approach based on interval calibration.
result Significant improvements in generalization error over widely-used loss functions.
We present a novel probabilistic programming framework that couples directly to existing large-scale simulators through a cross-platform probabilistic execution protocol, which allows general-purpose inference engines to record and control random number draws within simulators in a language-agnostic way. The execution …
The paper uses optimal transport to calibrate stochastic simulations.
problem Improper fidelity of stochastic simulators in scientific applications.
method Optimal transport theory applied to neural network corrections.
result Calibrated stochastic simulations improve fidelity to reality.
PDBAL targets experiments for probabilistic models to maximize insights.
problem Designing experiments to yield valuable insights efficiently.
method Combines user-specified risk function with probabilistic model to adaptively choose designs.
result PDBAL consistently outperforms standard approaches in simulations and real-world drug screen data.
LOL-GP model improves surrogate modeling of expensive simulators.
problem Costly computer simulations for complex systems.
method Local transfer learning Gaussian process.
result Improved surrogate performance over existing methods.
GAR generalizes autoregression for efficient multi-fidelity fusion.
problem Efficiently combining low-fidelity and high-fidelity simulation results.
method Generalized autoregression (GAR) using tensor formulation and latent features.
result GAR outperforms state-of-the-art methods with a large margin in RMSE.
Breiman's data analysis dichotomy is outdated, offering a third approach: mechanistic models.
problem Data analysis dichotomy between data modelers and algorithmic modelers.
method Interpolating between simple interpretable models and flexible function approximations using mechanistic models.
result Flexible, interpretable, and scientifically-informed hybrids can provide accurate and robust predictions.
TRIM improves interpretability of deep neural networks in cosmology.
problem Understanding which features a deep neural network uses in a transformed space.
method TRIM (Transformation IMportance) attributes importances to features in a transformed space.
result Combining TRIM with contextual decomposition helps identify physical features learned by DNNs.
OmniFold uses deep learning to deconvolve high-dimensional simulations.
problem Removing detector distortions and accounting for noise processes in high-dimensional simulations.
method OmniFold is a deep learning-based approach for maximum likelihood deconvolution.
result OmniFold can remove detector distortions and account for noise processes and acceptance effects.
Study uses machine learning to predict predator-prey dynamics without prior knowledge.
problem Predicting predator-prey interactions without prior knowledge of the system.
method Applied Neural Ordinary Differential Equations (Neural ODEs) and Universal Differential Equations (UDEs) to the Lotka-Volterra model.
result UDEs outperform Neural ODEs in predicting predator-prey dynamics, especially in noisy data.
Simulation-based inference methods can produce unreliable posterior approximations.
problem Reliability of simulation-based inference methods for scientific use cases.
method Benchmarked algorithms including Neural Posterior Estimation, Neural Ratio Estimation, Sequential Neural Likelihood, and Approximate Bayesian Computation.
result Ensembling posterior surrogates provides more reliable approximations.
Posterior inference with an intractable likelihood is becoming an increasingly common task in scientific domains which rely on sophisticated computer simulations. Typically, these forward models do not admit tractable densities forcing practitioners to make use of approximations. This work introduces a novel approach t…
This paper tackles infinite-dimensional diffusion bridge simulation using operator learning.
problem Challenges in simulating diffusion bridges for modeling natural data due to intractable drift terms and continuous data representations.
method Merges score matching techniques with operator learning to directly learn infinite-dimensional bridges.
result Demonstrates high efficacy in simulating diffusion bridges for various applications, including real-world biological data.
New model improves QGP simulation efficiency and accuracy.
problem Limited QGP simulation runs due to high computational cost.
method Additive Multi-Index Gaussian process (AdMIn-GP) model.
result Significantly improved surrogate modeling performance.
New method uses Diffusion Maps for latent space modeling of dynamical systems.
problem Building reduced dynamical models from time series data.
method Two rounds of Diffusion Maps on latent coordinates, with lifting back to ambient space.
result Approximation of full state functions in reduced coordinates.
We consider the problem of precision matrix estimation where, due to extraneous confounding of the underlying precision matrix, the data are independent but not identically distributed. While such confounding occurs in many scientific problems, our approach is inspired by recent neuroscientific research suggesting that…
Machine learning improves network classification and model selection.
problem Quantifying suitability of generative models for network structures.
method Interpretable machine learning to classify simulated networks based on features and interactions.
result Specific network features and their interactions are crucial for distinguishing generative models.
TSNPE improves SBI efficiency and scalability.
problem Efficient and scalable simulation-based inference for complex models.
method Sequential inference with truncated proposals.
result TSNPE performs on par with previous methods and scales to complex models.
Simulation-free VI closes the approximation gap in latent SDEs
problem Recovering dynamical systems from noisy observations
method Helmholtz-SDE
result Recovers dynamics more faithfully than prior methods
xVal tokenizes numbers continuously for better scientific model training.
problem Lack of continuous numerical tokenization for scientific datasets in LLMs.
method xVal: Continuous numerical tokenization strategy.
result xVal outperforms other numerical tokenization methods on scientific datasets.