Bayesian approach for learning spatiotemporal systems from noisy data.
problem Efficiently modeling and learning from spatiotemporal dynamical systems with noisy data.
method Hierarchical state-space models with Gaussian process regression for efficient interpolation and training.
result Efficient modeling and learning of spatiotemporal dynamics using Bayesian methods.
Gradient descent optimization improved by circuit perspective.
problem Efficient training of large AI systems.
method Utilizing circuit perspective for mechanistic interpretability.
result Designing a curriculum for efficient learning.
New methods for parameter estimation in mechanistic models using data-consistent inversion.
problem Parameter estimation bias in Bayesian analysis for mechanistic models.
method Data-consistent inversion methods based on rejection sampling, MCMC, GANs, and constrained optimization.
result Improved parameter estimation without bias from uninformative priors.
Physics-informed machine learning models improve biomolecular system simulations.
problem Modeling unresolved interactions beyond classical force fields.
method Physics-informed neural networks and operator learning.
result Accurate, mechanistic, generalizable models for long-timescale kinetics.
SGNNs use simulations to train neural networks, improving scientific forecasting and interpretability.
problem Combining precise theory and machine learning for robust scientific modeling.
method Pretraining neural networks on diverse mechanistic simulations as training data.
result SGNNs outperform data-driven and physics-constrained models in forecasting and interpretability.
This paper tackles interpretability of LLMs in finance.
problem Complexity and lack of transparency of LLMs in finance.
method Mechanistic interpretability to understand LLM behavior.
result Demonstrates practical relevance of mechanistic interpretability in financial use cases.
Purely data driven approaches for machine learning present difficulties when data is scarce relative to the complexity of the model or when the model is forced to extrapolate. On the other hand, purely mechanistic approaches need to identify and specify all the interactions in the problem at hand (which may not be feas…
Adapts MBDOE for real-time parameter estimation in complex systems.
problem Costly posterior inference and design optimization in nonlinear systems.
method Combines DAD with differentiable mechanistic models for real-time parameter estimation.
result Demonstrated on four systems, including a DC motor.
Study detects and explains positional bias in financial LLMs.
problem Positional bias in financial decision-making using LLMs.
method Unified framework and benchmark for detecting and quantifying bias in Qwen2.5 models.
result Positional bias is pervasive, scale-sensitive, and resurfaces under nuanced prompt designs.
Unified model bridges mechanistic and non-mechanistic epidemic approaches.
problem Understanding the dynamics of epidemics with flexibility and interpretability.
method A mixture-based model representing time series as Gaussian mixtures, derived from a networked SIR framework.
result The model provides interpretable parameters and low prediction error, validating its use in understanding interventions.
Statistical models outperform mechanistic models in short-term COVID-19 incidence forecasts.
problem Comparing accuracy of mechanistic vs statistical models for short-term COVID-19 incidence forecasts.
method Empirical comparison of forecasts from mechanistic and statistical models using daily incidence data from six US states.
result Statistical models are at least as accurate as mechanistic models and better capture volatility.
New method sparsifies hybrid neural ODEs for better performance and stability.
problem Excessive latent states and interactions from mechanistic models lead to training inefficiency and over-fitting.
method Automatic state selection and structure optimization combining domain-informed graph modifications with data-driven regularization.
result Improved predictive performance and robustness with desired sparsity.
Paper develops efficient Bayesian inference for enzymatic SRNs with LNA metamodel.
problem Bayesian inference for nonlinear SDE-based mechanistic models with partial observations and measurement errors.
method Interpretable Bayesian updating LNA metamodel and efficient posterior sampling.
result Proposed approach demonstrates promising performance in empirical studies.
Hybrid models combine domain knowledge and data-driven learning for Earth observation.
problem Challenges in modelling Earth observation data with either purely mechanistic or data-driven methods.
method Gaussian process convolution models, specifically latent force models (LFMs), integrating physical knowledge into multioutput GP models.
result Model automatically estimates soil moisture persistence and discovers latent forces related to precipitation.
Study forecasts cholera outbreaks in Malawi using dynamic models.
problem Cholera transmission forecasting in developing countries.
method Qualitative dynamics, Monte Carlo Markov Chain, sensitivity analysis, machine learning.
result Enhanced cholera forecasting models improve future trends prediction.
APD method decomposes neural network parameters into simple, faithful components.
problem Understanding the internal mechanisms learned by neural networks.
method Attribution-based Parameter Decomposition (APD) method.
result Demonstrated effectiveness in recovering features, separating computations, and identifying representations.
Natural language processing (NLP) can be done using either top-down (theory driven) and bottom-up (data driven) approaches, which we call mechanistic and phenomenological respectively. The approaches are frequently considered to stand in opposition to each other. Examining some recent approaches in deep learning we arg…
The paper argues for prioritizing identifying structure over complex models for scientific discovery.
problem Underdetermination of mechanisms in high-dimensional data, leading to unreliable explanations.
method Proposes concrete standards for 'mechanistic ML' to avoid collapsing explanations.
result Large language models (LLMs) can collapse large equivalence classes of explanations, making it hard to distinguish between mechanisms.
Accurately predicting the future capacity and remaining useful life of batteries is necessary to ensure reliable system operation and to minimise maintenance costs. The complex nature of battery degradation has meant that mechanistic modelling of capacity fade has thus far remained intractable; however, with the advent…
Latent force models are systems whereby there is a mechanistic model describing the dynamics of the system state, with some unknown forcing term that is approximated with a Gaussian process. If such dynamics are non-linear, it can be difficult to estimate the posterior state and forcing term jointly, particularly when …
Unified framework for disentangled representations using mechanistic independence.
problem Identifiability of disentangled latent factors under statistical dependencies.
method Introduces mechanistic independence to characterize latent factors by their actions on observed variables, proposing various independence criteria.
result Establishes conditions for identifiability of latent subspaces without statistical assumptions.
Hybrid model learns interpretable meal-level glycemic control.
problem Lack of flexible, interpretable meal-level glycemic control methods.
method Hybrid variational autoencoder grounding latent space to mechanistic differential equation.
result Unsupervised representation discovers separation between individuals based on disease severity.
Paper proposes a dual-level approach for multi-step forecasting of dynamical systems.
problem Accurate multi-step forecasting of time series systems for automatic control and optimization.
method Hybrid input forecasting using LSTM-STMs and physics-informed neural networks (PINNs).
result Hybrid models achieve higher log-likelihood and lower MSE compared to conventional methods.
Unified framework for blending ML and mechanistic models in dynamical systems.
problem Learning dynamical systems from noisy, partially observed data.
method A unifying framework that combines mechanistic and machine learning approaches.
result Proves that hybrid models can learn memory-dependent model error.
Adjoint SA speeds up bioprocess parameter learning.
problem Challenges in digital twin development for biomanufacturing.
method Adjoint sensitivity analysis on multi-scale enzymatic reaction networks.
result Resilient sensitivities reveal bioprocess regulatory mechanisms.
Breiman's data analysis dichotomy is outdated, offering a third approach: mechanistic models.
problem Data analysis dichotomy between data modelers and algorithmic modelers.
method Interpolating between simple interpretable models and flexible function approximations using mechanistic models.
result Flexible, interpretable, and scientifically-informed hybrids can provide accurate and robust predictions.
Triangulation filters spurious circuits in multilingual models.
problem Unreliable explanations of multilingual models across languages.
method Formalizes reference families and introduces triangulation as a causal acceptance rule.
result Triangulation provides a falsifiable standard for mechanistic claims.
Framework for imputing missing heart data to simulate brain-heart interactions.
problem Lack of multi-modal patient data representing heart and brain processes.
method Probabilistic framework for joint cardiac data imputation and mechanistic model personalization.
result Accurate imputation of missing cardiac features in incomplete datasets.
SAEs struggle with feature consistency across runs, hindering MI reliability.
problem Inconsistency of learned SAE features across different training runs.
method Propose using the Pairwise Dictionary Mean Correlation Coefficient (PW-MCC) to measure feature consistency.
result High levels of feature consistency (0.80 for TopK SAEs on LLM activations) are achievable with appropriate architectural choices.
nUDEs use neural networks to model biology without negative values.
problem Unrealistic negative values in hybrid models of biology.
method Developed non-negative UDEs (nUDEs) with regularization techniques.
result nUDEs provide realistic solutions for biological models.
Machine learning predicts COVID-19 activity in China.
problem Real-time forecasting of COVID-19 activity in Chinese provinces.
method Combines mechanistic disease models with digital traces (internet searches, news alerts). Uses clustering and data augmentation techniques.
result Stable and accurate forecasts 2 days ahead of current time, outperforming baseline models in 27 out of 32 provinces.
In this paper, we introduce a novel task for machine learning in healthcare, namely personalized modeling of the female hormonal cycle. The motivation for this work is to model the hormonal cycle and predict its phases in time, both for healthy individuals and for those with disorders of the reproductive system. Becaus…
Physics-based framework improves building energy forecasting.
problem Lack of physical correspondence in machine learning models for building energy systems.
method Combines LTI SSMs with subspace-based domain adaptation (SDA).
result Physics-derived subspaces align with data-derived subspaces for better forecasting.
The roundworm C. elegans exhibits robust escape behavior in response to rapidly rising temperature. The behavior lasts for a few seconds, shows history dependence, involves both sensory and motor systems, and is too complicated to model mechanistically using currently available knowledge. Instead we model the process p…
Unified framework for causal models at different levels of abstraction.
problem Relating causal models at varying levels of abstraction.
method Categorical framework using natural transformations between Markov functors.
result Generalized and unified causal abstractions with categorical proofs.
Optimizes biomanufacturing processes with a new digital twin calibration method.
problem Lack of interpretability and sample efficiency in traditional DoE methods.
method Developed a computational approach to calibrate Bio-SoS digital twin model.
result Guides sample-efficient and interpretable DoEs by quantifying sub-model parameter estimation errors.
Modeling dynamical systems with ordinary differential equations implies a mechanistic view of the process underlying the dynamics. However in many cases, this knowledge is not available. To overcome this issue, we introduce a general framework for nonparametric ODE models using penalized regression in Reproducing Kerne…
A theory of deep learning is emerging, focusing on training dynamics and statistics.
problem Develop a scientific theory to understand deep learning.
method Synthesize research into five areas: idealized settings, tractable limits, mathematical laws, hyperparameters, and universal behaviors.
result The emerging theory is a mechanics of the learning process, named learning mechanics.
This research formalizes uncertainty quantification for Universal Differential Equations models.
problem Quantifying uncertainties in Universal Differential Equations models.
method Formalized uncertainty quantification methods for UDEs, including frequentist and Bayesian approaches.
result Evaluation of ensemble, variational inference, and MCMC sampling methods for UDEs.
Mechanistic models of single-neuron dynamics have been extensively studied in computational neuroscience. However, identifying which models can quantitatively reproduce empirically measured data has been challenging. We propose to overcome this limitation by using likelihood-free inference approaches (also known as App…
Hybrid model improves forest growth predictions.
problem Misspecified assumptions in mechanistic models.
method Forest Informed Neural Networks (FINN) combining DVM and DNN.
result DNN learned improved growth process functional form.
Latent force models are a class of hybrid models for dynamic systems, combining simple mechanistic models with flexible Gaussian process (GP) perturbations. An extension of this framework to include multiplicative interactions between the state and GP terms allows strong a priori control of the model geometry at the ex…
Bayesian modelling of dynamic systems must achieve a compromise between providing a complete mechanistic specification of the process while retaining the flexibility to handle those situations in which data is sparse relative to model complexity, or a full specification is hard to motivate. Latent force models achieve …
New method learns SDEs with structured noise from data.
problem Learning SDEs with structured noise from data.
method Nonparametric framework for drift and diffusion terms.
result Accurately infers low-dimensional interaction kernels.
A model-based approach to forecasting chaotic dynamical systems utilizes knowledge of the physical processes governing the dynamics to build an approximate mathematical model of the system. In contrast, machine learning techniques have demonstrated promising results for forecasting chaotic systems purely from past time…
Cellular regulatory dynamics is driven by large and intricate networks of interactions at the molecular scale, whose sheer size obfuscates understanding. In light of limited experimental data, many parameters of such dynamics are unknown, and thus models built on the detailed, mechanistic viewpoint overfit and are not …
EGDL predicts TB outbreaks with deep learning, integrating epidemiological models.
problem Predicting TB outbreaks with complex spatiotemporal dynamics.
method Modified MN-SIR model with Bayesian inference, deep neural networks.
result EGDL delivers robust and accurate TB outbreak predictions.
G-Net uses deep learning for complex counterfactual outcome prediction.
problem Estimating counterfactual outcomes under dynamic treatment strategies.
method G-Net is a sequential deep learning framework for G-computation.
result G-Net can handle complex temporal data and provide accurate treatment effects.