Unified model bridges mechanistic and non-mechanistic epidemic approaches.
problem Understanding the dynamics of epidemics with flexibility and interpretability.
method A mixture-based model representing time series as Gaussian mixtures, derived from a networked SIR framework.
result The model provides interpretable parameters and low prediction error, validating its use in understanding interventions.
Bayesian approach for learning spatiotemporal systems from noisy data.
problem Efficiently modeling and learning from spatiotemporal dynamical systems with noisy data.
method Hierarchical state-space models with Gaussian process regression for efficient interpolation and training.
result Efficient modeling and learning of spatiotemporal dynamics using Bayesian methods.
Natural language processing (NLP) can be done using either top-down (theory driven) and bottom-up (data driven) approaches, which we call mechanistic and phenomenological respectively. The approaches are frequently considered to stand in opposition to each other. Examining some recent approaches in deep learning we arg…
The paper argues for prioritizing identifying structure over complex models for scientific discovery.
problem Underdetermination of mechanisms in high-dimensional data, leading to unreliable explanations.
method Proposes concrete standards for 'mechanistic ML' to avoid collapsing explanations.
result Large language models (LLMs) can collapse large equivalence classes of explanations, making it hard to distinguish between mechanisms.
Paper develops efficient Bayesian inference for enzymatic SRNs with LNA metamodel.
problem Bayesian inference for nonlinear SDE-based mechanistic models with partial observations and measurement errors.
method Interpretable Bayesian updating LNA metamodel and efficient posterior sampling.
result Proposed approach demonstrates promising performance in empirical studies.
Hybrid model learns interpretable meal-level glycemic control.
problem Lack of flexible, interpretable meal-level glycemic control methods.
method Hybrid variational autoencoder grounding latent space to mechanistic differential equation.
result Unsupervised representation discovers separation between individuals based on disease severity.
APD method decomposes neural network parameters into simple, faithful components.
problem Understanding the internal mechanisms learned by neural networks.
method Attribution-based Parameter Decomposition (APD) method.
result Demonstrated effectiveness in recovering features, separating computations, and identifying representations.
Statistical models outperform mechanistic models in short-term COVID-19 incidence forecasts.
problem Comparing accuracy of mechanistic vs statistical models for short-term COVID-19 incidence forecasts.
method Empirical comparison of forecasts from mechanistic and statistical models using daily incidence data from six US states.
result Statistical models are at least as accurate as mechanistic models and better capture volatility.
SGNNs use simulations to train neural networks, improving scientific forecasting and interpretability.
problem Combining precise theory and machine learning for robust scientific modeling.
method Pretraining neural networks on diverse mechanistic simulations as training data.
result SGNNs outperform data-driven and physics-constrained models in forecasting and interpretability.
Machine learning predicts COVID-19 activity in China.
problem Real-time forecasting of COVID-19 activity in Chinese provinces.
method Combines mechanistic disease models with digital traces (internet searches, news alerts). Uses clustering and data augmentation techniques.
result Stable and accurate forecasts 2 days ahead of current time, outperforming baseline models in 27 out of 32 provinces.
New method sparsifies hybrid neural ODEs for better performance and stability.
problem Excessive latent states and interactions from mechanistic models lead to training inefficiency and over-fitting.
method Automatic state selection and structure optimization combining domain-informed graph modifications with data-driven regularization.
result Improved predictive performance and robustness with desired sparsity.
Physics-informed machine learning models improve biomolecular system simulations.
problem Modeling unresolved interactions beyond classical force fields.
method Physics-informed neural networks and operator learning.
result Accurate, mechanistic, generalizable models for long-timescale kinetics.
SAEs struggle with feature consistency across runs, hindering MI reliability.
problem Inconsistency of learned SAE features across different training runs.
method Propose using the Pairwise Dictionary Mean Correlation Coefficient (PW-MCC) to measure feature consistency.
result High levels of feature consistency (0.80 for TopK SAEs on LLM activations) are achievable with appropriate architectural choices.
Study forecasts cholera outbreaks in Malawi using dynamic models.
problem Cholera transmission forecasting in developing countries.
method Qualitative dynamics, Monte Carlo Markov Chain, sensitivity analysis, machine learning.
result Enhanced cholera forecasting models improve future trends prediction.
Gradient descent optimization improved by circuit perspective.
problem Efficient training of large AI systems.
method Utilizing circuit perspective for mechanistic interpretability.
result Designing a curriculum for efficient learning.
This research formalizes uncertainty quantification for Universal Differential Equations models.
problem Quantifying uncertainties in Universal Differential Equations models.
method Formalized uncertainty quantification methods for UDEs, including frequentist and Bayesian approaches.
result Evaluation of ensemble, variational inference, and MCMC sampling methods for UDEs.
Unified framework for disentangled representations using mechanistic independence.
problem Identifiability of disentangled latent factors under statistical dependencies.
method Introduces mechanistic independence to characterize latent factors by their actions on observed variables, proposing various independence criteria.
result Establishes conditions for identifiability of latent subspaces without statistical assumptions.
Mechanistic models of single-neuron dynamics have been extensively studied in computational neuroscience. However, identifying which models can quantitatively reproduce empirically measured data has been challenging. We propose to overcome this limitation by using likelihood-free inference approaches (also known as App…
New methods for parameter estimation in mechanistic models using data-consistent inversion.
problem Parameter estimation bias in Bayesian analysis for mechanistic models.
method Data-consistent inversion methods based on rejection sampling, MCMC, GANs, and constrained optimization.
result Improved parameter estimation without bias from uninformative priors.
Breiman's data analysis dichotomy is outdated, offering a third approach: mechanistic models.
problem Data analysis dichotomy between data modelers and algorithmic modelers.
method Interpolating between simple interpretable models and flexible function approximations using mechanistic models.
result Flexible, interpretable, and scientifically-informed hybrids can provide accurate and robust predictions.
Triangulation filters spurious circuits in multilingual models.
problem Unreliable explanations of multilingual models across languages.
method Formalizes reference families and introduces triangulation as a causal acceptance rule.
result Triangulation provides a falsifiable standard for mechanistic claims.
Study detects and explains positional bias in financial LLMs.
problem Positional bias in financial decision-making using LLMs.
method Unified framework and benchmark for detecting and quantifying bias in Qwen2.5 models.
result Positional bias is pervasive, scale-sensitive, and resurfaces under nuanced prompt designs.
Hybrid model improves forest growth predictions.
problem Misspecified assumptions in mechanistic models.
method Forest Informed Neural Networks (FINN) combining DVM and DNN.
result DNN learned improved growth process functional form.
This paper tackles interpretability of LLMs in finance.
problem Complexity and lack of transparency of LLMs in finance.
method Mechanistic interpretability to understand LLM behavior.
result Demonstrates practical relevance of mechanistic interpretability in financial use cases.
Framework for imputing missing heart data to simulate brain-heart interactions.
problem Lack of multi-modal patient data representing heart and brain processes.
method Probabilistic framework for joint cardiac data imputation and mechanistic model personalization.
result Accurate imputation of missing cardiac features in incomplete datasets.
Purely data driven approaches for machine learning present difficulties when data is scarce relative to the complexity of the model or when the model is forced to extrapolate. On the other hand, purely mechanistic approaches need to identify and specify all the interactions in the problem at hand (which may not be feas…
ISAAC audits deep models for drug-target interactions, revealing structural differences.
problem Deep models for DTI often use irrelevant features, making them hard to evaluate.
method ISAAC uses intervention-based structural auditing to evaluate model sensitivity.
result ISAAC reveals significant structural differences in DTI models' reasoning.
Hybrid models combine domain knowledge and data-driven learning for Earth observation.
problem Challenges in modelling Earth observation data with either purely mechanistic or data-driven methods.
method Gaussian process convolution models, specifically latent force models (LFMs), integrating physical knowledge into multioutput GP models.
result Model automatically estimates soil moisture persistence and discovers latent forces related to precipitation.
Instrumented data enables causal scientific machine learning
problem Insufficient data for causal scientific machine learning
method Instrumented data with explicit model, uncertainty, and counterfactuals
result Supports causal interventions through Pearl's do-operator
Adjoint SA speeds up bioprocess parameter learning.
problem Challenges in digital twin development for biomanufacturing.
method Adjoint sensitivity analysis on multi-scale enzymatic reaction networks.
result Resilient sensitivities reveal bioprocess regulatory mechanisms.
Spectral measurements reveal hidden representation geometry in language model training.
problem Hidden internal representation in language model training is hard to examine.
method Empirical protocol using activation covariance and per-sample gradient SVD spectra.
result Batch size affects representation geometry, and activation spectra predict token efficiency.
Adapts MBDOE for real-time parameter estimation in complex systems.
problem Costly posterior inference and design optimization in nonlinear systems.
method Combines DAD with differentiable mechanistic models for real-time parameter estimation.
result Demonstrated on four systems, including a DC motor.
Bilinear MLPs offer a new way to interpret deep learning models without complex nonlinearities.
problem Lack of mechanistic understanding in how MLPs compute.
method Introduced bilinear MLPs without element-wise nonlinearities, analyzed their weights using tensor and eigendecomposition.
result Bilinear MLPs provide interpretable weight structures and enable adversarial attacks and overfitting analysis.
Transformers learn topic structure through embedding and attention mechanisms.
problem Understanding how transformers capture semantic structure in text.
method Combination of mathematical analysis and experiments on Wikipedia and synthetic data.
result Embedding and attention layers encode topic structure in transformers.
In this paper, we introduce a novel task for machine learning in healthcare, namely personalized modeling of the female hormonal cycle. The motivation for this work is to model the hormonal cycle and predict its phases in time, both for healthy individuals and for those with disorders of the reproductive system. Becaus…
Study shows how transformers learn to combine simple tasks into complex ones.
problem Understanding how transformers learn to perform complex tasks not seen during training.
method Controlled setting involving variable assignment and modular addition; partitioned training data analysis.
result Small transformers can generalize to unseen combinations of variables and numbers.
EGDL predicts TB outbreaks with deep learning, integrating epidemiological models.
problem Predicting TB outbreaks with complex spatiotemporal dynamics.
method Modified MN-SIR model with Bayesian inference, deep neural networks.
result EGDL delivers robust and accurate TB outbreak predictions.
New models explain heavy-tailed behavior in neural networks.
problem Heavy-tailed spectral densities in neural networks.
method High-temperature Marchenko-Pastur (HTMP) ensemble models.
result Heavy-tailed behavior arises from three factors: data structure, training temperature, and eigenvector entropy.
New framework evaluates model explanations based on decision task improvement.
problem Evaluation of model explanations often misses practical value.
method Decision-theoretic framework quantifying three key values.
result Provides benchmarks and interprets human-AI decision support.
New method uses Gaussian processes to improve PDE solver accuracy.
problem Uncertainty in PDE solver parameters and measurements.
method Physics-informed Gaussian process regression.
result Strictly generalizes weighted residual methods.
A theory of deep learning is emerging, focusing on training dynamics and statistics.
problem Develop a scientific theory to understand deep learning.
method Synthesize research into five areas: idealized settings, tractable limits, mathematical laws, hyperparameters, and universal behaviors.
result The emerging theory is a mechanics of the learning process, named learning mechanics.
I present a unified discussion of several recently published results concerning the escalation, timing and severity of violent events in human conflicts and global terrorism, and set them in the wider context of real-world and cyber-based collective violence and illicit activity. I point out how the borders distinguish…
Unified framework for blending ML and mechanistic models in dynamical systems.
problem Learning dynamical systems from noisy, partially observed data.
method A unifying framework that combines mechanistic and machine learning approaches.
result Proves that hybrid models can learn memory-dependent model error.
Neural networks learn simpler features first, then more complex ones; Fourier analysis reveals this pattern.
problem Understanding the learning dynamics of neural networks, especially with natural image data.
method Fourier analysis of translation-invariant and power-law spectra to study feature learning.
result Simple neural networks first rely on amplitude information, then phase information, and power-law spectra can accelerate learning phase information.
PKF improves KF for dynamic uncertainty tracking in time-course data.
problem Dynamic uncertainty tracking in time-course data.
method Pathspace Kalman Filter (PKF) using Bayesian methodology.
result PKF outperforms conventional KF methods, reducing MSE by several orders of magnitude.
Higher conservative training increases reward-hacking in reasoning models.
problem Reward hacking during online adaptation in reasoning models.
method Conservative offline training with varying levels of conservatism (β) was applied to a Qwen3-14B policy, and online adaptation was measured against a reward ensemble.
result Higher conservatism (β) increases reward-hacking damage, measured by the Goodhart gap and AUGC.
Physics-based framework improves building energy forecasting.
problem Lack of physical correspondence in machine learning models for building energy systems.
method Combines LTI SSMs with subspace-based domain adaptation (SDA).
result Physics-derived subspaces align with data-derived subspaces for better forecasting.
Accurately predicting the future capacity and remaining useful life of batteries is necessary to ensure reliable system operation and to minimise maintenance costs. The complex nature of battery degradation has meant that mechanistic modelling of capacity fade has thus far remained intractable; however, with the advent…