Data-driven model reduction captures non-Markovian dynamics using Koopman and Mori-Zwanzig formalisms.
problem Modeling complex, non-Markovian dynamics efficiently and understanding their underlying mechanisms.
method Formulates data-driven model reduction within Koopman and Mori-Zwanzig formalisms, deriving NARMAX models from dynamical systems.
result Shows how data-driven methods can represent non-Markovian dynamics using Koopman and Mori-Zwanzig formalisms.
Model Features improve transfer in reinforcement learning by clustering states.
problem Improving knowledge transfer between tasks with shared transition dynamics.
method Introduces Model Features, a feature representation that clusters behaviourally equivalent states.
result Learning Successor Features is equivalent to learning a Model-Reduction.
WeldNet reduces complex dynamics to simpler, manageable segments.
problem Complex, high-dimensional time-dependent datasets from physical processes are costly to simulate.
method Windowed Encoders for Learning Dynamics, splitting time domain into windows for nonlinear dimension reduction and propagator training.
result WeldNet captures nonlinear latent structures and dynamics, outperforming existing methods.
New method learns low-dimensional models for systems with non-polynomial terms.
problem Modeling systems with non-polynomial nonlinear terms that are spatially local and given in analytic form.
method Non-intrusive model reduction method that learns operators for linear and polynomially nonlinear dynamics via a least-squares problem incorporating given non-polynomial terms.
result Comparable accuracy to intrusive methods that require full knowledge of governing equations.
Active sampling selects few points for accurate model reduction of high-fidelity systems.
problem Efficiently identify dominant subspaces for model reduction of large training sets.
method Proposes an active sampling strategy to select a few points from the training set to estimate dominant subspaces accurately.
result Active sampling can provide 17x speed-up without sacrificing accuracy.
SVD-based methods reduce computational cost for stochastic systems.
problem High dimensionality and Monte Carlo runs in stochastic systems.
method Extending SVD-based model reduction to stochastic differential equations.
result Preserving symplectic structures improves accuracy and energy conservation.
New method uses neural networks to create models with memory effects.
problem Accurately modeling memory effects in reduced models.
method Analogies between recurrent neural networks and Mori-Zwanzig formalism to develop reduced models with memory.
result The proposed method produces reduced models with good performance on short-term and long-term predictions.
Bayesian sparsification reduces deep neural network complexity.
problem Complexity of deep neural networks limits their performance.
method Combines Bayesian shrinkage priors with stochastic variational inference.
result Bayesian model reduction (BMR) is a more efficient alternative for pruning model weights.
This paper concerns model reduction of dynamical systems using the nuclear norm of the Hankel matrix to make a trade-off between model fit and model complexity. This results in a convex optimization problem where this trade-off is determined by one crucial design parameter. The main contribution is a methodology to app…
RC flow learns molecular kinetics in low dimensions.
problem Discovering interpretable low-dimensional models of molecular kinetics.
method Normalizing flow for coordinate transformation and Brownian dynamics for kinetics approximation.
result Tractable and trainable model of reduced kinetics in continuous time and space.
BMRS offers a Bayesian approach to structured pruning of neural networks.
problem Overparameterized neural networks lead to high compute costs.
method Bayesian Model Reduction for Structured pruning (BMRS) based on two recent methods: Bayesian structured pruning with multiplicative noise and Bayesian model reduction.
result BMRS yields high compression rates and accuracy without tuning thresholds.
Develops neural network approximations for infinite-dimensional input-output maps.
problem Approximating input-output maps between infinite-dimensional spaces.
method Combines neural networks and model reduction techniques.
result Proves convergence of the proposed approximation methodology.
Adam optimizes DNNs to induce weight sparsity.
problem Large DNN models for edge devices.
method Using Adam optimizer with ReLU activations and L2 regularization.
result Deep learning automatically induces group sparsity of weights.
Paper learns predictive ROMs for combustion from high-fidelity simulations.
problem Predicting combustion dynamics from high-fidelity models.
method Combines physics-based model reduction and machine learning.
result ROMs accurately predict combustion dynamics with significant speedup.
This work examines consistency issues in Gaussian Mixture Model reduction algorithms.
problem Consistency issues in Gaussian Mixture Model reduction algorithms.
method Discussion of the importance of dissimilarity measure choice and consistency of GMR algorithms.
result Most existing GMR algorithms are not consistent with a unique measure, leading to suboptimal reduced GMs.
Efficiently simulates slow dynamics of high-dimensional stochastic systems.
problem Simulating high-dimensional stochastic systems with slow dynamics and fast modes.
method Designs an algorithm to estimate an invariant manifold and its dynamics, averaging out fast modes.
result Efficient simulator of effective dynamics on low-dimensional invariant manifold.
BRIEF reduces CNN models by 32.3% on ImageNet, removing redundant channels.
problem Redundant neural channels in CNN models.
method Backward reduction algorithm based on information flow analysis.
result Significant model reduction (32.3%) on ResNet-34 in ImageNet scale.
MBS reduces CNN model size with minimal accuracy loss.
problem Reducing CNN model size while maintaining accuracy.
method Adaptive macroblock scaling based on effective flops.
result Significant model size reduction across various CNN architectures.
Develops an oblique projection technique to approximate a foliation for non-normal dynamics.
problem Modeling dynamics far from a primary Spectral Submanifold (SSM) in non-normal systems.
method Oblique projection technique based on experimental data.
result Approximates a stable invariant foliation for non-normal dynamics efficiently.
American put options are among the most frequently traded single stock options, and their calibration is computationally challenging since no closed-form expression is available. Due to the higher flexibility in comparison to European options, the mathematical model involves additional constraints, and a variational in…
The paper develops a neural network for learning dynamics from data.
problem Trade-off between representational capacity and overfitting in EDMD.
method Linear recurrent autoencoder network for Koopman operator approximation.
result Improved model reduction and nonlinear reconstruction techniques.
Paper analyzes neural networks using active subspace for structural analysis and vulnerability, reducing model size and improving attacks.
problem Analyzing and reducing the complexity of neural networks.
method Active subspace method for measuring active neurons, network structure modification, and additive universal adversarial attack vector.
result ASNet achieves significant parameter and flops reduction, and improves universal adversarial attack performance.
Active inference selects actions to maximize information gain, aiding structure learning.
problem Learning the structure of underlying world models.
method Active inference selects actions based on expected free energy, which includes information gain and value.
result Actions that maximize information gain help disambiguate among alternative models.
Automates PDE model reduction with time-scale separation.
problem Computational expense in solving high-dimensional PDEs.
method Combines autoencoder and time-continuous model for latent dynamics.
result Automatically learns independent temporal scales in complex systems.
Paper presents a novel method to assess boundedness and stability of nonlinear systems with variable delays.
problem Challenges in assessing boundedness and stability of vector nonlinear systems with variable delays and coefficients.
method Develops a novel framework to evaluate the evolution of solution norms in such systems by constructing scalar counterparts.
result Introduces new criteria for boundedness and stability and estimates the radii of containing balls for history functions.
GPS model predicts subspace-valued functions efficiently.
problem Accurate and efficient prediction of subspace-valued functions.
method Gaussian Process Subspace regression (GPS) model, using multivariate Gaussian distributions on Euclidean space.
result GPS provides accurate, smooth predictions with uncertainty quantification.
New theory shows EDMD works well in chaotic systems.
problem Uncertainty in EDMD's properties in chaos.
method Developed rigorous theory of EDMD on chaotic maps using OPUC and transfer operator methods.
result EDMD converges to correct limits in chaotic systems with small polynomial dictionaries.
AXIOM learns games quickly with simple object models.
problem Data inefficiency in reinforcement learning.
method Expanding object-centric models with active inference.
result AXIOM learns games in 10,000 steps with minimal parameters.
Simplifies NL models by approximating them as LPV systems and identifying NL subterms.
problem Complex NL models are hard to interpret and impractical.
method Linear approximation around operating points, sparse estimation in RKHS, LPV model reduction.
result Identifies NL subterms and their input spaces in sparse additive NL models.
Improved recommendation systems using multi-layer embeddings reduce model size while maintaining accuracy.
problem Improving model accuracy in recommendation systems while minimizing model size.
method Introducing a multi-layer embedding training (MLET) architecture that trains embeddings via a sequence of linear layers.
result Substantial advantages in model accuracy and memory footprint are achieved with reduced embedding dimensions.
The paper tackles enforcing constraints in GANs for interpolation and extrapolation.
problem Enforcing constraints in GANs for interpolation and extrapolation.
method The approach involves adding noise to constraints for interpolation and using a projection step for extrapolation.
result The method improves the efficiency and accuracy of GAN training for constrained interpolation and extrapolation.
A framework learns multiscale dynamics from single trajectories using normalizing flows.
problem Learning effective stochastic dynamics from single observed paths of slow variables.
method Data-driven approach based on coupled multiscale SDEs, stochastic averaging, and normalizing flows for density modeling.
result Scalable approach to capturing epistemic uncertainty in multiscale systems.
Geometrically revisits and models homogeneous spaces of compact Lie group G2.
problem Classifying homogeneous reductive spaces of compact Lie group G2. method Geometrical approach to revisit and model the spaces.
result Explicit relations among geometric models of the spaces.
Adapts POD basis for parametric ROMs using pGP.
problem Updating POD basis for accurate system behavior over parameter space.
method Formulates problem as supervised statistical learning, uses pGP to learn mapping between parameter space and Grassmann manifold.
result Proposes pGP for optimal estimation of POD basis parameters and quantifies uncertainty.
Differentiable mask prunes deep networks for vision and text.
problem Efficiently compressing deep networks for edge devices.
method Introduces a differentiable mask for sparsity induction.
result Successfully prunes weights, filters, and nodes of convolutional and recurrent networks.
Efficiently transforms samples from various statistical models.
problem Approximately transforming samples from one statistical model to another without knowing the source model's parameters.
method Constructs computationally efficient procedures to reduce uniform, Erlang, and Laplace models to general target families.
result Establishes nonasymptotic reductions between canonical high-dimensional problems, such as mixtures of experts, phase retrieval, and signal denoising.
Families of exact solutions are found to a nonlinear modification of the Black-Scholes equation. This risk-adjusted pricing methodology model (RAPM) incorporates both transaction costs and the risk from a volatile portfolio. Using the Lie group analysis we obtain the Lie algebra admitted by the RAPM equation. It gives …
Kernel method approximates dynamical operators from data.
problem Estimating eigenfunctions of dynamical operators from data.
method Kernel-based approach in reproducing kernel Hilbert spaces.
result Eigenfunctions estimated via matrix eigenvalue problems.
New method sparsifies hybrid neural ODEs for better performance and stability.
problem Excessive latent states and interactions from mechanistic models lead to training inefficiency and over-fitting.
method Automatic state selection and structure optimization combining domain-informed graph modifications with data-driven regularization.
result Improved predictive performance and robustness with desired sparsity.
Spectral methods reduce the complexity of Markov processes.
problem Modeling and simplifying state-transition systems.
method Spectral decomposition and state aggregation.
result Developed methods to estimate low-rank Markov models.
Some model reduction techniques for multiple time-scale dynamical systems make use of the identification of low dimensional slow invariant attracting manifolds (SIAM) in order to reduce the dimensionality of the phase space by restriction to the slow flow. The focus of this work is on a proposition and discussion of a …
Regularization leads to balancedness in deep linear networks.
problem Balancedness in deep linear networks.
method Geometric invariant theory and Riemannian geometry of fibers.
result Balancing flows converge to the balanced manifold at a uniform exponential rate.
Data-driven method approximates Koopman generator for system identification and control.
problem Approximating Koopman generator for system identification and control.
method gEDMD (extended dynamic mode decomposition) for deterministic and stochastic systems.
result Data-driven approximation of Koopman generator for system identification and control.
Method generates resource-optimized ML models for different platforms.
problem Deployment of deep learning models on edge platforms requires model reduction and retraining.
method Conditional Neural Architecture Search using Generative Adversarial Networks (GAN)
result Successfully generates resource-optimized ML models for different platforms.
Reduced models derived from agent-based systems using Koopman theory.
problem Time-consuming simulations of large agent-based systems.
method Koopman operator theory applied to simulation data.
result Derived reduced models match known analytical results.
The problem of low-rank approximation with convex constraints, which appears in data analysis, system identification, model order reduction, low-order controller design and low-complexity modelling is considered. Given a matrix, the objective is to find a low-rank approximation that meets rank and convex constraints, w…
A new method reduces over-regularization in Bayesian deep learning.
problem Bayesian inference for over-parameterized models is challenging.
method Walsh-Hadamard Variational Inference (WHVI) using factorization strategies.
result WHVI avoids over-regularization issues and yields speedups and model reductions.
A new framework compresses neural networks using sparse optimization.
problem Efficiently reducing the size of deep neural networks for practical deployment.
method Sparse optimization for model compression, tailored for stochastic learning.
result Up to 7.2 and 2.9 times FLOPs reduction with comparable accuracy.