Dynamic Sparse Training finds efficient sparse networks from scratch.
problem Finding efficient sparse neural networks.
method Jointly optimizes network parameters and sparsity with trainable thresholds.
result Achieves state-of-the-art performance with minimal performance loss.
Predictive Sparse Manifold Transform learns dynamic video sequences.
problem Learning and predicting natural dynamics in video sequences.
method Two-layer framework: sparse coding and manifold learning.
result PSMT with dynamic embedding space outperforms static baselines in future frame prediction.
Deep learning predicts dynamics from sparse data.
problem Predicting spatiotemporal dynamics from sparse data.
method Spatially dimension-independent deep learning framework.
result Predicts dynamics from sparse data sites.
We address the issue of estimating the topology and dynamics of sparse linear dynamic networks in a hyperparameter-free setting. We propose a method to estimate the network dynamics in a computationally efficient and parameter tuning-free iterative framework known as SPICE (Sparse Iterative Covariance Estimation). The …
New pruning methods improve dynamic sparse training performance.
problem Improving dynamic sparse training performance.
method Design and empirical analysis of pruning criteria.
result Most pruning methods yield similar results, but magnitude-based pruning performs best in low-density regimes.
New method learns dynamics from sparse data using geometric constraints.
problem Learning dynamics from sparse, undersampled data.
method Reformulates inference as a stochastic control problem, using geometry-driven path augmentation.
result Accurately recovers stochastic dynamics from extremely undersampled data.
New techniques improve the accuracy of identifying nonlinear systems from noisy data.
problem Identifying nonlinear dynamical systems from noisy state measurements.
method Comparative study of local and global smoothing techniques to denoise state measurements and improve sparse regression methods.
result Global smoothing methods outperform local methods in improving the accuracy of governing equation recovery.
We consider the problem of modeling multivariate time series with parsimonious dynamical models which can be represented as sparse dynamic Bayesian networks with few latent nodes. This structure translates into a sparse plus low rank model. In this paper, we propose a Gaussian regression approach to identify such a mod…
Parsimonious Dynamic Mode Decomposition selects sparse modes robustly.
problem Manual tuning of sparsity parameters in traditional DMD.
method Time-delay embedding and Orthogonal Matching Pursuit.
result Autonomously determines optimally sparse subset of modes.
LD-EnSF speeds up data assimilation with sparse observations.
problem Efficiently assimilate sparse and noisy data into complex dynamical systems.
method LD-EnSF uses latent dynamics networks and history-aware LSTM encoders to process sparse observations without full-space simulations.
result Achieves significant speedups over existing methods while maintaining high accuracy.
Framework learns continuous dynamics from sparse trajectories.
problem Learning dynamics from sparsely sampled and high-dimensional trajectories.
method Interpolative Multi-Marginal Flow Matching (IMMFM) framework.
result IMMFM outperforms existing methods in forecasting and downstream tasks.
Improved SINDy autoencoder for identifying noisy dynamical systems.
problem Robust identification of noisy dynamical systems from data.
method Incorporates noise-separating neural network structures into SINDy autoencoder architecture.
result Accurately recovers latent dynamics and estimates measurement noise from noisy observations.
Neural networks have achieved state of the art performance across a wide variety of machine learning tasks, often with large and computation-heavy models. Inducing sparseness as a way to reduce the memory and computation footprint of these models has seen significant research attention in recent years. In this paper, w…
Despite the importance of sparsity signal models and the increasing prevalence of high-dimensional streaming data, there are relatively few algorithms for dynamic filtering of time-varying sparse signals. Of the existing algorithms, fewer still provide strong performance guarantees. This paper examines two algorithms f…
A nonparametric Bayesian sparse graph linear dynamical system (SGLDS) is proposed to model sequentially observed multivariate data. SGLDS uses the Bernoulli-Poisson link together with a gamma process to generate an infinite dimensional sparse random graph to model state transitions. Depending on the sparsity pattern of…
This research develops an evolutionary approach to discover non-Gaussian stochastic dynamical systems.
problem Discovering explicit governing equations of stochastic dynamical systems with Lévy noise from data.
method ESSR approach using genetic programming, sparse regression, and nonlocal Kramers-Moyal formulas.
result The approach effectively extracts non-Gaussian stochastic dynamical systems from sample path data.
The paper analyzes dynamics of momentum in high dimensions with sparse updates.
problem Theoretical analysis of momentum dynamics in high-dimensional sparse settings.
method Theoretical analysis of two models: least squares with sparse inputs and logistic regression with a rare class.
result Characterization of high-dimensional limits of momentum dynamics and phase structure.
Model infers latent variables in sparse coding models using Langevin dynamics.
problem Sampling posterior distribution in sparse coding models.
method Langevin dynamics for inference and simultaneous learning of parameters.
result Langevin dynamics efficiently sample from 'L0 sparse' posterior distribution.
The paper tackles reward-relevance in offline RL with sparse decision dynamics.
problem Offline reinforcement learning with sparse decision dynamics and estimation sparsity.
method Reward-filtered least-squares policy evaluation using thresholded lasso.
result The method provides theoretical guarantees with sample complexity dependent on sparse component size.
ADAPT method controls balance between sparse generator and discriminator in GANs.
problem High computational complexity in GANs training.
method Balanced Dynamic Sparse Training (ADAPT) method.
result ADAPT method achieves a good trade-off between performance and computational cost.
SDIFT generates full-field dynamics from sparse, irregular data.
problem Modeling and reconstructing physical dynamics from sparse, off-grid observations.
method SDIFT uses a functional Tucker model and sequential diffusion for generating full-field evolution from irregular sparse observations.
result Significant improvements in reconstruction accuracy and computational efficiency compared to state-of-the-art approaches.
Bayesian method identifies dynamical models with uncertainty quantification.
problem Uncertainty in selecting governing equations for dynamical systems.
method Bayesian sparse identification with model averaging.
result Accurately recovers sparse interaction structures with uncertainty quantification.
FedDST trains sparse sub-networks to improve efficiency in federated learning.
problem Efficiently train large models on resource-limited edge devices with limited network bandwidth.
method Dynamic sparse training to reduce model size and communication.
result FedDST outperforms existing methods in federated learning, especially in non-i.i.d. settings.
OKRidge solves sparse ridge regression problems for nonlinear systems.
problem Identifying sparse governing equations for nonlinear dynamical systems.
method OKRidge algorithm using saddle point formulation and ADMM-based approach with efficient proximal operators.
result OKRidge achieves provable optimality with significantly faster run times than Gurobi.
Sparse Hopfield model improves memory retrieval with fewer connections.
problem Memory retrieval efficiency with fewer connections.
method Sparse extension of Hopfield model, derived from sparse entropic regularizer.
result Sparse Hopfield model achieves tighter error bounds and better performance.
This paper considers portfolio construction in a dynamic setting. We specify a loss function comprised of utility and complexity components with an unknown tradeoff parameter. We develop a novel regret-based criterion for selecting the tradeoff parameter to construct optimal sparse portfolios over time.
Large SGD step sizes lead to sparse feature learning in neural networks.
problem Sparse feature learning in neural networks with large step sizes.
method Empirical observations and theoretical analysis of SGD dynamics.
result Large step sizes induce implicit regularization leading to sparse predictors.
Proposes a deep generative model for robust forecasting on sparse multivariate time series.
problem Forecasting on sparse multivariate time series with suboptimal results when sparsity is high.
method Dynamic Gaussian Mixture distribution for modeling latent clusters, using neural networks and gating mechanism.
result Demonstrates robust modeling of sparse multivariate time series with improved accuracy.
dLDS models neural dynamics as sparse combinations of simpler components.
problem Understanding complex neural dynamics at a population level.
method Proposes a decomposed dynamical system model trained through dictionary learning.
result Model efficiently captures and demix diverse neural dynamics.
SSINNs learn Hamiltonian systems from data with interpretable, low-memory models.
problem Learning Hamiltonian dynamical systems from data efficiently and accurately.
method Combines fourth-order symplectic integration with sparse regression for a learned Hamiltonian.
result Outperforms state-of-the-art techniques in system prediction and energy conservation.
Modern deep neural networks are typically highly overparameterized. Pruning techniques are able to remove a significant fraction of network parameters with little loss in accuracy. Recently, techniques based on dynamic reallocation of non-zero parameters have emerged, allowing direct training of sparse networks without…
New algorithm STCV improves sparse model discovery from normalised data.
problem Distortion of sparse model discovery due to data scaling.
method STCV, a novel sparse regression algorithm robust to data scaling.
result STCV outperforms standard methods on normalised, noisy datasets.
Method learns dynamics from sparse, irregular feature data.
problem Learn system dynamics from sparse, irregularly sampled feature time series.
method Formulates as high-dimensional linear regression using signatures.
result Oracle bound on prediction error with explicit sampling dependencies.
New method integrates sparse parametric and nonparametric techniques for complex system modeling.
problem Lack of accurate modeling for complex biological systems due to nonlinearities.
method Sparse nonparametric estimation framework combining parametric and nonparametric techniques.
result Accurately captures nonlinearities in complex systems without prior information.
With the rapid increase of available data for complex systems, there is great interest in the extraction of physically relevant information from massive datasets. Recently, a framework called Sparse Identification of Nonlinear Dynamics (SINDy) has been introduced to identify the governing equations of dynamical systems…
Study dynamic batch learning in high-dimensional sparse linear bandits.
problem Dynamic batch learning in high-dimensional sparse linear contextual bandits under batch constraints.
method Characterized fundamental learning limits via regret lower bound and provided matching upper bound.
result Prescribed an optimal scheme for dynamic batch learning in high-dimensional sparse linear contextual bandits.
A fast spectral algorithm detects community structure in evolving graphs.
problem Detecting community structure in time-evolving sparse graphs.
method Extension of the Bethe-Hessian matrix for spectral community detection.
result The algorithm reaches the optimal detectability threshold and outperforms other methods.
Paper learns Koopman operator from sparse data, escaping function space constraints.
problem Learning Koopman operator from non-closed function spaces.
method Operator stochastic approximation algorithm using conditional mean embeddings (CME).
result Online sparse learning algorithm with trajectory-based sampling guarantees.
Dynamic model pruning improves performance on deep neural networks without retraining.
problem High memory and latency requirements for deep neural networks on low-end devices.
method Dynamic allocation of sparsity pattern and feedback signal to reactivate pruned weights.
result Sparse models achieve state-of-the-art performance with no additional retraining.
Sparse Kernel Flows learns dynamical systems from data.
problem Learning dynamical systems from limited data.
method Sparse Kernel Flows: trains optimal kernel from a dictionary of kernels.
result Sparse Kernel Flows can learn from 132 chaotic systems.
ERFit identifies dynamic equations from data with minimal supervision.
problem Data-driven sparse system identification in science and engineering.
method Entropic Regression method.
result ERFit package simplifies sparse system identification for various applications.
Proposes a flexible MGP model for dynamic, sparse correlations.
problem Handling dynamic and sparse correlations in multivariate data.
method Non-stationary MGP with dynamic spike-and-slab prior and EM algorithm.
result Captures dynamic and sparse correlations effectively.
Novel data acquisition schemes have been an emerging need for scanning microscopy based imaging techniques to reduce the time in data acquisition and to minimize probing radiation in sample exposure. Varies sparse sampling schemes have been studied and are ideally suited for such applications where the images can be re…
Sparse-mode DMD disambiguates local and global modes in spatiotemporal data.
problem Disambiguating local and global modes in spatiotemporal data.
method Sparse-mode DMD with sparsity-promoting regularization.
result Explicitly constructs discrete and continuous spectra.
Method improves SINDy for noisy nonlinear systems.
problem Recover nonlinear dynamical systems from noisy data.
method Reweighted ℓ1-regularized least squares. result Improved accuracy and robustness in noisy conditions.
BINDy uses Bayesian methods to identify nonlinear dynamics from data.
problem Learning sparse representations of complex dynamics from data.
method Bayesian treatment of dictionary learning system identification using reversible-jump Markov-chain Monte-Carlo.
result BINDy produces models that are sparse in model space rather than parameter space.
CODE learns ODE dynamics from sparse data, outperforming neural and kernel methods.
problem Learning ODE dynamics from sparse and noisy data.
method CODE uses Polynomial Chaos Expansion (aPCE) for the ODE's RHS, enabling global orthonormal polynomial representation.
result CODE exhibits remarkable extrapolation capabilities even under novel initial conditions and measurement noise.
DDD reformulated for sparse matrices, integrating trajectory and snapshot time series data.
problem Efficiently integrate trajectory and snapshot time series data.
method Reformulate DDD to use compact basis functions, reducing parameter scaling.
result Inference of sparse matrices reduces the number of parameters in DDD.