Early neural networks can be simplified to linear models, revealing surprising simplicity.
problem Complexity of neural network learning dynamics.
method Formal proof and empirical verification of early-time learning dynamics of neural networks.
result Early learning dynamics of neural networks can be approximated by simple linear models.
Study reveals hidden infections and infection dynamics from early data.
problem Understanding early infection dynamics and hidden infections in COVID-19.
method Data-driven machine learning analysis focusing on infection counts over time.
result Significant asymptomatic infections, 10-day lag, and strong infectious force.
Novel approach detects early warning indicators in complex systems.
problem Detecting abrupt transitions in complex systems.
method Directed anisotropic diffusion map and latent stochastic dynamical systems.
result Early warning indicators can detect tipping points in state transitions.
Study proposes a new early-warning framework for high-dimensional complex systems.
problem Predicting critical transitions in complex systems like epileptic seizures.
method Integrates manifold learning with stochastic dynamical system modeling, using Schrödinger bridge theory.
result Demonstrates higher sensitivity and robustness in epilepsy prediction.
The paper analyzes early stopping in linear regression and shows it's equivalent to ridge regularization.
problem Understanding the effect of early stopping on linear regression models.
method Characterization of gradient descent dynamics and analysis of excess risk.
result Early stopped solution is equivalent to minimum norm solution for a generalized ridge regularized problem.
New method dynamically adjusts UTD ratio to balance under- and overfitting in RL.
problem Balancing under- and overfitting in world model learning for RL.
method Dynamic adjustment of UTD ratio based on validation performance on a small subset of experience data.
result Our method improves balance between under- and overfitting compared to default settings and competitive with extensive hyperparameter search.
Paper presents a method to diagnose schizophrenia using fMRI dynamics from healthy controls.
problem Early diagnosis of mental disorders like schizophrenia using fMRI.
method Self-supervised pre-training on fMRI dynamics of healthy controls for transfer learning.
result Effective classification of schizophrenia using fMRI dynamics with small datasets.
The study analyzes sharpness dynamics in neural networks, revealing mechanisms and conditions.
problem Understanding sharpness in neural network training.
method Fixed point analysis and edge of stability analysis in a simplified 2-layer linear network.
result Reveals mechanisms behind sharpness trends, conditions for edge of stability, and a period-doubling route to chaos.
The paper accelerates LLM inference by adding early exit heads trained in a self-supervised manner.
problem Inference speed in large language models (LLMs) is slow and resource-intensive.
method Adding self-supervised early exit heads at intermediate transformer layers to stop computation early based on confidence thresholds.
result Entropy provides the most reliable confidence metric for stopping computation early.
Dynamic Influence Tracker measures changing sample importance during model training.
problem Static influence measurements during training overlook how sample importance varies over time.
method Dynamic Influence Tracker (DIT) captures time-varying sample influence across arbitrary time windows.
result DIT reveals distinct learning phases with shifting priorities and detects corrupted samples more efficiently.
Study uses DNM theory to detect early warning signals of market instability.
problem Detecting early warning signals of financial market instability.
method Applying Dynamical Network Marker (DNM) theory to trading data from the Tokyo Stock Exchange.
result Early warning signals of large price movements can be detected on a daily time scale.
Remote sensing satellites capture the cyclic dynamics of our Planet in regular time intervals recorded in satellite time series data. End-to-end trained deep learning models use this time series data to make predictions at a large scale, for instance, to produce up-to-date crop cover maps. Most time series classificati…
Sepsis is a life-threatening host response to infection associated with high mortality, morbidity, and health costs. Its management is highly time-sensitive since each hour of delayed treatment increases mortality due to irreversible organ damage. Meanwhile, despite decades of clinical research, robust biomarkers for s…
Neural networks learn clean data patterns first, then noisy data, leading to improved performance initially but deteriorating later.
problem Improvement in prediction error on clean data during early training of neural networks with noisy labels.
method Theoretical analysis and experiments to explore the dynamics of gradient descent and the impact of clean and noisy data.
result Neural networks prioritize learning clean data patterns first, leading to improved performance initially but deteriorating later due to diminishing gradient dominance of clean samples over noisy ones.
Novel pricing method for equity-indexed annuities under uncertain volatility and stochastic interest rate.
problem Pricing equity-indexed annuities with early surrender risk under uncertain market conditions.
method Advanced financial modeling techniques, including uncertain volatility framework and Hull-White model for interest rate dynamics. Numerical algorithm using tree-based framework with local volatility optimization.
result High effectiveness of the proposed numerical algorithm compared to machine learning-based methods.
Early alignment in neural networks leads to sparse representations but hinders convergence.
problem The implicit bias of gradient descent during early training phases.
method Quantitative description of early alignment phase in small initialisation, one hidden layer networks.
result Early alignment induces a sparse representation but also hinders convergence to global minima.
This study uses high-frequency data to identify early warning signals for bank crises.
problem Identifying early warning signals for impending bank crises.
method Constructing multiple recurrence networks (MRNs) based on high-frequency stock returns to monitor nonlinear dynamics.
result Key indicators of MRNs, particularly average mutual information, provide valuable insights into periods of extreme volatility.
Early training of deep neural networks leads to small, directionally converging weights.
problem Training dynamics of deep homogeneous neural networks with small initializations.
method Gradient flow analysis and study of KKT points for neural correlation function.
result Weights converge in direction to KKT points during early training stages.
The paper develops a method to predict ICU mortality risk across diverse patient populations.
problem Improving patient survival by recognizing risky trajectories during ICU stays.
method Domain adaptation strategies to learn mortality prediction models robust to diverse ICU populations.
result The proposed model outperforms baselines, achieving AUC numbers up to 0.88 for the Cardiac ICU population.
Machine learning detects regime shifts in online game-experiments with high accuracy.
problem Detecting regime shifts in online social systems.
method Gradient-boosted decision trees with memory-retaining features.
result Significantly outperforms standard early warning indicators.
A new framework explains why early pruning works well.
problem Understanding why early pruning of neural networks leads to good performance.
method Gradient flow framework to unify pruning measures.
result Magnitude-based pruning removes least contributing parameters, leading to faster convergence.
Understanding the learning dynamics of neural networks is one of the key issues for the improvement of optimization algorithms as well as for the theoretical comprehension of why deep neural nets work so well today. In this paper, we introduce a random matrix-based framework to analyze the learning dynamics of a single…
Enhances early-exit neural networks for anytime classification.
problem Lack of guaranteed prediction quality improvement with longer computation time.
method Post-hoc modification based on Product-of-Experts to enforce conditional monotonicity.
result Achieves conditional monotonicity in prediction quality, enabling anytime classification.
In this paper, we propose a novel unsupervised learning method to learn the brain dynamics using a deep learning architecture named residual D-net. As it is often the case in medical research, in contrast to typical deep learning tasks, the size of the resting-state functional Magnetic Resonance Image (rs-fMRI) dataset…
Improved sample efficiency in reinforcement learning with deep Gaussian processes.
problem Efficiently learn to control actions with limited interaction data.
method Deep Gaussian processes that simulate dynamics with depth and prior knowledge.
result Significantly improved early sample efficiency across various tasks, including half-cheetah control.
Random matrix theory explains transient signal detectability in early-stopped gradient flow.
problem Transient signal detectability in early-stopped gradient flow.
method Random matrix theory applied to gradient flow in a linear teacher-student setting.
result Transient Baik-Ben Arous-Péché (BBP) transition in learning dynamics due to anisotropy and noise.
Paper presents a spatio-temporal Bayesian model for early detection of COVID-19 hotspots.
problem Understanding spatio-temporal dynamics of COVID-19 hotspots to prevent outbreaks.
method Spatio-temporal Bayesian framework with a zero-mean Gaussian process and non-stationary kernel function enhanced by deep neural networks.
result Model demonstrates superior hotspot-detection performance compared to baseline methods.
Study builds an EWS for early detection of stock market turbulence.
problem Early detection of stock market turbulence.
method SWARCH filtering probabilities, two-peak method, LSTM network.
result 96.6% test-set accuracy and 2.4 days of forewarned period.
Regularization timing affects deep network performance, not just its presence.
problem The timing of regularization in deep networks impacts their performance.
method Analysis of different datasets, architectures, regularization methods, and learning rate schedules.
result The critical period for regularization in deep networks is decisive of final performance.
Energy-efficient DL inference for IoT devices reduces power consumption and improves performance.
problem Energy inefficiency in deep learning models for IoT devices.
method Energy-aware early exiting policy to balance energy consumption and inference accuracy.
result Accuracy and service rate improved up to 25% and 35% respectively.
Paper proposes a new descriptor for early trajectory characterization in matrix iterations.
problem Comparing early behavior of high-dimensional trajectories in nonlinear matrix iterations.
method Develops a two-channel fuzzy coordinate system using F-transform for compact representation.
result The descriptor achieves high R^2 values (mean = 0.6480) in approximating convergence lengths.
With the proliferation of algorithmic high-frequency trading in financial markets, the Limit Order Book has generated increased research interest. Research is still at an early stage and there is much we do not understand about the dynamics of Limit Order Books. In this paper, we employ a machine learning approach to i…
Dynamic treatment strategies on networks amplify policy impact through spillovers.
problem Effective dynamic treatment allocation in network settings.
method Q-Ising, a three-stage pipeline integrating Bayesian dynamic Ising model, treatment adoption histories, and offline reinforcement learning.
result Adaptive targeting outperforms static centrality benchmarks in Indian village microfinance networks and synthetic data.
New method stops experiments early for harm in diverse groups.
problem Early stopping of experiments for harmful treatment effects in diverse populations.
method Causal machine learning approach (CLASH) for early stopping.
result CLASH effectively stops experiments early for harmful treatment effects in diverse groups.
The IM effect helps detect anomalies by memorizing inliers early.
problem Challenges in fully unsupervised outlier detection.
method Theoretical study of a simple autoencoder, focusing on early training dynamics.
result Characterization of IM effect emergence, strength, and persistence.
ABS dynamically adjusts batch size based on policy stability, improving RL performance.
problem Diminishing returns with large batch sizes in RL due to non-stationary data.
method Adaptive Batch Scaling (ABS) with Behavioral Divergence metric.
result Larger batch sizes can improve RL performance, contrary to conventional wisdom.
Paper proposes deep learning model for dynamic stock repurchase forecasting.
problem Complex temporal dependencies in corporate financial conditions.
method Hybrid Temporal Convolutional Network (TCN) and Attention-based LSTM.
result Model significantly outperforms static baselines in stock repurchase forecasting.
The study reveals optimal early stopping behaviors in deep learning models.
problem Understanding optimal early stopping in deep learning models.
method Theoretical analysis of linear models and experimental validation.
result Two distinct behaviors of optimal early stopping time depending on model dimension relative to dataset features.
Study on how optimal representations emerge during deep learning training, focusing on the role of implicit regularization.
problem Understanding how optimal representations for tasks are learned during training.
method Investigates the role of implicit regularization in learning minimal sufficient representations, analyzing changes in representation content during training.
result Semantically meaningful but ultimately irrelevant information is encoded in early transient dynamics of training, which is later discarded.
New model predicts neural network performance from early training epochs, incorporating architecture impact.
problem Predicting neural network performance from early training epochs, neglecting architecture impact.
method Architecture-aware graph ordinary differential equation model.
result Model outperforms state-of-the-art methods for MLP and CNN learning curves.
E2CM uses class means for efficient early exits in neural networks.
problem Efficient early exits in neural networks with low computational cost.
method Early Exit Class Means (E2CM) based on class means of samples, without gradient-based training. result E2CM achieves higher accuracy with fixed training time budget and boosts existing early exit schemes. Minimizing non-convex and high-dimensional objective functions is challenging, especially when training modern deep neural networks. In this paper, a novel approach is proposed which divides the training process into two consecutive phases to obtain better generalization performance: Bayesian sampling and stochastic op…
Early stopping in meta-learning improved by analyzing neural activation patterns.
problem Early stopping in few-shot learning is challenging due to distributional shifts between meta-validation and meta-test sets.
method Activation-Based Early-stopping (ABE) analyzes hidden layer activations from unlabelled support examples to detect when target generalization diverges from source data.
result Simple activation statistics can effectively estimate target generalization, improving few-shot transfer learning across various algorithms and datasets.
Estimates RL data for dynamic treatment effects using GMM.
problem Estimating dynamic treatment effects from RL data with nonstationary behavior policies.
method Weighted GMM approach to stabilize variance in adaptive RL settings.
result Valid hypothesis testing and confidence regions for dynamic treatment effects.
Gradient-flow optimization is reinterpreted as a statistical inference problem.
problem Optimizing training duration and assessing model performance in deep learning.
method Develops a statistical framework for gradient-flow training, treating it as a random-effects model.
result Establishes asymptotic optimality for prediction and reduces reliance on validation splits.
Framework prevents deep learning models from memorizing noisy labels.
problem Deep learning models memorize noisy labels during early learning phase.
method Develops a technique that exploits early learning phase via regularization.
result Framework achieves robustness to noisy annotations on benchmarks and real-world datasets.
Enhances early risk assessments for pediatric outcomes using contrastive learning.
problem Improving risk assessments in early stages of pediatric development.
method Contrastive multi-modal framework that treats each time window as a distinct modality, training on all available data.
result Consistent improvements in early-stage risk assessments validated on real-world tasks.
TM-GCN learns dynamic graph embeddings using tensor algebra.
problem Handling dynamic graphs in graph neural networks.
method Tensor M-product for dynamic graph convolution.
result TM-GCN outperforms existing methods on edge classification and link prediction.