The correspondence between residual networks and dynamical systems motivates researchers to unravel the physics of ResNets with well-developed tools in numeral methods of ODE systems. The Runge-Kutta-Fehlberg method is an adaptive time stepping that renders a good trade-off between the stability and efficiency. Can we …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Adaptive time stepping”
Paper adapts ACI for online multi-step time-series forecasting with coverage guarantees.
problem Achieving reliable error bounds in online multi-step time-series forecasting.
method Adaptive conformal inference (ACI) adapted for multi-step forecasting with dynamic significance levels.
result Proposes a multi-step ACI algorithm with finite-sample coverage guarantees for non-exchangeable data.
A neural RNN model adapts time steps for non-stationary time series data.
problem Modeling and forecasting non-stationary time series with sharp changes.
method RNN-ODE-Adap model using neural ODE and adaptive time steps.
result Consistent estimation of intensity function for Hawkes-type data.
JANET improves time series prediction with adaptive uncertainty regions.
problem Time series data's lack of exchangeability and multi-step prediction challenges.
method Proposes JANET, a framework for joint adaptive prediction regions with controlled error rates.
result Demonstrates superior performance in multi-step prediction tasks across diverse datasets.
Step-DAD improves BED by periodically updating a design policy during experiments.
problem Improving flexibility and robustness in Bayesian experimental design.
method Semi-amortized, policy-based approach that updates a design policy during data collection.
result Consistently superior decision-making and robustness compared to current BED methods.
The CSA-ES is an Evolution Strategy with Cumulative Step size Adaptation, where the step size is adapted measuring the length of a so-called cumulative path. The cumulative path is a combination of the previous steps realized by the algorithm, where the importance of each step decreases with time. This article studies …
In this paper, we present an online adaptive PCA algorithm that is able to compute the full dimensional eigenspace per new time-step of sequential data. The algorithm is based on a one-step update rule that considers all second order correlations between previous samples and the new time-step. Our algorithm has O(n) co…
New solver avoids memory issues for long differential equations.
problem Memory constraints in adaptive probabilistic ODE solvers.
method Fixed memory demands adaptive probabilistic solver using robust state estimation.
result Eliminates memory issues for long time series simulations.
Dynamic risk measures follow law invariance principles over time.
problem Tackles dynamic risk measurement principles.
method Shows equivalence between adapted law invariance and recursive one-step conditional-law representation for time-consistent risk measures.
result Identifies adapted law invariance as the dynamic counterpart of ordinary law invariance.
A framework is introduced for actively and adaptively solving a sequence of machine learning problems, which are changing in bounded manner from one time step to the next. An algorithm is developed that actively queries the labels of the most informative samples from an unlabeled data pool, and that adapts to the chang…
Test-time training adapts a pretrained model to each prompt via parameter updates, improving accuracy under pretraining-to-test distribution shifts.
problem Improving accuracy of pretrained models under distribution shifts.
method Explaining TTT behavior through a decision-theoretic lens.
result TTT reduces prediction error when updates are spectrally matched to the prompt's signal-to-noise ratio and aligned with query-relevant eigen-directions.
Reinforcement learning has attracted great attention recently, especially policy gradient algorithms, which have been demonstrated on challenging decision making and control tasks. In this paper, we propose an active multi-step TD algorithm with adaptive stepsizes to learn actor and critic. Specifically, our model cons…
ISALT: Inference-based schemes adaptive to large time-stepping for locally Lipschitz ergodic systemsmath.NA
ISALT uses inference to simulate SDEs with large time-steps, improving efficiency.
problem Efficiently simulating ergodic SDEs with large time-steps.
method Inference-based schemes adaptive to large time-steps (ISALT) from data.
result ISALT achieves significant time reduction and optimal accuracy.
The paper analyzes and validates two step size schedules for SGD: exponential and cosine, proving their adaptivity and performance.
problem The variability of SGD performance due to step size choice.
method Analysis and empirical evaluation of exponential and cosine step sizes.
result Exponential and cosine step sizes are adaptive to noise and achieve optimal performance without tuning hyperparameters.
Enhances CEV model pricing with high-order scheme and adaptive time stepping.
problem Improving accuracy in pricing American CEV models with irregularities.
method High-order time adapted scheme, local mesh refinement, adaptive time stepping, fifth-order 5(4) Dormand-Prince method.
result Highly accurate solution with reduced computational runtime.
Reward-poisoning attacks can force RL agents to learn bad policies, and we categorize and quantify their feasibility.
problem Reward-poisoning attacks can manipulate RL agents to learn undesirable policies.
method Categorize attacks by infinity-norm constraint, provide thresholds for feasibility, and develop adaptive attack strategies.
result Adaptive reward-poisoning attacks can achieve the nefarious policy in polynomial steps, while non-adaptive attacks require exponential steps.
Multi-hop inference is necessary for machine learning systems to successfully solve tasks such as Recognising Textual Entailment and Machine Reading. In this work, we demonstrate the effectiveness of adaptive computation for learning the number of inference steps required for examples of different complexity and that l…
Study improves BN TTA under distribution shift using higher-order asymptotics.
problem Improving BN TTA for changing data distributions.
method Integrates Edgeworth expansion and saddlepoint approximation with one-step M-estimation.
result Derives optimal weighting parameter for minimized mean-squared error.
SPAQL improves RL by adaptively partitioning state-action space and learning a time-invariant policy.
problem Efficient model-free reinforcement learning with scalable algorithms.
method Adaptive Q-learning with UCB and Boltzmann exploration, automatically tuning temperature.
result SPAQL converges faster and uses fewer resources than AQL, showing higher sample efficiency.
Deep recurrent neural networks perform well on sequence data and are the model of choice. However, it is a daunting task to decide the structure of the networks, i.e. the number of layers, especially considering different computational needs of a sequence. We propose a layer flexible recurrent neural network with adapt…
New algorithm improves stability of optimization algorithms by adapting step-size.
problem Optimization algorithms' effectiveness is sensitive to step-size hyperparameters.
method Adapts NGN step-size method with momentum to enhance stability.
result Achieves convergence rate of O(1/√K) without restrictive assumptions.
WALNUTS improves sampling efficiency and robustness for multi-scale distributions.
problem Adapting leapfrog step size for multi-scale posterior distributions.
method Adapts leapfrog step size at fixed intervals of simulated time, selecting the largest step size to keep energy error below a threshold.
result Substantial improvements in sampling efficiency and robustness compared to standard NUTS.
New RL algorithms adapt to time limits, improving task performance.
problem Fixed RL behaviors cannot adapt to different time restrictions.
method Introduced two algorithms for time adaptive RL: Independent Gamma-Ensemble and n-Step Ensemble.
result Zero-shot adaptation between different time restrictions.
New insights into SGD and SGD-M in high dimensions.
problem Understanding and comparing SGD and SGD-M in high-dimensional settings.
method Developed high-dimensional scaling limits for SGD-M and online SGD, examining their dynamics and performance.
result SGD-M amplifies high-dimensional effects, potentially degrading performance compared to online SGD.
An online learning framework for survival analysis with real-time adaptation.
problem Real-time adaptation to dynamic environments and censored data in survival analysis.
method Online Newton Step (ONS) for optimal second order online convex optimization.
result Logarithmic stochastic regret for ONS with adaptive aggregation method.
M-L2O adapts fast to new tasks by self-adapting during test-time.
problem L2O optimizers struggle with out-of-distribution tasks.
method Meta-training an L2O optimizer to adapt quickly to new tasks.
result M-L2O converges significantly faster than vanilla L2O with only 5 steps of adaptation.
Paper analyzes convergence rates for multi-agent learning in games.
problem Convergence rates for multi-agent learning in games.
method Characterizes finite-time convergence rates for joint OGD learning on -cocoercive games and develops adaptive algorithms.
result Adaptive algorithms achieve same convergence rates as non-adaptive counterparts.
Adaptive denoising models adjust the number of steps based on noise level.
problem Generating data with lower intrinsic dimensions.
method Adaptive diffusion models using Doob's h-transform to terminate at a random time.
result Adaptive models simplify termination to a first-hitting rule, enhancing adaptability.
GradaGrad adapts learning rate non-monotonically, overcoming AdaGrad's step size decrease.
problem Fixed learning rate in AdaGrad leads to step size decrease over time.
method Introduces GradaGrad, which grows or shrinks the learning rate based on a different accumulation in the denominator.
result GradaGrad achieves similar convergence rates as AdaGrad and demonstrates non-monotone adaptation.
LatentTrack generates model parameters online for nonstationary data.
problem Online probabilistic prediction under nonstationary dynamics.
method Sequential neural architecture with latent filtering and amortized inference.
result Consistently lower negative log-likelihood and mean squared error than baselines.
This work proposes ACTC for adaptive distributed learning under communication constraints.
problem Adaptive distributed learning in networks with communication constraints.
method ACTC (Adapt-Compress-Then-Combine) strategy with diffusion exchange of compressed updates.
result ACTC iterates converge to the optimizer with significant bit savings.
A general framework for multi-step ahead adaptive conformal heteroscedastic time series forecastingstat.ML
AEnbMIMOCQR generates robust multi-step ahead prediction intervals for time series data.
problem Generating reliable multi-step ahead prediction intervals for time series data.
method Adaptive ensemble batch multi-input multi-output conformalized quantile regression (AEnbMIMOCQR) based on conformal prediction principles.
result AEnbMIMOCQR provides close to exact coverage and robustness to distribution shifts.
Adaptive step-size improves optimization in complex geometries.
problem Optimizing functions with non-Euclidean geometries.
method Adaptive step-size strategy for optimization algorithms.
result Guaranteed convergence for Adaptive Conditional Gradient Descent.
New adaptive step-size method for convex optimization without tuning.
problem Optimizing convex functions efficiently with stochastic gradients.
method Adapted Adaptive Gradient Descent Without Descent to stochastic setting.
result Stochastic gradient descent converges under various assumptions.
We present a numerical framework for approximating unknown governing equations using observation data and deep neural networks (DNN). In particular, we propose to use residual network (ResNet) as the basic building block for equation approximation. We demonstrate that the ResNet block can be considered as a one-step me…
AdaptOn achieves logarithmic regret in adaptive control of unknown partially observable linear systems.
problem Adaptive control in partially observable linear dynamical systems.
method AdaptOn algorithm that estimates system dynamics through online learning and gradient descent.
result AdaptOn achieves a logarithmic regret bound of polylog(T) after T steps.
In this paper, we introduce a method for adapting the step-sizes of temporal difference (TD) learning. The performance of TD methods often depends on well chosen step-sizes, yet few algorithms have been developed for setting the step-size automatically for TD learning. An important limitation of current methods is that…
ANT improves TS diffusion models by automatically determining noise schedules.
problem Suboptimal performance of TS diffusion models due to lack of domain-specific noise schedules.
method ANT proposes an adaptive noise schedule that automatically determines proper noise schedules for TS datasets based on their statistics.
result ANT achieves state-of-the-art performance on various TS tasks, including forecasting, refinement, and generation.
Online Tensor Learning: Computational and Statistical Trade-offs, Adaptivity and Optimal Regretstat.ML
Unified online tensor learning algorithm reduces computational and memory costs.
problem Efficiently learning from large tensors with minimal data storage and timely predictions.
method oRGrad algorithm for online tensor learning.
result oRGrad achieves optimal regret and adaptive regret.
Sparse Meta Networks for Sequential Adaptation and its Application to Adaptive Language Modellingcs.NE
Sparse Meta Networks adapt deep neural networks incrementally for fast learning.
problem Training deep neural networks is slow and impractical for complex, changing environments.
method Sparse Meta Networks use a memory layer to learn online sequential adaptation, accumulating fast-weights incrementally.
result Sparse Meta Networks achieve strong performance in various sequential adaptation scenarios.
We develop a projected Nesterov's proximal-gradient (PNPG) approach for sparse signal reconstruction that combines adaptive step size with Nesterov's momentum acceleration. The objective function that we wish to minimize is the sum of a convex differentiable data-fidelity (negative log-likelihood (NLL)) term and a conv…
This paper analyzes SGD with biased gradients for deep learning models.
problem Analyzing SGD with biased gradients for deep learning models.
method Non-asymptotic analysis of SGD with adaptive steps for non-convex smooth functions.
result Adagrad, RMSProp, and AMSGRAD converge to critical points at a similar rate to unbiased case.
Adaptive step-size method improves compressed SGD performance in machine learning.
problem Communication bottleneck in distributed and decentralized optimization.
method Developed an adaptive step-size method for compressed SGD.
result Order-optimal convergence rates for various objective functions.
This paper presents, evaluates, and discusses a new software tool to automatically build Dynamic Bayesian Networks (DBNs) from ordinary differential equations (ODEs) entered by the user. The DBNs generated from ODE models can handle both data uncertainty and model uncertainty in a principled manner. The application, na…
The article examines in some detail the convergence rate and mean-square-error performance of momentum stochastic gradient methods in the constant step-size and slow adaptation regime. The results establish that momentum methods are equivalent to the standard stochastic gradient method with a re-scaled (larger) step-si…
Few-step protein backbone generators reduce sampling time by over 20x.
problem Computational bottleneck in diffusion-based protein generation models.
method Score distillation adapted for protein backbone generation, combined with inference time noise modulation.
result Significant reduction in sampling time (20+ fold) while maintaining comparable performance.
Proposes QDF to improve multi-step time-series forecasting.
problem Ignoring label autocorrelation and unequal task weights in training objectives.
method Quadratic-form weighted training objective and QDF learning algorithm.
result Improves performance of various forecast models, achieving state-of-the-art results.
Adaptive sequential testing optimizes epidemic control by learning optimal test strategies.
problem Optimizing test allocation in epidemics with network and temporal dependence.
method Adaptive sequential design with Online Super Learner for optimal test strategies.
result Superior performance in simulated university COVID-19 pandemic.