A new method uses higher-order Langevin dynamics with critical damping for better generative modeling.
problem Improving generative models using Langevin dynamics with auxiliary variables.
method Introducing higher-order Langevin dynamics with critical damping, providing closed-form solutions.
result Improved generative models with better performance as measured by FID metric.
TOLD++ improves convergence of diffusion models by critically damping the forward transition matrix.
problem Improving the convergence of Denoising Diffusion Probabilistic Models.
method Critically damping the Third-Order Langevin Dynamics (TOLD) forward transition matrix using eigen-analysis.
result TOLD++ converges faster than TOLD, verified on toy and real datasets.
Improved sampling in generative models using CLDs with a hyperparameter.
problem Improving sampling performance in generative models.
method Extending Critically-damped Langevin Diffusions with a hyperparameter to control noise.
result Derivation of a novel upper bound on Wasserstein sampling error.
Improved generative models using critically-damped Langevin diffusion.
problem Current score-based generative models (SGMs) use overly simplistic diffusion processes, leading to complex denoising tasks and suboptimal performance.
method Proposed a novel critically-damped Langevin diffusion (CLD) and derived a score matching objective and sampling scheme.
result CLD-based SGMs achieve superior performance in synthesis quality compared to previous methods.
Introduces VSMD to improve generative diffusion processes without high costs.
problem High training costs and scalability issues in generative diffusion processes.
method Introduces variational Schrödinger momentum diffusion (VSMD) with adaptively transport-optimized variational scores and critical-damping transform.
result Efficiently generates anisotropic shapes while maintaining transport efficacy, outperforming alternatives.
New optimization method helps models generalize better after achieving near-perfect training performance.
problem Models can achieve near-perfect training performance but fail to generalize well to unseen examples.
method GROKtimizer combines rapid convergence to interpolation with post-interpolation norm minimization using Critically Damped Momentum.
result GROKtimizer provides a quadratic speedup over classical gradient descent, offering a natural solution for selecting low-norm interpolating solutions.
New damping technique improves deep learning models by reducing noise in flat directions.
problem Improving generalization in deep learning models by reducing estimation noise in flat directions.
method Developed a novel random matrix theory based damping learner to reduce the shrinkage coefficient and improve generalization.
result Significant generalization improvements in logistic regression and deep neural networks experiments.
We prove a Weyl-type fractal upper bound for the spectrum of the damped wave equation, on a negatively curved compact manifold. It is known that most of the eigenvalues have an imaginary part close to the average of the damping function. We count the number of eigenvalues in a given horizontal strip deviating from this…
Autoencoder estimates parameters of noisy, multi-component damped signals.
problem Parameter estimation of damped sinusoidal signals under rapid decay and noise.
method Autoencoder-based approach using latent space for frequency, phase, decay, and amplitude estimation.
result High accuracy in parameter estimation, robustness to subdominant components and phase differences.
Unified method for calculating financial option prices from characteristic functions.
problem Calculating financial option prices from characteristic functions in high dimensions.
method Damped Fourier-cosine expansion (COS) method.
result The method converges exponentially if the characteristic function decays exponentially.
Study shows how neural networks learn eigenfunctions of the NTK in underparameterized settings.
problem Understanding the dynamics of MSE optimization in underparameterized neural networks.
method Analysis of gradient flow dynamics, focusing on eigenfunctions of the NTK.
result Eigenfunctions of the NTK determine the learning dynamics in underparameterized networks.
DistillKac generates images quickly using damped wave equations.
problem Generating high-quality images efficiently.
method Uses damped wave equations and Kac dynamics for finite speed transport.
result Fast image generation with high quality and numerical stability.
Spatial statisticians and quantitative investors use the same mathematical object: a Schur complement, damped by one parameter.
problem The Schur complement is used in both spatial modeling and portfolio allocation, but the parameters are different.
method The Schur complement is interpreted as reliability shrinkage of a conditional Gaussian.
result The Schur complement is the same in both applications.
The maximum a posteriori (MAP) configuration of binary variable models with submodular graph-structured energy functions can be found efficiently and exactly by graph cuts. Max-product belief propagation (MP) has been shown to be suboptimal on this class of energy functions by a canonical counterexample where MP conver…
A new method optimizes Fourier pricing for multi-asset options using adaptive quadrature.
problem Efficiently pricing multi-asset options in Lévy models.
method Optimized damping parameters and hierarchical adaptive quadrature.
result Significant speed-up in computational time for up to six dimensions.
In this paper, we present an algorithm for the sparse signal recovery problem that incorporates damped Gaussian generalized approximate message passing (GGAMP) into Expectation-Maximization (EM)-based sparse Bayesian learning (SBL). In particular, GGAMP is used to implement the E-step in SBL in place of matrix inversio…
Framework corrects model form errors in structural dynamics predictions.
problem Model form errors in parametric models of structural dynamics.
method Gaussian Process Latent Force Model (GPLFM) for non-parametric discrepancy representation, linear Bayesian filtering for state and discrepancy estimation, modal reduction for computational tractability.
result Significant reduction of displacement and rotation prediction errors under unseen excitations.
Develops geometric framework for dissipative field equations.
problem Dissipative field equations and their geometric analysis.
method Canonical k-contact manifolds, k-contactifications, splitting results, regularity conditions, criteria for PDEs. result Explicit Hamiltonian descriptions for various nonlinear PDEs.
The energy in a square membrane Ω subject to constant viscous damping on a subset ω⊂Ω decays exponentially in time as soon as ω satisfies a geometrical condition known as the "Bardos-Lebeau-Rauch" condition. The rate τ(ω) of this decay satisfies τ(ω)=2min(−μ(ω),g(ω)) (see Lebeau [Math. Phys. Stud. …
Unified ODE model explains residual and non-residual networks.
problem Unclear relationship between residual and non-residual networks.
method Introducing a damping term in an ODE model to interpolate between ResNet and CNN architectures.
result Unified framework for understanding residual and non-residual networks.
New non-separable covariance kernels for spatiotemporal data derived from harmonic oscillator physics.
problem Capturing complex spatiotemporal dependencies in Gaussian processes.
method Hybrid spectral method based on the harmonic oscillator, deriving explicit covariance kernels.
result Explicit non-separable covariance kernels with space-time interactions.
Newton's method solves variational problems on manifolds.
problem Solving variational equations on manifolds.
method Newton's method with affine covariant damping strategy.
result Numerical results for variational problems demonstrated.
We analyse four consecutive cycles observed in the USA for employment and inflation. They are driven by three oil price shocks and an intended interest rate shock. Non-linear coupling between the rate equations for consumer products as prey and consumers as predators provides the required instability, but its natural d…
For dynamical systems that can be modelled as asymptotically stable linear systems forced by Gaussian noise, this paper develops methods to infer or estimate their modes from observations in real time. The modes can be real or complex. For a real mode, we wish to infer its damping rate and mode shape. For a complex mod…
This paper proposes an alternative to the classical price-adjustment mechanism (called "tâtonnement" after Walras) that is second-order in time. The proposed mechanism, an analogue to the damped harmonic oscillator, provides a dynamic equilibration process that depends only on local information. We show how such a proc…
We discuss stochastic modeling of volatility persistence and anti-correlations in electricity spot prices, and for this purpose we present two mean-reverting versions of the multifractal random walk (MRW). In the first model the anti-correlations are modeled in the same way as in an Ornstein-Uhlenbeck process, i.e. via…
Stability of black holes proven in full subextremal range with positive cosmological constant.
problem Stability of Kerr-de Sitter black holes in the full subextremal range.
method Similar to previous proof in slowly rotating case, with implementation of constraint damping and verification of subprincipal symbol condition.
result Stability of Kerr-de Sitter black holes proven in the full subextremal range.
In this paper we consider the composite self-concordant (CSC) minimization problem, which minimizes the sum of a self-concordant function f and a (possibly nonsmooth) proper closed convex function g. The CSC minimization is the cornerstone of the path-following interior point methods for solving a broad class of co…
Improved numerical solution for BSDEs with reduced boundary errors.
problem Boundary errors in numerical solution of BSDEs.
method Modified damping and shifting schemes to transform target function into a bounded periodic function, applying Fourier transforms.
result Significant reduction in boundary errors with improved accuracy and convergence.
Logarithmic-time schedules boost large-scale language model training efficiency.
problem Improving performance and efficiency in large-scale language model training.
method Designing time-varying hyperparameters (β1,β2,λ) for AdamW, specifically logarithmic-time scheduling with damping mechanisms. result ADANA optimizer achieves up to 40% compute efficiency compared to tuned AdamW, with gains persisting as model scale increases.
New algorithm for computing Wasserstein barycenters with guarantees.
problem Computing Wasserstein barycenters with varying regularization strengths.
method Damped Sinkhorn iterations followed by exact maximization/minimization steps.
result First non-asymptotic convergence guarantees for approximating Wasserstein barycenters.
Momentum is a simple and widely used trick which allows gradient-based optimizers to pick up speed along low curvature directions. Its performance depends crucially on a damping coefficient β. Large β values can potentially deliver much larger speedups, but are prone to oscillations and instability; hence one typic…
New method predicts quasar continuum near Lyman-α with high precision and accuracy.
problem Precise measurement of quasar red damping wing for epoch of reionization.
method Fully probabilistic approach using conditional neural spline flows.
result Achieved state-of-the-art precision and accuracy in predicting quasar continua.
Paper defends diffusion models from membership inference attacks using Langevin dynamics.
problem Defending diffusion models against membership inference attacks.
method Uses critically-damped higher-order Langevin dynamics with auxiliary variables.
result Demonstrates improved resistance to membership inference attacks through theoretical investigation and validation.
Using a modified damped harmonic oscillator model equivalent to a model of market dynamics with price expectations, we analyze the reaction of financial markets to shocks. In order to do this, we gather data from indices of a variety of financial markets for the 1987 Black Monday, the Russian crisis of 1998, the crash …
The log-periodic power law (LPPL) is a model of asset prices during endogenous bubbles. If the on-going development of a bubble is suspected, asset prices can be fit numerically to the LPPL law. The best solutions can then indicate whether a bubble is in progress and, if so, the bubble critical time (i.e., when the bub…
A new real-time method estimates system matrices and states using Kalman filter.
problem Real-time estimation of system matrices and states using Kalman filter.
method Estimation in an observation space, time-invariant interval, and online learning framework.
result Extraordinary performance in estimating matrices for damped oscillation model.
By using Hsu's multiplicative functional for the Neumann heat equation, a natural damped gradient operator is defined for the reflecting Brownian motion on compact manifolds with boundary. This operator is linked to quasi-invariant flows in terms of a integration by parts formula, which leads to the standard log-Sobole…
AdamQLR optimizes Adam with K-FAC heuristics, achieving comparable performance to tuned benchmarks.
problem Improving the performance of Adam optimizers with stabilizing heuristics.
method Combining Adam's update directions with K-FAC's heuristics (damping and learning rate selection).
result Untuned AdamQLR can achieve comparable performance to tuned benchmarks.
New model explains GAN training dynamics and mode collapse.
problem Mode collapse in GANs, where generators fail to reproduce diversity.
method Simplified model using particles in output space, coupled by universal kernel.
result Gradient regularizers can optimally yield convergence through critical damping.
We prove a global smooth isometric immersion for negatively curved surfaces with finite total curvature.
problem Finding a sufficient condition for a complete negatively curved surface to be isometrically embedded in R^3.
method Developed new techniques to overcome slow decay and oscillations of Gauss curvature, reformulating the Gauss-Codazzi equations as a symmetric hyperbolic system.
result Proved the global existence of a smooth solution to the Gauss-Codazzi system, achieving a global smooth isometric immersion of the surface into R^3.
Mathematical study of learning long-term integration in linear RNNs.
problem How do linear recurrent neural networks learn to integrate over long timescales?
method Analytical study of linear RNNs trained to integrate white noise and damped oscillatory filters.
result Learning dynamics are described by low-dimensional effective equations for outlier eigenvalues.
We present a detailed methodological study of the application of the modified profile likelihood method for the calibration of nonlinear financial models characterised by a large number of parameters. We apply the general approach to the Log-Periodic Power Law Singularity (LPPLS) model of financial bubbles. This model …
We develop a new geometric framework suitable for dealing with Hamiltonian field theories with dissipation. To this end we define the notions of k-contact structure and k-contact Hamiltonian system. This is a generalization of both the contact Hamiltonian systems in mechanics and the k-symplectic Hamiltonian syst…
In this paper, a complete Lie symmetry analysis of the damped wave equation with time-dependent coefficients is investigated. Then the invariant solutions and the exact solutions generated from the symmetries are presented. Moreover, a Lie algebraic classifications and the optimal system are discussed. Finally, using C…
PrecGD restores linear convergence in over-parameterized nonconvex matrix factorization.
problem Slow convergence of local search algorithms in over-parameterized nonconvex matrix factorization.
method Preconditioned Gradient Descent (PrecGD) with an inexpensive ℓ2 regularization. result PrecGD restores linear convergence rate even in the over-parameterized case.
LSTM Networks accurately forecast COVID-19 cases in Turkey with lower error than other methods.
problem Forecasting total COVID-19 cases in Turkey using machine learning.
method Long Short-Term Memory (LSTM) Networks for forecasting.
result LSTM Networks outperform other methods in forecasting accuracy.
We present a dynamical system framework for understanding Nesterov's accelerated gradient method. In contrast to earlier work, our derivation does not rely on a vanishing step size argument. We show that Nesterov acceleration arises from discretizing an ordinary differential equation with a semi-implicit Euler integrat…