Paper optimizes MVE network convergence and regularization.
problem Optimizing Mean Variance Estimation networks for better performance.
method Presented two key insights: warm-up period for mean optimization and separate regularization of mean and variance.
result Warm-up period and separate regularization improve MVE network performance.
Warm-up improves training by adapting learning rate based on curvature.
problem Improving training efficiency in deep learning models.
method Introducing a curvature condition to explain warm-up, and showing empirically that it leads to faster convergence.
result Adapting learning rate based on curvature condition naturally induces warm-up-like schedule, leading to faster convergence.
Paper eliminates warm-up phase for PO in linear MDPs, achieving optimal regret.
problem Costly warm-up phase in PO algorithms for linear MDPs.
method Simple contraction mechanism replaces warm-up phase.
result Achieves rate-optimal regret with improved dependence on problem parameters.
Layer normalization placement affects training stability and warm-up stage necessity.
problem Training instability and the necessity of a learning rate warm-up stage in Transformers.
method Theoretical analysis and mean field theory to prove gradient behavior at initialization.
result Removing the warm-up stage for Pre-LN Transformers can achieve comparable results with less time and tuning.
The paper shows how warming up the learning rate improves deep learning performance.
problem Improving deep learning performance through better handling of larger learning rates.
method Systematic experiments with SGD and Adam showing the benefits of warmup and different regimes of operation.
result Properly choosing ηextinit can eliminate the need for warmup and improve performance. DP-SGD can update fewer coordinates while maintaining privacy.
problem How to update fewer coordinates in DP-SGD without losing optimization signal.
method TP-TopK (Two-Phase TopK DP-SGD), a two-phase method for coordinate-sparse private training.
result Private training can update fewer coordinates without losing optimization signal, scaling noise with active dimension \(k\) instead of full dimension \(d\).
Click-through rate (CTR) prediction has been one of the most central problems in computational advertising. Lately, embedding techniques that produce low-dimensional representations of ad IDs drastically improve CTR prediction accuracies. However, such learning techniques are data demanding and work poorly on new ads w…
Dual model combines HMM and neural networks for energy trading during volatile periods.
problem Optimizing energy trading performance during market volatility.
method Integrates Hidden Markov Models and neural networks with Black-Litterman portfolio optimization.
result Achieved 83% return with Sharpe ratio 0.77 during COVID period.
New algorithm finds critical points in non-convex optimization with heavy-tailed gradients.
problem Non-convex stochastic optimization with heavy-tailed gradient estimates.
method Gradient clipping, momentum, and normalized gradient descent.
result High-probability convergence to critical points with best-known rates.
Active learning tackles cold start and imbalanced data issues.
problem Cold start problem and imbalanced data in ML models.
method ODAL approach with 3-stage AL policies.
result ODAL can quickly reach high performance models with limited labels.
AdaScale SGD adapts learning rates for large-batch training efficiently.
problem Adapting learning rates for large-batch training to balance speed-ups and model quality.
method Adaptive learning rate adaptation based on gradient variance.
result AdaScale achieves reliable speed-ups for a wide range of batch sizes without degrading model quality.
Optimal learning rate schedules derived for various tasks.
problem Inadequate learning rate schedules in practice compared to theory.
method Refined analysis of learning rate schedules for optimization algorithms.
result Derives new problem-adaptive learning rate schedules.
Despite the advances in the representational capacity of approximate distributions for variational inference, the optimization process can still limit the density that is ultimately learned. We demonstrate the drawbacks of biasing the true posterior to be unimodal, and introduce Annealed Variational Objectives (AVO) in…
We provide a differential cocycle model for elliptic cohomology with complex coefficients and use analytic methods to construct a cocycle representative for the Witten class in this language. Our motivation stems from the conjectural connection between 2-dimensional field theories and elliptic cohomology originally due…
The paper provides examples of geometric transitions in low dimensions.
problem Exploring geometric transitions between different types of structures in low dimensions.
method Explicit examples and computations of transitions from hyperbolic to Euclidean, spherical, and Anti-de Sitter structures.
result Details of elementary computations and techniques are provided to explain geometric transitions.
We prove that in CAT(0) spaces a quasi-geodesic is Morse if and only if it is contracting. Specifically, in our main theorem we prove that for γ a quasi-geodesic in a CAT(0) space X, the following four statements are equivalent: (i) γ is Morse, (ii) γ is (b,c)--contracting, (iii), γ is strongly contracting, and…
This work analyzes the statistical properties of adaptive gradient methods.
problem Lack of understanding of the statistical properties of adaptive gradient methods.
method Theoretical analyses and experiments on the variance of update magnitudes.
result The variance of update magnitudes is an increasing and bounded function of time, not diverging.
Variational Autoencoders are powerful models for unsupervised learning. However deep models with several layers of dependent stochastic variables are difficult to train which limits the improvements obtained using these highly expressive models. We propose a new inference model, the Ladder Variational Autoencoder, that…
SALSA automatically adjusts learning rates in stochastic gradient methods.
problem Automatic adjustment of learning rates in stochastic gradient methods.
method SALSA uses a line-search procedure to gradually increase the learning rate, then a statistical test to decrease it.
result SALSA matches the performance of best hand-tuned learning rate schedules in deep learning tasks.
A new dynamic learning-rate scheme for optimization.
problem Manual tuning of learning rates in optimization is time-consuming and error-prone.
method Locally optimal stepsize estimation for dynamic learning-rate scheduling.
result Our method achieves comparable performance with minimal tuning.
The aim of multi-task reinforcement learning is two-fold: (1) efficiently learn by training against multiple tasks and (2) quickly adapt, using limited samples, to a variety of new tasks. In this work, the tasks correspond to reward functions for environments with the same (or similar) dynamical models. We propose to l…
Although exploration in reinforcement learning is well understood from a theoretical point of view, provably correct methods remain impractical. In this paper we study the interplay between exploration and approximation, what we call approximate exploration. Our main goal is to further our theoretical understanding of …
The paper extends gradient flow and relaxation studies to non-flat Riemannian manifolds.
problem Understanding gradient flows and relaxation in non-flat Riemannian manifolds.
method Developed a criterion for comparing relaxation along gradient descent curves using non-metricity tensor.
result Revealed a universal asymmetry: warming up is faster than cooling down.
Large-scale distributed training requires significant communication bandwidth for gradient exchange that limits the scalability of multi-node training, and requires expensive high-bandwidth network infrastructure. The situation gets even worse with distributed training on mobile devices (federated learning), which suff…
VCoTTA uses variational Bayesian methods to adapt models under continuous domain shifts.
problem Error accumulation in continual test-time adaptation.
method VCoTTA employs variational Bayesian techniques to update a Bayesian Neural Network (BNN) during testing, combining priors from source and teacher models.
result VCoTTA effectively mitigates error accumulation in CTTA, as shown by experimental results on three datasets.
Optimizes neural network training by dynamically updating Tucker decomposition ranks.
problem Redundant parameters in neural network architectures.
method Geometry-aware training of factorized layers in tensor Tucker format.
result Optimal locally approximating the original dynamics without initial rank knowledge.
The paper finds linked periodic orbits in disc homeomorphisms using braids.
problem Finding linked periodic orbits in disc homeomorphisms.
method Interpreting linking of orbits by induced braids and using forcing relations.
result New examples of linked orbits of periods at most 4 for pseudo-Anosov braid types.
Paper proposes a robust framework for detecting multiple periodic components in time series.
problem Detecting multiple periodic components in time series with interlaced patterns and external noise.
method Applying maximal overlap discrete wavelet transform to isolate periodic components, ranking them by wavelet variance, and detecting single periodicity robustly.
result The proposed algorithm outperforms other methods for both single and multiple periodicity detection.
Enhances financial time series forecasting with a multi-period learning framework.
problem Accurate financial time series forecasting requires considering both short-term and long-term trends.
method Proposes a Multi-period Learning Framework (MLF) with three modules: Inter-period Redundancy Filtering, Learnable Weighted-average Integration, and Multi-period self-Adaptive Patching.
result Improves financial time series forecasting accuracy and efficiency.
Nonorientable spanning surfaces of periodic knots can have arbitrarily high first Betti number.
problem Periodic knots do not always have nonorientable spanning surfaces of high genus.
method Examples and calculations of nonorientable spanning surfaces of periodic knots.
result The first Betti number of nonorientable spanning surfaces can be arbitrarily large.
We give criteria for framed links and 3-manifolds to be periodic of prime order. As applications we show that the Poincare sphere is of periodicity 2, 3, 5 only and the Brieskorn sphere (2,3,7) is of periodicity 2, 3, 7 only.
Model for multi-period carbon market pricing with allowances.
problem Carbon market pricing with multiple trading periods and compliance times.
method Singular forward-backward stochastic differential equations (SDEs).
result Value function convergence to infinite period model under certain conditions.
We derive new obstructions to periodicity of classical knots by employing the Heegaard Floer correction terms of the finite cyclic branched covers of the knots. Applying our results to two fold covers, we demonstrate through numerous examples that our obstructions are successful where many existing periodicity obstruct…
Paper defines linking numbers for periodic tangles.
problem Defining invariant linking numbers for periodic tangles.
method Defined linking numbers using motifs and showed invariance.
result Linking numbers are well-defined and invariant.
Khovanov-Rozansky homology shows periodic links have group actions.
problem Understanding periodic links through homology.
method Showed Khovanov-Rozansky slN-homology has group actions on m-periodic links. result Proved an analog of periodicity criterion using slN-homology. Maps minimize periodic points for high periods, but not for low periods.
problem Understanding periodic points of maps on infinite type surfaces.
method Analysis of isotopic maps, spun pseudo-Anosov maps, and Handel-Miller maps.
result Spun pseudo-Anosov maps minimize periodic points for sufficiently high periods.
We prove the existence of infinitely many periodic points of symplectomorphisms isotopic to the identity if they admit at least one (non-contractible) hyperbolic periodic orbit and satisfy some condition on its flux. The obtained periodic points correspond to periodic orbits whose free homotopy classes are formed by it…
Study on periodic knots, proving limitations on their Alexander polynomials.
problem Understanding Alexander polynomials of periodic knots.
method Polynomial factorization, number theory interpretation, computational methods.
result Alexander polynomials of freely periodic knots are restricted to products of cyclotomic polynomials.
Paper disproves Wright's periodic map conjecture.
problem Existence of periodic maps on closed hyperbolic surfaces.
method Analyzes automorphisms and curve behavior on surfaces.
result No periodic map can fill every curve with its image.
Derives an index formula for families of end-periodic Dirac operators.
problem Calculating the index of families of end-periodic Dirac operators.
method Using the renormalized Chern character and Fourier-Laplace transform of the Bismut superconnection.
result Establishes an index formula involving a new end-periodic eta form.
In this paper, we give a classification of orientation reversing periodic maps on closed surfaces which generalizes the theory of Nielsen for the orientation preserving periodic maps. On one hand, we give a group of data for each orientation reversing periodic map such that two periodic maps with the same data must be …
DEPTS learns to forecast periodic time series with improved accuracy.
problem Forecasting periodic time series is challenging due to complex dependencies and diverse periods.
method DEPTS uses a decoupled formulation with an expansion module and a periodicity module to handle these challenges.
result DEPTS significantly improves forecasting accuracy, reducing errors by up to 20%.
Study periodic geodesics on contact 3D manifolds, proving existence and precise properties.
problem Existence and properties of periodic geodesics in contact sub-Riemannian metrics.
method Develops two independent subjects: existence of spiraling geodesics and precise study of geodesics on quotient of SL2(R).
result Proves existence and precise properties of periodic geodesics.
This paper is devoted to prove the existence of q-periodic alternating projections of prime alternating q-periodic knots. The main tool is the Menasco-Thistlethwaite's Flyping theorem. Let K be an oriented prime alternating knot that is q-periodic with q≥3, i.e. K admits a symmetry that is a rotation of…
Paper introduces dynamic strategies for multi-period investment models.
problem Optimizing investment strategies over multiple periods with risk and return considerations.
method Developed a Bellman principle for discrete time multi-period mean-variance models, leading to dynamic optimal strategies and efficient frontiers.
result Dynamic optimal strategies can achieve higher returns with lower risk compared to the 1/n strategy.
We show that any virtual or welded period of a classical knot K can be realized as a classical period. A direct consequence is that a classical knot admits only finitely many virtual or welded periods.
We study the asymptotic behavior of the sequence of the Nielsen numbers {N(fk)}, the essential periodic orbits of f and the homotopy minimal periods of f by using the Nielsen theory of maps f on infra-solvmanifolds of type R. We give a linear lower bound for the number of essential periodic orbits of such …
The aim of this paper is to compare statistical properties of a bubble period with those of the anti-bubble period in stock markets. We investigate the statistical properties of daily data for the Nikkei 225 index in the 28-year period from January 1975 to April 2003, corresponded to the periods of bubbles and anti-bub…