arXiv research

Cyclical learning rates improve DRL performance without manual tuning.

Cyclical learning rate improves neural machine translation performance.

The paper investigates cyclic arbitrage opportunities in decentralized exchanges.

CycleFQI tackles offline reinforcement learning for cyclic MDPs, mitigating state distribution mismatch.

Random learning rate improves neural network training without extra cost.

Model financial dynamics using 2-manifold geometries, revealing the torus as best for cyclical data.

Mini-batch optimization has proven to be a powerful paradigm for large-scale learning. However, the state of the art parallel mini-batch algorithms assume synchronous operation or cyclic update orders. When worker nodes are heterogeneous (due to different computational capabilities or different communication delays), s…

Resolving Schwartz's quadratic meander number conjecture

This paper introduces 'General Cyclical Training' for neural networks.

A nonpolycyclic nilpotent-by-cyclic group Gamma can be expressed as the HNN extension of a finitely-generated nilpotent group N. The first main result is that quasi-isometric nilpotent-by-cyclic groups are HNN extensions of quasi-isometric nilpotent groups. The nonsurjective injection defining such an extension induces…

Cyclic coordinate descent identifies models in finite time and converges linearly.

We solve structure learning for cyclic linear causal models using observational data.

Cyclical MCMC tackles high-dimensional multimodal distributions, showing convergence under certain conditions.

New algorithms reduce bias in federated learning for non-i.i.d. data.

This paper characterizes the equilibrium in a continuous time financial market populated by heterogeneous agents who differ in their rate of relative risk aversion and face convex portfolio constraints. The model is studied in an application to margin constraints and found to match real world observations about financi…

Study develops a data-based model for in-cylinder pressure and cyclic variations in RCCI engines.

MissNODAG learns cyclic causal graphs from incomplete data.

Cyclic and randomized stepsizes can lead to heavier tails in SGD, improving generalization.

The posteriors over neural network weights are high dimensional and multimodal. Each mode typically characterizes a meaningfully different representation of the data. We develop Cyclical Stochastic Gradient MCMC (SG-MCMC) to automatically explore such distributions. In particular, we propose a cyclical stepsize schedul…

Gaussian Belief Propagation (BP) algorithm is one of the most important distributed algorithms in signal processing and statistical learning involving Markov networks. It is well known that the algorithm correctly computes marginal density functions from a high dimensional joint density function over a Markov network i…

Deep neural networks are typically trained by optimizing a loss function with an SGD variant, in conjunction with a decaying learning rate, until convergence. We show that simple averaging of multiple points along the trajectory of SGD, with a cyclical or constant learning rate, leads to better generalization than conv…

We consider convex SGD updates with a block-cyclic structure, i.e. where each cycle consists of a small number of blocks, each with many samples from a possibly different, block-specific, distribution. This situation arises, e.g., in Federated Learning where the mobile devices available for updates at different times d…

The proximal inertial gradient descent is efficient for the composite minimization and applicable for broad of machine learning problems. In this paper, we revisit the computational complexity of this algorithm and present other novel results, especially on the convergence rates of the objective function values. The no…

Cycles in causal learning cause feedback loops under intervention.

New framework learns nonlinear cyclic causal models from data.

AdaS adapts SGD learning rate based on knowledge gain metrics.

Paper analyzes convergence of continual learning with adaptive methods.

The paper generalizes cyclic metrics in homogeneous Finsler geometry.

Unified analysis for shuffling-type gradient methods in optimization.

In this paper we study the connections between cyclic presentations of groups and branched cyclic coverings of (1,1)-knots. In particular, we prove that every n-fold strongly-cyclic branched covering of a (1,1)-knot admits a cyclic presentation for the fundamental group encoded by a Heegaard diagram of genus n.

Characterizes non-degenerate cyclic metric Lie algebras.

Deep learning models have become state of the art for natural language processing (NLP) tasks, however deploying these models in production system poses significant memory constraints. Existing compression methods are either lossy or introduce significant latency. We propose a compression method that leverages low rank…

We analyse four consecutive cycles observed in the USA for employment and inflation. They are driven by three oil price shocks and an intended interest rate shock. Non-linear coupling between the rate equations for consumer products as prey and consumers as predators provides the required instability, but its natural d…

Proves almost profinite rigidity for certain free-by-cyclic groups.

We study the connections among the mapping class group of the twice punctured torus, the cyclic branched coverings of (1,1)-knots and the cyclic presentations of groups. We give the necessary and sufficient conditions for the existence and uniqueness of the n-fold strongly-cyclic branched coverings of (1,1)-knots, thro…

New cyclicity measures defined in weighted Besov spaces, with stability and geometric analysis.

A new learning rule consistently reduces error over data samples.

The method of block coordinate gradient descent (BCD) has been a powerful method for large-scale optimization. This paper considers the BCD method that successively updates a series of blocks selected according to a Markov chain. This kind of block selection is neither i.i.d. random nor cyclic. On the other hand, it is…

A new framework learns cyclic causal graphs from incomplete data.

We refine the cyclic cohomological apparatus for computing the Hopf cyclic cohomology of the Hopf algebras associated to infinite primitive Cartan-Lie pseudogroups, and for the transfer of their characteristic classes to foliations. The main novel feature is the precise identification as a Hopf cyclic complex of the im…

Develop a variational framework for statistical inference on cyclic interactions.

WSD schedule improves model training efficiency by adapting learning rates dynamically.

In this paper, we study quandles of cyclic type, which form a particular subclass of finite quandles. The main result of this paper describes the set of isomorphism classes of quandles of cyclic type in terms of certain cyclic permutations. By using our description, we give a direct classification of quandles of cyclic…

Paper proves CLT for quantile SGD with constant learning rate.

Unified rigidity theorem for cyclic and alternating surfaces.

In spin geometry, traceless cyclic homogeneous Riemannian manifolds equipped with a homogeneous spin structure can be viewed as the simplest manifolds after Riemannian symmetric spin spaces. In this paper, we give some characterizations and properties of cyclic and traceless cyclic homogeneous Riemannian manifolds and …

The abstract introduces a new cyclic structure for surfaces.

New method uses resurgent analysis to determine growth rate of quantum field theory coefficients.