A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New algorithm uses PSO to optimize DNN training parameters in distributed systems.
Machine learning predicts synchronization transitions in unknown systems.
This paper proposes a general model for synchronized crowding behavior. An order parameter is introduced to quantify the level of synchronization which is shown a function of percentage of agents in reactive state. Further, synchronization is shown to be driven by the most active agents with the highest volatility. A t…
The sectoral synchronization observed for the Japanese business cycle in the Indices of Industrial Production data is an example of synchronization. The stability of this synchronization under a shock, e.g., fluctuation of supply or demand, is a matter of interest in physics and economics. We consider an economic syste…
Stochastic Gradient Descent (SGD) is the most popular algorithm for training deep neural networks (DNNs). As larger networks and datasets cause longer training times, training on distributed systems is common and distributed SGD variants, mainly asynchronous and synchronous SGD, are widely used. Asynchronous SGD is com…
Deep learning is a popular machine learning technique and has been applied to many real-world problems. However, training a deep neural network is very time-consuming, especially on big data. It has become difficult for a single machine to train a large model over large datasets. A popular solution is to distribute and…
Study on synchronization in financial markets with time delays.
This work studies clustering in transformer models, proving exponential convergence to a single token state.
In this paper, we show synchronization for a group of output passive agents that communicate with each other according to an underlying communication graph to achieve a common goal. We propose a distributed event-triggered control framework that will guarantee synchronization and considerably decrease the required comm…
Paper proposes GPM for simultaneous community detection and group synchronization.
Extends angular synchronization to heterogeneous groups, improving accuracy in multiple applications.
Improved KLMC for sampling under various conditions.
Word embeddings are a powerful approach for analyzing language and have been widely popular in numerous tasks in information retrieval and text mining. Training embeddings over huge corpora is computationally expensive because the input is typically sequentially processed and parameters are synchronously updated. Distr…
Novel higher-order group synchronization for noisy local measurements on hypergraphs.
Trade is a fundamental pillar of economy and a form of social organization. Its empirical characterization at the worldwide scale is represented by the World Trade Web (WTW), the network built upon the trade relationships between the different countries. Several scientific studies have focused on the structural charact…
Data parallelism has become the de facto standard for training Deep Neural Network on multiple processing units. In this work we propose DC-S3GD, a decentralized (without Parameter Server) stale-synchronous version of the Delay-Compensated Asynchronous Stochastic Gradient Descent (DC-ASGD) algorithm. In our approach, w…
Most commonly used distributed machine learning systems are either synchronous or centralized asynchronous. Synchronous algorithms like AllReduce-SGD perform poorly in a heterogeneous environment, while asynchronous algorithms using a parameter server suffer from 1) communication bottleneck at parameter servers when wo…
Distributed training of deep nets is an important technique to address some of the present day computing challenges like memory consumption and computational demands. Classical distributed approaches, synchronous or asynchronous, are based on the parameter server architecture, i.e., worker nodes compute gradients which…
New method uses neural networks for accurate angle estimation in noisy conditions.
Machine learning with big data often involves large optimization models. For distributed optimization over a cluster of machines, frequent communication and synchronization of all model parameters (optimization variables) can be very costly. A promising solution is to use parameter servers to store different subsets of…
We analyze how an observer synchronizes to the internal state of a finite-state information source, using the epsilon-machine causal representation. Here, we treat the case of exact synchronization, when it is possible for the observer to synchronize completely after a finite number of observations. The more difficult …
ShadowSync separates background synchronization for scalable distributed training.
In this work, we propose a hybrid approach to synchronize large scale networks. In particular, we draw on Kalman Filtering (KF) along with time-stamps generated by the Precision Time Protocol (PTP) for pairwise node synchronization. Furthermore, we investigate the merit of Factor Graphs (FGs) along with Belief Propagat…
New method synchronizes graphs with probability measures on rotations.
In order to use the advanced inference techniques available for Ising models, we transform complex data (real vectors) into binary strings, by local averaging and thresholding. This transformation introduces parameters, which must be varied to characterize the behaviour of the system. The approach is illustrated on fin…
Study predicts synchronization state of financial time series using cross-recurrence plots.
Efficiently estimates rotations with corrupted data.
GCMM improves clustering and fits un-synchronized data.
Study optimizes estimation of orthogonal and rotation matrices from noisy data.
Solves complex clustering and rotation synchronization problem.
We introduce a new, high-throughput, synchronous, distributed, data-parallel, stochastic-gradient-descent learning algorithm. This algorithm uses amortized inference in a compute-cluster-specific, deep, generative, dynamical model to perform joint posterior predictive inference of the mini-batch gradient computation ti…
Spectral method for joint community detection and group synchronization.
Study financial markets using synchronization measures and clustering algorithms.
Spectral methods achieve near-optimal performance in orthogonal and permutation group synchronization.
Networks of coupled dynamical systems provide a powerful way to model systems with enormously complex dynamics, such as the human brain. Control of synchronization in such networked systems has far reaching applications in many domains, including engineering and medicine. In this paper, we formulate the synchronization…
New method linearizes nonlinear coupled oscillators on graphs.
New approach predicts stock price synchronization using RNNs and LSTMs.
Adaptive synchronization improves deep reinforcement learning performance.
KuramotoGNN uses Kuramoto model to prevent over-smoothing in graph neural networks.
Deep learning models can take weeks to train on a single GPU-equipped machine, necessitating scaling out DL training to a GPU-cluster. However, current distributed DL implementations can scale poorly due to substantial parameter synchronization over the network, because the high throughput of GPUs allows more data batc…
The paper studies how noise synchronizes tokens in deep transformer models.
Paper presents a probabilistic framework for diffusion synchronization.
This paper presents Rudra, a parameter server based distributed computing framework tuned for training large-scale deep neural networks. Using variants of the asynchronous stochastic gradient descent algorithm we study the impact of synchronization protocol, stale gradient updates, minibatch size, learning rates, and n…
We extend a recent synchronization analysis of exact finite-state sources to nonexact sources for which synchronization occurs only asymptotically. Although the proof methods are quite different, the primary results remain the same. We find that an observer's average uncertainty in the source state vanishes exponential…
Transitive consistency is an intrinsic property for collections of linear invertible transformations between Euclidean coordinate frames. In practice, when the transformations are estimated from data, this property is lacking. This work addresses the problem of synchronizing transformations that are not transitively co…
Paper addresses group synchronization with incomplete measurements and proves linear convergence of GPM.
We propose three new robust aggregation rules for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model. The attackers can arbitrarily manipulate the data transferred between the servers and the workers in the parameter server~(PS) architecture. We prove the Byzantine resilie…
Researchers compare two GAN models for lip-synchronization tasks.