The paper studies how noise synchronizes tokens in deep transformer models.
problem Understanding synchronization in deep learning models with noise.
method Proves convergence to a stochastic particle system and identifies the limiting SDE.
result The limiting model displays synchronization by noise and exponential dissipation of interaction energy.
New method uses neural networks for accurate angle estimation in noisy conditions.
problem Accurately estimate angles from noisy measurements in various applications.
method Directed Graph Neural Networks (GNNSync) for end-to-end trainable framework.
result GNNSync achieves competitive performance, even at high noise levels.
Paper addresses group synchronization with incomplete measurements and proves linear convergence of GPM.
problem Orthogonal group synchronization with incomplete measurements and additive noise.
method Generalized power method (GPM) with local error bound analysis.
result Linear convergence of GPM to a global maximizer under general additive noise model.
Spectral methods achieve near-optimal performance in orthogonal and permutation group synchronization.
problem Recovering group elements from pairwise measurements in computer vision.
method Spectral methods applied with the leave-one-out technique.
result Near-optimal performance bounds for orthogonal and permutation group synchronization established.
We propose a general framework for solving the group synchronization problem, where we focus on the setting of adversarial or uniform corruption and sufficiently small noise. Specifically, we apply a novel message passing procedure that uses cycle consistency information in order to estimate the corruption levels of gr…
Efficiently estimates rotations with corrupted data.
problem Rotation synchronization under high corruption and noise.
method Message passing algorithm with reweighted least squares.
result Superior performance over state-of-the-art methods.
Novel higher-order group synchronization for noisy local measurements on hypergraphs.
problem Synchronizing higher-order local measurements on hyperedges to global estimates on nodes.
method Message passing algorithm for global synchronization of higher-order measurements.
result Higher-order method outperforms standard pairwise synchronization methods in certain applications.
This paper investigates the effect of cross-shareholding on stock price synchronicity, as a measure of price informativeness, of the listed firms in the Chinese stock market. We gauge firms' levels of cross-shareholdings in terms of centrality in the cross-shareholding network. It is confirmed that it is through a nois…
A probing scheme is considered with an accessible and controllable qubit, used to probe an out-of equilibrium system consisting of a second qubit interacting with an environment. Quantum spontaneous synchronization between the probe and the system emerges in this model and, by tuning the probe frequency, can occur both…
The paper develops robust tests for detecting independence in synchronous stochastic systems with finite sample guarantees.
problem Detecting independence in synchronous stochastic systems with finite sample guarantees.
method Combines confidence region estimates with permutation tests and dependence measures to detect nonlinear dependence.
result Consistent hypothesis tests for detecting independence under mild assumptions.
NS-RGS improves orthogonal group synchronization with faster convergence.
problem Orthogonal group synchronization from pairwise measurements.
method Newton-Schulz iteration for Riemannian gradient optimization.
result NS-RGS achieves linear convergence and near-optimal accuracy.
Improved stock return prediction model handles noise and non-stationarity.
problem Predicting stock returns with robustness to noise and non-stationarity.
method Extended AROW algorithm to handle synchronous mini-batch updates and applied it to stock return prediction.
result The new model outperforms classical approaches in backtesting on S\&P500 stocks.
Dynamic angles estimated from noisy measurements over time with smoothness constraints.
problem Recovering angles from noisy pairwise measurements over time.
method Three algorithms for joint estimation of angles under smoothness constraints.
result MSE converges to zero as T increases under milder conditions. Extends angular synchronization to heterogeneous groups, improving accuracy in multiple applications.
problem Recovering angles from noisy pairwise measurements in a heterogeneous setting.
method Probabilistic generative model and spectral algorithm with robustness analysis.
result Spectral algorithm provides improved recovery accuracy in various parameter regimes.
New algorithm improves PPS for multi-object matching.
problem Efficiently synchronize partial permutations for multi-object matching.
method Proposed CEMP-Partial algorithm for partial permutation synchronization (PPS). Uses sparse matrix operations and nonconvex weighted projected power method.
result Proves CEMP-Partial can exactly classify corrupted and clean partial permutations under adversarial corruption.
Given a measurement graph G=(V,E) and an unknown signal r∈Rn, we investigate algorithms for recovering r from pairwise measurements of the form ri−rj; {i,j}∈E. This problem arises in a variety of applications, such as ranking teams in sports data and time synchronization of distribute…
Robust Q-learning for mean-field control under Wasserstein uncertainty
problem Mean-field control under Wasserstein uncertainty
method Quantization-and-projection scheme with Wasserstein dual reformulation
result Convergence and finite-time iteration bounds
We consider the classic problem of establishing a statistical ranking of a set of n items given a set of inconsistent and incomplete pairwise comparisons between such items. Instantiations of this problem occur in numerous applications in data analysis (e.g., ranking teams in sports data), computer vision, and machine …
Various alignment problems arising in cryo-electron microscopy, community detection, time synchronization, computer vision, and other fields fall into a common framework of synchronization problems over compact groups such as Z/L, U(1), or SO(3). The goal of such problems is to estimate an unknown vector of group eleme…
The sectoral synchronization observed for the Japanese business cycle in the Indices of Industrial Production data is an example of synchronization. The stability of this synchronization under a shock, e.g., fluctuation of supply or demand, is a matter of interest in physics and economics. We consider an economic syste…
Hybrid approach for large-scale network synchronization using KF and PTP.
problem Synchronization of large-scale networks in 5G.
method Combines Kalman Filtering and PTP for pairwise synchronization, and Factor Graphs and Belief Propagation for end-to-end synchronization.
result Error in offset estimation remains below 5 ns in simulations.
We analyze how an observer synchronizes to the internal state of a finite-state information source, using the epsilon-machine causal representation. Here, we treat the case of exact synchronization, when it is possible for the observer to synchronize completely after a finite number of observations. The more difficult …
Transitive consistency is an intrinsic property for collections of linear invertible transformations between Euclidean coordinate frames. In practice, when the transformations are estimated from data, this property is lacking. This work addresses the problem of synchronizing transformations that are not transitively co…
New algorithm uses PSO to optimize DNN training parameters in distributed systems.
problem Reducing synchronization frequency in DNN training leads to poor convergence.
method Integrates PSO into distributed training to automatically compute new parameters.
result Proposed algorithm outperforms synchronous methods in distributed DNN training.
ShadowSync separates background synchronization for scalable distributed training.
problem Reducing synchronization overhead in distributed training for high scalability.
method Separates synchronization from training and runs it in the background.
result Achieves both high throughput and excellent model quality at scale.
New method synchronizes graphs with probability measures on rotations.
problem Synchronizing graphs with measure-valued edges over rotations.
method Formulated as maximization of cycle-consistency in probability measures over rotations, using Sinkhorn divergences.
result Proposes a nonparametric Riemannian particle optimization approach converging to global optimum under certain conditions.
Study predicts synchronization state of financial time series using cross-recurrence plots.
problem Predicting the state of synchronization of financial time series.
method Cross-correlation analysis and deep learning framework for predicting synchronization state based on cross-recurrence plots.
result Satisfactory performance in predicting synchronization state for certain pairs of stocks.
This paper proposes a general model for synchronized crowding behavior. An order parameter is introduced to quantify the level of synchronization which is shown a function of percentage of agents in reactive state. Further, synchronization is shown to be driven by the most active agents with the highest volatility. A t…
New model stabilizes asynchronous LTI systems, independent of synchronous stability.
problem Stability of asynchronous LTI systems under randomization and asynchrony.
method Introduced a new model for random asynchronous LTI systems and developed a method for system identification.
result Stability of random asynchronous LTI systems is independent of synchronous stability.
Study optimizes estimation of orthogonal and rotation matrices from noisy data.
problem Estimating orthogonal and rotation matrices from noisy data.
method Iterative polar decomposition algorithm initialized by spectral methods.
result Algorithm achieves optimal error rate of $(1+o(1)) rac{σ^2 d(d-1)}{2np}$.
Solves complex clustering and rotation synchronization problem.
problem Challenges in classifying and synchronizing rotated objects into multiple categories.
method Semidefinite programming relaxations to solve the joint problem of community detection and synchronization.
result Exact recovery of community detection and synchronization when extending stochastic block model.
Spectral method for joint community detection and group synchronization.
problem Jointly detecting communities and synchronizing orthogonal groups in graphs.
method Spectral decomposition followed by CPQR factorization.
result Near-optimal guarantees for exact and stable recovery of cluster memberships and orthogonal transforms.
Study financial markets using synchronization measures and clustering algorithms.
problem Analyze high-frequency trading dynamics and market states.
method Ordinal pattern series, information-theoretic synchronization measure, clustering algorithms, Markov model.
result Identify two coherent seasons of centralized and decentralized synchronicity.
Networks of coupled dynamical systems provide a powerful way to model systems with enormously complex dynamics, such as the human brain. Control of synchronization in such networked systems has far reaching applications in many domains, including engineering and medicine. In this paper, we formulate the synchronization…
New algorithm reduces federated learning rounds and improves privacy.
problem Inefficient synchronous federated learning causing scalability issues.
method Asynchronous federated learning with reduced communication and differential privacy via Gaussian noise.
result The algorithm reduces waiting times and network communication, making federated learning more scalable and private.
Machine learning predicts synchronization transitions in unknown systems.
problem Predicting synchronization transitions in systems with unknown equations.
method Developed a 'parameter-aware' machine learning scheme using reservoir computing or echo state networks.
result Machine learning accurately predicts synchronization transitions, including hysteresis loops.
New method linearizes nonlinear coupled oscillators on graphs.
problem Predicting global synchronization in nonlinear coupled oscillators on graphs.
method Latent dynamic filters learned through supervised matrix factorization.
result Latent dynamics filters enable effective prediction of global synchronization.
New approach predicts stock price synchronization using RNNs and LSTMs.
problem Forecasting synchronization of stock prices in the Indian market.
method Utilizing recurrence plots and CRQA for non-linear analysis, RNNs and LSTMs for prediction.
result Accuracy of 0.98 and F1 score of 0.83 in predicting stock price synchronization.
Adaptive synchronization improves deep reinforcement learning performance.
problem Fixed step size synchronization can cause loss of properly learned networks.
method Adaptive synchronization based on recent agent behavior.
result Adaptive synchronization leads to better performance in games.
KuramotoGNN uses Kuramoto model to prevent over-smoothing in graph neural networks.
problem Over-smoothing in graph neural networks where node features become indistinguishable.
method Integrates Kuramoto model to prevent phase synchronization and instead achieve frequency synchronization.
result KuramotoGNN reduces over-smoothing on various graph deep learning tasks.
We study the statistical performance of semidefinite programming (SDP) relaxations for clustering under random graph models. Under the Z2 Synchronization model, Censored Block Model and Stochastic Block Model, we show that SDP achieves an error rate of the form \[ \exp\Big[-\big(1-o(1)\big)\bar{n} I^* \Bi…
New MAB model for online caching costs.
problem Learning costs of cached items online.
method Synchronization bandits, MirrorSync algorithm.
result Adversarial regret of O(T2/3) for MirrorSync. We provide tight finite-time convergence bounds for gradient descent and stochastic gradient descent on quadratic functions, when the gradients are delayed and reflect iterates from τ rounds ago. First, we show that without stochastic noise, delays strongly affect the attainable optimization error: In fact, the error…
Paper presents a probabilistic framework for diffusion synchronization.
problem Improper application of heuristics leads to suboptimal results in diffusion synchronization.
method Develops a probabilistic framework to analyze and adapt correlation models for each specific task.
result Achieves better results by identifying optimal correlation models per task.
We extend a recent synchronization analysis of exact finite-state sources to nonexact sources for which synchronization occurs only asymptotically. Although the proof methods are quite different, the primary results remain the same. We find that an observer's average uncertainty in the source state vanishes exponential…
Researchers compare two GAN models for lip-synchronization tasks.
problem Improving the quality of generated lip-sync videos.
method Reimplemented and trained two GAN models (LipGAN and L1WGAN-GP) on the GRID dataset.
result L1WGAN-GP outperformed LipGAN in generating more realistic lip-sync videos.
The bulk synchronous parallel (BSP) is a celebrated synchronization model for general-purpose parallel computing that has successfully been employed for distributed training of machine learning models. A prevalent shortcoming of the BSP is that it requires workers to wait for the straggler at every iteration. To amelio…
We study the quantum synchronization between a pair of two-level systems inside two coupled cavities. By using a digital-analog decomposition of the master equation that rules the system dynamics, we show that this approach leads to quantum synchronization between both two-level systems. Moreover, we can identify in th…