Deep Leakage from Gradients can reveal private training data from shared gradients.
problem The safety of gradients in multi-node machine learning systems is questionable.
method Empirical validation of Deep Leakage from Gradient attack on computer vision and natural language processing tasks.
result The attack is more effective than previous methods, achieving pixel-wise accuracy for images and token-wise matching for texts.
Paper proposes a new algorithm combining gradient descent and Langevin dynamics.
problem Gradient descent can get stuck in local minima, while Langevin dynamics can explore but is slow.
method Replica exchange mechanism swaps positions if Langevin yields a lower objective function.
result New algorithm converges to global minimum linearly with high probability.
Asynchronous computation and gradient compression have emerged as two key techniques for achieving scalability in distributed optimization for large-scale machine learning. This paper presents a unified analysis framework for distributed gradient methods operating with staled and compressed gradients. Non-asymptotic bo…
Gradient flows on graphons converge to curves on graphon space.
problem Optimizing functions on large, exchangeable graphs.
method Euclidean gradient flow on edge weights converges to a curve on graphon space.
result Gradient flows on graphons can be described as curves of maximal slope on graphon space.
Proposes r2SGLD for efficient constrained exploration in non-convex learning.
problem Stagnation in high-temperature chains of reSGLD in distribution tails.
method r2SGLD: replica exchange with reflection steps in a bounded domain.
result Reflection steps enhance mixing rates with quadratic improvement in domain diameter.
Study on communication delays in decentralized learning networks.
problem Optimizing communication latency in decentralized learning networks.
method Utilized network information theory and random geometric graph theory.
result Communication delay scales as O(n^(2-3β)/βlog n).
Improved reSGLD accelerates convergence in non-convex learning problems.
problem Inefficient swaps due to noisy energy estimators in reSGLD.
method Variance reduction for noisy energy estimators, theoretical analysis, and numerical experiments.
result Exponential acceleration in convergence for non-convex learning problems.
BLADE uses Bayesian methods to discover complex systems from scarce data.
problem Efficiently discovering governing equations of complex dynamical systems from limited data.
method Combines replica-exchange stochastic gradient Langevin Monte Carlo with active learning.
result Reduces measurement requirements by 60% for Lotka-Volterra and 40% for Burgers' equation.
Large-scale distributed training requires significant communication bandwidth for gradient exchange that limits the scalability of multi-node training, and requires expensive high-bandwidth network infrastructure. The situation gets even worse with distributed training on mobile devices (federated learning), which suff…
New algorithm speeds up MCMC for deep learning models.
problem Large biases in SGMCMC for big data.
method Adaptive replica exchange SGMCMC (reSGMCMC).
result Achieves state-of-the-art results on various datasets.
Replica exchange Langevin diffusion accelerates nonconvex optimization.
problem Nonconvex optimization challenges in machine learning.
method Replica exchange Langevin diffusion, discretization analysis.
result Replica exchange accelerates convergence to global minima.
This paper deals with distributed policy optimization in reinforcement learning, which involves a central controller and a group of learners. In particular, two typical settings encountered in several applications are considered: multi-agent reinforcement learning (RL) and parallel RL, where frequent information exchan…
Study improves forex forecasting accuracy using machine learning models.
problem Improving accuracy in predicting foreign exchange rates.
method Employed LSTM neural networks and Gradient Boosting Classifier for forecasting.
result Achieved 99.449% accuracy in forecasting USD/BDT exchange rates.
Novel approach to Bayesian experimental design for non-exchangeable data.
problem Optimal experimental design for non-exchangeable data.
method Inside-Out SMC2 algorithm embedded in particle Markov chain Monte Carlo framework. result Efficacy demonstrated on a set of dynamical systems.
Improved sample efficiency in reinforcement learning with object exchangeability.
problem Sample inefficiency in reinforcement learning, especially with complex input structures.
method Attention-based method to project inputs into an efficient representation space invariant under input ordering.
result Our representation reduces the search space by a factor of m! for m objects, improving sample efficiency.
We address the issue of speeding up the training of convolutional neural networks by studying a distributed method adapted to stochastic gradient descent. Our parallel optimization setup uses several threads, each applying individual gradient descents on a local variable. We propose a new way of sharing information bet…
Fault-tolerant federated learning for non-uniform data.
problem Faulty workers corrupting data in federated learning.
method Fault-resilient proximal gradient (FRPG) algorithm with Nesterov's acceleration and local FRPG for reduced communication.
result FRPG and LFRPG converge faster than robust stochastic aggregation.
Modern large scale machine learning applications require stochastic optimization algorithms to be implemented on distributed computational architectures. A key bottleneck is the communication overhead for exchanging information such as stochastic gradients among different workers. In this paper, to reduce the communica…
New sampler tackles complex discrete energy landscapes efficiently.
problem Stagnation in gradient-based discrete samplers for non-convex settings.
method DREXEL sampler with Replica Exchange and Adjusted Metropolis.
result Proves samplers satisfy detailed balance and converge to target distribution.
Paper predicts stock market values using machine learning.
problem Predicting stock market values for Tehran stock exchange groups.
method Used machine learning algorithms including Decision Tree, Bagging, Random Forest, Adaptive Boosting, Gradient Boosting, XGBoost, Artificial neural network, Recurrent Neural Network, and Long short-term memory (LSTM).
result LSTM shows highest accuracy among all algorithms tested.
New method reduces communication in deep learning training.
problem Communication overhead in distributed deep learning training.
method Random-block sparsification to reduce gradients communicated.
result Performance close to standard SGD with reduced communication.
Detect anomalies in complex networks using topological subspace detectors.
problem Detect anomalies in complex networks defined by simplicial complexes.
method Formulate a hypothesis testing framework using Neyman-Pearson matched topological subspace detectors.
result Effective detection of anomalies in foreign currency exchange networks and other real-world data.
We unify subsampling methods for network embeddings and prove their asymptotic distribution.
problem Understanding and improving the performance of network embeddings learned via subsampling.
method Unified framework for node2vec-like methods, proving asymptotic distribution under exchangeable graph assumption.
result Asymptotic distribution of learned embedding vectors decouples and provides rates of convergence.
A networked learning method for correlated data outperforms federated learning in precision.
problem Estimating models from correlated data distributed across a network.
method Local linear model estimation with network regularization and information exchange.
result The weighted ensemble average estimate converges faster and more precisely than federated learning.
The exchange algorithm is studied for its convergence and asymptotic variance.
problem Theoretical limitations of the exchange algorithm in sampling from doubly-intractable distributions.
method Theoretical analysis of the exchange algorithm's convergence speed and asymptotic variance.
result The exchange algorithm converges at a geometric rate and satisfies a Central Limit Theorem.
This thesis tackles non-convex Bayesian learning via scalable dynamic importance sampling algorithms.
problem Non-convex Bayesian learning problem in deep neural networks.
method Replica exchange Langevin Monte Carlo, control variates method, population-chain replica exchange, scalable dynamic importance sampling.
result Control variates method reduces variance and accelerates convergence in non-convex Bayesian learning.
Study on pricing American Exchange options using Lévy processes.
problem Pricing American Exchange options driven by Lévy processes.
method Represented American Exchange options as European options plus early exercise premium; studied properties of free boundary and provided an approximative formula.
result Developed an approximative formula for American Exchange options.
As artificial intelligence systems spread to more diverse and larger tasks in many domains, the machine learning algorithms, and in particular the deep learning models and the databases required to train them are getting bigger themselves. Some algorithms do allow for some scaling of large computations by leveraging da…
Optimal crypto order execution using cross-exchange signals.
problem Maximizing order execution efficiency in cryptocurrency markets.
method Reinforcement learning applied to cross-exchange data.
result Cross-exchange signals improve optimal execution outcomes.
Adaptive conformal inference without data exchangeability assumptions.
problem Real-world scenarios often violate the data exchangeability assumption for conformal prediction.
method Parameter-free online convex optimization for adaptive conformal inference.
result Controls long-term miscoverage frequency at a nominal level empirically.
The thesis presents a new perspective on high-dimensional optimization.
problem The failure point of classical optimization methods in high dimensions.
method A distributional view of optimization, focusing on random objective functions and Bayesian Optimization.
result The distributional view explains predictable progress in high-dimensional optimization and provides insights into optimal step size control.
Study exchange option pricing with stochastic volatility and correlation.
problem Pricing exchange options under stochastic volatility and correlation.
method Approximation using a closed-form solution with Taylor expansion.
result Numerical results show the effectiveness of the proposed method.
New models reduce regional inequality by adjusting exchange range and asset distribution bias.
problem Reduction of regional inequality in economic systems.
method Proposed new asset exchange models with spatial exchange range and local support bias to adjust asset distribution and circulation rates.
result Achieved asset distribution from over-concentration to exponential and eventually normal, reducing Gini coefficient.
A known failing of many popular random graph models is that the Aldous-Hoover Theorem guarantees these graphs are dense with probability one; that is, the number of edges grows quadratically with the number of nodes. This behavior is considered unrealistic in observed graphs. We define a notion of edge exchangeability …
Develops a gradient-enhanced approach for online estimation in high-dimensional generalized linear models with streaming data.
problem Online estimation for high-dimensional generalized linear models with streaming data.
method Proposes a gradient-enhanced surrogate loss for non-distributed setting and extends to distributed streaming data.
result Derives non-asymptotic error bounds under high-dimensional scaling without batch-number constraint.
Improves sample efficiency in reinforcement learning with input representation.
problem Poor sample efficiency in reinforcement learning.
method Attention-based method to project inputs into an invariant representation space.
result Representation space is m! smaller for inputs of m objects, improving sample efficiency. This paper tackles collision avoidance for many UAVs using MFG and ML.
problem Collision avoidance for many UAVs in real-time missions.
method Mean-field game (MFG) theory combined with machine learning (ML) to reduce computation and communication energy.
result The proposed MFG learning control method achieves collision avoidance with low communication and acceptable computation energy.
New priors for deep neural networks converge to Gaussian processes.
problem Improving the performance and stability of deep neural networks.
method Extending prior distributions to include non-zero means and partially exchangeable priors, leading to a new Gaussian process model.
result The new Gaussian process model avoids pathologies and improves performance on regression problems.
How do individuals accumulate wealth as they interact economically? We outline the consequences of a simple microscopic model in which repeated pairwise exchanges of assets between individuals build the wealth distribution of a population. This distribution is determined for generic exchange rules --- transactions that…
The team predicts foreign exchange rates using clustering and attention models.
problem Complexity and unexpected events in foreign exchange markets.
method Clustering and attention models applied to historical data.
result Improved event-driven price prediction for oversold scenarios.
This work proposes ACTC for adaptive distributed learning under communication constraints.
problem Adaptive distributed learning in networks with communication constraints.
method ACTC (Adapt-Compress-Then-Combine) strategy with diffusion exchange of compressed updates.
result ACTC iterates converge to the optimizer with significant bit savings.
A dynamical model of capital exchange is introduced in which a specified amount of capital is exchanged between two individuals when they meet. The resulting time dependent wealth distributions are determined for a variety of exchange rules. For ``greedy'' exchange, an interaction between a rich and a poor individual r…
Study finds recurring patterns in cryptocurrency volatility and liquidity.
problem Recurring patterns in volatility and liquidity of major cryptocurrencies.
method Data from two centralized exchanges and a decentralized exchange analyzed for patterns.
result Systematic patterns in volatility and liquidity across different timeframes.
The article provides representations of exchange option prices under SVJD dynamics.
problem Modeling and pricing exchange options under stochastic volatility and jumps.
method Develops representations for European and American exchange options using SVJD dynamics and equivalent martingale measures.
result Derives integro-partial differential equations and representations for exchange option prices.
This paper introduces cluster exchange groupoids for Coxeter-Dynkin diagrams and finds their fundamental groups are braid groups.
problem Understanding the fundamental groups of cluster exchange groupoids for Coxeter-Dynkin diagrams.
method Introduced cluster exchange groupoids for Coxeter-Dynkin diagrams and showed the fundamental group isomorphic to braid groups.
result The fundamental group of the exchange groupoid for a Coxeter-Dynkin diagram is the braid group associated with the diagram.
Modeling exchange rates and options using entropic dynamics.
problem Modeling the dynamics of exchange rates and European options.
method Entropic Dynamics, entropic inference, scale invariance, logarithm of exchange rate.
result Derives the Geometric Brownian Motion and the Garman-Kohlhagen model for European options.
Study finds relevance of exchange and inflation rates to economic factors.
problem Determining the relevance of exchange and inflation rates to economic factors.
method Introduced concept of adequacy, established positive relation between exchange and inflation rates and other economic factors.
result Close positive relation found between exchange and inflation rates and other economic factors.
Machine learning models outperform traditional technical analysis in Bitcoin trading.
problem Maximizing profits in the Bitcoin market using trading signals.
method Comparison of machine learning models (LightGBM, LSTM) and technical analysis strategies (EMA, MACD+ADX).
result LSTM model achieved a 65.23% cumulative return over a year, significantly outperforming other strategies.