A blockchain-based federated learning system with latency analysis.
problem Latency issues in decentralized federated learning architectures.
method Introduced a consortium blockchain and a latency model to analyze the workflow.
result The latency model accurately quantifies actual delays in the system.
CryptoNAS improves PI accuracy by 3.4% with 2.4x less latency.
problem Private inference on machine learning models with limited latency.
method Developed CryptoNAS, a novel NAS method for finding models that maximize accuracy within a ReLU budget.
result Improves accuracy by 3.4% and latency by 2.4x over state-of-the-art methods.
PHAZE framework uses zkML and hashing for fast, verifiable LHC trigger decisions.
problem Inefficient inference on large machine learning models for LHC trigger performance.
method Cryptographic techniques like hashing and zkML for low latency, certifiable inference.
result Achieves nanosecond-order latency for LHC triggers, enabling dynamic low-level triggers.
This paper studies optimal market making for large-tick assets in the presence of latency. We consider a random walk model for the asset price, and formulate the market maker's optimization problem using Markov Decision Processes (MDP). We characterize the value of an order and show that it plays the role of one-period…
Automated tool reduces FPGA inference latency to 5 μs for deep neural networks.
problem Deploying ultra low-latency, low-power deep neural networks on FPGAs.
method Extending hls4ml library, using model compression techniques like pruning and quantization-aware training.
result Achieved inference latency of 5 μs with 97% resource reduction.
TinyML models detect RF and cyber threats in spacecraft with low latency.
problem Detecting cyber-RF threats in autonomous spacecraft with low latency.
method Analysis of classical models (RF, LR, SVM, MLP) for latency-accuracy trade-offs.
result Logistic Regression achieves microsecond-level inference with minimal accuracy loss.
Optimizes latency and false alarm probability in change detection problems.
problem Balancing latency and false alarms in non-stationary environments.
method Develops order-optimal change detectors under specified latency and false alarm levels.
result Derives a universal lower bound on latency and develops order-optimal detectors.
The possibility of latency arbitrage in financial markets has led to the deployment of high-speed communication links between distant financial centers. These links are noisy and so there is a need for coding. In this paper, we develop a gametheoretic model of trading behavior where two traders compete to capture laten…
Study validates low latency's impact on trading profits.
problem Determining the impact of low latency on trading profits.
method Agent-based simulation of trading strategies in a controlled environment.
result Latency inversely affects trading profits; latency rank is key.
Research examines how strategic latency manipulation impacts Ethereum's network efficiency and decentralization.
problem Impact of artificial latency on Ethereum's network efficiency and decentralization.
method Comprehensive analysis of MEV-Boost auction system and empirical validation with a pilot.
result Increased profitability for node operators and significant systemic challenges like heightened network inefficiencies and centralization risks.
This paper proposes an EM approach to reduce inference latency in NAR sequence generation.
problem High inference latency in NAR models due to multi-modality in sequence generation.
method A unified EM framework that jointly optimizes AR and NAR models, with iterative refinement.
result The proposed approach achieves competitive performance with existing NAR models and significantly reduces inference latency.
The Tick library simulates and learns Hawkes processes with latency effects.
problem Correctly modeling causality in order book events with latency.
method Exponential kernels shifted by latency, derived log-likelihood expressions.
result Latency determines most decays in real data, showing decay relationships.
This study analyzes satellite communication latency using a stochastic geometry model.
problem Latency analysis of LEO satellite relay communication systems.
method Stochastic geometry framework with spherical BPP models, suboptimal satellite relay selection strategy.
result Derives distance distributions and analytical expressions for transmission delays.
Research optimizes C++ patterns for HFT, reducing latency and improving profitability.
problem Optimizing latency-critical code for high-frequency trading systems.
method Creation of a Low-Latency Programming Repository, optimisation of trading strategy, implementation of Disruptor pattern.
result Significant performance improvements in speed and profitability.
In this paper, a novel experienced deep reinforcement learning (deep-RL) framework is proposed to provide model-free resource allocation for ultra reliable low latency communication (URLLC). The proposed, experienced deep-RL framework can guarantee high end-to-end reliability and low end-to-end latency, under explicit …
MASnet enhances speech on mobile devices with low latency.
problem Efficiently enhancing speech on mobile devices with low latency.
method MASnet processes linear-scale spectrograms, using ratio masks to enhance noisy frames, and operates in low-latency incremental inference mode.
result MASnet achieves efficient speech enhancement with low latency, reducing FMA/s operations.
Improves text-to-speech speed by interleaving character reading and audio synthesis.
problem Latency in text-to-speech models limits their use in time-sensitive tasks.
method Reinforcement learning to train an agent to choose the order of character reading and audio synthesis.
result The proposed method successfully balances latency and audio quality.
HOLMES improves real-time model serving for ICU patients, balancing accuracy and speed.
problem Real-time model serving is crucial in ICU due to urgency and cost.
method Online model ensemble serving framework for healthcare applications.
result HOLMES achieves high accuracy and sub-second latency for model ensemble serving.
MLaaS (ML-as-a-Service) offerings by cloud computing platforms are becoming increasingly popular. Hosting pre-trained machine learning models in the cloud enables elastic scalability as the demand grows. But providing low latency and reducing the latency variance is a key requirement. Variance is harder to control in a…
Paper presents FPGA implementation for efficient recurrent neural networks.
problem Implementing recurrent neural networks on FPGAs for low latency.
method Developed hls4ml framework to implement LSTM and GRU layers.
result Demonstrated effective designs for both small and large models.
A DRL approach optimizes resource allocation in BFL to reduce latency and energy consumption.
problem Energy and CPU constraints of mobile devices and increased training latency due to blockchain mining.
method Deep Reinforcement Learning (DRL) to derive optimal decisions for MLMO.
result Optimal resource allocation and block generation rate to minimize system latency, energy consumption, and incentive cost.
Reducing the latency variance in machine learning inference is a key requirement in many applications. Variance is harder to control in a cloud deployment in the presence of stragglers. In spite of this challenge, inference is increasingly being done in the cloud, due to the advent of affordable machine learning as a s…
Paper analyzes how latency affects optimal order execution in markets.
problem Balancing profit from optimal order execution against execution risk.
method Stochastic optimal control problem with risk-averse trader.
result Closed-form approximations of fill probability, limit price, and latency.
Study replicates market model, finds replication hindered by missing details.
problem Replicating a market model with missing details and limited quantitative reporting.
method Increased simulation runs, bootstrap confidence intervals, and code analysis.
result Achieved relational equivalence for most metrics but rejected quantitative alignment.
New attacks exploit neural network energy and latency, increasing costs by 10-200x.
problem Energy and latency vulnerabilities in neural networks.
method Crafted sponge examples to maximize energy and latency.
result Adversaries can increase energy consumption by 10-200x and delay decisions.
Power-SMC reduces inference latency for training-free LLM reasoning.
problem Training-free LLM reasoning with low latency.
method Power-SMC, a training-free Sequential Monte Carlo scheme targeting sequence-level power distribution.
result Power-SMC reduces inference latency from 16-28× to 1.4-3.3× over baseline decoding.
We resolve the fundamental problem of online decoding with general nth order ergodic Markov chain models. Specifically, we provide deterministic and randomized algorithms whose performance is close to that of the optimal offline algorithm even when latency is small. Our algorithms admit efficient implementation vi…
Latency (i.e., time delay) in electronic markets affects the efficacy of liquidity taking strategies. During the time liquidity takers process information and send marketable limit orders (MLOs) to the exchange, the limit order book (LOB) might undergo updates, so there is no guarantee that MLOs are filled. We develop …
LITE models improve query-document relevance with learnable late interactions.
problem Improving query-document relevance with lower latency and storage.
method Proposes learnable late-interaction models (LITE) that use factorized query and document embeddings followed by a learnable scorer.
result Empirically, LITE outperforms previous late-interaction models in re-ranking tasks.
FlashIV solves Black-Scholes implied volatility efficiently and accurately.
problem Efficiently calculating implied volatility for financial models.
method Normalizes inputs, uses asymptotic seed, and combines Householder refinement.
result Runs faster than existing methods while maintaining accuracy.
A new algorithm for cryo-EM data collection that balances reward and latency.
problem Optimizing data collection in cryo-EM experiments with action delays.
method Latency-aware contextual bandit framework and COAF algorithm.
result The COAF algorithm efficiently maximizes reward over time in cryo-EM experiments.
Interactive applications incorporating high-data rate sensing and computer vision are becoming possible due to novel runtime systems and the use of parallel computation resources. To allow interactive use, such applications require careful tuning of multiple application parameters to meet required fidelity and latency …
High frequency trading has led to widespread efforts to reduce information propagation delays between physically distant exchanges. Using relativistically correct millisecond-resolution tick data, we document a 3-millisecond decrease in one-way communication time between the Chicago and New York areas that has occurred…
PolySwarm uses a swarm of LLMs to predict and arbitrage prediction markets.
problem Real-time prediction market trading and latency arbitrage inefficiencies.
method PolySwarm employs a swarm of 50 diverse LLMs, Bayesian combination, and risk-controlled execution.
result Swarm aggregation outperforms single-model baselines in prediction tasks.
Kolmogorov-Arnold Networks enable ultrafast online learning with fixed-point quantization.
problem Efficient online learning for high-frequency systems with strict memory constraints.
method Fixed-point online training on FPGAs exploiting B-spline locality in KANs.
result Kolmogorov-Arnold Networks are more efficient and expressive than MLPs for low-latency tasks.
S3NAS finds high-accuracy CNN architectures for NPUs in 3 hours.
problem Finding optimal CNN architectures for NPUs with high accuracy and low latency.
method Supernet design, Single-Path NAS, scaling, analytical latency estimation.
result 82.72% top-1 accuracy on ImageNet with 11.66 ms latency.
LaRT models LLMs' response accuracy and CoT length to evaluate reasoning ability and speed.
problem Valid evaluation of Large Language Models (LLMs) via response accuracy and chain-of-thought length.
method Introduces Latency-Response Theory (LaRT) to jointly model response accuracy and CoT length using latent ability and latent speed.
result LaRT yields higher estimation accuracy and shorter confidence intervals for latent traits compared to IRT.
This paper tackles URLLC in 6G networks with deep learning.
problem Stringent requirements on end-to-end delay and reliability for mission-critical applications.
method Develops a multi-level architecture combining theoretical models and real-world data, using deep transfer learning and federated learning.
result Demonstrates improved performance in URLLC for mission-critical applications.
DCT-SNN uses DCT to reduce inference latency in SNNs.
problem High inference latency in SNNs.
method Proposes a time-based encoding scheme using DCT to reduce timesteps.
result Achieves top-1 accuracy comparable to standard deep learning while reducing inference latency.
DIET-SNN optimizes SNNs for faster, lower-energy image classification.
problem High inference latency and inefficient input encoding in SNNs.
method End-to-end backpropagation to optimize membrane leak and firing threshold.
result Achieves top-1 accuracy of 69% on ImageNet with 5 timesteps and 12x less compute energy.
CoinTossX is a low-latency, open-source matching engine for financial trading.
problem Efficiently matching orders in financial markets with low latency and high throughput.
method Developed in Java, orders submitted via UDP SBE, low-latency message transport (Aeron Media Driver). Separates order generation and matching.
result Demonstrated low-latency, high-throughput performance in various deployment scenarios.
A power-law fit to the empirical inference-compute frontier in LOB prediction suggests a scaling-law-style frontier.
problem Limit order book prediction
method Using a suite of models ranging from small decision trees to neural LOB architectures
result A power-law fit to the low- and mid-compute non-MLPLOB frontier extrapolates across multiple orders of magnitude and attains R2=0.941 on the excluded high-compute MLPLOB target frontier. As Deep Learning (DL) models have been increasingly used in latency-sensitive applications, there has been a growing interest in improving their response time. An important venue for such improvement is to profile the execution of these models and characterize their performance to identify possible optimization opportu…
PolyLUT uses polynomials to reduce FPGA latency.
problem Reducing latency in FPGA-based neural network inference.
method Training neural networks using multivariate polynomials as basic building blocks.
result Achieved significant latency and area improvements.
We present a systematic analysis on the performance of a phonetic recogniser when the window of input features is not symmetric with respect to the current frame. The recogniser is based on Context Dependent Deep Neural Networks (CD-DNNs) and Hidden Markov Models (HMMs). The objective is to reduce the latency of the sy…
We develop a supervised machine learning model that detects anomalies in systems in real time. Our model processes unbounded streams of data into time series which then form the basis of a low-latency anomaly detection model. Moreover, we extend our preliminary goal of just anomaly detection to simultaneous anomaly pre…
Density-Softmax improves uncertainty estimation and robustness without sampling, reducing model size and latency.
problem Sampling-based uncertainty estimation methods suffer from large model size and high latency.
method Combines a Lipschitz-constrained feature extractor with the softmax layer to create a sampling-free deterministic framework.
result Density-Softmax reduces over-confidence under distribution shifts and achieves competitive results in uncertainty and robustness.
Select-DC reduces GFLOPS for uncertainty estimation in neural networks.
problem Computational inefficiency in estimating model uncertainty for low-latency applications.
method Select-DC uses a subset of layers to model epistemic uncertainty with MCDC, reducing GFLOPS.
result Significant reduction in GFLOPS required for uncertainty estimation with marginal performance loss.