This paper introduces intermittent learning - the goal of which is to enable energy harvested computing platforms capable of executing certain classes of machine learning tasks effectively and efficiently. We identify unique challenges to intermittent learning relating to the data and application semantics of machine l…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “data intermittency”
Intermittent demand, where demand occurrences appear sporadically in time, is a common and challenging problem in forecasting. In this paper, we first make the connections between renewal processes, and a collection of current models used for intermittent demand forecasting. We then develop a set of models that benefit…
Unified framework for intermittent demand forecasting using renewal processes.
problem Intermittency in demand forecasting.
method Unified framework based on extensions of discrete-time renewal processes.
result Efficacy demonstrated in forecasting practice with favorable predictive accuracy.
Maximum likelihood estimation applied to high-frequency data allows us to quantify intermittency in the fluctu- ations of asset prices. From time records as short as one month these methods permit extraction of a meaningful intermittency parameter λ characterising the degree of volatility clustering of asset prices. We…
The study uses Gaussian Processes with Tweedie likelihood for forecasting intermittent time series.
problem Forecasting intermittent time series with high accuracy and flexibility.
method The approach combines Gaussian Processes with two forecast distributions: negative binomial and Tweedie.
result TweedieGP provides better probabilistic forecasts, especially for high quantiles.
NeuralPrefix fills in missing sensor data without additional training.
problem Data intermittency in real-world sensing.
method NeuralPrefix is a task-agnostic, zero-shot imputation framework.
result NeuralPrefix accurately recovers missing samples and generalizes to unseen datasets.
Dynamical model of financial markets: fluctuating `temperature' causes intermittent behavior of price changescond-mat.stat-mech
We present a model of financial markets originally proposed for a turbulent flow, as a dynamic basis of its intermittent behavior. Time evolution of the price change is assumed to be described by Brownian motion in a power-law potential, where the `temperature' fluctuates slowly. The model generally yields a fat-tailed…
Distributed Non-Convex Optimization with Sublinear Speedup under Intermittent Client Availabilitystat.ML
Federated learning algorithm improves with intermittent client availability.
problem Performance degradation in Federated Averaging due to client availability changes.
method Federated Latest Averaging (FedLaAvg) uses latest gradients from all clients, even when unavailable.
result FedLaAvg achieves sublinear speedup compared to classical Federated Averaging.
Global models outperform local models in forecasting intermittent time series.
problem Forecasting intermittent time series with zeros in supply chains.
method Comparison of state-of-the-art probabilistic local and global models on five datasets.
result TiDE, a simple neural network architecture, achieves the best accuracy among global models.
A new model captures multifractal volatility in stock returns.
problem Capturing multifractal volatility in stock returns.
method Introduced mLog S-fBM model, defined mS-fBM, and developed calibration procedure.
result Model captures multifractal behavior in stock returns, validating on real data.
A new model captures multifractal volatility in stock returns.
problem Capturing multifractal volatility in stock returns.
method Introduced mLog S-fBM model, defined mS-fBM, and developed calibration procedure.
result Validated model on synthetic and real data, showing multifractal behavior.
A new Bayesian model improves forecasting for intermittent demand.
problem Sparse observations, cold-start items, and obsolescence in intermittent demand forecasting.
method Hierarchical Bayesian TSB model with partial pooling and calibrated probabilistic configuration.
result TSB-HB achieves the lowest RMSE and RMSSE on the UCI Online Retail dataset.
Paper addresses state estimation in sensor networks with intermittent data.
problem State estimation in sensor networks with packet dropouts and corrupted observations.
method Bayesian variational inference with a dual-mask generative model.
result The method effectively identifies system states and noise parameters.
Multiplicative random cascade model naturally reproduces the intermittency or multifractality, which is frequently shown among hierarchical complex systems such as turbulence and financial markets. As described herein, we investigate the validity of a multiplicative hierarchical random cascade model through an empirica…
A new metric optimizes forecasts for lumpy, intermittent demand.
problem Inaccurate demand forecasts lead to suboptimal logistics and production.
method Developed a novel metric that considers both statistical and business aspects.
result The new metric yields more accurate predictions for lumpy and intermittent demand.
Factorial moments are convenient tools in nuclear physics to characterize the multiplicity distributions when phase-space resolution () becomes small. For uncorrelated particle production within , Gaussian statistics holds and factorial moments are equal to unity for all orders . Correlations between par…
In this paper we study the price dynamics in a simple model of financial markets with heterogeneous agents. We concentrate on how increases in the total number of active traders influences fluctuations of asset prices. We find that a curious route to chaos is observed when the total number of [active traders] increases…
ACI uses Bayesian data assimilation to trace causes from effects in complex systems.
problem Capturing instantaneous, time-evolving causal relationships in complex, high-dimensional systems.
method Assimilative causal inference (ACI) leverages Bayesian data assimilation to trace causes backward from observed effects.
result ACI provides online tracking of causal roles that may reverse intermittently and reveals how far effects propagate.
We present a scalable and robust Bayesian inference method for linear state space models. The method is applied to demand forecasting in the context of a large e-commerce platform, paying special attention to intermittent and bursty target statistics. Inference is approximated by the Newton-Raphson algorithm, reduced t…
Paper shows local SGD outperforms mini-batch SGD under certain conditions.
problem Proving local SGD's superiority in distributed learning with heterogeneous data.
method New lower and upper bounds for local SGD under first-order heterogeneity assumptions.
result Local SGD is min-max optimal under certain conditions, resolving understanding of distributed optimization.
Sharp heat kernel estimates on manifolds lead to solutions of the Parabolic Anderson model.
problem Well-posedness and intermittency of solutions to the Parabolic Anderson model on Riemannian manifolds.
method Sharp global heat kernel bounds and geodesic comparison geometry.
result Upper and lower moment bounds for solutions of the Parabolic Anderson model on general compact Riemannian manifolds.
SurvSurf predicts first hitting times for intermittent events without monotonic violations.
problem Predicting first hitting times for intermittent events with monotonicity guarantees.
method Partially monotonic neural network for sequential events, incorporating unobserved events.
result SurvSurf outperforms existing models in MSE and IBS metrics.
Proposes a pricing agent using reinforcement learning to balance renewable energy demand.
problem Intermittent renewable energy sources challenge carbon-free electricity generation.
method Reinforcement learning approach to balance customer demand with renewable energy generation.
result Demonstrates improved electricity pricing strategy for renewable energy integration.
A new hierarchical forecasting method improves overall accuracy.
problem Hierarchical forecasting challenges, especially for intermittent time series.
method Top-down alignment of independent level forecasts using deep learning and tree-based algorithms.
result Improves overall forecasting accuracy compared to existing methods.
The Minority Game framework was recently generalized to account for the possibility that agents adapt not only through strategy selection but also by diversifying their response according to the kind of dynamical regime, or the risk, they perceive. Here we study the effects of this mechanism in different information st…
Successful implementation of California's Renewable Portfolio Standard (RPS) mandating 33 percent renewable energy generation by 2020 requires inclusion of a robust strategy to mitigate increased risk of energy deficits (blackouts) due to short time-scale (sub 1 hour) intermittencies in renewable energy sources. Of the…
We consider several estimation and learning problems that networked agents face when making decisions given their uncertainty about an unknown variable. Our methods are designed to efficiently deal with heterogeneity in both size and quality of the observed data, as well as heterogeneity over time (intermittence). The …
Non Poisson intermittent events in price formationphysics.data-an
The formation of price in a financial market is modelled as a chain of Ising spin with three fundamental figures of trading. We investigate the time behaviour of the model, and we compare the results with the real EURO/USD change rate. By using the test of local Poisson hypothesis, we show that this minimal model leads…
Studying the impact of climate change on precipitation is constrained by finding a way to evaluate the evolution of precipitation variability over time. Classical approaches (feature-based) have shown their limitations for this issue due to the intermittent and irregular nature of precipitation. In this study, we prese…
Paper optimizes FL communication efficiency with stochastic optimization.
problem Intermittent connectivity and non-i.i.d. data in FL.
method Convergence analysis of non-convex loss functions, stochastic optimization for client selection and power allocation.
result Significant reduction in communication time compared to random participation.
Data-driven ODE modeling of the high-frequency complex dynamics via a low-frequency dynamics modelnlin.CD
New method models complex dynamics using a base variable.
problem Modeling complex high-frequency dynamics from time series.
method Constructing a joint model with a base variable and a target variable.
result Successfully models chaotic behavior and reconstructs statistical properties.
Intermittency and Localizationphysics.soc-ph
In this paper, we show how simple logistic growth that was studied intensively during the last 200 years in many domains of science could be extended in a rather simple way and with these extensions is capable to produce a collection of behaviors widely observed in an enormous number of real-life systems in Economics, …
Federated Learning is a powerful machine learning paradigm to cooperatively train a global model with highly distributed data. A major bottleneck on the performance of distributed Stochastic Gradient Descent (SGD) algorithm for large-scale Federated Learning is the communication overhead on pushing local gradients and …
Scaling analysis of multivariate intermittent time seriescond-mat.stat-mech
The scaling properties of the time series of asset prices and trading volumes of stock markets are analysed. It is shown that similarly to the asset prices, the trading volume data obey multi-scaling length-distribution of low-variability periods. In the case of asset prices, such scaling behaviour can be used for risk…
NN-Turb generates turbulent velocity statistics using neural networks.
problem Creating a 1D field with turbulent velocity statistics.
method Fully-convolutional neural network (NN-Turb) to generate the field.
result NN-Turb generates a 1D field that satisfies Kolmogorov's 2/3 and 4/5 laws, exhibiting intermittency.
The sharp peak-flat trough pattern and critical speculationcond-mat.stat-mech
We find empirically a characteristic sharp peak-flat trough pattern in a large set of commodity prices. We argue that the sharp peak structure reflects an endogenous inter-market organization, and that peaks may be seen as local ``singularities'' resulting from imitation and herding. These findings impose a novel strin…
We consider the problem of recovering an expert's reward function with inverse reinforcement learning (IRL) when there are missing/incomplete state-action pairs or observations in the demonstrated trajectories. This issue of missing trajectory data or information occurs in many situations, e.g., GPS signals from vehicl…
Minimal Agent Based Model for Financial Markets I: Origin and Self-Organization of Stylized Factsq-fin.TR
We introduce a minimal Agent Based Model for financial markets to understand the nature and Self-Organization of the Stylized Facts. The model is minimal in the sense that we try to identify the essential ingredients to reproduce the main most important deviations of price time series from a Random Walk behavior. We fo…
We suggest a general oracle-based framework that captures different parallel stochastic optimization settings described by a dependency graph, and derive generic lower bounds in terms of this graph. We then use the framework and derive lower bounds for several specific parallel optimization settings, including delayed …
This study is a detailed analysis of Speculation Game, a minimal agent-based model of financial markets, in which the round-trip trading and the dynamic wealth evolution with variable trading volumes are implemented. Instead of herding behavior, we find that the emergence of volatility clustering can be induced by the …
We perform an extensive empirical analysis of scaling properties of equity returns, suggesting that financial data show time varying multifractal properties. This is obtained by comparing empirical observations of the weighted generalised Hurst exponent (wGHE) with time series simulated via Multifractal Random Walk (MR…
We show that the statistics of spreads in real order books is characterized by an intrinsic asymmetry due to discreteness effects for even or odd values of the spread. An analysis of data from the NYSE order book points out that traders' strategies contribute to this asymmetry. We also investigate this phenomenon in th…
Study on collaborative vs. non-collaborative online and bandit convex optimization.
problem Minimizing average regret in distributed online and bandit convex optimization.
method Analyzes the impact of collaboration in adaptive and zeroth-order feedback settings.
result Collaboration is beneficial in high-dimensional federated online optimization with limited feedback.
The Stock Market is a complex self-interacting system, characterized by an intermittent behaviour. Periods of high activity alternate with periods of relative calm. In the present work we investigate empirically about the possibility that the market is in a self-organized critical state (SOC). A wavelet transform metho…
We study a phenomenological model for the continuous double auction, equivalent to two independent queues. The continuous double auction defines a continuous-time random walk for trade prices. The conditions for ergodicity of the auction are derived and, as a consequence, three possible regimes in the behavior …
A new asynchronous method for vertical federated learning improves privacy and efficiency.
problem Solving vertical federated learning in an asynchronous manner with privacy and efficiency.
method A simple FL method that allows clients to run stochastic gradient algorithms asynchronously with a new perturbed local embedding technique.
result The method improves privacy and communication efficiency compared to centralized and synchronous FL methods.
Distribution grids are currently challenged by frequent voltage excursions induced by intermittent solar generation. Smart inverters have been advocated as a fast-responding means to regulate voltage and minimize ohmic losses. Since optimal inverter coordination may be computationally challenging and preset local contr…
We have presented a novel technique of detecting intermittencies in a financial time series of the foreign exchange rate data of U.S.- Euro dollar(US/EUR) using a combination of both statistical and spectral techniques. This has been possible due to Continuous Wavelet Transform (CWT) analysis which has been popularly a…