Accelerates data loading in deep neural network training by 30x.
problem Data loading is a bottleneck in deep neural network training.
method Locality-aware data loading method using software caches.
result More than 30x speedup in data loading.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “data loading”
Paper proposes a new method for hourly load forecasting using smart meter data.
problem Challenges in short-term load forecasting at fine granularity.
method Forecasting using Matrix Factorization (fmf) for hourly load forecasting.
result Significantly outperforms state-of-the-art methods in load forecasting.
This paper monetizes customer load data to boost energy retailer profits.
problem Improving load forecasts to reduce energy imbalance costs.
method Cooperative game theory approach to quantify and distribute profits.
result Retailer gains significant profit from customer load data.
HIV RNA viral load (VL) is an important outcome variable in studies of HIV infected persons. There exists only a handful of methods which classify patients by viral load patterns. Most methods place limits on the use of viral load measurements, are often specific to a particular study design, and do not account for com…
Study finds cherry-picking load shaping strategies outperforms others in reducing grid CO2 emissions.
problem Lack of detailed counterfactual data makes it hard to assess load shaping strategies' effectiveness.
method Calibrated granular ERCOT simulations for counterfactual analysis of load shaping strategies.
result LMP-based load shaping outperforms other strategies in reducing grid CO2 emissions.
Develops a method to analyze SARS-CoV-2 viral load vs. age, finding a significant increase.
problem Determining if SARS-CoV-2 viral load depends on patient age.
method Flexible, non-parametric, hierarchical, Bayesian, causal model.
result Statistically significant increase in viral load with age, but only for one dataset.
Simple 1D-CNN network predicts electricity loads 36 hours ahead.
problem Forecasting electricity loads for future time periods.
method Used a one-dimensional CNN with parameter scanning to optimize kernel size, filters, and dense size.
result Good forecast quality achieved with basic CNN architectures.
Machine learning reconstructs aerodynamic forces from noisy data.
problem Accurately modeling aerodynamic forces with limited or noisy data.
method Physics-informed Gaussian processes trained on noisy structural responses.
result Strong agreement between true and predicted aerodynamic loads.
Deep learning boosts building energy load forecasting.
problem Short-term load forecasting in buildings.
method Stacked Boosters Network architecture with sparse interactions, parameter sharing, and equivariant representations.
result Outperforms state-of-the-art models in short-term load forecasting tasks.
Proposes ML-RBM for non-intrusive load monitoring without appliance-level data.
problem Non-intrusive load monitoring without appliance-level data.
method Multi-label Restricted Boltzmann Machine (ML-RBM)
result Experimental evaluation of proposed and state-of-the-art techniques.
Short-term load forecasting (STLF) is essential for the reliable and economic operation of power systems. Though many STLF methods were proposed over the past decades, most of them focused on loads at high aggregation levels only. Thus, low-aggregation load forecast still requires further research and development. Comp…
Forecasting Short-term load using Econometrics time series model with T-student Distributionq-fin.ST
Paper uses econometrics time series model with T-student Distribution for short-term load forecasting.
problem Accurate short-term load forecasting for optimizing electrical sources and protecting energy.
method Uses SARIMA-GARCH model with T-student Distribution to forecast electric load.
result The proposed model outperforms the ARIMA model with Normal Distribution.
Active learning reduces smart meter data needs for better electric load predictions.
problem Inaccurate and costly electric load predictions due to insufficient data.
method Active learning to collect more informative data subsets.
result Electric load predictions can be made with about half the data using active learning.
New method predicts heat load in thermal grids using latent variables.
problem Predicting heat load in district energy systems.
method Combines nominal model for outdoor temperature with latent variable model for residual heat load.
result Proposed method achieves better prediction accuracy than artificial neural networks.
Several statistical and machine learning methods are proposed to estimate the type and intensity of physical load and accumulated fatigue . They are based on the statistical analysis of accumulated and moving window data subsets with construction of a kurtosis-skewness diagram. This approach was applied to the data gat…
Traditional load analysis is facing challenges with the new electricity usage patterns due to demand response as well as increasing deployment of distributed generations, including photovoltaics (PV), electric vehicles (EV), and energy storage systems (ESS). At the transmission system, despite of irregular load behavio…
Paper proposes dense average network for improved power load forecasting.
problem Improving power load forecasting accuracy to save millions for the power industry.
method Introduces dense average connection and constructs dense average network for power load forecasting.
result Proposed model outperforms existing methods on public datasets.
Model uses GAMs to forecast hourly electricity load weeks to one year ahead.
problem Accurate mid-term hourly load forecasting for power plant operation and energy management.
method Generalized Additive Models (GAMs) with P-splines and autoregressive post-processing.
result Significantly enhanced forecasting accuracy compared to state-of-the-art methods.
EGPR method forecasts power grid load and generation with high accuracy.
problem Accurate week-long forecasting of load demand and power generation for efficient power grid operation.
method Gaussian process regression with ensembles of preceding weeks' data.
result EGPR method outperforms traditional methods in forecasting weekly load and generation.
To investigate whether training load monitoring data could be used to predict injuries in elite Australian football players, data were collected from elite athletes over 3 seasons at an Australian football club. Loads were quantified using GPS devices, accelerometers and player perceived exertion ratings. Absolute and …
Deep learning improves weather modeling for electricity load forecasting.
problem Accurate load and renewable energy forecasting requires complex spatio-temporal weather modeling.
method Automated spatio-temporal feature extraction using deep neural networks.
result Deep learning outperforms traditional methods in French national load forecasting.
Adaptive probabilistic load forecasting improves performance in power systems.
problem Complexity of electricity load forecasting due to changing drivers and local generation.
method Adaptive probabilistic approach using Kalman filter and online gradient descent.
result Adaptive probabilistic forecasts improve performance in both point and probabilistic forecasting.
Short-term load forecasting is a critical element of power systems energy management systems. In recent years, probabilistic load forecasting (PLF) has gained increased attention for its ability to provide uncertainty information that helps to improve the reliability and economics of system operation performances. This…
Enhances load forecasting for multiple entities with dynamic similarities.
problem Inaccurate probabilistic load predictions due to uncertainties and dynamic changes.
method Online multi-task learning for probabilistic load forecasting.
result Significantly enhances load forecasting accuracy across various scenarios.
Many post-disaster and -conflict regions do not have sufficient data on their transportation infrastructure assets, hindering both mobility and reconstruction. In particular, as the number of aging and deteriorating bridges increase, it is necessary to quantify their load characteristics in order to inform maintenance …
In this paper, we propose and study opportunistic bandits - a new variant of bandits where the regret of pulling a suboptimal arm varies under different environmental conditions, such as network load or produce price. When the load/price is low, so is the cost/regret of pulling a suboptimal arm (e.g., trying a suboptim…
Power load forecast with Machine Learning is a fairly mature application of artificial intelligence and it is indispensable in operation, control and planning. Data selection techniqies have been hardly used in this application. However, the use of such techniques could be beneficial provided the assumption that the da…
Novel method improves load estimation in power grids using anomaly and change point detection.
problem Improving load estimation in power grid systems.
method Combining unsupervised anomaly and change point detection methods for automatic filtering.
result Automatic load estimation is accurate with 90% estimates within a 10% error margin.
Improved non-intrusive load monitoring with a novel neural network.
problem Accurately disaggregating household electricity consumption without dedicated meters.
method Developed a scale- and context-aware network with multi-scale features and contextual information.
result Significantly improved accuracy compared to state-of-the-art methods.
b-LOAD extends local causal discovery with prior knowledge, improving causal effect estimation.
problem Local causal discovery struggles in data-scarce settings due to uncertainty and incomplete neighborhoods.
method b-LOAD incorporates prior knowledge directly into local structure learning, using Meek's rules to refine discovery.
result b-LOAD refines the admissible equivalence class and enlarges identifiable causal queries, improving causal effect estimation.
In (exploratory) factor analysis, the loading matrix is identified only up to orthogonal rotation. For identifiability, one thus often takes the loading matrix to be lower triangular with positive diagonal entries. In Bayesian inference, a standard practice is then to specify a prior under which the loadings are indepe…
New methods improve electricity load forecasting using hierarchical transfer learning.
problem Improving electricity load forecasts at national scale using smart meter data.
method Developed two hierarchical transfer learning methods based on stacking and aggregation of experts.
result Significant improvement in predictions compared to benchmark algorithms.
A new framework uses DDQN to simplify WECC CLM for efficient load modeling.
problem Complexity and high parameter count in WECC CLM.
method Two-stage approach with DDQN for load composition and parameter selection.
result The framework efficiently approximates WECC CLM transient responses.
Paper benchmarks and customizes energy forecasting methods.
problem Energy forecasting challenges and differences from traditional time series.
method Collected large-scale load datasets and renewable energy datasets. Developed feature engineering and customized loss functions.
result Comprehensive evaluation of 21 forecasting methods in energy datasets.
This two-part work puts forth the idea of engaging power electronics to probe an electric grid to infer non-metered loads. Probing can be accomplished by commanding inverters to perturb their power injections and record the induced voltage response. Once a probing setup is deemed topologically observable by the tests o…
High-Resolution Peak Demand Estimation Using Generalized Additive Models and Deep Neural Networkscs.LG
Paper proposes combining GAM and DNN for accurate peak demand estimation from lower-resolution data.
problem Predicting high-resolution peak demand from limited lower-resolution data.
method Combines generalized additive models (GAM) and deep neural networks (DNN) for half-hourly load forecasting.
result Proposed method reduces out-of-sample RMSE by 57.4% compared to benchmark.
We present a simple quantile regression-based forecasting method that was applied in a probabilistic load forecasting framework of the Global Energy Forecasting Competition 2017 (GEFCom2017). The hourly load data is log transformed and split into a long-term trend component and a remainder term. The key forecasting ele…
Viral Load Inference in Non-Adaptive Pooled Testingcond-mat.stat-mech
New method infers viral load from pooled tests.
problem Inefficient viral load inference in pooled testing.
method Message passing algorithm with PCR noise function.
result Accurate viral load inference possible.
Paper assesses the market value of sharing privacy-protected smart meter data.
problem Value of sharing privacy-protected smart meter data between consumers and load serving entities.
method Discounted differential privacy model, ANN-based load forecasting, optimal procurement problem.
result Significant value in sharing smart meter data while retaining individual consumer privacy.
We use online convex optimization (OCO) for setpoint tracking with uncertain, flexible loads. We consider full feedback from the loads, bandit feedback, and two intermediate types of feedback: partial bandit where a subset of the loads are individually observed and the rest are observed in aggregate, and Bernoulli feed…
This paper uses a diffusion model to forecast electrical loads with uncertainty.
problem Uncertainties in electrical load forecasting due to renewable energy and external events.
method Diffusion-based Seq2Seq structure for epistemic uncertainty and robust additive Cauchy distribution for aleatoric uncertainty.
result Ability to separate and quantify both types of uncertainties in load forecasting.
Deep Learning is applied to energy markets to predict extreme loads observed in energy grids. Forecasting energy loads and prices is challenging due to sharp peaks and troughs that arise due to supply and demand fluctuations from intraday system constraints. We propose deep spatio-temporal models and extreme value theo…
Paper presents a method for probabilistic load forecasting using adaptive online learning.
problem Inability to assess intrinsic uncertainties and capture dynamic changes in consumption patterns.
method Adaptive online learning of hidden Markov models for recursive parameter updates and sequential prediction.
result Significant improvement in performance compared to existing techniques across various scenarios.
An accurate load forecasting has always been one of the main indispensable parts in the operation and planning of power systems. Among different time horizons of forecasting, while short-term load forecasting (STLF) and long-term load forecasting (LTLF) have respectively got benefits of accurate predictors and probabil…
New method relaxes PCA orthogonality constraints using explained variance of correlated components.
problem Difficulty in using PCA for sparse design due to orthogonality constraints and non-differentiable penalty.
method Introduce expvar(Y) to measure variance explained by correlated components, relax orthogonality constraints.
result Two expvar(Y) definitions suitable for block PCA formulations without orthogonality constraints.
New LSTM model predicts disaggregated electricity loads accurately.
problem Forecasting disaggregated electricity loads from smart meters.
method Single complex LSTM model capturing individual consumption patterns.
result Model accurately predicts future loads of new consumers.
Log-supermodularity of weight functions and the loading monotonicity of weighted insurance premiumsq-fin.RM
The paper is motivated by a problem concerning the monotonicity of insurance premiums with respect to their loading parameter: the larger the parameter, the larger the insurance premium is expected to be. This property, usually called loading monotonicity, is satisfied by premiums that appear in the literature. The inc…
Study compares forecasting methods for distribution grid loads.
problem Improving load prediction accuracy in distribution grids.
method Evaluation of probabilistic forecasting techniques on distribution grid load prediction.
result Hierarchical techniques enhance bottom-level forecast accuracy.