Study uses CNNs to upscale wind speed data from 100 km to 3 km, improving subgrid-scale variability.
problem Recovering fine-scale wind speed information from coarse data.
method Convolutional neural networks (CNNs) with different input configurations (coarse wind speed, fine-scale topography, diurnal cycle) were tested.
result CNN models with coarse wind and fine topography inputs perform best in generalizing to unseen regions.
DistPre predicts traffic speeds efficiently for large networks.
problem Fine-grained, accurate speed prediction for large-scale transportation networks.
method Customizes LSTM models on a cluster, sharing trained models between detectors.
result Efficient and accurate fine-grained traffic-speed prediction.
The paper categorizes four types of scale-up: smart, dumb, forced, and fumbled.
problem Growing ventures in size and maintaining efficiency.
method Identifying modularity and speed as key factors, categorizing four types of scale-up.
result Modularity and speed are crucial for successful scale-up.
New method improves traffic speed estimation from sparse data.
problem Incomplete and noisy traffic speed data from sparse sensors.
method Laplacian-enhanced low-rank tensor kriging (LETC) framework.
result LETC achieves state-of-the-art kriging performance under low observation rates.
HollowFlow speeds up likelihood evaluation for large-scale models.
problem Prohibitive scaling of sample likelihood computations in flow-based models.
method Introduces HollowFlow, a flow-based generative model using a NoBGNN with a block-diagonal Jacobian structure.
result Achieves up to O(n^2) speed-up in likelihood evaluation for large systems.
Scale of data and scale of computation infrastructures together enable the current deep learning renaissance. However, training large-scale deep architectures demands both algorithmic improvement and careful system configuration. In this paper, we focus on employing the system approach to speed up large-scale training.…
New limits found for training deep learning models efficiently.
problem Optimizing the training speed of deep learning models without sacrificing accuracy.
method Applied stochastic thermodynamics to set speed limits for neural network training.
result Training neural networks is optimal within certain scaling assumptions.
AcceleratedLiNGAM speeds up causal discovery methods for large datasets.
problem Slow causal discovery methods for large-scale datasets.
method Parallelized LiNGAM method with GPU acceleration.
result Up to 32-fold speed-up on benchmark datasets.
Accurately modeling traffic speeds is a fundamental part of efficient intelligent transportation systems. Nowadays, with the widespread deployment of GPS-enabled devices, it has become possible to crowdsource the collection of speed information to road users (e.g. through mobile applications or dedicated in-vehicle dev…
A new algorithm speeds up CP decomposition for large tensors.
problem Efficiently processing large-scale tensors in real-time.
method Randomized online CP decomposition (ROCP) algorithm.
result ROCP reduces computing time and memory usage significantly.
Extends hyperparameter transfer across model sizes and modules, improving training speed.
problem Training stability and performance of large-scale models with optimal hyperparameters.
method Complete(d) Parameterisation, per-module hyperparameter optimisation and transfer. result Hyperparameter transfer holds even in the per-module hyperparameter regime, improving training speed.
AdaScale SGD adapts learning rates for large-batch training efficiently.
problem Adapting learning rates for large-batch training to balance speed-ups and model quality.
method Adaptive learning rate adaptation based on gradient variance.
result AdaScale achieves reliable speed-ups for a wide range of batch sizes without degrading model quality.
New method speeds up training of large kernel models.
problem Scaling kernel machines to large datasets and model sizes.
method Delayed projections in Preconditioned Stochastic Gradient Descent (PSGD).
result Significant training speed up over existing methods.
Paper investigates separating times for general diffusions, providing new insights.
problem Understanding phase transitions between equivalence and singularity in diffusions.
method Representation of separating time as hitting time of a deterministic set, characterized by speed and scale.
result Explicit and easy-to-check conditions for absolute continuity and singularity of diffusions.
This paper proposes a convolutional neural network (CNN)-based method that learns traffic as images and predicts large-scale, network-wide traffic speed with a high accuracy. Spatiotemporal traffic dynamics are converted to images describing the time and space relations of traffic flow via a two-dimensional time-space …
Traffic speed prediction is a critically important component of intelligent transportation systems (ITS). Recently, with the rapid development of deep learning and transportation data science, a growing body of new traffic speed prediction models have been designed, which achieved high accuracy and large-scale predicti…
One of the difficulties of training deep neural networks is caused by improper scaling between layers. Scaling issues introduce exploding / gradient problems, and have typically been addressed by careful scale-preserving initialization. We investigate the value of preserving scale, or isometry, beyond the initial weigh…
In this paper, we propose the idea of radial scaling in frequency domain and activation functions with compact support to produce a multi-scale DNN (MscaleDNN), which will have the multi-scale capability in approximating high frequency and high dimensional functions and speeding up the solution of high dimensional PDEs…
The paper analyzes why Gaussianization slows down with higher dimensions and proposes a solution.
problem The convergence rate of Gaussianization slows down as the dimension increases.
method Analytical and empirical analysis of Gaussianization with random rotations.
result The number of required layers scales linearly with the dimension for Gaussian input.
Large-scale Hierarchical Classification (HC) involves datasets consisting of thousands of classes and millions of training instances with high-dimensional features posing several big data challenges. Feature selection that aims to select the subset of discriminant features is an effective strategy to deal with large-sc…
New method speeds up Gaussian process training and inference for large datasets.
problem Training and inference in Gaussian processes are computationally expensive for large datasets.
method Iterative alternating projection method that accesses subblocks of the kernel matrix, reducing time and space complexity.
result Empirically, the method accelerates GP training and inference by up to 72x compared to conjugate gradients.
Stochastic variational inference (SVI) employs stochastic optimization to scale up Bayesian computation to massive data. Since SVI is at its core a stochastic gradient-based algorithm, horizontal parallelism can be harnessed to allow larger scale inference. We propose a lock-free parallel implementation for SVI which a…
SIVF k-means algorithm speeds up sparse data clustering.
problem Efficiently clustering large-scale high-dimensional sparse data.
method Inverted-file structure for centroids, filter-based similarity reduction.
result SIVF achieves higher speed and lower memory consumption.
Wind energy resource quantification, air pollution monitoring, and weather forecasting all rely on rapid, accurate measurement of local wind conditions. Visual observations of the effects of wind---the swaying of trees and flapping of flags, for example---encode information regarding local wind conditions that can pote…
Study short-term wind power and speed predictions using machine learning.
problem Accurate short-term wind power and speed predictions for energy systems.
method Combining numerical weather prediction models with local observations, using machine learning for variable selection and forecasting.
result Improved wind power and speed predictions for 4-hour ahead using machine learning.
SCAFFLSA reduces communication complexity for federated learning with heterogeneous clients.
problem Quantifying and reducing communication complexity in federated learning with heterogeneous clients.
method Proposes SCAFFLSA, a variant of FedLSA using control variates to correct for client drift.
result SCAFFLSA achieves logarithmic communication complexity for statistically heterogeneous agents, scaling with the inverse of the desired accuracy.
Deep learning models can take weeks to train on a single GPU-equipped machine, necessitating scaling out DL training to a GPU-cluster. However, current distributed DL implementations can scale poorly due to substantial parameter synchronization over the network, because the high throughput of GPUs allows more data batc…
TabPFN-3 scales tabular prediction models to large datasets and improves performance and speed.
problem Scaling tabular prediction models to large datasets with high performance and efficiency.
method Pretrained on synthetic data, introduces test-time compute scaling, and uses row-chunking.
result Significantly outperforms other models on TabArena and diverse datasets.
A new method for reinforcement learning scales errors without tuning.
problem Error scaling varies across reinforcement learning tasks and stages.
method A simple scaling mechanism for temporal-difference learning.
result The method effectively mitigates interference between learning tasks.
Quantum computing speeds up Bermudan option pricing.
problem Efficient pricing of financial derivatives, especially Bermudan options.
method Quantum amplitude estimation combined with Chebyshev interpolation.
result Quadratic speed-up over classical methods.
Unconstrained MLIPs outperform constrained ones in accuracy and speed.
problem Improving the efficiency and accuracy of machine-learned interatomic potentials.
method Investigated unconstrained models trained on large datasets compared to physically constrained models.
result Unconstrained MLIPs can be superior in accuracy and speed compared to physically constrained models.
Study no-arbitrage conditions in 1D diffusion markets with interest rates.
problem Determining no-arbitrage conditions in 1D diffusion markets with interest rates.
method Established deterministic criteria for no-arbitrage notions in terms of scale function and speed measure.
result Revealed various effects, e.g., NIP not excluded by reflecting boundaries.
A new sampling method called Restart improves both speed and quality of generative processes.
problem Balancing speed and quality in generative processes involving differential equations.
method Alternates between adding noise and following ODE, improving both speed and quality.
result Surpasses previous SDE and ODE samplers in both speed and accuracy.
The implementation of artificial neural networks in hardware substrates is a major interdisciplinary enterprise. Well suited candidates for physical implementations must combine nonlinear neurons with dedicated and efficient hardware solutions for both connectivity and training. Reservoir computing addresses the proble…
Efficient SVD algorithm robust to outliers.
problem Outliers in data matrix affect SVD accuracy and speed.
method Spherically Normalized SVD (SpherSVD) algorithm.
result Significantly faster and more robust than existing methods.
The paper extends a Harnack inequality to noncompact evolving hypersurfaces.
problem Proving a Harnack inequality for noncompact evolving hypersurfaces.
method Using a differential Harnack inequality for noncompact convex hypersurfaces flowing with normal speed based on their principal curvatures.
result The extension of Andrews' result to noncompact hypersurfaces.
Unified model improves sampling speed and quality.
problem Difficult to balance sampling speed and quality.
method Multistep Consistency Models combining consistency and diffusion models.
result Improved sampling quality with reduced steps.
PPRGo uses approximate PageRank to speed up GNNs on large graphs.
problem Efficiently learning on large graphs using GNNs.
method Approximates PageRank for efficient information diffusion in GNNs.
result PPRGo outperforms other methods in speed and scalability.
Graph autoencoders (AE) and variational autoencoders (VAE) are powerful node embedding methods, but suffer from scalability issues. In this paper, we introduce FastGAE, a general framework to scale graph AE and VAE to large graphs with millions of nodes and edges. Our strategy, based on an effective stochastic subgraph…
New algorithm speeds up Bayesian UQ for high-dimensional inverse problems.
problem Computational inefficiency in Bayesian inference for high-dimensional inverse problems.
method Deep neural network-based autoencoder for dimension reduction and emulation phase.
result Computational efficiency up to three orders of magnitude with scalable Bayesian UQ.
A neural network speeds up computation of Wasserstein barycenters by 60x.
problem Computing Wasserstein barycenters is computationally demanding.
method Trained a deep convolutional neural network to compute Wasserstein barycenters.
result Computational times reduced from milliseconds to seconds.
New algorithm speeds up large-scale statistical inference.
problem Efficiently solving large-scale mean-field variational inference problems.
method Developed a novel primal-dual algorithm (PD-VI) and a block-preconditioned extension (P2D-VI) for mean-field variational inference. result PD-VI and P2D-VI achieve faster convergence and better solution quality compared to existing methods. This paper speeds up OCSSVM training using SMO.
problem Training One-Class Slab SVMs is slow.
method Uses updated SMO to divide large problems into smaller, analytically solvable subproblems.
result Training OCSSVMs scales better with large datasets.
Communication networks have evolved from specialized, research and tactical transmission systems to large-scale and highly complex interconnections of intelligent devices, increasingly becoming more commercial, consumer-oriented, and heterogeneous. Propelled by emergent social networking services and high-definition st…
We propose HAMSI (Hessian Approximated Multiple Subsets Iteration), which is a provably convergent, second order incremental algorithm for solving large-scale partially separable optimization problems. The algorithm is based on a local quadratic approximation, and hence, allows incorporating curvature information to sp…
Multirate training speeds up neural network fine-tuning.
problem Efficiently fine-tuning deep neural networks.
method Partitioning neural network parameters into fast and slow parts, updating slowly over longer intervals.
result Significant computational speed-up for transfer learning tasks.
We study empirical scaling laws for language model performance on the cross-entropy loss. The loss scales as a power-law with model size, dataset size, and the amount of compute used for training, with some trends spanning more than seven orders of magnitude. Other architectural details such as network width or depth h…
We derive scaling laws for optimizing neural networks in hardware.
problem Optimizing the large parameter space of neural networks in hardware.
method Analytical derivation of scaling laws for Coordinate Descent optimization.
result Convergence is exponential and scales linearly with the number of neurons.