The event-driven and elastic nature of serverless runtimes makes them a very efficient and cost-effective alternative for scaling up computations. So far, they have mostly been used for stateless, data parallel and ephemeral computations. In this work, we propose using serverless runtimes to solve generic, large-scale …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “runtime scaling”
Vecchia approximations provide the best accuracy-runtime trade-off for Gaussian process approximations.
problem High computational cost of Gaussian processes for large data sets.
method Systematic comparison of different Gaussian process approximations.
result Vecchia approximations consistently provide the best accuracy-runtime trade-off.
Scorio.jl ranks systems from repeated tasks using various methods.
problem Evaluating and ranking systems from repeated responses to shared tasks.
method Common tensor-based interface for multiple ranking methods.
result Pilot experiments show stability and runtime scaling.
Improves Gaussian process factor models for multi-population recordings.
problem Cubic runtime scaling with trial length and group number limits application to large-scale recordings.
method Two approximate approaches: inducing variables and frequency domain.
result Achieved orders of magnitude speed-up with minimal statistical performance impact.
We introduce "TriMap"; a dimensionality reduction technique based on triplet constraints, which preserves the global structure of the data better than the other commonly used methods such as t-SNE, LargeVis, and UMAP. To quantify the global accuracy of the embedding, we introduce a score that roughly reflects the relat…
Stochastic momentum methods trade compute efficiency for serial runtime.
problem Stochastic momentum methods trade compute efficiency for serial runtime.
method Stochastic HB and ASGD for consistent linear regression with Gaussian covariates.
result HB preserves SGD-level CE over a larger batch-size window, allowing larger batches to reduce serial runtime until HB reaches its deterministic accelerated scale.
New model scales MHPs for analyzing large-scale diffusion processes.
problem Complex temporal dependencies in MHPs make them hard to scale.
method Exploits sparsity in diffusion processes to compute MHP likelihood and gradients efficiently.
result Improves runtime performance by multiple orders of magnitude on sparse event sequences.
This paper presents Rudra, a parameter server based distributed computing framework tuned for training large-scale deep neural networks. Using variants of the asynchronous stochastic gradient descent algorithm we study the impact of synchronization protocol, stale gradient updates, minibatch size, learning rates, and n…
We present a structural clustering algorithm for large-scale datasets of small labeled graphs, utilizing a frequent subgraph sampling strategy. A set of representatives provides an intuitive description of each cluster, supports the clustering process, and helps to interpret the clustering results. The projection-based…
Improved kernel Stein discrepancy for large-scale data.
problem Efficiently testing probability distributions with kernel methods.
method Nyström approximation to reduce runtime complexity.
result Nyström-based KSD is -consistent and applicable for large datasets.
A universal learner achieves best rates for all distributions.
problem Improving learning algorithm rates under various settings.
method Simple extension of Levin's universal search.
result Achieves best-possible rates for all distributions.
Improved EXACT strategy reduces GNN memory consumption and runtime.
problem Efficiently training large-scale GNNs with reduced memory usage.
method Block-wise quantization of intermediate activation maps with improved variance minimization.
result Further reduction in memory consumption (>15%) and runtime speedup (5%) with similar performance trade-offs.
Improved computational efficiency for estimating Wasserstein distance.
problem Inefficient computation of Wasserstein distance for large samples.
method Developed Sample-Sketch-Solve paradigm using grid sketches.
result Approximates Wasserstein distance within ε error in ε^(-max(2, (d+1+o(1))/(1+α))) time.
New algorithm scales NDPP learning and inference to large item collections.
problem Memory and runtime limitations in existing NDPP learning and inference algorithms.
method Introduced a new NDPP kernel decomposition for learning and a linear-complexity MAP inference algorithm.
result Our algorithms scale linearly in , matching prior work's predictive performance.
Model predicts counterfactuals under domain shift and inaccessible variables.
problem Runtime domain corruption impairs counterfactual prediction.
method Subsumes counterfactual prediction under domain adaptation, uses adversarial domain adaptation to reduce distribution disparity.
result VEGAN outperforms baselines in individual-level treatment effect estimation.
SNS accelerates Sinkhorn algorithm with sparse Newton iterations.
problem Slow runtime of Sinkhorn algorithm for optimal transport.
method Early stopping and Newton-type subroutine for matrix scaling steps.
result SNS converges orders of magnitude faster than Sinkhorn.
Exact Gaussian Process (GP) regression has O(N^3) runtime for data size N, making it intractable for large N. Many algorithms for improving GP scaling approximate the covariance with lower rank matrices. Other work has exploited structure inherent in particular covariance functions, including GPs with implied Markov st…
Optimal transportation, or computing the Wasserstein or ``earth mover's'' distance between two distributions, is a fundamental primitive which arises in many learning and statistical settings. We give an algorithm which solves this problem to additive with parallel depth, and $\tilde{O}\left(n^2/ε\…
PClean automates Bayesian data cleaning for specific datasets.
problem Bayesian inference for diverse and complex data cleaning.
method Domain-specific probabilistic programming language with custom models and inference.
result PClean programs outperform general-purpose PPLs in accuracy and runtime.
We introduce the C++ application and R package ranger. The software is a fast implementation of random forests for high dimensional data. Ensembles of classification, regression and survival trees are supported. We describe the implementation, provide examples, validate the package with a reference implementation, and …
Improved Frank-Wolfe algorithm speeds up training of differentially private LASSO models.
problem Training differentially private LASSO models on sparse data.
method Adapted Frank-Wolfe algorithm for sparse inputs, reducing runtime.
result Training time reduced from to .
Support vector machine (SVM) is a particularly powerful and flexible supervised learning model that analyzes data for both classification and regression, whose usual algorithm complexity scales polynomially with the dimension of data space and the number of data points. To tackle the big data challenge, a quantum SVM a…
New scalable MCMC sampling for nonsymmetric DPPs speeds up computations.
problem Efficient sampling for nonsymmetric DPPs with low-rank kernels.
method Enhanced rejection sampling with efficient proposal distribution construction.
result Sublinear runtime for scalable MCMC sampling of -NDPPs.
This paper explains the theoretical inductive bias of Isolation Forest.
problem Lack of theoretical foundation explaining Isolation Forest's success.
method Formulated the growth process of iForest as a random walk, derived expected depth function using transition probabilities.
result Established a theoretical understanding of iForest's effectiveness and parameter adaptability.
We consider online learning with linear models, where the algorithm predicts on sequentially revealed instances (feature vectors), and is compared against the best linear function (comparator) in hindsight. Popular algorithms in this framework, such as Online Gradient Descent (OGD), have parameters (learning rates), wh…
We extend Neural Processes (NPs) to sequential data through Recurrent NPs or RNPs, a family of conditional state space models. RNPs model the state space with Neural Processes. Given time series observed on fast real-world time scales but containing slow long-term variabilities, RNPs may derive appropriate slow latent …
Bayesian method predicts runtime metrics for fog manufacturing.
problem Accurate prediction of runtime performance metrics in fog manufacturing.
method Bayesian sparse regression for multivariate mixed responses.
result Enhanced prediction and statistical inferences of runtime metrics.
We describe and analyze a simple algorithm for principal component analysis and singular value decomposition, VR-PCA, which uses computationally cheap stochastic iterations, yet converges exponentially fast to the optimal solution. In contrast, existing algorithms suffer either from slow convergence, or computationally…
The runtime for Kernel Partial Least Squares (KPLS) to compute the fit is quadratic in the number of examples. However, the necessity of obtaining sensitivity measures as degrees of freedom for model selection or confidence intervals for more detailed analysis requires cubic runtime, and thus constitutes a computationa…
Asynchronous SGD can speed up training with a trade-off of gradient staleness.
problem Asynchronous SGD suffers from gradient staleness, affecting convergence error.
method Theoretical analysis of the error-runtime trade-off considering random straggling delays.
result A method of gradually varying synchronicity in distributed SGD is proposed and demonstrated.
0-1 knapsack is of fundamental importance in computer science, business, operations research, etc. In this paper, we present a deep learning technique-based method to solve large-scale 0-1 knapsack problems where the number of products (items) is large and/or the values of products are not necessarily predetermined but…
Algorithm decides if two hyperbolic 3-manifolds are homeomorphic.
problem Determining if two hyperbolic 3-manifolds are homeomorphic.
method Algorithm finds hyperbolic structures and compares geometric manifolds.
result Algorithm runs in bounded time for triangulations with at most tetrahedra.
pySigLib speeds up signature-based computations on CPUs and GPUs.
problem Efficient signature-based computations on large datasets and long sequences.
method Optimised Python library for CPU and GPU, novel differentiation scheme.
result Accurate gradients at a fraction of the runtime of existing libraries.
A new method for efficient causal structure learning at scale.
problem Causal structure learning is computationally challenging at scale.
method Relaxed sparsest-permutation formulation with support-level relaxation and masked zero-fill incomplete Cholesky factorization.
result The method enables scalable comparison of candidate orderings and matches the accuracy of slower baselines.
Efficiently plans large MDPs with weak function approximations.
problem Planning in large MDPs with limited function approximation capabilities.
method Uses linear value function approximation with weak requirements and a generative oracle.
result Produces almost-optimal actions for any state with polynomial computation time.
New methods prune unpromising rules from KGs, improving scalability and runtime.
problem Scalability issues in walk-based rule learning from KGs.
method Rule Hierarchy Framework (RHF) and Hierarchical Pruning (HPMs).
result Significant reductions in runtime and number of learned rules without compromising predictive performance.
CausalMan simulates complex causality for fair benchmarking.
problem Lack of realistic causal models with known data generating processes.
method Developed a physics-based simulator, CausalMan, for large-scale causality.
result Demonstrated inadequacy of state-of-the-art approaches and analyzed their performance.
Develops a game-theoretic approach to solve SGEP efficiently.
problem Efficiently solving the symmetric generalized eigenvalue problem for large datasets.
method Formulates SGEP as a Nash equilibrium in a game-theoretic context and develops a parallelizable algorithm.
result Achieves runtime complexity, making it feasible for large-scale problems.
For well over a quarter century, detection systems have been driven by models learned from input features collected from real or simulated environments. An artifact (e.g., network event, potential malware sample, suspicious email) is deemed malicious or non-malicious based on its similarity to the learned model at runt…
In many recent applications, data is plentiful. By now, we have a rather clear understanding of how more data can be used to improve the accuracy of learning algorithms. Recently, there has been a growing interest in understanding how more data can be leveraged to reduce the required training runtime. In this paper, we…
Separating the short jobs from the long is a known technique to improve scheduling performance. In this paper we describe a method we developed for accurately predicting the runtimes classes of the jobs to enable this separation. Our method uses the fact that the runtimes can be represented as a mixture of overlapping …
GNMR controls runtime stability in low-precision language model training.
problem Efficient low-precision training faces numerical risks at specific operators.
method GNMR compares gradient norms to historical means, applying bounded recovery actions.
result GNMR preserves high-fidelity quality with sparse, budgeted recovery.
New algorithms solve large-scale convex regression problems.
problem Large-scale convex regression with subgradient regularization.
method Active set type algorithm on dual QP, approximate optimization, randomized augmentation.
result Solves problems with n=10^5 and d=10 in minutes.
Perhaps surprisingly, it is possible to predict how long an algorithm will take to run on a previously unseen input, using machine learning techniques to build a model of the algorithm's runtime as a function of problem-specific instance features. Such models have important applications to algorithm analysis, portfolio…
Mixture models and topic models generate each observation from a single cluster, but standard variational posteriors for each observation assign positive probability to all possible clusters. This requires dense storage and runtime costs that scale with the total number of clusters, even though typically only a few clu…
Adversarial training, in which a network is trained on adversarial examples, is one of the few defenses against adversarial attacks that withstands strong attacks. Unfortunately, the high cost of generating strong adversarial examples makes standard adversarial training impractical on large-scale problems like ImageNet…
Quantum Ridgelet Transform: Winning Lottery Ticket of Neural Networks with Quantum Computationquant-ph
Quantum Ridgelet Transform speeds up neural network learning.
problem Efficiently finding sparse trainable subnetworks in neural networks.
method Developed a quantum ridgelet transform (QRT) for linear runtime.
result Quantum Ridgelet Transform efficiently finds sparse trainable subnetworks.
We develop an online learning method for prediction, which is important in problems with large and/or streaming data sets. We formulate the learning approach using a covariance-fitting methodology, and show that the resulting predictor has desirable computational and distribution-free properties: It is implemented onli…