Serverless runtimes boost large-scale optimization efficiency.
problem Efficiently solving large-scale optimization problems.
method Master-worker setup with AWS Lambda, parallel optimization algorithm.
result Relative speedups up to 256 workers and efficiencies above 70% up to 64 workers.
Paper introduces HISA for efficient FHE computations.
problem Efficiently evaluating encrypted neural networks.
method Developed HISA for FHE applications, including compiler and runtime.
result Generated code is faster than hand-optimized implementations.
New DP algorithm improves privacy and efficiency for convex optimization.
problem Efficient, DP algorithms for convex optimization with strong excess risk bounds.
method Output perturbation for a broad class of tilted loss functions.
result Near optimal DP excess risk and runtime bounds for convex optimization.
A new algorithm reduces online exp-concave optimization runtime.
problem Minimizing regret in online learning with exponentially concave losses.
method LightONS, a variant of Online Newton Step (ONS), reduces runtime to O ( d 2 T + d ω T log T ) O(d^2 T + d^ω\sqrt{T \log T}) O ( d 2 T + d ω T log T ) . result Optimal regret with reduced runtime to O ( d 2 T + d ω T log T ) O(d^2 T + d^ω\sqrt{T \log T}) O ( d 2 T + d ω T log T ) . Improved algorithm for optimal stopping problems reduces runtime.
problem Optimal stopping problems with infinite time horizon and random discounting.
method Flexible forward improvement iteration with a variable look-ahead distance.
result The new algorithm converges and can significantly reduce runtime.
LeapsAndBounds configures solvers efficiently for unknown problem distributions.
problem Configuring general-purpose solvers to run efficiently on unknown problem instances.
method LeapsAndBounds tests configurations on randomly selected problem instances for longer and longer time.
result LeapsAndBounds achieves a runtime close to the optimal expected runtime with near-optimal running time.
A distributed algorithm for online multi-task learning reduces communication and runtime costs.
problem Heavy communication and high runtime complexity in online multi-task learning.
method Adaptive primal-dual algorithm that synchronizes data across geographically distributed tasks.
result The proposed algorithm achieves optimal regret and is effective on real-world datasets.
Improved computational efficiency for estimating Wasserstein distance.
problem Inefficient computation of Wasserstein distance for large samples.
method Developed Sample-Sketch-Solve paradigm using grid sketches.
result Approximates Wasserstein distance within ε error in ε^(-max(2, (d+1+o(1))/(1+α))) time.
This paper improves neural network training performance by optimizing concurrency and operation scheduling.
problem Managing and scheduling fine-grained operations in neural network training for high performance.
method Extending TensorFlow runtime to enable automatic concurrency control and scheduling, using performance modeling.
result Achieved 33% average performance improvement on neural network models, up to 49%.
Direct parallel algorithm for optimal transport with optimal runtime.
problem Computing the Wasserstein distance between two distributions efficiently.
method Primal-dual extragradient method for first-order iterations.
result Solves optimal transport to additive ε with O(1/ε) parallel depth.
ZICO learns DAGs from zero-inflated count data efficiently.
problem Learning network structures from zero-inflated count data.
method ZICO uses node-wise likelihoods with canonical links and a differentiable surrogate constraint for acyclicity.
result ZICO achieves superior performance and faster runtimes on simulated data.
Model predicts counterfactuals under domain shift and inaccessible variables.
problem Runtime domain corruption impairs counterfactual prediction.
method Subsumes counterfactual prediction under domain adaptation, uses adversarial domain adaptation to reduce distribution disparity.
result VEGAN outperforms baselines in individual-level treatment effect estimation.
Median sampling reduces the runtime of noisy evolutionary optimization problems.
problem Reduction of noise's negative effect in evolutionary optimization.
method Introducing median sampling into evolutionary algorithms and analyzing its performance.
result Median sampling reduces the expected runtime exponentially under onebit noise.
New methods improve solving linear systems and preconditioning with reduced complexity.
problem Efficiently solving linear systems and preconditioning matrices.
method Developed structured semidefinite programming algorithms.
result Improved runtimes for preconditioning and solving linear systems.
QBSD optimizes KPI forecasting for RAN networks with fast runtime and accuracy.
problem Efficiently forecasting KPIs for RAN networks with dynamic operating ranges.
method Quartile-Based Seasonality Decomposition (QBSD) for live single-step forecasting.
result QBSD outperforms other methods in runtime efficiency and forecast accuracy.
Improved SVRC algorithm reduces complexity for nonconvex optimization.
problem Finding local minima for nonconvex finite-sum optimization with improved complexity.
method Stochastic Recursive Variance-Reduced Cubic regularization (SRVRC) using recursively updated semi-stochastic gradient and Hessian estimators.
result SRVRC achieves improved gradient and Hessian complexities to find ( ε , ε ) (ε, \sqrtε) ( ε , ε ) -approximate local minimum. Bayesian optimization sped up with importance sampling.
problem Efficiently tuning hyperparameters for neural networks.
method Bayesian optimization with importance sampling.
result Significantly improved runtime and validation error.
Paper explores stochastic algorithms for PCA, focusing on convergence and runtime.
problem Finding top k eigen vectors of data covariance matrix.
method Revisits and analyzes stochastic approaches to PCA optimization.
result Stochastic methods offer comparable or superior empirical performance to direct non-convex methods.
Compress++ speeds up distribution compression to near-linear time.
problem Accurately summarize a probability distribution using a small number of points efficiently.
method Introduces Compress++, a meta-procedure to speed up any thinning algorithm.
result Achieves n \sqrt{n} n points with O ( log n / n ) \mathcal{O}(\sqrt{\log n/n}) O ( log n / n ) integration error in O ( n log 3 n ) \mathcal{O}(n \log^3 n) O ( n log 3 n ) time and O ( n log 2 n ) \mathcal{O}( \sqrt{n} \log^2 n ) O ( n log 2 n ) space. New algorithm speeds up feature selection and experimental design.
problem Designing efficient parallel algorithms for statistical subset selection.
method Differential submodularity and adaptive sampling.
result Logarithmic parallel runtime for feature selection and experimental design.
Daydream predicts DNN optimization efficacy efficiently.
problem Inefficiency and error in evaluating DNN optimizations.
method Models DNN execution with a dependency graph, predicts runtime based on simulation.
result Accurately predicts performance improvements from DNN optimizations.
Vecchia approximations provide the best accuracy-runtime trade-off for Gaussian process approximations.
problem High computational cost of Gaussian processes for large data sets.
method Systematic comparison of different Gaussian process approximations.
result Vecchia approximations consistently provide the best accuracy-runtime trade-off.
Bayesian optimization (BO) aims to minimize a given blackbox function using a model that is updated whenever new evidence about the function becomes available. Here, we address the problem of BO under partially right-censored response data, where in some evaluations we only obtain a lower bound on the function value. T…
Improved robust regression algorithms with faster runtime and better estimation rates.
problem Statistical regression problems under strong contamination model.
method Nearly-linear time algorithms using robust gradient descent and Sever framework.
result Improved estimation rates and runtime compared to state-of-the-art.
Two scalable methods for PSL structure learning improve runtime and AUC.
problem Efficiently learning clauses for probabilistic soft logic models.
method Greedy search and a novel optimization method combining data-driven clause generation and PPLL objective.
result PPLL achieves up to 15% AUC gains and an order of magnitude runtime speedup.
Stochastic momentum methods trade compute efficiency for serial runtime.
problem Stochastic momentum methods trade compute efficiency for serial runtime.
method Stochastic HB and ASGD for consistent linear regression with Gaussian covariates.
result HB preserves SGD-level CE over a larger batch-size window, allowing larger batches to reduce serial runtime until HB reaches its deterministic accelerated scale.
New method speeds up Bayesian inference for complex simulators.
problem Challenges in Bayesian inference for complex stochastic simulators with intractable likelihood functions.
method Optimization Monte Carlo framework reformulated as deterministic optimization problems with gradient-based methods.
result Accurate posterior inference with reduced runtimes compared to existing methods.
Bayesian method predicts runtime metrics for fog manufacturing.
problem Accurate prediction of runtime performance metrics in fog manufacturing.
method Bayesian sparse regression for multivariate mixed responses.
result Enhanced prediction and statistical inferences of runtime metrics.
The runtime for Kernel Partial Least Squares (KPLS) to compute the fit is quadratic in the number of examples. However, the necessity of obtaining sensitivity measures as degrees of freedom for model selection or confidence intervals for more detailed analysis requires cubic runtime, and thus constitutes a computationa…
Improved iterative hard thresholding for faster, sparser solutions.
problem Finding sparser solutions without sacrificing runtime.
method Adaptive regularization framework applied to iterative hard thresholding.
result Returns solutions with sparsity O ( s κ ) O(sκ) O ( s κ ) , improving over existing methods. Limbo is an open-source C++11 library for Bayesian optimization which is designed to be both highly flexible and very fast. It can be used to optimize functions for which the gradient is unknown, evaluations are expensive, and runtime cost matters (e.g., on embedded systems or robots). Benchmarks on standard functions …
New algorithm for online portfolio selection with reduced runtime.
problem Maximizing total return in online portfolio selection.
method Minimizes current logarithmic loss regularized by log-determinant of Hessian.
result Achieves regret guarantee similar to Universal Portfolios with reduced runtime.
Asynchronous SGD can speed up training with a trade-off of gradient staleness.
problem Asynchronous SGD suffers from gradient staleness, affecting convergence error.
method Theoretical analysis of the error-runtime trade-off considering random straggling delays.
result A method of gradually varying synchronicity in distributed SGD is proposed and demonstrated.
Algorithm learns to prune search space for repeated computations.
problem Exploit common structure in repeated similar problems.
method Exploit explore-exploit technique for pruning search space.
result Reduces runtime while provably outputting correct solutions.
Efficiently solves large portfolio optimization problems by reducing and sparsifying covariance matrices.
problem Large and dense covariance matrices limit efficient portfolio optimization.
method Dimension reduction and increased sparsity based on machine learning predictions.
result Improved portfolio performance and reduced runtime compared to full dense covariance matrices.
Novel algorithm optimizes decision trees for nonlinear metrics.
problem Optimizing decision trees for nonlinear metrics like F1-score.
method Bi-objective optimisation approach to find optimal trees on Pareto frontier.
result The optimal tree for nonlinear metrics lies on the Pareto frontier.
Sublinear LSVI via LSH reduces runtime to sublinear in actions.
problem Efficiently estimating value functions in reinforcement learning with sublinear runtime.
method Formulated as approximate maximum inner product search, used LSH to solve with sublinear time complexity.
result Sublinear runtime while maintaining LSVI's regret.
NPOD algorithm improves efficiency in estimating pharmacokinetic parameters.
problem Efficiently estimating joint distribution of model parameters in population pharmacokinetics.
method Uses gradient approach to suggest new support points, reducing evaluation time.
result Achieves similar solutions to NPAG but with significantly fewer cycles and runtime.
DBSCAN++ speeds up density clustering for large datasets.
problem Slow runtime of DBSCAN for large datasets.
method DBSCAN++ computes densities for a subset of points instead of all.
result DBSCAN++ provides competitive performance and robustness.
Algorithm decides if two hyperbolic 3-manifolds are homeomorphic.
problem Determining if two hyperbolic 3-manifolds are homeomorphic.
method Algorithm finds hyperbolic structures and compares geometric manifolds.
result Algorithm runs in bounded time for triangulations with at most t t t tetrahedra. This work tackles runtime complexity prediction for code, using machine learning and a new dataset.
problem Predicting runtime complexity of code is hard and mathematically impossible.
method Modelled as a machine learning task, using feature engineering and code embeddings, with a new dataset.
result Achieved state-of-the-art results in runtime complexity prediction.
Deep RL optimizes compiler passes for better performance.
problem Designing optimal compiler optimization sequences is hard and often suboptimal.
method Employed deep reinforcement learning to learn optimal pass orderings.
result Achieved up to 1.32x speedup on previously unseen programs.
Improves zeroth-order optimization for private machine learning with public data.
problem High computation and memory cost of first-order DP methods.
method PAZO (Public Data Assisted Zeroth-order Optimization) framework.
result Achieves superior privacy/utility tradeoffs across tasks.
Some methods based on simple regularizing geometric element transformations have heuristically been shown to give runtime efficient and quality effective smoothing algorithms for meshes. We describe the mathematical framework and a systematic approach to global optimization-based versions of such methods for mixed volu…
ACE models allow flexible conditioning and prediction of latent variables.
problem Lack of flexibility in conditioning and prediction of latent variables in probabilistic models.
method Introduces Amortized Conditioning Engine (ACE) that explicitly represents latent variables and allows runtime conditioning and prediction.
result ACE models outperform existing methods in diverse tasks like image completion, classification, Bayesian optimization, and simulation-based inference.
Hierarchical Block Sparse Neural Networks improve both accuracy and runtime efficiency of sparse DNNs.
problem Inefficiency of sparse DNNs on regular parallel hardware due to irregular computation.
method Introducing HBsNN, a structured sparse neural network that balances accuracy and runtime efficiency.
result HBsNN achieves better runtime performance and accuracy than unstructured and highly structured sparse models.
Near-optimal algorithms for mean estimation and linear regression with Gaussian covariates and Huber contamination.
problem Gaussian mean estimation and linear regression with Gaussian covariates in the presence of Huber contamination.
method Near-optimal algorithms with optimal error guarantees, achieving sample complexity n = i l d e O ( d / ε 2 ) n = ilde{O}(d/ε^2) n = i l d e O ( d / ε 2 ) and almost linear runtime. result First sample near-optimal and almost linear-time algorithms with optimal error guarantees for both problems.
For well over a quarter century, detection systems have been driven by models learned from input features collected from real or simulated environments. An artifact (e.g., network event, potential malware sample, suspicious email) is deemed malicious or non-malicious based on its similarity to the learned model at runt…