Improved Frank-Wolfe algorithms for large-scale optimization.
problem Efficiently solving large-scale optimization problems.
method Modifications to Frank-Wolfe algorithm using stochastic gradients, approximate solutions, and sketched variables.
result Achieves optimal convergence rate of O(k1) for large problems. Efficiently solves large-scale robust portfolio optimization problems.
problem High computational demands in large-scale robust portfolio optimization.
method Extended supporting hyperplane approximation for distributionally robust portfolio problems.
result Significantly reduces computational time from several thousand seconds to just a few.
BanditLP optimizes personalized recommendations for large-scale systems.
problem Optimizing personalized recommendations for large-scale systems with constraints.
method Unified neural Thompson Sampling for learning and large-scale linear programming for action selection.
result Consistent gains over strong baselines in experiments and business win in LinkedIn's email marketing system.
Serverless runtimes boost large-scale optimization efficiency.
problem Efficiently solving large-scale optimization problems.
method Master-worker setup with AWS Lambda, parallel optimization algorithm.
result Relative speedups up to 256 workers and efficiencies above 70% up to 64 workers.
Proposes MamBO for efficient high-dimensional large-scale optimization.
problem High-dimensional and large-scale optimization problems in machine learning and simulation.
method Combines subsampling and subspace embeddings with model aggregation to address uncertainty in surrogate models.
result Improves robustness of Bayesian optimization algorithm and achieves superior performance.
Paper proposes PPMM for fast estimation of large-scale OTM.
problem Estimation of large-scale optimal transport maps (OTM) is challenging due to the curse of dimensionality.
method Combines projection pursuit regression and sufficient dimension reduction to adaptively select projection directions.
result PPMM consistently estimates the most informative projection direction and weakly converges to the target OTM.
A new L-BFGS method tackles large-scale optimization with fewer evaluations.
problem Efficiently solving large-scale unconstrained optimization problems.
method Proposes a regularized L-BFGS method with line search techniques.
result Shows global convergence and robust performance in numerical tests.
Paper proposes a faster SPIDER-EM variant for large-scale nonconvex optimization.
problem High computational cost of EM algorithm in large-scale learning.
method Extension of SPIDER-EM for nonconvex finite-sum optimization problems.
result Achieves state-of-the-art complexity bounds and linear convergence under certain conditions.
We optimize saddle-point problems for large-scale Markov decision processes.
problem Optimizing policies in large-scale Markov decision processes.
method Characterized conditions for convergence and designed an optimization algorithm.
result Our algorithm converges faster and is state-space independent.
A new framework speeds up submodular optimization for large problems.
problem Large scale submodular optimization problems are hard to solve.
method Memoization framework for submodular functions with precomputed statistics.
result Significant speedups for various submodular optimization problems.
Proposes a probabilistic optimization method for large-scale problems.
problem Large-scale regularized optimization problems.
method Develops a probabilistic interpretation of the incremental proximal gradient algorithm and uses Bayesian filtering.
result Makes it possible to solve large-scale problems using well-known Bayesian filters.
Adaptive IHS improves sketching for large-scale data.
problem Efficiently modeling large-scale data with iterative Hessian sketch.
method Deterministic A-optimal subsampling for improved IHS.
result A-optimal IHS outperforms existing accelerated IHS methods.
Memory-efficient learning for large-scale imaging systems.
problem Memory limitations in GPUs for real-world large-scale inverse problems.
method Exploits reversibility of network layers to enable data-driven design.
result Demonstrated on small-scale and large-scale real-world systems.
This paper analyzes convergence of large-scale Transformers with weight decay.
problem Understanding optimization guarantees in large-scale Transformer training.
method Construct mean-field limit, show gradient flow convergence to PDE, demonstrate global minimum consistency.
result Gradient flow reaches global minimum in large-scale Transformers with small weight decay.
This paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications. Through case studies on text classification and the training of deep neural networks, we discuss how optimization problems arise in machine learning and what…
RL solves large-scale MV portfolio allocation with high returns.
problem Large-scale mean-variance portfolio optimization.
method Continuous-time reinforcement learning with a multivariate Gaussian policy.
result Our method outperforms econometric and deep RL methods by significant margins.
Kernel methods provide a principled way to perform non linear, nonparametric learning. They rely on solid functional analytic foundations and enjoy optimal statistical properties. However, at least in their basic form, they have limited applicability in large scale scenarios because of stringent computational requireme…
Optimized online learning with kernels for large-scale adversarial data.
problem Efficient online learning for large-scale, potentially adversarial datasets.
method Online variations of kernel Ridge regression using approximated basis functions.
result Optimal regret for a wide range of kernels with low per-round complexity.
Paper presents an efficient algorithm for learning minimax risk classifiers with large-scale data.
problem Efficient learning of minimax risk classifiers for large-scale data with multiple classes.
method Combination of constraint and column generation for efficient learning.
result 10x speedup for general large-scale data and 100x speedup with many classes.
This paper surveys large-scale machine learning methods for efficient data analysis.
problem Efficiently processing large-scale data with machine learning models.
method Divided into three categories: model simplification, optimization approximation, and computation parallelism.
result Blueprint for future developments in large-scale machine learning.
Paper introduces slow kill for efficient large-scale variable screening.
problem Challenges in variable selection and parameter estimation for big data.
method Nonconvex constrained optimization, adaptive \(\ell_2\)-shrinkage, and increasing learning rates.
result Slow kill outperforms state-of-the-art algorithms in various situations.
ParK efficiently solves kernel ridge regression for large datasets.
problem Large-scale kernel ridge regression efficiency and accuracy.
method Partitioning feature space with random projections and iterative optimization.
result Provably maintains statistical accuracy with reduced space and time complexity.
Unified framework for fast large-scale portfolio optimization.
problem Efficient portfolio optimization for large-scale financial data.
method Incorporates shrinkage and regularization techniques, addressing multiple objectives.
result AP-Trees and PCA-based factor models consistently outperform other approaches in out-of-sample portfolio performance.
Unified model predicts structure of neural network loss landscapes.
problem Understanding the structure of neural network loss landscapes.
method Modeling loss landscape as high-dimensional wedges, analyzing hyperparameters' effects.
result Existence of low-loss subspaces connecting solutions.
We propose HAMSI (Hessian Approximated Multiple Subsets Iteration), which is a provably convergent, second order incremental algorithm for solving large-scale partially separable optimization problems. The algorithm is based on a local quadratic approximation, and hence, allows incorporating curvature information to sp…
New algorithms optimize risk for large datasets, improving efficiency.
problem Optimizing risk for large datasets with robust methods.
method Proposed algorithms for distributionally robust optimization with CVaR and χ² divergence uncertainty sets.
result Algorithms require independent gradient evaluations of training set size and parameters, suitable for large-scale applications.
LargeMvC-Net improves scalability of multi-view clustering.
problem Scalability issues in multi-view clustering.
method Deep unfolding of multi-view clustering into a network architecture with three modules.
result LargeMvC-Net consistently outperforms state-of-the-art methods in scalability and effectiveness.
Many real-world regression problems demand a measure of the uncertainty associated with each prediction. Standard decision forests deliver efficient state-of-the-art predictive performance, but high-quality uncertainty estimates are lacking. Gaussian processes (GPs) deliver uncertainty estimates, but scaling GPs to lar…
Paper tackles entity matching over multi-source data, optimizing alignment and mitigating negative transfer.
problem Learning effective entity matching models over multi-source large-scale data with relaxed assumptions.
method Proposes a Relaxed Multi-source Large-scale Entity-matching (RMLE) problem and Incentive Compatible Pareto Alignment (ICPA) method.
result Optimized cross-source alignments and mitigated negative transfer, improving entity matching accuracy.
Paper tackles non-convex constrained DRO with a stochastic algorithm for large-scale applications.
problem Training robust models against data distribution shifts with non-convex loss functions.
method Developed a stochastic algorithm for non-convex constrained DRO with a complexity independent of dataset size.
result Algorithm finds ε-stationary points with computational complexity of O(ε^(-3k_*-5)) for general Cressie-Read divergence.
Faster convergence and handling larger mini-batches for deep neural networks.
problem Generalization gap in large-scale distributed training of deep neural networks.
method Second-order optimization using Kronecker-factored approximate curvature.
result Achieved 75% Top-1 validation accuracy with mini-batch size of 131,072 in 978 iterations.
This paper extends Mirror Descent to Riemannian manifolds for optimization.
problem Optimization on Riemannian manifolds.
method Developed a Riemannian Mirror Descent (RMD) framework and a stochastic variant.
result Established non-asymptotic convergence guarantees for RMD and stochastic RMD.
Improved ridge regression with Frequent Directions for large-scale tasks.
problem Improving performance of ridge regression for large-scale data.
method Combines Frequent Directions with iterative optimization schemes.
result Achieves high accuracy in estimating bias and variance for sketched ridge regression.
This paper proposes a new method for learning covers of geometric datasets to improve topological inference and visualization.
problem Improving topological inference and visualization of large-scale geometric datasets.
method Proposes a method for learning topologically-faithful covers of geometric datasets using optimization.
result Simplicial complexes obtained from learned covers outperform standard methods in terms of size and representation of large-scale topology.
Majorization-minimization algorithms consist of iteratively minimizing a majorizing surrogate of an objective function. Because of its simplicity and its wide applicability, this principle has been very popular in statistics and in signal processing. In this paper, we intend to make this principle scalable. We introduc…
New Bayesian optimization models for efficient material screening.
problem Efficiently screening materials with expensive and cheap tests.
method Flexible multi-test Bayesian optimization models with complex relationships.
result Demonstrated power on synthetic and real data.
Algorithm tackles large-scale portfolio optimization with higher moments, improving computational efficiency.
problem Optimizing portfolios with higher moments (variance, skewness, kurtosis) for large asset universes is computationally infeasible.
method Developed a structure-exploiting algorithm based on Yau's affine-normal descent, working directly with return matrix.
result Algorithm avoids explicit higher-order tensors and exploits quartic structure for efficient computation.
Develops minibatch stochastic proximal gradient for large-scale learning models.
problem Finding optimal predictors with complex regularizers in large-scale learning models.
method Minibatch variants of stochastic proximal gradient algorithm for composite objective functions.
result Minibatch size N after O(Nε1) iterations achieves ε−suboptimality in expected quadratic distance. A new OT discrepancy derived from GW for faster large-scale comparisons.
problem Comparing distributions with different metric spaces.
method Closed-form solution for 1D GW, slicing approach for large-scale optimization.
result Sliced Gromov-Wasserstein (SGW) is faster and similar to GW in performance.
We study large-scale classification problems in changing environments where a small part of the dataset is modified, and the effect of the data modification must be quickly incorporated into the classifier. When the entire dataset is large, even if the amount of the data modification is fairly small, the computational …
New DAM method improves AUC scores in medical image classification.
problem Maximizing AUC in large-scale medical image classification.
method Proposes AUC margin loss for robust optimization, conducts extensive empirical studies.
result Improves performance on four medical image classification tasks, achieving 1st place on Stanford CheXpert.
A new framework SPOT efficiently solves large scale optimal transport problems.
problem Heavy computational burden in optimal transport limits its use.
method Implicit generative learning framework (SPOT) approximates optimal transport plan and solves it using stochastic gradient algorithms.
result SPOT efficiently solves optimal transport problems and can recover the density of the plan.
Sparse Convex Biclustering improves accuracy and robustness in high-dimensional datasets.
problem Challenges in clustering rows and columns of large-scale datasets due to noise and computational complexity.
method Sparse Convex Biclustering (SpaCoBi) using convex optimization and stability-based tuning.
result Significantly outperforms state-of-the-art methods in accuracy for high-dimensional datasets.
A topology on a set X is the same as a projection (i.e. an idempotent linear operator) cl:2X→2X satisfying A⊂cl(A) for all A⊂X. That's a good way to summarize Kuratowski's closure operator. Basic geometry on a set X is a dot product ⋅:2X×2X→2Y. Its equivalent form is an or…
For large scale on-line inference problems the update strategy is critical for performance. We derive an adaptive scan Gibbs sampler that optimizes the update frequency by selecting an optimum mini-batch size. We demonstrate performance of our adaptive batch-size Gibbs sampler by comparing it against the collapsed Gibb…
The paper optimizes A/B tests by balancing lift and cost in large-scale settings.
problem Balancing lift and cost in A/B tests for large-scale experimentation.
method Empirical Bayes approach using a greedy knapsack algorithm to rank experiments based on lift-to-cost ratio, incorporating local false discovery rate (lfdr).
result The proposed method maximizes expected profit while controlling false discovery rate, demonstrating superior performance in large-scale settings.
HyperAgent improves RL exploration in large-scale problems.
problem Efficient exploration in large-scale reinforcement learning problems.
method Hypermodel framework for incremental posterior approximation without conjugacy.
result HyperAgent achieves logarithmic per-step computational complexity and sublinear regret.
A new method reduces memory usage for large models by three orders of magnitude.
problem Memory limitations in training large models.
method Extreme tensoring for adaptive preconditioning.
result Significant reduction in memory usage without performance degradation.