Bayesian optimization sped up with scalable Gaussian processes.
problem Optimizing functions with derivative information and large datasets.
method Combines derivative acceleration and scalable Gaussian process models.
result Significant speedup in optimization convergence for large datasets.
Scalable3-BO tackles scalability issues in Bayesian optimization for big data and high dimensions.
problem Bayesian optimization scalability issues in big data and high dimensions.
method Sparse Gaussian process, random embedding, asynchronous parallelization.
result Scalable3-BO framework optimizes high-dimensional problems with 1 million data points and 10,000 dimensions.
A scalable parallel BO method for asynchronous settings.
problem Expensive-to-evaluate problems in machine learning.
method Simple and scalable Bayesian optimization method for asynchronous parallel settings.
result Demonstrated promising performance on benchmark functions and hyperparameter optimization.
Unified framework for scalable black-box optimization.
problem Expensive black-box evaluations in scientific and engineering domains.
method Integrates active learning, multi-armed bandits, and distributed computing.
result Consistently outperforms state-of-the-art black-box optimizers.
Novel method for scalable neural network-based blackbox optimization.
problem Scalability challenges in high-dimensional Bayesian Optimization.
method SNBO: Adds new samples using separate criteria for exploration and exploitation, adaptively controlling the sampling region.
result SNBO achieves better function values with 40-60% fewer function evaluations and reduced runtime.
New GP-VAE model improves scalability and performance.
problem Inability of conventional VAEs to model correlations between data points.
method Principled sparse inference approaches to improve scalability of GP-VAEs.
result New model outperforms existing approaches in runtime and memory usage.
Scalable TS using sparse GPs improves efficiency without sacrificing performance.
problem Efficiently applying TS to complex, multi-modal problems.
method Sparse Gaussian Process models for scalable TS.
result Theoretical and empirical validation of scalable TS's effectiveness.
Bayesian optimization method tackles combinatorial spaces, scalable for large data.
problem Optimization over combinatorial categorical spaces in natural sciences.
method Combines variational optimization and continuous relaxations for gradient-based optimization.
result Method performs comparably to state-of-the-art methods while scaling well.
A scalable method for training prediction models in predict-then-optimize
problem Training prediction models in the predict-then-optimize paradigm
method Decision-focused learning pipeline
result Decision quality competitive with state-of-the-art methods while reducing training time
A dissertation on scalable projection-free optimization methods.
problem Efficient optimization algorithms for large-scale machine learning problems.
method Study of Frank-Wolfe variants and their extensions to distributed and derivative-free settings.
result Development of 1-SFW and QFW, achieving state-of-the-art complexity and efficiency.
Meta-learning interpretable decision trees with synthetic data.
problem Lack of efficient, scalable methods for generating synthetic data for decision tree meta-learning.
method Synthetic generation of near-optimal decision trees using the MetaTree transformer architecture.
result Meta-learning of decision trees achieves performance comparable to real-world data or optimal decision trees, with significant computational cost reduction.
TensorHyper-VQC improves VQC scalability and robustness.
problem Scalability and noise sensitivity in VQC.
method Tensor-train-guided hypernetwork framework.
result TensorHyper-VQC achieves superior performance and robust noise tolerance.
New method for scalable barycenter computation using Wasserstein gradient flows.
problem Scalability and integration of label information in barycenter computation.
method Gradient flows in Wasserstein space, time discretization, mini-batch optimal transport, modular regularization, task-aware functions, supervised information integration.
result Empirically validated new state-of-the-art barycenter solver with labeled barycenters outperforming unlabeled ones.
Optimal Transport (OT) naturally arises in many machine learning applications, yet the heavy computational burden limits its wide-spread uses. To address the scalability issue, we propose an implicit generative learning-based framework called SPOT (Scalable Push-forward of Optimal Transport). Specifically, we approxima…
Develops scalable differentiable physics for complex object interactions.
problem Limited scalability of existing differentiable physics solvers.
method Adopting meshes for arbitrary geometry, localized collision handling, and accelerated implicit differentiation.
result Significantly reduces memory and computation requirements compared to particle-based methods.
A scalable framework optimizes multi-asset portfolios with constraints.
problem Optimizing multi-asset portfolios with inequality constraints.
method Integrates neural policies with Pontryagin's Maximum Principle, enforcing feasibility via log-barrier regularization.
result Recover KKT-optimal policies in high-dimensional problems without violating constraints.
New scalable methods for robust model learning from large datasets.
problem Training robust models resistant to data distribution shifts.
method Composite optimization for distributionally robust optimization (DRO).
result Scalable methods for learning robust models from large datasets.
ScaML-GP efficiently learns from few meta-tasks using Gaussian processes.
problem Exploiting historical data for quick task solving in low-data regimes.
method Modular Gaussian process model with a carefully designed multi-task kernel.
result ScaML-GP learns efficiently with few and many meta-tasks.
New algorithm improves clustering accuracy without sacrificing scalability.
problem Improving clustering accuracy for large datasets.
method Nonnegative low-rank semidefinite programming with Burer-Monteiro factorization.
result Significantly smaller mis-clustering errors compared to existing methods.
Scalable algorithm for computing Wasserstein-2 barycenters without bias.
problem Computing Wasserstein-2 barycenters efficiently and accurately.
method Input convex neural networks and cycle-consistency regularization.
result Our approach avoids introducing bias and does not require minimax optimization.
Bayesian optimization technique scaled using Vecchia approximations.
problem Scalability issue with Gaussian process surrogate models in Bayesian optimization.
method Adapted Vecchia approximation from spatial statistics to Gaussian processes, developed improvements and extensions.
result Methods compared favorably to state-of-the-art on various test functions and reinforcement learning problems.
LargeMvC-Net improves scalability of multi-view clustering.
problem Scalability issues in multi-view clustering.
method Deep unfolding of multi-view clustering into a network architecture with three modules.
result LargeMvC-Net consistently outperforms state-of-the-art methods in scalability and effectiveness.
Thompson sampling (TS) is a class of algorithms for sequential decision-making, which requires maintaining a posterior distribution over a model. However, calculating exact posterior distributions is intractable for all but the simplest models. Consequently, efficient computation of an approximate posterior distributio…
This paper develops scalable control variates for Monte Carlo methods using stochastic optimization.
problem Reducing variance in Monte Carlo estimators for large-scale problems.
method Control variates based on Stein operators, optimized through stochastic optimization.
result Novel theoretical results and empirical validations show effective variance reduction.
LOAD discovers optimal adjustments locally for scalable causal inference.
problem Scalable causal inference for unknown causal graphs.
method Local Optimal Adjustments Discovery (LOAD) method.
result LOAD combines local and global approaches for efficient and accurate causal effect estimation.
L0Learn solves sparse learning problems with millions of features.
problem Sparse learning problems with millions of features.
method Approximate algorithms based on coordinate descent and local combinatorial optimization.
result Achieves competitive run times and statistical performance.
BO method improved by density-ratio estimation for better efficiency and scalability.
problem Limitations in Bayesian optimization due to analytical tractability of predictive models.
method Reformulated Bayesian optimization by casting expected improvement as a binary classification problem.
result Improved efficiency and scalability of Bayesian optimization.
Scalable tools for nested optimization in deep learning.
problem Solving nested optimization problems on a large scale in deep learning.
method Building scalable tools for bilevel optimization.
result Tools for nested optimization scale to deep learning setups.
FSBM improves matching efficiency with minimal supervision.
problem Scalability vs. minimal supervision in matching frameworks.
method FSBM uses a small portion of pre-aligned pairs as state feedback to guide non-coupled samples.
result FSBM accelerates training and enhances generalization.
New algorithm optimizes interventions under network interference, scaling to large networks.
problem Optimal policy learning under network interference where one individual's treatment affects others.
method Developed a scalable Thompson sampling algorithm for dynamic networks.
result Proved a Bayesian regret bound that is sublinear in network size and rounds.
ECPv2 optimizes Lipschitz functions efficiently and scalably.
problem Global optimization of Lipschitz-continuous functions with unknown Lipschitz constants.
method Adapting the Every Call is Precious (ECP) framework, ECPv2 introduces adaptive lower bounds, Worst-m memory, and random projections to reduce computational cost and improve acceptance regions.
result ECPv2 retains ECP's no-regret guarantees with optimal finite-time bounds and expands the acceptance region with high probability.
Statistical relational frameworks such as Markov logic networks and probabilistic soft logic (PSL) encode model structure with weighted first-order logical clauses. Learning these clauses from data is referred to as structure learning. Structure learning alleviates the manual cost of specifying models. However, this be…
Bayesian optimization improved for nanophotonic device design.
problem Scalability and derivative information limitations in Bayesian optimization.
method Combining forward shape derivatives and iterative inversion scheme.
result Optimal designs of nanophotonic devices achieved with fewer iterations.
Bayesian optimization has emerged as a strong candidate tool for global optimization of functions with expensive evaluation costs. However, due to the dynamic nature of research in Bayesian approaches, and the evolution of computing technology, using Bayesian optimization in a parallel computing environment remains a c…
Gradient-informed BNNs improve Bayesian optimization performance.
problem Improving Bayesian optimization with gradient information.
method Gradient-informed loss function for Bayesian neural networks.
result Gradient information enhances BNN predictions and BO convergence.
As a non-parametric Bayesian model which produces informative predictive distribution, Gaussian process (GP) has been widely used in various fields, like regression, classification and optimization. The cubic complexity of standard GP however leads to poor scalability, which poses challenges in the era of big data. Hen…
We propose a new scalable method to optimize the architecture of an artificial neural network. The proposed algorithm, called Greedy Search for Neural Network Architecture, aims to determine a neural network with minimal number of layers that is at least as performant as neural networks of the same structure identified…
Scalable model for slate recommendation learns reward probabilities.
problem Scalable personalized slate recommendation in large action spaces.
method Probabilistic Rank and Reward (PRR) model combining reward, interaction, and rank.
result PRR outperforms existing methods and is scalable to large action spaces.
Improves scalability of Bayesian optimization for combinatorial spaces.
problem Optimizing expensive functions over large combinatorial spaces.
method Parametrized Submodular Relaxation (PSR) to solve AFO problems for BOCS.
result Significant improvements in scalability and accuracy for BOCS model.
The optimization of expensive-to-evaluate black-box functions over combinatorial structures is an ubiquitous task in machine learning, engineering and the natural sciences. The combinatorial explosion of the search space and costly evaluations pose challenges for current techniques in discrete optimization and machine …
A scalable portfolio approach speeds up Bayesian optimization for noisy functions.
problem Efficiently selecting multiple designs in parallel for noisy, expensive black-box optimization.
method A portfolio approach that balances exploration and exploitation, using a scalable allocation strategy.
result Significant speed improvements over existing methods, with similar or better performance.
FSGD uses latent factors to scale SGD for high-dimensional learning.
problem Scalable optimization in high-dimensional machine learning.
method Factor-Augmented SGD (FSGD) that operates on streaming data.
result Established theoretical framework for latent factor estimation error in SGD.
Proposes SHORE model for efficient MOR with sparsity and scalability.
problem Challenges of interpretability and scalability in MOR with high-dimensional outputs.
method Incorporates sparsity requirements and a two-stage optimization framework for efficient compression.
result Theoretical and empirical validation of the proposed framework's efficiency and accuracy.
New method calibrates LLMs for safety-critical tasks with scalable Bayesian inference.
problem Overconfidence in LLMs after fine-tuning for specific tasks.
method Orthogonalized Low-Rank Adapters (PoLAR) with variational Bayesian inference.
result Scalable and well-calibrated uncertainty estimation for LLMs.
A scalable algorithm for GP regression selects relevant covariates efficiently.
problem Scalable variable selection in large GP regression models.
method VGPR algorithm using Vecchia approximation for sparse precision matrix, mini-batch subsampling.
result Improved scalability and accuracy in selecting relevant covariates.
Meta-BO method clusters and learns from prior tasks to optimize heterogeneous functions.
problem Optimizing multiple functions with historical data and scalability issues.
method Clustering-based meta-learning, surrogate prototypes, adaptive weighting policies.
result Scalable and robust meta-BO method improves convergence to global optimum.
Bayesian approach improves performance in Gaussian process models.
problem Scalable posterior estimation in Gaussian process models.
method Revisiting variational inference techniques with Bayesian treatment of inducing variables and hyper-parameters.
result State-of-the-art performance demonstrated across various regression and classification problems.
New rule-based method for classification with scalability, interpretability, and fairness.
problem Developing a scalable and fair classification method.
method Column generation for linear programming, decision tree-based heuristic, and rule-based optimization.
result The method returns interpretable rules with optimal weights and addresses fairness constraints.