Scalable tools for nested optimization in deep learning.
problem Solving nested optimization problems on a large scale in deep learning.
method Building scalable tools for bilevel optimization.
result Tools for nested optimization scale to deep learning setups.
Unified SGD method improves convergence for nested optimization problems.
problem Stochastic nested optimization problems.
method ALTERNATE dESCEN (ALSET) method leveraging hidden smoothness.
result Requires O(ε−2) samples to achieve an ε-stationary point. Quantum algorithm speeds up nested expectation estimation by nearly quadratically.
problem Estimating repeatedly nested expectations with quantum computing.
method Proposes a quantum algorithm achieving nearly quadratic speedup over classical methods.
result Achieves nearly quadratic speedup for RNEs, up to logarithmic factors.
Paper tackles robust optimization under uncertainty using nested distance.
problem Optimizing under distributionally robust uncertainty with nested distance.
method Equivalent recursive and dynamic programming reformulations for tractable optimization.
result Optimal robust policies can be found efficiently using convex optimization.
New unbiased gradient estimators for complex optimization problems.
problem Unbiased and variance-limited gradient estimation for conditional stochastic optimization.
method Developed multilevel Monte Carlo gradient estimators for conditional stochastic optimization problems.
result Unbiased and finite variance gradient estimators for conditional stochastic optimization problems.
Paper tackles robust model training with a new stochastic algorithm.
problem Training robust models against data distribution shift.
method Derives a novel dual formulation and proposes a nested stochastic gradient descent algorithm.
result Establishes polynomial iteration and sample complexities for large-scale DRO problems.
New method uses nested optimal transport for financial time series evaluation.
problem Lack of consensus metric for evaluating generative models in finance.
method Nested optimal transport distance for time-causal tasks, with a parallelizable algorithm.
result Substantial speedups and robustness to financial tasks.
Develops a method for learning proposals in nested importance samplers.
problem Improving sampling quality in complex distributions.
method Nested Variational Inference (NVI) using forward or reverse KL divergence.
result Optimizing nested objectives leads to improved sample quality.
Nested model averaging improves high-dimensional linear regression performance.
problem High-dimensional linear regression with predictor ordering impact.
method Combining model averaging with regularized estimators on the solution path.
result Nested model averaging with lasso and SLOPE outperforms competing methods.
The paper proposes an efficient nested simulation design using likelihood ratio method.
problem Designing nested simulations with fixed outer scenarios and minimizing simulation effort.
method Proposes a bi-level optimization problem to decide inner replications and pooling strategies.
result Optimized design achieves $\cO(Γ^{-1})$ mean squared error of estimators.
New estimator reduces nested expectation estimation costs.
problem Estimating repeatedly nested expectations is computationally expensive.
method Recursive Estimator for Arbitrary Depth (READ) using randomized multilevel Monte Carlo.
result Optimal computational cost of O(ε^(-2)) for every fixed D.
NEST optimizes deep learning training by placing devices efficiently across networks and memory.
problem Inefficient device placement in distributed deep learning leads to high communication and memory overhead.
method NEST uses network-, compute-, and memory-aware dynamic programming to optimize device placement.
result NEST achieves up to 2.43 times higher throughput and better memory efficiency.
Nested Slice Sampling accelerates Nested Sampling for GPU acceleration.
problem Challenging inference for complex, multimodal targets.
method Vectorized Nested Slice Sampling using Hit-and-Run Slice Sampling.
result NSS maintains accurate evidence estimates and high-quality posterior samples, robust on multimodal problems.
Unified GP model optimizes hyperparameters with conditional dependence.
problem Efficient tuning of hyperparameters in neural networks.
method Unified Bayesian optimization framework based on a new Gaussian process (GP) model.
result Higher prediction accuracy and better optimization efficiency observed.
Develops new optimization techniques for decision-making under uncertainty.
problem Decision-making under uncertainty with complex cost functions and nested expectations.
method Introduces Multistage Conditional Compositional Optimization (MCCO) and develops multilevel Monte Carlo techniques.
result New optimization techniques reduce scenario complexity from exponential to polynomial growth.
Proposes a method to enforce nestedness in subspace learning methods.
problem Consistency between data representations when choosing different subspaces dimensions.
method Lifts Grassmannian optimization criteria to flag manifolds via nested projectors.
result Successfully addresses the nestedness issue in several machine learning methods.
New algorithm tackles nested bi-level optimization problems for robust feature learning.
problem Nested compositional bi-level optimization problems in machine learning.
method Stochastic approximation algorithms for solving nested compositional bi-level optimization problems without matrix inversions.
result Achieves an ε-stationary solution with an oracle complexity of approximately O_T(1/ε^2).
We show that deliberately introducing a nested simulation stage can lead to significant variance reductions when comparing two stopping times by Monte Carlo. We derive the optimal number of nested simulations and prove that the algorithm is remarkably robust to misspecifications of this number. The method is applied to…
In science and engineering, intelligent processing of complex signals such as images, sound or language is often performed by a parameterized hierarchy of nonlinear processing layers, sometimes biologically inspired. Hierarchical systems (or, more generally, nested systems) offer a way to generate complex mappings usin…
BAxUS optimizes high-dimensional functions adaptively, avoiding performance degradation and failure.
problem State-of-the-art HDBO methods degrade or fail with increasing dimensions.
method BAxUS uses nested random subspaces to adaptively optimize high-dimensional functions.
result BAxUS outperforms state-of-the-art methods across various applications.
Enhances financial risk quantification in classical models.
problem Risk quantification in classical finance models.
method Nested risk measures, limiting behavior analysis.
result Uniqueness of risk-averse limit in classical models.
MLMC boosts Bayesian optimization's look-ahead efficiency.
problem Efficiently computing nested expectations in Bayesian optimization.
method Multilevel Monte Carlo (MLMC) for nested operations.
result MLMC achieves MC convergence rate for nested operations, improving BO performance.
Optimizes SGD for anytime neural networks, improving accuracy.
problem Training networks that produce increasingly accurate outputs over time.
method Orthogonalized SGD optimizer for nested architectures.
result Significantly improves generalization accuracy of anytime networks.
Proposes methods for online conformal prediction with nested prediction sets across multiple confidence levels.
problem Need for uncertainty quantification with multiple confidence levels in diverse applications.
method Online optimization perspective to enforce nestedness of prediction sets while controlling quantile estimation error.
result Achieves stable coverage across all levels, strictly nested prediction sets, and improved efficiency.
A new algorithm estimates VaR and ES for financial risks.
problem Estimating Value-at-Risk and Expected Shortfall for financial losses.
method Multilevel Stochastic Approximation (MLSA) scheme for nested stochastic approximation problems.
result Optimal complexities for VaR and ES estimation are derived.
Study cobordisms of nested manifolds and their invariants.
problem Understanding cobordisms of nested manifolds and their invariants.
method Identify a nested analog of the Pontryagin-Thom construction and find spaces homotopy equivalent to nested Pontryagin-Thom spaces.
result Discover nested cobordism invariants and provide an alternative proof of Wall's splitting result.
Nested sampling is a powerful technique for exploring high-likelihood regions, but its theoretical derivation is complex and involves approximations.
problem Sampling from likelihood-constrained priors in nested sampling
method Providing a comprehensive and detailed exposition of nested sampling derivation and practical challenges
result Deepening understanding of nested sampling and fostering future enhancements
We study a stylized dynamic assortment planning problem during a selling season of finite length T. At each time period, the seller offers an arriving customer an assortment of substitutable products and the customer makes the purchase among offered products according to a discrete choice model. The goal of the selle…
New method improves simulation efficiency in high dimensions.
problem Efficiency in estimating functionals of conditional expectations in high dimensions.
method Kernel ridge regression exploiting smoothness of conditional expectation.
result Effective reduction of the curse of dimensionality, bridging convergence rates.
Simple algorithms identify best items or full rankings from choice-based feedback.
problem Learning to identify the best item or full ranking from choice-based feedback.
method Nested Elimination (NE) and Nested Partition (NP) algorithms.
result NE is worst-case asymptotically optimal, NP is optimal up to a constant factor.
New algorithm improves understanding of decentralized SBO transient iteration complexity.
problem Limited understanding of how network topology, data heterogeneity, and nested structures affect SBO.
method D-SOBA framework with two variants: D-SOBA-SO and D-SOBA-FO, providing non-asymptotic convergence analysis and transient iteration complexity.
result First theoretical understanding of how network topology, data heterogeneity, and nested structures influence decentralized SBO.
Improved nested simulation for financial risk measurement.
problem Efficiently estimating nested risk measures in financial engineering.
method Reusing inner simulation outputs to improve efficiency and accuracy.
result The proposed approach outperforms standard nested simulation and regression methods.
Paper proposes a new estimator for nested expectations with faster convergence.
problem Estimating nested expectations is computationally challenging.
method Nested kernel quadrature estimators with proof of faster convergence rate.
result The proposed method requires fewer samples for accurate estimation.
Let R be an o-minimal expansion of the real field, and let L(R) be the language consisting of all nested Rolle leaves over R. We call a set nested subpfaffian over R if it is the projection of a boolean combination of definable sets and nested Rolle leaves over R. Assuming that R admits analytic cell decomposition, we …
VCSMC improves efficiency in Bayesian phylogenetic inference.
problem Inefficient exploration of phylogenetic state space.
method Variational Combinatorial Sequential Monte Carlo (VCSMC) and nested CSMC.
result VCSMC and VNCSMC explore higher probability spaces efficiently.
Many problems in machine learning and statistics involve nested expectations and thus do not permit conventional Monte Carlo (MC) estimation. For such problems, one must nest estimators, such that terms in an outer estimator themselves involve calculation of a separate, nested, estimation. We investigate the statistica…
We study finite-sum nonconvex optimization problems, where the objective function is an average of n nonconvex functions. We propose a new stochastic gradient descent algorithm based on nested variance reduction. Compared with conventional stochastic variance reduced gradient (SVRG) algorithm that uses two reference …
Gradient-guided nested sampling improves posterior inference efficiency.
problem Efficiently sampling from complex posterior distributions.
method Gradient-guided nested sampling combining differentiable programming, Hamiltonian slice sampling, clustering, mode separation, dynamic nested sampling, and parallelization.
result Significantly faster mode discovery and more accurate partition function estimates.
This work connects IRL methods from ML and economics.
problem Solving the inverse reinforcement learning problem.
method Shows connections and differences between various IRL methods.
result Identifies key computational and algorithmic differences.
The paper proposes a new portfolio allocation method combining RMT and machine learning.
problem Optimal allocation instability in high-dimensional portfolios.
method Combines Random Matrix Theory covariance estimators with Nested Clustered Optimization.
result The modified NCO algorithm achieves stable allocations without risky short positions.
New method for estimating treatment effects without complex propensity models.
problem Estimating treatment effects in dynamic treatment regimes.
method Recursive Riesz representer estimation for de-biasing corrections.
result Directly estimates de-biasing corrections without auxiliary models.
New discrete cobordism category for nested manifolds and relations to algebraic structures.
problem Discrete cobordism category for nested manifolds.
method Stratified Morse theory, Cyl-objects, doubling construction, cylindrical bar construction.
result Relations between Cyl-objects and algebraic structures like Temperley-Lieb algebras.
Nested sampling improved for arbitrary priors.
problem Technical obstacle to using nested sampling with arbitrary priors.
method Parametric bijectors trained on samples from a desired prior density.
result Nested sampling can be used with arbitrary priors.
Proposes a method for explaining black-box models with nested feature attributions.
problem Making black-box models transparent and trustworthy.
method Model-agnostic local explanation method exploiting nested feature structure and consistency property.
result Accurate and consistent HiFAs and LoFAs estimated using fewer model queries.
We consider the stochastic nested composition optimization problem where the objective is a composition of two expected-value functions. We proposed the stochastic ADMM to solve this complicated objective. In order to find an ε stationary point where the expected norm of the subgradient of corresponding augmented Lag…
Efficiently optimizes expensive functions with multi-step lookahead using one-shot optimization.
problem Optimizing expensive functions with long-term impacts using myopic approaches.
method Formulated as nested optimization problems within a multi-step scenario tree, optimized in one-shot fashion.
result Multi-step expected improvement is computationally tractable and outperforms existing methods.
Novel approach to Bayesian experimental design for non-exchangeable data.
problem Optimal experimental design for non-exchangeable data.
method Inside-Out SMC2 algorithm embedded in particle Markov chain Monte Carlo framework. result Efficacy demonstrated on a set of dynamical systems.
Study of loops in sums of Laplace eigenfunctions on surfaces.
problem Uniform bound for the number of nested loops in sums of Laplace eigenfunctions.
method Real-analytic category analysis and biharmonic function construction.
result Uniform bound for the number of rooted double nests in terms of surface, root, and spectral cutoff.