Optimal controls for conformal Laplacian obstacle problems on spheres and manifolds.
problem Optimal control of conformal metrics with constant scalar curvature.
method Analysis of optimal control problem on Riemannian manifolds with positive Yamabe invariant.
result Existence of smooth optimal controls inducing metrics with constant scalar curvature.
Standard optimizers perform as well as LARS and LAMB at large batch sizes.
problem Comparing optimizers for neural network training at large batch sizes.
method Used standard optimizers like Nesterov momentum and Adam to match or exceed LARS and LAMB results.
result Standard optimizers can match or exceed LARS and LAMB at large batch sizes.
Standard acquisition functions are sufficient for asynchronous Bayesian optimization.
problem Redundant and repeated queries in asynchronous Bayesian optimization.
method Conceptual analysis and theoretical guarantees of standard acquisitions.
result Standard acquisition functions achieve theoretical guarantees equivalent to Thompson sampling in asynchronous settings.
Optimizes sharp curvature inequality on spheres, proving near-minimizers are close to standard metric.
problem Optimizing total σ2-curvature on spheres with positive scalar curvature. method Analyzes metrics conformal to the standard sphere, uses Sobolev norms to measure closeness.
result Near-minimizers of total σ2-curvature are almost the standard metric (up to Möbius transformations). Standard Gaussian Process outperforms in high-dimensional Bayesian Optimization.
problem Standard Gaussian Process underperforms in high-dimensional optimization problems.
method Comprehensive evaluation of twelve benchmarks, use of Matérn kernels, probabilistic bounds, robust initialization strategy.
result Standard Gaussian Process can consistently achieve top-tier results in high-dimensional optimization problems.
Convex optimization with sparsity-promoting convex regularization is a standard approach for estimating sparse signals in noise. In order to promote sparsity more strongly than convex regularization, it is also standard practice to employ non-convex optimization. In this paper, we take a third approach. We utilize a no…
Revisits PPO design choices, exposing failure modes and proposing alternatives.
problem Failure modes of standard PPO in new environments.
method Revisits standard PPO design choices, exposes failure modes, and proposes alternative approaches.
result Alternative design choices prevent failure modes in new environments.
Continuous optimization is an important problem in many areas of AI, including vision, robotics, probabilistic inference, and machine learning. Unfortunately, most real-world optimization problems are nonconvex, causing standard convex techniques to find only local optima, even with extensions like random restarts and …
Inference models are a key component in scaling variational inference to deep latent variable models, most notably as encoder networks in variational auto-encoders (VAEs). By replacing conventional optimization-based inference with a learned model, inference is amortized over data examples and therefore more computatio…
Bayesian optimization sped up to linear time.
problem Expensive function evaluations and cubic computational complexity.
method Flexible binary partitioning of the search space.
result Linear computational complexity and superior optimization performance.
This work precisely characterizes and improves the tradeoff between robustness and accuracy in linear regression.
problem Tradeoff between robustness and accuracy in adversarial training.
method Characterizes the effect of augmentation on standard error in linear regression; proves RST improves robust error without sacrificing standard error.
result RST improves both standard and robust error for neural networks under various perturbations.
Standard Transformers approximate Hölder functions and achieve optimal nonparametric regression rate.
problem Approximating Hölder functions and achieving optimal nonparametric regression rate with Transformers.
method Using the size tuple and dimension vector metrics, the paper characterizes Transformer structures and derives upper bounds for their Lipschitz constant and memorization capacity.
result Standard Transformers achieve the minimax optimal rate in nonparametric regression for Hölder target functions.
Paper proves global optimality of a simple optimization scheme for learning DAG models.
problem Learning acyclic directed graphical models from data.
method Path-following optimization scheme for bivariate setting.
result Simple optimization scheme globally converges to global minimum.
The standard interpretation of importance-weighted autoencoders is that they maximize a tighter lower bound on the marginal likelihood than the standard evidence lower bound. We give an alternate interpretation of this procedure: that it optimizes the standard variational lower bound, but using a more complex distribut…
This paper shows how to learn variational inequalities fast with strong monotonicity.
problem Learning variational inequalities efficiently.
method Extending convex optimization techniques to variational inequalities with strong monotonicity.
result Fast generalization rates of Θ(1/ε) for learning variational inequalities. Unified approach to non-standard classification tasks.
problem Non-standard classification tasks like semi-supervised, positive-unlabelled, multi-positive-unlabelled and noisy-label learning.
method Probabilistic, unified approach training a classifier to predict label-distributions, then inferring class-distributions.
result Unified model for various non-standard classification tasks.
Cheap permutation tests speed up distribution testing without sacrificing accuracy.
problem Efficiently testing distribution differences and independence.
method Group datapoints into bins and permute only these bins, using stored sufficient statistics.
result Cheap permutation tests maintain the accuracy and optimality of standard tests but are significantly faster.
Graph neural networks improve solving linear optimization problems.
problem Improving the efficiency of solving linear optimization problems.
method Using graph neural networks to simulate standard interior-point methods for linear optimization problems.
result Graph neural networks can solve linear optimization problems close to optimality, often outperforming conventional solvers.
In the standard models for optimal multiple stopping problems it is assumed that between two exercises there is always a time period of deterministic length δ, the so called refraction period. This prevents the optimal exercise times from bunching up together on top of the optimal stopping time for the one-exercise c…
Ricci flow preserves standard sphere's curvature for certain conditions.
problem Preserving curvature of a manifold during Ricci flow.
method Normalized Ricci flow on manifolds with specific curvature integrals.
result Ricci flow preserves standard sphere's curvature under certain conditions.
Meta Optimal Transport learns from past problems to solve similar OT problems faster.
problem Solving similar optimal transport problems repeatedly from scratch is inefficient.
method Amortized optimization to predict optimal transport maps from past solutions.
result Meta OT models can solve new problems faster than standard methods.
Automated trading systems on developed and emerging capital markets are studied in this paper. The standard for developed market is automated trading system with 40-days simple moving average. We tested it for the index SIX Industrial for 1000 and 730 trading days of the slovak emerging capital market. The Buy and Hold…
The purpose of this paper is to generalize the regular Optimal Reduction Theorem to general proper Dirac actions, formulated both in terms of point and orbit reduction. A comparison to general standard singular Dirac reduction is given emphasizing the desingularization role played by optimal reduction.
This work extends reinforcement learning to handle non-cumulative objectives.
problem Optimizing functions of rewards rather than their sum in decision processes.
method Mapping NCMDPs to standard MDPs for reinforcement learning.
result Reinforcement learning techniques can be applied to NCMDPs.
New model improves data augmentation for causal tasks.
problem Optimizing causal models robustly under Wasserstein distances.
method Proposes a new G-Causal Normalizing Flow architecture.
result Empirically outperforms standard generative models.
Deriving and applying Proximal Policy Optimization to GFlowNets for efficient training of discrete sampling policies
problem Training stochastic policies to sample from structured discrete probability distributions
method Deriving policy gradient algorithms for GFlowNets and applying Proximal Policy Optimization
result Improved convergence speed and data efficiency compared to standard GFlowNet training objectives
Unified approach for fair classification with overlapping groups.
problem Ensuring fairness across multiple overlapping groups in prediction problems.
method Probabilistic population analysis leading to Bayes-optimal classifier, unifying existing methods.
result Outperforms baselines in fairness-performance tradeoff on real datasets.
Real-world applications often combine learning and optimization problems on graphs. For instance, our objective may be to cluster the graph in order to detect meaningful communities (or solve other common graph optimization problems such as facility location, maxcut, and so on). However, graphs or related attributes ar…
Unified approach to RLHF tackles uncertainty in reward function.
problem Uncertainty in reward function learned from human feedback.
method Value-incentivized preference optimization (VPO) that regularizes the reward function with value function.
result Theoretical and practical guarantees for both online and offline RLHF settings.
A new Riemannian framework optimizes LoRA for faster convergence and better performance.
problem Optimizing low-rank adapters in neural networks to improve convergence and performance.
method Integrates Riemannion optimizer, LoRA initialization, and efficient implementation for geometrically treating low-rank adapters.
result Consistent and noticeable improvements in convergence speed and final task performance over standard LoRA and its modifications.
Derives optimal control conditions using calculus of variations.
problem Optimizing Markov control in stochastic control problems.
method Calculus of variations approach to derive necessary conditions.
result Solves the Merton portfolio optimization problem.
Proposes MCBO for causal Bayesian optimization with model learning and regret bounds.
problem Maximizing downstream variables in unknown structural models.
method Model-based causal Bayesian optimization (MCBO) that learns full system models and trades off exploration and exploitation.
result First non-asymptotic bounds for CBO and practical implementation showing superior performance.
Bayesian optimization uses BNNs as efficient surrogate models for expensive function evaluations.
problem Optimizing expensive objective functions using Gaussian process surrogates.
method Study of Bayesian neural networks (BNNs) as alternatives to standard Gaussian process (GP) surrogates for optimization.
result Infinite-width BNNs are particularly promising, especially in high dimensions.
Reinforcement Learning improves insulin bolus decisions for type-I diabetes patients.
problem Optimal insulin bolus decisions for type-I diabetes patients are not well-established.
method Applied Reinforcement Learning to simulated T1DM data.
result Optimal bolus rule differs from standard advisors and can prevent hypoglycemia.
GOAT improves attention mechanisms by learning better priors.
problem Standard attention mechanisms use a naive uniform prior, limiting flexibility and generalization.
method GOAT introduces a trainable, continuous prior that replaces the uniform assumption, maintaining compatibility with optimized kernels.
result GOAT avoids representational trade-offs and learns an extrapolatable prior that combines positional flexibility with length generalization.
Training deep neural networks (DNNs) efficiently is a challenge due to the associated highly nonconvex optimization. The backpropagation (backprop) algorithm has long been the most widely used algorithm for gradient computation of parameters of DNNs and is used along with gradient descent-type algorithms for this optim…
Optimizes data acquisition in high-dimensional Bayesian optimization.
problem Suboptimal data acquisition in high-dimensional Bayesian optimization tasks.
method Utility-calibrated variational inference to align approximations with BO goals.
result Optimal data acquisition decisions under a limited computational budget.
Paper develops exact convex optimization for neural networks with polynomial activations.
problem Training two-layer neural networks with nonlinear polynomial activations.
method Exact convex optimization using semidefinite programming.
result Global optimization of neural networks is polynomial-time computable.
This paper calibrates Gaussian process predictive distributions for Bayesian optimization to improve sampling decisions.
problem Lower-tail miscalibration in GP predictive distributions affects BO sampling decisions.
method Introduces goal-oriented calibration for GP predictive distributions below a threshold t. result Post-hoc method tcGP improves lower-tail calibration and BO performance.
We study the effect of the stochastic gradient noise on the training of generative adversarial networks (GANs) and show that it can prevent the convergence of standard game optimization methods, while the batch version converges. We address this issue with a novel stochastic variance-reduced extragradient (SVRE) optimi…
Improved Random Search for hyperparameter optimization.
problem Optimizing machine learning hyperparameters efficiently.
method Weighted Random Search with probabilistic hyperparameter updates.
result Our method outperforms standard Random Search within the same budget.
New method balances multivariate model fitting for mixed likelihoods.
problem Multivariate models often fit only a subset of observed variables.
method Lipschitz standardization for data preprocessing.
result Lipschitz standardization leads to more accurate multivariate models.
DenseHMM improves HMMs by learning dense representations that enable gradient-based optimization.
problem Learning dense representations for hidden states and observables in HMMs.
method DenseHMM uses kernelized transition probabilities and two optimization schemes.
result DenseHMM achieves superior performance and expressiveness compared to standard HMMs.
Simplifies fair PCA with fast, efficient solution.
problem Learning fair low-rank approximations of data.
method Conceptually simple approach with analytic solution.
result Faster and similar results to existing fair PCA methods.
A new model corrects inhomogeneity in Optimal Transport with Boundary.
problem Inhomogeneity in UROT models for Optimal Transport with Boundary.
method Proposed a modified entropic regularization term to make UROT models homogeneous.
result Homogeneous UROT model preserves properties of standard UROT while correcting inhomogeneity.
In this paper, we work in the framework of the Merton problem but we impose a drawdown constraint on the consumption process. This means that consumption can never fall below a fixed proportion of the running maximum of past consumption. In terms of economic motivation, this constraint represents a type of habit format…
As the complexity of neural network models has grown, it has become increasingly important to optimize their design automatically through metalearning. Methods for discovering hyperparameters, topologies, and learning rate schedules have lead to significant increases in performance. This paper shows that loss functions…
Abstract notes on generative modeling techniques.
problem Improving generative modeling techniques.
method Connections between optimal transport and Schrödinger bridge, flow matching.
result Showed connections between mathematical principles and generative modeling techniques.