New method uses neural operators for efficient function space optimization.
problem Optimization over function spaces with costly function evaluations.
method Sample-then-optimize approach with neural operator surrogates.
result Better sample efficiency and significant performance gains in experiments.
This research proves that quadratic regularized optimal transport can approximate the Laplace-Beltrami operator on smooth manifolds.
problem Approximating the Laplace-Beltrami operator using optimal transport with quadratic regularization.
method Deriving first-order optimal potentials and analyzing the convergence of discrete Laplace operators.
result The discrete Laplace operators converge to the Laplace-Beltrami operator on smooth manifolds.
OpEvo automates tensor operator optimization for better efficiency.
problem Manual optimization of tensor operators is inefficient and limited.
method OpEvo uses evolutionary computation with topology-aware mutation.
result OpEvo finds optimal configurations with less effort and variance.
Optimal lower bounds for eigenvalues of Dirac-Witten operator on certain submanifolds.
problem Estimating eigenvalues of the Dirac-Witten operator on specific submanifolds.
method Optimal lower bounds derived using intrinsic and extrinsic expressions.
result Limiting-cases of eigenvalues studied and optimal bounds obtained.
New algorithm solves composite optimization problems with unknown expectations.
problem Solving composite optimization problems with unknown statistical expectations.
method Proposes a new stochastic primal-dual algorithm for composite optimization problems with unknown statistical expectations.
result Converges to a saddle point of the Lagrangian function.
Optimizes eigenvalue bounds for submanifold Dirac operators.
problem Estimating eigenvalues of submanifold Dirac operators.
method Optimal lower bounds derived using intrinsic and extrinsic expressions.
result Optimal eigenvalue bounds established for submanifold Dirac operators.
Optimal iterative thresholding algorithms improve upon hard and soft thresholding.
problem Optimizing sparsity or rank constraints in optimization problems.
method Developed the notion of relative concavity for thresholding operators, finding a new class of operators that are optimal.
result A new class of thresholding operators, including ℓq thresholding and reciprocal thresholding, achieves the strongest convergence guarantee. New reinforcement learning operators improve performance and robustness.
problem Improving reinforcement learning algorithms to handle approximation errors.
method Developed a new family of robust stochastic operators.
result Preserves optimality and increases action gap on sample paths.
OPVI uses operators to optimize variational objectives, improving scalability and approximation quality.
problem Statistical properties of classical variational inference can be undesirable.
method OPVI redefines variational inference using operators to optimize variational objectives.
result OPVI enables data subsampling and variational programs, improving scalability and approximation quality.
Automates GPU kernel optimization for diverse applications.
problem Lack of systematic evaluation for multi-scenario GPU kernel optimization.
method Introduces MSKernelBench and CUDAMaster for automated optimization.
result Significant speedups across various operators, outperforming existing tools.
Neural operators achieve fast convergence rates for solving PDEs.
problem Solving partial differential equations (PDEs) efficiently.
method Two-layer neural operators with gradient descent analysis in RKHS.
result Fast convergence rates are minimax optimal for early-stopped GD.
Smoothed top-k operator improves model training efficiency.
problem Discontinuous top-k operation makes models untrainable end-to-end.
method SOFT top-k operator approximates top-k as EOT solution.
result Improved performance in k-nearest neighbors and beam search.
Study optimal holomorphic extensions on complex manifolds with transitivity property.
problem Optimal holomorphic extensions on complex manifolds with transitivity property.
method Use Toeplitz operators and transitivity property for optimal holomorphic extensions.
result Transitivity property of optimal holomorphic extensions with small defect.
Operator calculus for population-based optimization provides a unified framework for analyzing convergence of various methods.
problem Convergence analysis of population-based optimization methods
method Introduce an operator calculus for describing composite mean-field algorithms as compositions of elementary operators acting on probability measures.
result Establish a modular Lyapunov principle for certifying exponential decay of state-space Lyapunov function and search errors.
New optimization algorithms for neural networks using operator splitting.
problem Training efficiency and convergence in neural networks.
method Sequential operator splitting technique applied to neural network training.
result Empirical rate of convergence towards local minimum of loss function validated.
Softmax Bellman operator improves Q-function performance in RL despite sub-optimality.
problem Softmax Bellman operator's impact on value functions in RL is problematic.
method Revisited theoretical properties of softmax Bellman operator, proving convergence and overestimation reduction.
result Softmax Bellman operator leads to superior policies in practice, even outperforming double Q-learning.
Optimal multiscale learning of linear operators
problem Statistical and computational limits of learning bounded linear operators between Sobolev spaces
method Reformulate as an infinite-dimensional matrix regression problem with heterogeneous multiscale structure
result Establish minimax rates and construct a finite-resolution blockwise least-squares estimator attaining these rates
Alternative proof for convergence rate of three operator splitting scheme.
problem Optimizing composite functions with specific operator properties.
method Three operator splitting scheme for optimizing composite functions.
result Sublinear rate of convergence proved for the method.
Near-optimal rates for multi-task learning with shared representations.
problem Approximation and statistical complexity of learning multiple operators.
method Multiple Neural Operators (MNO) architecture and comparison with DeepONet.
result Near-optimal upper and lower bounds for approximation and generalization.
The paper tackles data-driven optimal control of unknown nonlinear systems using RKHS.
problem Unknown nonlinear dynamics and stage cost functions.
method Embed state densities into RKHS, learn Markov operators, solve Hamilton-Jacobi-Bellman recursions.
result Solves a wide range of nonlinear control problems, including depth regulation.
Agent learns optimal control inputs for plants with unknown parameters.
problem Optimal control for systems with unknown and changing parameters.
method Personalized control inputs based on stochastic dynamics.
result Demonstrated effectiveness on simulated system.
Optimizes learning Hilbert-Schmidt operators between Sobolev spaces.
problem Statistical limits of learning mappings between infinite-dimensional function spaces.
method Minimax optimal regularization and multilevel training.
result Multilevel kernel operator learning achieves optimal learning rate.
Optimal upper bound found for eigenvalue of Jacobi operator on hypersurfaces in spheres.
problem Estimating the first eigenvalue of Jacobi operator on hypersurfaces with constant mean curvature.
method Analyzing the eigenvalue of Jacobi operator on compact hypersurfaces in spheres with constant mean curvature.
result An optimal upper bound for the first eigenvalue is derived, depending only on mean curvature and dimension.
Kernel operators help detect patterns in complex data.
problem Detecting long-lived coherent patterns in high-dimensional time-series data.
method Dominant eigenfunctions of kernel transfer operators combined with gradient-based optimization.
result Effective detection of long-lived coherent patterns in high-dimensional time-series data.
Study solves optimal portfolio selection using HJB equation.
problem Optimal portfolio selection problem.
method Maximal monotone operator method, Banach fixed-point theorem, Fourier transform, monotone operators technique.
result Existence and uniqueness of solution to HJB equation.
DeepCO uses deep learning for offline combinatorial optimization in warehouse operations.
problem Optimizing warehouse operation sequences in offline settings.
method DeepCO framework utilizing distribution regularized optimization for TSP.
result DeepCO reduces route length by 5.7% on average for TSP problems.
DVA framework attributes value of predictive models to features, configurations, and interactions.
problem Lack of explanation for how predictive models influence operational decisions.
method Shapley-based cooperative game theory applied to predict-then-optimize systems.
result DVA can guide targeted interventions to align model beliefs with operational performance.
Study eigenvalues and eigenfunctions of fourth-order operators in annuli, proving optimal estimates and non-radiality.
problem Eigenvalue and eigenfunction analysis of fourth-order operators in degenerating annuli.
method Optimal estimates and non-radiality results for eigenfunctions in annuli.
result Nigh optimal estimate for the first eigenvalue and non-radiality of eigenfunctions in degenerating annuli.
This paper uses t-SNE to visualize multi-objective electric machine optimization at various operating points.
problem Visualization of multi-objective electric machine optimization at multiple operating points is challenging.
method Utilizes t-distributed stochastic neighbor embedding (t-SNE) to visualize high-dimensional data.
result t-SNE provides better visualizations of electric machine design candidates and their performance.
Paper finds a fast method for a matrix norm proximal operator.
problem Optimizing mixed ℓ1,∞ matrix norms efficiently. method Closed-form computation using soft-thresholding, iterative algorithm for thresholds.
result Mixed ℓ1,∞ prox can be computed in closed form. Framework learns to optimize tensor programs for various hardware.
problem Manual optimization of tensor operators for deep learning limits applicability and increases engineering costs.
method Learning-based statistical cost models guide tensor operator implementations over billions of variants.
result Framework delivers performance competitive with hand-tuned libraries across multiple hardware targets.
A note on setting swap parameters for traders.
problem Determining optimal slippage parameters and trade size for wealth swapping.
method Theoretical solution and framework for optimal slippage parameters and trade size.
result Offers a method to solve optimal slippage parameters and trade size for wealth swapping.
Paper proposes a novel metric learning algorithm using Riemannian optimization.
problem Optimizing a smooth, convex function in Riemannian space with constraints.
method Developed a primal-dual algorithm with proximal operator for iterative optimization.
result Demonstrated the efficacy of the proposed metric learning algorithm on fund selection.
Spectral Inference Networks learn eigenfunctions from data using optimization.
problem Learning eigenfunctions of linear operators from data.
method Spectral Inference Networks generalize Slow Feature Analysis to generic symmetric operators and use stochastic optimization.
result Spectral Inference Networks accurately recover eigenfunctions and discover interpretable representations from video data.
Optimal proof of finite small eigenvalues for specific geometric manifolds.
problem Proving finiteness of small eigenvalues for geometrically finite manifolds.
method Analyzing the spectrum of the Laplace operator on geometrically finite rank one locally symmetric manifolds.
result Optimal proof of finite small eigenvalues in a specific interval.
Self-ONNs adapt nodal operators during training for higher diversity and efficiency.
problem Limited network heterogeneity and high computational demand in ONNs.
method Self-organized ONNs with generative neurons that adapt nodal operators during training.
result Self-ONNs achieve utmost heterogeneity and computational efficiency.
Accelerates stochastic optimization for convex and strongly convex problems.
problem Improving convergence rates in noisy stochastic optimization.
method Extends Catalyst approach to stochastic settings, handles inexact proximal operators.
result Achieves optimal worst-case complexity for noise-dominated regions.
NEON uses neural networks to optimize functions in infinite-dimensional spaces.
problem Optimizing composite functions in function spaces.
method NEON (Neural Epistemic Operator Networks) for sequential decision-making.
result NEON achieves state-of-the-art performance with fewer parameters.
New optimizers control network width scaling, improving stability and transfer across different model sizes.
problem Designing stable optimizers for networks of varying widths.
method Interpreting optimizers as steepest descent under mean-normalized operator norms, enabling layerwise composability and width-independent bounds.
result New optimizers like row normalization and column normalization provide stable learning-rate transfer across different model widths.
Optimal scaling found to depend on operator norm across large models and datasets.
problem Lack of unifying principle for optimal hyperparameter scaling across models and datasets.
method Discovered that optimal scaling is conditioned on the operator norm of the output layer.
result The optimal learning rate/batch size pair (η∗,B∗) consistently has the same operator norm value. ICON-OCnet solves optimal execution problems with neural networks and few examples.
problem Optimal order execution in markets with unknown price impact.
method Transformer-based neural network architecture (ICON-OCnet) that learns price impact from few examples and applies it to optimal execution strategies.
result ICON-OCnet accurately infers price impact models and retrieves optimal execution strategies for various propagator kernels.
Generative operators solve many convex problems with minimal parameters.
problem Worst-case parameter bounds limit the practical use of neural operators.
method Developed generative equilibrium operators (GEOs) using realizable finite-dimensional layers.
result GEOs can uniformly approximate solutions to convex optimization problems with logarithmic growth in parameters.
Paper proposes a new optimization framework for learning eigenfunctions of operators.
problem Computing eigenvalue decomposition of high-dimensional operators.
method Operator SVD with Neural Networks via Nested Low-Rank Approximation.
result Proposed method efficiently learns top-L singular values and functions in the correct order.
Proposes differentiable and sparse top-k operators for neural networks.
problem Discontinuity of top-k operator makes it unsuitable for end-to-end training with backpropagation.
method Formulates top-k as a linear program over permutahedron, introduces p-norm regularization, and uses isotonic optimization.
result Successfully applied to neural network pruning, fine-tuning, and routing.
New method solves blind inverse problems by optimizing both operator and image parameters.
problem Solving blind inverse problems with known forward operator.
method Parallel reverse diffusion guided by gradients from intermediate stages.
result State-of-the-art performance on blind deblurring and imaging through turbulence.
Researchers compute heat kernel coefficients for 2D diffusion operators.
problem Analyzing heat kernel coefficients for 2D hypoelliptic operators.
method Explicit computation of heat kernel coefficients and interpretation in terms of curvature.
result Interpretation of heat kernel asymptotics for non-sub-Riemannian operators.
Proposes glocal hypergradient estimation for hyperparameter optimization.
problem Combining reliability and efficiency in hyperparameter optimization.
method Uses Koopman operator theory to approximate global hypergradients from local ones.
result Achieves both reliability and efficiency in hyperparameter optimization.
Optimal transport for functional data using Hilbert-Schmidt operators.
problem Optimal transport for distributions on function spaces with partially represented stochastic maps.
method Regularization technique to restrict transport maps to Hilbert-Schmidt operators, developing an efficient algorithm.
result Existence, uniqueness, and consistency of the Hilbert-Schmidt operator estimate for the transport map.