A new optimizer, MVO, improves nonlinear regression performance.
problem Finding optimal coefficients in nonlinear regression models.
method Multi-Verse Optimizer (MVO) compared to Particle Swarm Optimizer (PSO).
result MVO statistically outperforms PSO in 10 nonlinear regression problems.
This paper tackles efficient optimization for nonlinear embeddings in similarity learning.
problem Learning similarity with nonlinear embeddings is challenging due to the large number of pairs.
method Detailed derivations and efficient optimization methods for nonlinear embeddings are developed.
result Efficient optimization methods for nonlinear embeddings are shown to be highly effective.
Unified analysis for nonlinear parametric models in Bayesian optimization.
problem Limited theoretical guarantees for nonlinear parametric models in Bayesian optimization.
method Kernel-based framework for analyzing regularized nonlinear parametric models trained on adaptively collected data.
result Unified convergence guarantees for nonlinear acquisition and surrogate models.
Optimal trading strategy derived for nonlinear price impact models.
problem Optimal trading with nonlinear price impact induced by alpha signals.
method Variational approach, nonlinear Fredholm equation, iterative scheme.
result Existence and uniqueness of optimal trading strategy under monotonicity condition.
Optimized nonlinearities enhance generalization in random feature models.
problem Improving generalization performance in random feature models.
method Analyzed and defined optimal nonlinearities from Gaussian model parameters.
result Optimized nonlinearities achieve better generalization performance than ReLU.
We find a convex model for traditional nonlinear regression under L2 loss.
problem Nonlinear regression under L2 loss with non-convex optimization.
method Showed a convex nonlinear regression model for least squares problem.
result Existence of a convex model simplifies training complex systems.
NCG methods improve shape optimization efficiency.
problem Shape optimization problems
method Nonlinear conjugate gradient methods
result NCG methods are efficient for shape optimization
New algorithms optimize nonlinear Kalman filtering using divergence measures.
problem Nonlinear Kalman filtering without closed-form solutions.
method Proposes novel algorithms for KL and α-divergence optimization.
result Improves performance on radar and sensor tracking problems.
Estimates input from output of nonlinear systems using ANN.
problem Estimating unknown compositional input from system output.
method Artificial Neural Networks (ANNs) for nonlinear system inversion.
result ANNs can compete with optimal bounds for linear systems and demonstrate promising results for nonlinear systems.
The paper tackles data-driven optimal control of unknown nonlinear systems using RKHS.
problem Unknown nonlinear dynamics and stage cost functions.
method Embed state densities into RKHS, learn Markov operators, solve Hamilton-Jacobi-Bellman recursions.
result Solves a wide range of nonlinear control problems, including depth regulation.
Novel algorithm optimizes decision trees for nonlinear metrics.
problem Optimizing decision trees for nonlinear metrics like F1-score.
method Bi-objective optimisation approach to find optimal trees on Pareto frontier.
result The optimal tree for nonlinear metrics lies on the Pareto frontier.
Proposes a new derivative concept for nonlinear DRO problems.
problem Optimizing nonlinear functions in probability space with distributionally robust optimization.
method Introduces Gateaux derivative for smoothness and proposes a Frank-Wolfe algorithm.
result Validates theoretical results on portfolio selection problems with numerical validation.
New neural architectures with multivariate nonlinearities are optimal in function space.
problem Optimality of neural architectures with multivariate nonlinearities.
method Construction of Banach spaces via k k k -plane transform and sparsity-promoting norm, proving representer theorem. result Neural architectures with multivariate nonlinearities are optimal in function space.
Method solves complex optimization problems with high probability bounds.
problem Nonlinear equality constrained stochastic optimization problems.
method Step-search sequential quadratic programming method.
result High-probability bound on iteration complexity for first-order stationarity.
Policy gradient converges to globally optimal policy in nearly linear-quadratic systems.
problem Finding optimal policies in nonlinear control systems with partial information.
method Policy gradient algorithm designed for nearly linear-quadratic regulators with small Lipschitz nonlinear components.
result Policy gradient algorithm converges to globally optimal policy with linear rate.
New method tackles nonlinear, infinite-dimensional signal processing problems.
problem Nonlinear, infinite-dimensional signal processing challenges.
method Directly addresses continuous, nonlinear problems as sparse functional optimization.
result Proves no duality gap for non-atomic problems, allowing efficient solution.
Study optimal investment and consumption in incomplete markets with nonlinear expectations.
problem Utility maximization in incomplete markets with general constraints.
method Utilizes g g g -martingale method to solve optimization problem for various utility functions. result Characterizes optimal investment-consumption strategy through quadratic BSDE solutions.
Optimizes exploration for nonlinear systems to learn controllers efficiently.
problem Learning optimal controllers for unknown nonlinear systems.
method Formally quantifies which parameters are most critical, and develops an algorithm to efficiently explore these parameters.
result Proves a near-instance-optimal rate for learning controllers.
Metaheuristics improve yield curve estimation for Costa Rica.
problem Estimating the yield curve for Costa Rica using historical data.
method Used Nelson-Siegel and Svensson models with four metaheuristics (Ant colony, Genetic, Particle Swarm, Simulated Annealing) for optimization.
result Metaheuristics achieved better results than classical methods, especially Particle Swarm and Simulated Annealing.
In this paper, we present a generic framework to extend existing uniformly optimal convex programming algorithms to solve more general nonlinear, possibly nonconvex, optimization problems. The basic idea is to incorporate a local search step (gradient descent or Quasi-Newton iteration) into these uniformly optimal conv…
Greedy MI maximization method outperforms existing approaches in nonlinear models.
problem Maximizing mutual information in nonlinear models with non-Gaussian noise.
method Greedy approaches based on log-Sobolev inequalities for computationally inexpensive MI lower bounds.
result Proposed method outperforms random selection and Gaussian approximations.
Study optimal control in unknown nonlinear systems with near-optimal regret bound.
problem Sequential control in unknown, nonlinear dynamical systems.
method LC^3 algorithm, based on information theory.
result Near-optimal O ( T ) O(\sqrt{T}) O ( T ) regret bound for episodic settings. This paper analyzes a simplified strategy for nonlinear control using local linear models and iLQR updates.
problem Nonlinear policy optimization in control systems.
method Iterative estimation of local linear models and iLQR-like policy updates.
result Demonstrates polynomial sample complexity and overcomes exponential problem horizon dependence.
Gradient descent and SGD solve nonlinear inverse problems efficiently.
problem Solving nonlinear inverse problems with random design.
method Gradient descent and SGD with mini-batching, under classical assumptions.
result Achieves optimal convergence rates in RKHS framework.
Optimizes learning from experiments with nonlinear belief models.
problem Maximizing expected value of information with unknown nonlinear parameters.
method Uses sampled approximation to guide experiments and resampling to adapt to new information.
result The method converges to true parameters while maximizing the metric.
New method shapes constellation to reduce fiber nonlinearities.
problem Nonlinear fiber effects in deep learning.
method Unsupervised machine learning for constellation optimization.
result Gains up to 0.13 bit/4D in fiber nonlinearities.
NLCG optimizes DNN training, especially with large mini-batches.
problem Improving convergence speed in large-scale DNN training.
method Stochastic Preconditioned Nonlinear Conjugate Gradient (SP-NLCG) algorithm.
result NLCG improves DNN training accuracy by over 10 percentage points at large mini-batch sizes.
New algorithm optimizes nonlinear SDEs online with convergence guarantees.
problem Optimizing nonlinear stochastic differential equations (SDEs) is computationally challenging.
method Forward propagation algorithm that solves an SDE derived using forward differentiation.
result Convergence theorem for nonlinear dissipative SDEs with bounds on stochastic fluctuations.
Proposes a new deep neural network training scheme combining different loss functions.
problem Improving generalization ability of deep neural networks.
method Integrates multiple loss functions in a nonlinear manner.
result The new objective function enhances optimization and generalization.
ResNets with depth and nonlinearity avoid bad local minima.
problem Avoiding bad local minima in deep learning models.
method Proving depth and nonlinearity in ResNets create no bad local minima.
result ResNets with depth and nonlinearity have values no worse than global minimum and can improve further.
A new DDR framework learns low-dimensional data representations using dynamical systems.
problem Learning efficient low-dimensional data representations.
method DDR framework based on nonlinear dynamical systems, using linear combinations of functions and regularization.
result DDR method outperforms other methods on synthetic and real datasets.
The paper develops adaptive deep learning methods for nonlinear time series models.
problem Estimating mean functions of non-stationary and nonlinear time series models.
method Develops non-penalized and sparse-penalized DNN estimators for general non-stationary time series, derives minimax lower bounds, and shows the sparse-penalized DNN estimator is adaptive and optimal.
result Sparse-penalized DNN estimator achieves minimax optimal rates for many nonlinear AR models.
Two new Koopman models improve nonlinear system prediction.
problem Predicting nonlinear, nonconvex dynamic systems.
method Convex and Extended Koopman Models using deep learning.
result Significantly improved predictive performance.
Framework for designing nonlinearities in neural networks with slope constraints.
problem Designing nonlinearities with specific properties for signal processing.
method Variational framework with regularization for slope constraints and optimization of adaptive splines.
result Adaptive nonuniform linear splines achieve global optimum in constrained optimization.
New method recovers matrices with nonlinear structures using optimization on Grassmann manifold.
problem Recovering high-rank matrices with nonlinear structures like subspaces or clusters.
method Formulated as rank minimization of a nonlinear feature map, approximated by constrained non-convex optimization on the Grassmann manifold, using Riemannian and alternating minimization schemes.
result Global convergence and worst-case complexity bounds for alternating minimization scheme, leading to unique limit point.
GO-OED maximizes predictive information gain on nonlinear QoIs.
problem Maximizing information gain on nonlinear predictive quantities.
method Nested Monte Carlo estimator, Markov chain Monte Carlo, kernel density estimation, Bayesian optimization.
result GO-OED outperforms conventional OED in nonlinear settings.
New method designs experiments robustly for nonlinear estimation, improving parameter knowledge.
problem Designing robust experiments for nonlinear estimation under parametric uncertainty.
method Multi-stage robust optimization framework for sequential experiments.
result Identifies experiments better conducted early for improved parameter knowledge.
A hybrid algorithm combines optimization and enumeration for symbolic regression.
problem Finding any function from a set of operators without prior specification.
method Mixed-integer nonlinear optimization with explicit enumeration and constraints.
result The hybrid algorithm is competitive with state-of-the-art methods.
JULIA combines multi-linear and nonlinear models for tensor completion.
problem Complex patterns in real-world tensors require a unified model.
method JULIA unifies multi-linear and nonlinear models with flexible component assignment and efficient alternating optimization.
result JULIA outperforms existing methods in large-scale tensor completion.
Investigates stock correlations during market crises, finds nonlinear dependencies increase, and optimizes portfolios.
problem Investigating stock correlations during market crises.
method Pearson correlation and mutual information based complex networks, surrogate data for nonlinear dependencies, Markowitz mean variance portfolio optimization.
result Nonlinear dependencies increase during financial market crises, not reducing to linear correlations.
Linear principal component analysis (PCA) can be extended to a nonlinear PCA by using artificial neural networks. But the benefit of curved components requires a careful control of the model complexity. Moreover, standard techniques for model selection, including cross-validation and more generally the use of an indepe…
Develops a new duality between entropy martingale optimal transport and nonlinear pricing-hedging.
problem Entropy Martingale Optimal Transport problem and its associated optimization problem.
method Combines Entropy Optimal Transport and Martingale Optimal Transport theories, with novel penalization terms and constraints.
result Establishes a nonlinear robust pricing-hedging duality, covering various known robust results.
Method solves high-dimensional nonlinear PDEs using neural networks.
problem Solving high-dimensional fully nonlinear PDEs.
method Backward induction with multi-layer neural networks to estimate solution and its gradient, with Hessian approximated by automatic differentiation.
result Method extends previous work on semi-linear PDEs to fully nonlinear cases, demonstrating accuracy on various examples.
Dual optimization connects ERM-fDR to normalization function.
problem Empirical risk minimization with f-divergence regularization.
method Dual formulation, Legendre-Fenchel transform, implicit function theorem, nonlinear ODE.
result Computational method to calculate normalization function efficiently.
The paper analyzes equity market dynamics and optimal portfolios using time-varying optimization.
problem Analyzing the time-varying structure of equity markets, particularly market capitalization inequality and concentration.
method The study employs mathematical functionals of time-varying portfolios and a Sharpe optimization procedure.
result Optimal portfolios exhibit varying market capitalization exposure over time.
New algorithm tackles stochastic optimization with inequality constraints.
problem Stochastic optimization with inequality constraints in various applications.
method Active-set stochastic sequential quadratic programming (StoSQP) with a differentiable exact augmented Lagrangian.
result Global convergence for any initialization, KKT residuals converge to zero almost surely.
Optimizes experiment design for nonlinear systems with exact confidence regions.
problem Optimizing experiment design for nonlinear systems with exact confidence regions.
method Explicitly considers exact confidence regions, uses inner- and outer-approximating ellipsoids, and solves as a bilevel optimization problem.
result Optimal designs provide more informative measurements than linearized approaches.
We solve the Dirichlet problem for fully nonlinear elliptic equations on Riemannian manifolds under essentially optimal structure conditions, especially with no restrictions to the curvature of the underlying manifold and the second fundamental form of its boundary. The main result (Theorem 1.1) includes a new (and opt…