CoNES optimizes blackbox functions using convex optimization and information geometry.
problem Optimizing high-dimensional blackbox functions efficiently.
method Formulated as a convex program that adapts evolutionary strategies gradient estimates.
result Vastly outperforms conventional blackbox optimization methods on benchmarks and MuJoCo tasks.
Novel evolutionary strategy solves stochastic constrained optimization problems.
problem Optimizing objective functions with stochastic constraints in reinforcement learning.
method Design of a novel optimization algorithm with a sufficient decrease mechanism for stochastic constrained problems.
result Demonstrated convergence of the algorithm on control tasks and constrained optimization problems.
In this note, we extend an evolutionary stochastic portfolio optimization framework to include probabilistic constraints. Both the stochastic programming-based modeling environment as well as the evolutionary optimization environment are ideally suited for an integration of various types of probabilistic constraints. W…
Proves FR-NGD optimally approximates evolutionary dynamics and continuous Bayesian inference.
problem Optimizing continuous time replicator equations and continuous Bayesian inference.
method Fisher-Rao natural gradient descent (FR-NGD) and its correspondence with evolutionary dynamics.
result FR-NGD optimally approximates continuous time replicator equations and continuous Bayesian inference.
New algorithm improves game learning with randomised optimism.
problem Learning in matrix games with unknown payoffs and bandit feedback.
method Integrates evolutionary algorithms into bandit framework for randomised optimism.
result Achieves sublinear regret, outperforming classical methods.
Combines variational and evolutionary optimization for generative models.
problem Optimizing generative models with discrete latent variables.
method Truncated posteriors as variational distributions, evolutionary algorithms applied to variational parameters.
result Evolutionary algorithms effectively optimize variational bounds for generative models.
Evolutionary algorithms improve neural network performance by discovering better activation functions.
problem The choice of activation function affects neural network performance, but ReLU remains dominant.
method Defined a tree-based search space of candidate activation functions and used evolutionary algorithms (mutation, crossover, exhaustive search) to explore and discover better functions.
result Replacing ReLU with evolved activation functions statistically significantly increases network accuracy.
Enhanced evolutionary algorithms solve NP-hard portfolio optimization with cardinality constraints.
problem Portfolio optimization under cardinality constraints with real-world conditions.
method Strengthened multi-objective evolutionary algorithms with new representations, operators, and repair mechanisms.
result The proposed algorithms converge faster and provide better approximations with no performance loss.
The paper improves evolutionary computation by optimizing selection rates.
problem Choosing the right selection rate in evolutionary computation.
method The paper proves mathematically that a selection rate of μ/λ leads to better progress rates and provides a theoretical basis for this.
result The theoretical selection rate μ/λ leads to a better progress rate of order O(λ^-1) compared to O(λ^-2/d).
Evolutionary algorithms improve decision tree ensembles.
problem Improving predictive performance of decision trees.
method Real-valued vector representation of decision trees, evolutionary algorithms (Differential evolution, Evolution strategies).
result Proposed methods outperform classical decision tree induction algorithms.
Many cooperative multiagent reinforcement learning environments provide agents with a sparse team-based reward, as well as a dense agent-specific reward that incentivizes learning basic skills. Training policies solely on the team-based reward is often difficult due to its sparsity. Furthermore, relying solely on the a…
Evolutionary Strategies optimize hyper-parameters for off-policy learning.
problem Hyper-parameter sensitivity in off-policy learning.
method Application of Evolutionary Strategies for online hyper-parameter tuning.
result Our method outperforms state-of-the-art baselines.
In many practical applications of clustering, the objects to be clustered evolve over time, and a clustering result is desired at each time step. In such applications, evolutionary clustering typically outperforms traditional static clustering by producing clustering results that reflect long-term trends while being ro…
BoGA combines evolutionary search with Bayesian optimization for efficient protein design.
problem Designing novel proteins with specific characteristics is challenging due to sequence space complexity.
method BoGA integrates a genetic algorithm with Bayesian optimization to efficiently explore sequence space.
result BoGA accelerates discovery of high-confidence binders for diverse protein design objectives.
Many applications in machine learning require optimizing a function whose true gradient is unknown, but where surrogate gradient information (directions that may be correlated with, but not necessarily identical to, the true gradient) is available instead. This arises when an approximate gradient is easier to compute t…
VisEvol uses evolutionary optimization to find optimal hyperparameters for machine learning models.
problem Finding the best hyperparameters for complex machine learning models is computationally intensive and challenging.
method VisEvol employs evolutionary optimization, storing performant models and improving others through crossover and mutation processes.
result VisEvol generates a voting ensemble of models with improved predictive performance.
EFS uses LLMs to optimize sparse portfolios by evolving alpha factors.
problem Sparse portfolio optimization in dynamic market regimes.
method Evolutionary feedback loop with LLM-generated alpha factors.
result Significantly outperforms baselines in diverse datasets.
We propose a population-based Evolutionary Stochastic Gradient Descent (ESGD) framework for optimizing deep neural networks. ESGD combines SGD and gradient-free evolutionary algorithms as complementary algorithms in one framework in which the optimization alternates between the SGD step and evolution step to improve th…
Evolutionary algorithm finds optimal pixel perturbations to improve neural network generalization.
problem Minimal data corruption by pixel modifications causes overfitting in neural networks.
method Evolutionary algorithm with a novel cost function to maximize generalization gap and domain divergence.
result Method outperforms previous pixel-based data distribution shift methods on CNNs.
In this paper we apply evolutionary optimization techniques to compute optimal rule-based trading strategies based on financial sentiment data. The sentiment data was extracted from the social media service StockTwits to accommodate the level of bullishness or bearishness of the online trading community towards certain…
Multi-stage financial decision optimization under uncertainty depends on a careful numerical approximation of the underlying stochastic process, which describes the future returns of the selected assets or asset categories. Various approaches towards an optimal generation of discrete-time, discrete-state approximations…
In this paper we present an evolutionary optimization approach to solve the risk parity portfolio selection problem. While there exist convex optimization approaches to solve this problem when long-only portfolios are considered, the optimization problem becomes non-trivial in the long-short case. To solve this problem…
MadEvolve optimizes trading algorithms using LLMs, achieving significant improvements in feature generation and trading strategy optimization.
problem Optimizing trading algorithms for better performance and feature generation.
method A framework inspired by Alpha-Evolve, using LLMs to evolve trading strategies and feature pipelines.
result Significant improvements in trading performance across various tasks, including feature generation and trading strategy optimization.
A large number of engineering, science and computational problems have yet to be solved in a computationally efficient way. One of the emerging challenges is how evolving technologies grow towards autonomy and intelligent decision making. This leads to collection of large amounts of data from various sensing and measur…
OpEvo automates tensor operator optimization for better efficiency.
problem Manual optimization of tensor operators is inefficient and limited.
method OpEvo uses evolutionary computation with topology-aware mutation.
result OpEvo finds optimal configurations with less effort and variance.
Paper proposes EMO-based AE generation for black-box settings.
problem Designing robust adversarial examples for black-box models.
method Evolutionary Multi-objective Optimization (EMO) for generating various types of adversarial examples.
result EMO-based method generates AEs that are robust and applicable to high-resolution images.
Evolutionary clustering aims at capturing the temporal evolution of clusters. This issue is particularly important in the context of social media data that are naturally temporally driven. In this paper, we propose a new probabilistic model-based evolutionary clustering technique. The Temporal Multinomial Mixture (TMM)…
The paper proposes a technique to speed up evolutionary algorithms by using lower-cost approximations of the objective function.
problem Evolutionary algorithms require many evaluations to solve computationally expensive black-box optimization problems.
method The paper introduces a technique to choose an appropriate approximate function cost during the execution of the optimization algorithm.
result The proposed approach can reach the same objective value in less than half the time in certain cases.
EPNE models evolving network patterns for better predictions.
problem Capturing evolving patterns in dynamic networks.
method EPNE models temporal network evolution using causal convolutions and a temporal objective function.
result EPNE outperforms other methods in various prediction tasks.
Evolution and learning are two of the fundamental mechanisms by which life adapts in order to survive and to transcend limitations. These biological phenomena inspired successful computational methods such as evolutionary algorithms and deep learning. Evolution relies on random mutations and on random genetic recombina…
Evolutionary strategy optimizes quantum circuit design and parameters.
problem Optimizing quantum circuit design and parameters for NISQ devices.
method Simple evolutionary strategy to optimize both circuit architecture and parameters.
result Minor slowdown on actual quantum hardware compared to simulations, with insights into mutation operations.
A real-time federated neural architecture search approach reduces costs and improves performance.
problem High communication and computational demands in federated learning for large models.
method Evolutionary approach with double-sampling technique to optimize model performance and reduce costs.
result Effective real-time federated neural architecture search for deep models on edge devices.
Time series prediction with deep learning methods, especially long short-term memory neural networks (LSTMs), have scored significant achievements in recent years. Despite the fact that the LSTMs can help to capture long-term dependencies, its ability to pay different degree of attention on sub-window feature within mu…
It is known that evolution strategies in continuous domains might not converge in the presence of noise. It is also known that, under mild assumptions, and using an increasing number of resamplings, one can mitigate the effect of additive noise and recover convergence. We show new sufficient conditions for the converge…
Proposes Genetic Thompson Sampling for multi-armed bandits, improving performance in nonstationary settings.
problem Improving sequential decision making tasks of online learning agents using multi-armed bandits.
method Integrates genetic principles into Thompson Sampling for multi-armed bandits.
result Significantly outperforms baselines in nonstationary settings.
A new approach for efficient batch multiobjective optimization using Thompson sampling.
problem Inefficient batch multiobjective optimization due to expensive oracles and hard inner optimization.
method Proposes a Thompson sampling approach (qextttPOTS) that chooses Pareto optimal candidates sequentially. result Empirically superior performance compared to classical evolutionary approaches and MOBO.
LRSAO uses RL to dynamically select and unlearn auxiliary objectives for EA optimization.
problem Optimizing complex landscapes with evolutionary algorithms.
method Local Reinforcement Learning (RL) to dynamically select and unlearn auxiliary objectives.
result Improved optimization complexity from O(n2log(n)/ℓ) to Θ(n2/ℓ2+nlog(n)). The paper analyzes cryptocurrency and equity markets using advanced statistical methods.
problem Comparing dynamics and strategies between cryptocurrency and equity markets.
method Random matrix theory, PCA, spectral dynamics, structural break analysis, portfolio simulation.
result Cryptocurrency and equity markets exhibit distinct evolutionary dynamics and time-varying sector behaviors.
Evolutionary methods improve neural network loss functions, reducing overfitting.
problem Improving neural network performance and preventing overfitting.
method Evolutionary computation to optimize loss functions, balancing error pull and overfitting push.
result Evolved loss functions effectively reduce overfitting, leading to better performance and robustness.
Evolutionary methods improve understanding of LLMs and their relationships.
problem Improving understanding of LLMs and their relationships.
method Relating weights to genotypes and output text to phenotypes using evolutionary methods.
result Estimated evolutionary trees reliably recover the topology of the ground-truth training tree.
Optimizes master faces for 2D and 3D face verification using evolutionary algorithms and neural networks.
problem Impersonation attacks using master faces for face-based identity authentication.
method Evolutionary algorithm in latent space of StyleGAN, neural network to direct search, 2D and 3D face reconstruction.
result Obtains high impersonation rates with fewer master faces for 2D and 3D face verification.
Deep neural network learning can be formulated as a non-convex optimization problem. Existing optimization algorithms, e.g., Adam, can learn the models fast, but may get stuck in local optima easily. In this paper, we introduce a novel optimization algorithm, namely GADAM (Genetic-Evolutionary Adam). GADAM learns deep …
evo-RL combines evolutionary computation with reinforcement learning for better adaptability.
problem Improving reinforcement learning algorithms' adaptability and performance in environments with rewardless states.
method Embedding reinforcement learning in an evolutionary cycle, distinguishing instinctive from learnable behavior.
result evo-RL leads to state-of-the-art performance on OpenAI Gym control problems with rewardless states.
Momentum speeds up evolutionary processes in machine learning.
problem Accelerating convergence in evolutionary dynamics.
method Combining momentum from machine learning with evolutionary dynamics using information divergences as Lyapunov functions.
result Momentum accelerates convergence of evolutionary dynamics, including the replicator equation and Euclidean gradient descent.
In [1], we have explored the theoretical aspects of feature selection and evolutionary algorithms. In this chapter, we focus on optimization algorithms for enhancing data analytic process, i.e., we propose to explore applications of nature-inspired algorithms in data science. Feature selection optimization is a hybrid …
In this article, we present the elitist particle filter based on evolutionary strategies (EPFES) as an efficient approach for nonlinear system identification. The EPFES is derived from the frequently-employed state-space model, where the relevant information of the nonlinear system is captured by an unknown state vecto…
We define Lie algebroids over infinite jet spaces and establish their equivalent representation through homological evolutionary vector fields.
Multi-task learning uses auxiliary data or knowledge from relevant tasks to facilitate the learning in a new task. Multi-task optimization applies multi-task learning to optimization to study how to effectively and efficiently tackle multiple optimization problems simultaneously. Evolutionary multi-tasking, or multi-fa…