Latest algorithms for automatic neural architecture search perform remarkable but are basically directionless in search space and computational expensive in training of every intermediate architecture. In this paper, we propose a method for efficient architecture search called EENA (Efficient Evolution of Neural Archit…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “evolution search”
Differentiable NAS frameworks grow networks wider and deeper, revealing biases in wiring evolution.
problem Understanding the evolution of neural architecture wiring in differentiable NAS methods.
method Unified view on searching algorithms, local cost minimization, empirical and theoretical analyses.
result Implicit inductive biases cause observed searching patterns in differentiable NAS methods.
NGE uses neural graphs to efficiently design robots.
problem Designing robots is hard due to combinatorial search space and evaluation costs.
method Formulated as graph search, NGE uses neural networks for policy parameterization and graph mutation with uncertainty.
result NGE significantly outperforms previous methods, discovering kinematically preferred structures.
Context: One of the black arts of data mining is learning the magic parameters which control the learners. In software analytics, at least for defect prediction, several methods, like grid search and differential evolution (DE), have been proposed to learn these parameters, which has been proved to be able to improve t…
Gradients help find global optima in complex functions.
problem Finding global optima in functions with many local minima.
method A principle for generating search directions from non-local quadratic approximants based on gradients.
result The proposed algorithm and CMA-ES perform better than random reinitialized BFGS.
Guided Evolution improves NAS efficiency and accuracy.
problem NAS methods converge to local minima and are complex.
method G-EA: guided evolutionary approach with initialization evaluation and continuous knowledge extraction.
result G-EA achieves state-of-the-art results in CIFAR-10, CIFAR-100, and ImageNet16-120.
This tutorial introduces the CMA Evolution Strategy (ES), where CMA stands for Covariance Matrix Adaptation. The CMA-ES is a stochastic, or randomized, method for real-parameter (continuous domain) optimization of non-linear, non-convex functions. We try to motivate and derive the algorithm from intuitive concepts and …
Contrastive embeddings improve neural architecture search performance.
problem Improving performance of neural architecture search algorithms.
method Contrastive learning to identify networks based on data Jacobians and produce embeddings.
result Traditional black-box optimization algorithms can reach state-of-the-art performance with contrastive embeddings.
This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner. Unlike conventional approaches of applying evolution or reinforcement learning over a discrete and non-differentiable search space, our method is based on the continuous relaxation of the architectu…
This work tackles the problem of robust zero-shot planning in non-stationary stochastic environments. We study Markov Decision Processes (MDPs) evolving over time and consider Model-Based Reinforcement Learning algorithms in this setting. We make two hypotheses: 1) the environment evolves continuously with a bounded ev…
EFS: Evolutionary Factor Searching for Sparse Portfolio Optimization Using Large Language Modelsq-fin.PM
EFS uses LLMs to optimize sparse portfolios by evolving alpha factors.
problem Sparse portfolio optimization in dynamic market regimes.
method Evolutionary feedback loop with LLM-generated alpha factors.
result Significantly outperforms baselines in diverse datasets.
MoGA optimizes mobile networks for GPU efficiency.
problem Lack of GPU-aware optimization in mobile network design.
method Proposes MoGA, a GPU-aware neural architecture search.
result Achieves higher accuracy with similar latency constraints.
Backlund transformations are used to search for solutions, particularly soliton solutions, of non-linear differential equations. In this paper we present an invariant geometrical theory of Backlund transformations for second order evolution equations with one space variable. The main concept is that of connection defin…
Paper proposes MCTSPO for better reinforcement learning policy optimization.
problem Local optima and saddle points in gradient-based methods and poor initialization in gradient-free methods.
method Monte-Carlo tree search combined with gradient-free optimization.
result Improved performance on reinforcement learning tasks with deceptive or sparse reward functions.
NEMoTS improves time series analysis by deriving efficient, interpretable models.
problem Lack of comprehensive understanding and insightful explanations in time series analysis.
method Neural-enhanced Monte-Carlo Tree Search (NEMoTS) for symbolic regression.
result NEMoTS provides efficient and interpretable models for time series analysis.
We propose Stochastic Neural Architecture Search (SNAS), an economical end-to-end solution to Neural Architecture Search (NAS) that trains neural operation parameters and architecture distribution parameters in same round of back-propagation, while maintaining the completeness and differentiability of the NAS pipeline.…
Paper proposes a method to monitor research topic evolution.
problem Difficulty in tracking research topic diffusion and evolution.
method Deep Non-negative Autoencoder with information divergence measurement.
result Identifies evolution of research topics and discovers topic diffusions.
Simpler neural architecture search method using random architectures and regression.
problem Complex algorithms in neural architecture search.
method Train N random architectures, use them to train a regression model, predict validation accuracies, and select top-K architectures.
result More sample efficient and competitive with complex approaches.
We present a broad agenda for meaningful banking regulation reform aiming the creation of evolutive competitive environment to maximize the effectiveness of international financial system through the introduction of fair competition process among the banks in free market capitalism. We assume that the international fin…
NPENAS improves neural architecture search efficiency and accuracy.
problem Efficient and accurate neural architecture search (NAS) for minimizing search costs.
method Proposes NPENAS, a neural predictor guided evolutionary algorithm that enhances exploration ability of evolutionary algorithms.
result NPENAS-BO and NPENAS-NP outperform existing NAS algorithms on NASBench-201, NASBench-101, and DARTS.
NVA combines variational posteriors, annealing, and natural-gradient learning for multimodal optimization.
problem Finding multiple global and local modes in nonconvex objectives.
method NVA integrates variational posteriors, annealing, and natural-gradient learning.
result NVA outperforms gradient descent and evolution strategies on simulations and real-world problems.
A new ES method improves reinforcement learning speed and accuracy.
problem Slow convergence and local maxima in reinforcement learning.
method Directional Gaussian Smoothing Evolution Strategy (DGS-ES)
result DGS-ES accelerates RL training with high accuracy and nonlocal search direction.
Predict accuracy of neural architectures using non-neural models.
problem Improving efficiency and effectiveness of neural architecture search.
method Used gradient boosting decision tree (GBDT) for accuracy prediction and pruned the search space based on features learned from GBDT.
result GBDT-based predictor achieves comparable or better accuracy than neural network-based predictors and is 22x more sample efficient on NASBench-101.
Neural Architecture Search (NAS) has shown great success in automating the design of neural networks, but the prohibitive amount of computations behind current NAS methods requires further investigations in improving the sample efficiency and the network evaluation cost to get better results in a shorter time. In this …
Neural Architecture Search aims at automatically finding neural architectures that are competitive with architectures designed by human experts. While recent approaches have achieved state-of-the-art predictive performance for image recognition, they are problematic under resource constraints for two reasons: (1)the ne…
Natural Evolution Strategiesstat.ML
This paper presents Natural Evolution Strategies (NES), a recent family of algorithms that constitute a more principled approach to black-box optimization than established evolutionary algorithms. NES maintains a parameterized distribution on the set of solution candidates, and the natural gradient is used to update th…
Enhances inverse design optimization with machine learning and reduced fidelity simulations.
problem Limited compute resources in inverse design optimization.
method Synergy of multi-fidelity simulations, machine learning, and search space reduction.
result Significant computational resource savings and improved optimization performance.
Contextual policy search (CPS) is a class of multi-task reinforcement learning algorithms that is particularly useful for robotic applications. A recent state-of-the-art method is Contextual Covariance Matrix Adaptation Evolution Strategies (C-CMA-ES). It is based on the standard black-box optimization algorithm CMA-ES…
As an emerging field, Automated Machine Learning (AutoML) aims to reduce or eliminate manual operations that require expertise in machine learning. In this paper, a graph-based architecture is employed to represent flexible combinations of ML models, which provides a large searching space compared to tree-based and sta…
We investigate the use of attentional neural network layers in order to learn a `behavior characterization' which can be used to drive novelty search and curiosity-based policies. The space is structured towards answering a particular distribution of questions, which are used in a supervised way to train the attentiona…
Study uses web search data to analyze tech startups growth.
problem Analyzing growth dynamics of tech startups.
method Utilized Google Trends data for 241 US-based tech startups.
result Web search traffic correlates positively with tech startup growth.
AutoML discovers complete machine learning algorithms from basic operations.
problem Automating the discovery of machine learning algorithms from scratch.
method Evolutionary search on a generic search space of basic mathematical operations.
result Simple neural networks can be surpassed by evolving directly on tasks of interest.
New model predicts neural network performance from early training epochs, incorporating architecture impact.
problem Predicting neural network performance from early training epochs, neglecting architecture impact.
method Architecture-aware graph ordinary differential equation model.
result Model outperforms state-of-the-art methods for MLP and CNN learning curves.
In iterative supervised learning algorithms it is common to reach a point in the search where no further induction seems to be possible with the available data. If the search is continued beyond this point, the risk of overfitting increases significantly. Following the recent developments in inductive semantic stochast…
CMA-ME combines CMA-ES and MAP-Elites for better quality and diversity in continuous domains.
problem Finding a diverse set of high-quality solutions in complex continuous domains.
method Combines CMA-ES self-adaptation with MAP-Elites archiving and mapping.
result CMA-ME outperforms MAP-Elites in both quality and diversity of solutions.
Due to recent explosion of text data, researchers have been overwhelmed by ever-increasing volume of articles produced by different research communities. Various scholarly search websites, citation recommendation engines, and research databases have been created to simplify the text search tasks. However, it is still d…
Proposes EDESH-SA for better inventory management under uncertainty.
problem Inventory management under uncertainty.
method Ensemble Differential Evolution with simulation-based hybridization and self-adaptation.
result Improves financial performance and optimizes search spaces.
Non-parametric Determination of Real-Time Lag Structure between Two Time Series: the "Optimal Thermal Causal Path" Methodcond-mat.stat-mech
We introduce a novel non-parametric methodology to test for the dynamical time evolution of the lag-lead structure between two arbitrary time series. The method consists in constructing a distance matrix based on the matching of all sample data pairs between the two time series. Then, the lag-lead structure is searched…
NAS improves gliomas segmentation on MRI scans.
problem Designing optimal deep learning architectures for brain tumor segmentation.
method Neural architecture search with probabilistic parameter learning for MRI brain tumor segmentation.
result Two optimal neural architectures for brain tumor segmentation were discovered.
Time series prediction with deep learning methods, especially long short-term memory neural networks (LSTMs), have scored significant achievements in recent years. Despite the fact that the LSTMs can help to capture long-term dependencies, its ability to pay different degree of attention on sub-window feature within mu…
-invariant Laplacian flowmath.DG
The study explores -invariant Laplacian flow on 6-manifolds.
problem Finding -invariant solutions to the Laplacian flow.
method Derive and analyze -invariant evolution equations.
result Discovery of first inhomogeneous shrinking solitons.
Paper proposes verifier engineering for improving foundation models.
problem Challenges in providing effective supervision signals for foundation models.
method Leverages automated verifiers to perform verification tasks and deliver feedback.
result Verifier engineering can enhance foundation models' capabilities.
Improved genetic programming by optimizing mutation operators for continuous program search.
problem Small syntactic mutations in genetic programming can lead to unpredictable behavioral shifts.
method Learned a compact trading-strategy DSL, created a block-factorized embedding, and designed geometry-compiled mutation operators.
result Geometry-compiled mutation operators discover strong strategies using fewer evaluations and achieve higher Sharpe ratios.
Novel RL-based NPG improves multi-objective NAS efficiency and performance.
problem Discovering optimal neural architectures with multiple conflicting objectives.
method Non-stationary policy gradient with adaptive reward functions and shared model.
result Framework efficiently approximates full Pareto front and achieves superior performance.
Investment strategies in financial markets can lead to instability due to market impacts.
problem Market impacts make it impossible for investors to accurately optimize their strategies.
method Built an agent-based model with technical analysis strategy agents to investigate optimization instability.
result Investment strategies' parameters never converged but continued to change, leading to unstable market price evolution.
Evolutionary algorithms improve neural network performance by discovering better activation functions.
problem The choice of activation function affects neural network performance, but ReLU remains dominant.
method Defined a tree-based search space of candidate activation functions and used evolutionary algorithms (mutation, crossover, exhaustive search) to explore and discover better functions.
result Replacing ReLU with evolved activation functions statistically significantly increases network accuracy.
Study on adversarial training dynamics in high dimensions using SGD.
problem Analyzing adversarial training of models in high-dimensional settings.
method Deriving deterministic equivalents for SGD iterates under Gaussian mixtures.
result No constant learning rate guarantees monotone descent in adversarial training.
AgEBO-Tabular combines NAS and hyperparameter tuning for fast, high-performing tabular models.
problem Developing high-performing predictive models for large tabular data sets is challenging.
method Combines aging evolution NAS and asynchronous Bayesian optimization for hyperparameter tuning in data-parallel training.
result Automatically discovered neural network models outperform state-of-the-art AutoML ensembles in inference speed by two orders of magnitude.