A method to control results of gradient descent unsupervised learning in a deep neural network by using evolutionary algorithm is proposed. To process crossover of unsupervisedly trained models, the algorithm evaluates pointwise fitness of individual nodes in neural network. Labeled training data is randomly sampled an…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Evolutionary Neural Networks”
Evolutionary algorithms improve neural network performance by discovering better activation functions.
problem The choice of activation function affects neural network performance, but ReLU remains dominant.
method Defined a tree-based search space of candidate activation functions and used evolutionary algorithms (mutation, crossover, exhaustive search) to explore and discover better functions.
result Replacing ReLU with evolved activation functions statistically significantly increases network accuracy.
Evolutionary methods improve understanding of LLMs and their relationships.
problem Improving understanding of LLMs and their relationships.
method Relating weights to genotypes and output text to phenotypes using evolutionary methods.
result Estimated evolutionary trees reliably recover the topology of the ground-truth training tree.
Evolutionary algorithm finds optimal pixel perturbations to improve neural network generalization.
problem Minimal data corruption by pixel modifications causes overfitting in neural networks.
method Evolutionary algorithm with a novel cost function to maximize generalization gap and domain divergence.
result Method outperforms previous pixel-based data distribution shift methods on CNNs.
Neural Architecture Search has shown potential to automate the design of neural networks. Deep Reinforcement Learning based agents can learn complex architectural patterns, as well as explore a vast and compositional search space. On the other hand, evolutionary algorithms offer higher sample efficiency, which is criti…
LR-EDNN reduces PDE solver complexity by limiting network weights to low-rank subspace.
problem Efficiently solving time-dependent PDEs with deep neural networks.
method Low-rank constraint on network weights using SVD for efficient parameter updates.
result LR-EDNN achieves comparable accuracy to full EDNN with fewer parameters and lower cost.
Evolutionary methods improve neural network loss functions, reducing overfitting.
problem Improving neural network performance and preventing overfitting.
method Evolutionary computation to optimize loss functions, balancing error pull and overfitting push.
result Evolved loss functions effectively reduce overfitting, leading to better performance and robustness.
EvoNUDGE uses graph neural networks to improve genetic programming performance.
problem Efficiency in evolutionary computation for problem solving.
method Graph neural network to elicit additional knowledge from symbolic regression problems.
result EvoNUDGE significantly outperforms conventional and neural genetic programming methods.
A hybrid method improves Convolutional Neural Networks training.
problem Training Convolutional Neural Networks efficiently and avoiding local minima.
method Combines backpropagation with evolutionary strategies.
result Improves accuracy by 0.61% on CIFAR-10 image classification.
There has been significant recent interest towards achieving highly efficient deep neural network architectures. A promising paradigm for achieving this is the concept of evolutionary deep intelligence, which attempts to mimic biological evolution processes to synthesize highly-efficient deep neural networks over succe…
A neural network and evolutionary algorithm framework designs nonlinear optical molecules.
problem Designing efficient nonlinear optical materials.
method Multi-stage Bayesian neural network (msBNN) and corrected Lewis-mode group contribution method (cLGC) combined with evolutionary algorithm (EA).
result Accurately and efficiently designs molecules with different optical properties using a small data set.
Neural networks play an increasingly important role in the field of machine learning and are included in many applications in society. Unfortunately, neural networks suffer from adversarial samples generated to attack them. However, most of the generation approaches either assume that the attacker has full knowledge of…
Deep neural networks continue to show improved performance with increasing depth, an encouraging trend that implies an explosion in the possible permutations of network architectures and hyperparameters for which there is little intuitive guidance. To address this increasing complexity, we propose Evolutionary DEep Net…
Neuro-Evolution is a field of study that has recently gained significantly increased traction in the deep learning community. It combines deep neural networks and evolutionary algorithms to improve and/or automate the construction of neural networks. Recent Neuro-Evolution approaches have shown promising results, rival…
A real-time federated neural architecture search approach reduces costs and improves performance.
problem High communication and computational demands in federated learning for large models.
method Evolutionary approach with double-sampling technique to optimize model performance and reduce costs.
result Effective real-time federated neural architecture search for deep models on edge devices.
BEKAN uses RBFs and evolutionary methods to solve PDEs with boundary conditions.
problem Enforcing boundary conditions in neural networks for PDE solutions.
method Boundary condition-guaranteed evolutionary Kolmogorov-Arnold Network (BEKAN) with radial basis functions (RBFs). Incorporates Dirichlet, periodic, and Neumann conditions.
result BEKAN outperforms MLP and B-splines KAN in solving PDEs with boundary conditions.
We propose a population-based Evolutionary Stochastic Gradient Descent (ESGD) framework for optimizing deep neural networks. ESGD combines SGD and gradient-free evolutionary algorithms as complementary algorithms in one framework in which the optimization alternates between the SGD step and evolution step to improve th…
Automates design of lightweight neural networks for image classification.
problem Designing efficient neural networks for edge devices with limited computational resources.
method Uses the Mesh Adaptive Direct Search (MADS) algorithm to optimize network architecture.
result Achieves comparable performance to standard methods with fewer design trials.
NHC learns scalable algorithmic solutions from diverse tasks.
problem Neural networks struggle to learn algorithmic strategies.
method Memory-augmented network with abstraction mechanism and evolutionary training.
result Reliable learning of robust and scalable algorithmic solutions.
This paper is a review of the evolutionary history of deep learning models. It covers from the genesis of neural networks when associationism modeling of the brain is studied, to the models that dominate the last decade of research in deep learning like convolutional neural networks, deep belief networks, and recurrent…
Taking inspiration from biological evolution, we explore the idea of "Can deep neural networks evolve naturally over successive generations into highly efficient deep neural networks?" by introducing the notion of synthesizing new highly efficient, yet powerful deep neural networks over successive generations via an ev…
New bound for neural nets on non-iid data.
problem Generalization of deep nets for dependent data.
method Establishes a generalization bound for feed-forward neural networks on -mixing data.
result Proves neural nets can generalize well on non-iid data.
Deep neural network learning can be formulated as a non-convex optimization problem. Existing optimization algorithms, e.g., Adam, can learn the models fast, but may get stuck in local optima easily. In this paper, we introduce a novel optimization algorithm, namely GADAM (Genetic-Evolutionary Adam). GADAM learns deep …
The accurate and interpretable prediction of future events in time-series data often requires the capturing of representative patterns (or referred to as states) underpinning the observed data. To this end, most existing studies focus on the representation and recognition of states, but ignore the changing transitional…
NPENAS improves neural architecture search efficiency and accuracy.
problem Efficient and accurate neural architecture search (NAS) for minimizing search costs.
method Proposes NPENAS, a neural predictor guided evolutionary algorithm that enhances exploration ability of evolutionary algorithms.
result NPENAS-BO and NPENAS-NP outperform existing NAS algorithms on NASBench-201, NASBench-101, and DARTS.
Optimizes master faces for 2D and 3D face verification using evolutionary algorithms and neural networks.
problem Impersonation attacks using master faces for face-based identity authentication.
method Evolutionary algorithm in latent space of StyleGAN, neural network to direct search, 2D and 3D face reconstruction.
result Obtains high impersonation rates with fewer master faces for 2D and 3D face verification.
Federated learning is an emerging technique used to prevent the leakage of private information. Unlike centralized learning that needs to collect data from users and store them collectively on a cloud server, federated learning makes it possible to learn a global model while the data are distributed on the users' devic…
DyHATR learns dynamic heterogeneous networks for better link prediction.
problem Learning effective representations of dynamic heterogeneous networks for link prediction.
method Hierarchical attention for heterogeneous information and temporal RNN for evolutionary patterns.
result DyHATR significantly outperforms state-of-the-art baselines on link prediction tasks.
Regression or classification? This is perhaps the most basic question faced when tackling a new supervised learning problem. We present an Evolutionary Deep Learning (EDL) algorithm that automatically solves this by identifying the question type with high accuracy, along with a proposed deep architecture. Typically, a …
Time series prediction with deep learning methods, especially long short-term memory neural networks (LSTMs), have scored significant achievements in recent years. Despite the fact that the LSTMs can help to capture long-term dependencies, its ability to pay different degree of attention on sub-window feature within mu…
EPNE models evolving network patterns for better predictions.
problem Capturing evolving patterns in dynamic networks.
method EPNE models temporal network evolution using causal convolutions and a temporal objective function.
result EPNE outperforms other methods in various prediction tasks.
New method speeds up image denoising models without sacrificing performance.
problem Efficiently training models for image denoising tasks.
method Introduces superkernel techniques for fast training of dense prediction models.
result Demonstrates effectiveness on SIDD+ benchmark with 6-8 RTX2080 GPU hours.
Evolution in Groups: A deeper look at synaptic cluster driven evolution of deep neural networkscs.NE
A promising paradigm for achieving highly efficient deep neural networks is the idea of evolutionary deep intelligence, which mimics biological evolution processes to progressively synthesize more efficient networks. A crucial design factor in evolutionary deep intelligence is the genetic encoding scheme used to simula…
Automatically discovers effective activation functions for deep learning.
problem Inconsistent performance of novel activation functions in deep learning networks.
method Evolutionary search for general form, gradient descent for parameters.
result Significant performance improvements over ReLU and other functions.
A co-evolutionary approach for Heston model calibration reduces overfitting with diverse datasets.
problem Overfitting and lack of generalization in Heston model calibration.
method Coupling a genetic algorithm with an evolving neural inverse map, using both GA-history sampling and Latin hypercube sampling.
result Diverse datasets improve out-of-sample stability and calibration accuracy.
Hard constraint learning approaches with trainable influence functions for evolutionary equationscs.LG
Paper improves deep learning for solving evolutionary equations with trainable hard constraints.
problem Low computational accuracy of standard PINNs in large temporal domains.
method Sequential learning strategies and trainable influence functions for hard constraints.
result Significantly improved computational accuracy and universality of the method.
DHRL learns interpretable features from visual data.
problem Limited use of deep learning in basic research for interpretable features.
method Generative model chaining, ladder network architecture, latent space regularization.
result DHRL generates disentangled hierarchical features from small datasets.
DONNA rapidly finds optimal neural networks across diverse spaces.
problem Efficient scaling and handling of diverse architectural search-spaces in NAS.
method Three-phase pipeline: accuracy predictor, rapid evolutionary search, and optimal model finetuning.
result 100x faster than MNasNet in finding state-of-the-art architectures on-device.
AutoML discovers complete machine learning algorithms from basic operations.
problem Automating the discovery of machine learning algorithms from scratch.
method Evolutionary search on a generic search space of basic mathematical operations.
result Simple neural networks can be surpassed by evolving directly on tasks of interest.
This paper presents a framework for estimating the remaining useful life (RUL) of mechanical systems. The framework consists of a multi-layer perceptron and an evolutionary algorithm for optimizing the data-related parameters. The framework makes use of a strided time window to estimate the RUL for mechanical component…
Study improves chiral photonic metasurface design using neural networks and genetic algorithms.
problem Optimizing chiral photonic metasurfaces for high chiral dichroism and reflectivity.
method Combines neural network and genetic algorithm approaches with improved fitness functions and data augmentation.
result Demonstrates a significant increase in chiral dichroism and reflectivity.
The Zone of Avoidance makes it difficult for astronomers to catalogue galaxies at low latitudes to our galactic plane due to high star densities and extinction. However, having a complete sky map of galaxies is important in a number of fields of research in astronomy. There are many unclassified sources of light in the…
Machine Learning-Based Optimization of Chiral Photonic Nanostructures: Evolution- and Neural Network-Based Designphysics.optics
Designs chiral photonic structures using machine learning for efficient optical properties.
problem Optimizing chiral photonic nanostructures for light-matter interactions.
method Evolutionary algorithm and neural network approach for rapid optimization.
result Frequency-dependent modification in reflected light's degree of circular polarization.
DeepONet learns operators for PDEs with varying parameters and initial conditions.
problem Learning operators for partial differential equations with different parameters or initial conditions.
method DeepONet uses a Branch net and Trunk net to minimize error between evaluated and expected outputs, incorporating a scalar auxiliary variable approach for energy dissipation.
result DeepONet can accurately approximate operators for PDEs with varying parameters or initial conditions.
The commercialization of deep learning creates a compelling need for intellectual property (IP) protection. Deep neural network (DNN) watermarking has been proposed as a promising tool to help model owners prove ownership and fight piracy. A popular approach of watermarking is to train a DNN to recognize images with ce…
Deep neuroevolution, that is evolutionary policy search methods based on deep neural networks, have recently emerged as a competitor to deep reinforcement learning algorithms due to their better parallelization capabilities. However, these methods still suffer from a far worse sample efficiency. In this paper we invest…
This paper proposes AI-based solutions for optimizing semiconductor manufacturing processes.
problem Optimizing semiconductor manufacturing processes with advanced analytics.
method Evolutionary Computing and Deep Learning algorithms for feature selection and neural networks.
result Advanced algorithm for intelligent feature selection in semiconductor manufacturing.
OpEvo automates tensor operator optimization for better efficiency.
problem Manual optimization of tensor operators is inefficient and limited.
method OpEvo uses evolutionary computation with topology-aware mutation.
result OpEvo finds optimal configurations with less effort and variance.