Automated design using neural networks and gradient descent.
problem Optimizing complex engineering tasks.
method Training neural networks to mimic fitness functions and using gradient descent for optimization.
result Demonstrated effectiveness in designing optimized heat sinks and airfoils.
PrototypeML simplifies neural network design and development.
problem Error-prone code and time-consuming model design.
method Visual interface for neural networks, abstracting PyTorch.
result Reduces model design and development time, easier debugging.
New neural network design improves performance.
problem Improving neural network performance.
method Guided by traditional algorithm design, two heuristics are presented to incorporate additional weights.
result Performance of the new networks can match or exceed traditional neural networks.
Introduces MLP algebra for designing neural networks.
problem Designing neural networks relies on developer experience.
method Introduces MLP algebra as a mathematical structure.
result Guiding principle to build MLPs for specific data sets.
Neural network ensembles predict design rule violations from early stages of IC design.
problem Predicting design rule violations from placement and global routing stages in IC design.
method Proposes a framework using neural network ensembles with soft voting and PCA-based subset selection.
result Significant improvement in model performance compared to baseline, including better performance than random forest.
DeepESN models efficiently design deep neural networks for temporal data.
problem Designing efficient deep neural networks for temporal data.
method Hierarchical compositions of recurrent layers.
result Intrinsic properties of state dynamics in deep RNNs.
Automates design of lightweight neural networks for image classification.
problem Designing efficient neural networks for edge devices with limited computational resources.
method Uses the Mesh Adaptive Direct Search (MADS) algorithm to optimize network architecture.
result Achieves comparable performance to standard methods with fewer design trials.
Automated neural network synthesis engine for custom designs.
problem Difficulty in creating custom neural networks due to lack of skills and time.
method Automated synthesis engine NeuNetS for text and image domains.
result Builds neural networks faster and with similar accuracy to human-designed models.
Generative neural network designs novel 3D molecules with specified properties.
problem Designing molecules with desired properties in chemistry.
method Conditional generative neural network for 3D molecular structures.
result Demonstrated utility in generating novel molecules with specified motifs or composition.
New method designs multilayer nanoparticles using AI.
problem Difficult to design multilayer nanoparticles by trial and error.
method Combines genetic algorithm and neural network for inverse design.
result Successfully designs multilayer nanoparticles efficiently.
GraphNAS uses reinforcement learning to automatically design graph neural network architectures.
problem Designing effective graph neural network architectures requires manual work and domain knowledge.
method GraphNAS generates variable-length strings to describe architectures and trains a recurrent network with reinforcement learning to maximize validation accuracy.
result GraphNAS achieves consistently better performance on various citation and protein networks.
A new scaling calculus helps design and initialize ReLU networks more effectively.
problem Optimizing the design and initialization of ReLU neural networks.
method Proposes a scaling constant for neural network layers and weights, relating it to optimizability.
result A network with a uniform scaling constant is easier to train, and the geometric mean of fan-in and fan-out is a better initialization variance.
This thesis aims to automate deep neural network design for efficiency and complexity reduction.
problem Manual design of deep neural networks is inefficient and complex.
method Examines and proposes automated approaches to neural network design.
result Creation of less complex models with good performance through automation.
Simple method finds efficient CNN architectures without manual design.
problem Manual design of neural network architectures is cumbersome and time-consuming.
method Simple hill climbing procedure with network morphisms and cosine annealing optimization.
result Simple method yields competitive CNN architectures with minimal resources.
New layer improves neural networks' flexibility and speed.
problem Designing efficient neural networks for real-time learning.
method Sparse Associative Memories for incremental learning.
result Improved Neural Networks with faster training and real-time support.
Optimized neural networks for Edge TPU achieve high accuracy in real-time image classification.
problem Designing neural networks for hardware accelerators to achieve optimal performance.
method Hardware-aware neural architecture search and model customization for Edge TPU.
result Improved accuracy-latency tradeoff on Pixel 4's Edge TPU compared to existing models.
IL-Net uses domain knowledge to improve neural network design.
problem Designing optimal neural network architectures from a principled approach has been challenging.
method Developed a furcated neural network architecture that incorporates domain knowledge.
result Furcated networks can improve model accuracy by 20-35% without additional labeled data.
Neural networks predict EV charging station usage from network layout.
problem Designing optimal EV charging station networks.
method Used neural networks to predict usage from station layout.
result Quickly estimates average usage statistics from proposed station placements.
Flexible VHDL design for multiple neural networks on FPGAs.
problem Inflexible neural network designs for FPGAs.
method Proposes a flexible VHDL structure with multiple processor groups.
result Allows training and testing of multiple neural networks on multiple FPGAs.
A novel method designs multi-block neural networks using Q-learning.
problem Designing optimal neural network architectures efficiently.
method Reinforcement learning (Q-learning) to sequentially pick different types of blocks.
result Effective in creating multi-block neural networks with comparable or better performance.
Automates neural network design without training, speeding up search by seconds.
problem Time and effort in hand-designing deep neural networks.
method Predict trained accuracy from untrained network state using activation overlap.
result Search for powerful networks in seconds, verified on various benchmarks.
New neural network architecture for auction design exploiting permutation symmetry.
problem Designing incentive-compatible auctions that maximize expected revenue.
method Constructed a permutation-equivariant neural network architecture.
result Permutation-equivariant architectures can perfectly recover optimal mechanisms.
Graph neural networks struggle with heterophily, but new designs improve their performance.
problem Graph neural networks struggle with heterophily (networks where connected nodes may have different class labels and dissimilar features).
method Ego- and neighbor-embedding separation, higher-order neighborhoods, and combination of intermediate representations.
result The identified designs increase the accuracy of GNNs by up to 40% and 27% over models without them on synthetic and real networks with heterophily, respectively.
A neural network and evolutionary algorithm framework designs nonlinear optical molecules.
problem Designing efficient nonlinear optical materials.
method Multi-stage Bayesian neural network (msBNN) and corrected Lewis-mode group contribution method (cLGC) combined with evolutionary algorithm (EA).
result Accurately and efficiently designs molecules with different optical properties using a small data set.
Algorithm optimizes neural network design with progression and random weights.
problem Systematic design of large neural networks with optimal regularization and size.
method Progressive learning with convex optimization, random weight matrices.
result Developed network shows good generalization power.
Design automation optimizes deep learning models for various hardware.
problem Designing efficient deep learning models requires balancing algorithm and hardware.
method Proposes design automation techniques for specialized neural networks, including auto pruning and quantization.
result Learning-based automation achieves superior performance and efficiency compared to human design.
New method uses neural networks to optimize experimental designs for complex models.
problem Designing experiments for complex, intractable models with high computational cost.
method Neural mutual information estimation for mutual information maximization.
result Optimal experimental designs and posterior inference can be jointly determined.
A novel neural network approach for optimization problems.
problem Constrained optimization problems.
method Neural Optimization Machine (NOM) using a specially designed NN architecture and training procedure.
result Solves optimization problems efficiently, especially in high-dimensional spaces.
RID-Noise improves robust design under noisy conditions using neural networks.
problem Design robustness under noisy environments.
method Robust Inverse Design under Noise (RID-Noise) using conditional invertible neural networks (cINNs).
result RID-Noise achieves more effective robust design compared to state-of-the-art methods.
A new neural network for efficient density estimation.
problem Efficient density estimation for high-dimensional data.
method Triangular neural network implementation of neural autoregressive flow (NAF).
result Achieves state-of-the-art bits-per-dimension indices on MNIST and CIFAR-10.
Visualizes 3D CNNs for protein-ligand scoring.
problem Interpreting complex neural network decisions for protein-ligand scoring.
method Three visualization methods for 3D CNNs, including filters and weights.
result Visualizations aid in tuning and designing neural networks.
Serenity optimizes neural network execution for edge devices by scheduling with optimal memory footprint.
problem Order of nodes in irregular neural networks affects memory footprint, complicating execution under resource constraints.
method Memory-aware compiler using dynamic programming and graph rewriting to find optimal schedules.
result Achieves optimal peak memory and further improves it with graph rewriting, reducing memory usage by 1.68x-1.86x compared to TensorFlow Lite.
Automates GNN design for molecular property prediction.
problem Designing and tuning GNN architectures for molecular property prediction is labor-intensive.
method Developed a NAS approach to automatically discover high-performing GNN architectures for MPNNs.
result Automatically discovered MPNNs outperform manually designed GNNs in molecular property prediction.
New RNN reconstructs video frames from sparse measurements.
problem Sequential signal reconstruction from compressive measurements.
method Unfolding proximal gradient method for l1-l1 minimization.
result Outperforms state-of-the-art RNN models in video frame reconstruction.
Paper presents FPGA implementation for efficient recurrent neural networks.
problem Implementing recurrent neural networks on FPGAs for low latency.
method Developed hls4ml framework to implement LSTM and GRU layers.
result Demonstrated effective designs for both small and large models.
This paper designs neural associative memories that can correct many adversarial errors.
problem Designing neural associative memories that can correct many adversarial errors.
method Mapping the learning phase and recall phase to dictionary learning with a square dictionary and iterative error correction in an expander code.
result The designed associative memories can store datasets with exp(n) vectors and tolerate Ω(mpolylognn) adversarial errors. Optimizes experimental designs for intractable models using mutual information bounds.
problem Finding optimal experimental designs for models with intractable data-generating distributions.
method Maximizes mutual information lower bounds parametrized by neural networks, updating network parameters and designs simultaneously.
result Framework enables experimental design for various tasks including parameter estimation and model discrimination.
Sparse neural networks can improve performance with less memory.
problem Lack of fast memory limits deep neural network performance.
method Experimented with sparse neural network topologies, including pruning-based and RadiX-Nets.
result Sparse networks achieve comparable accuracy to dense networks but suffer instability at extreme sparsity.
iDAD uses neural networks to quickly adapt experiments without likelihoods.
problem Performing adaptive experiments in real-time with implicit models.
method iDAD learns a design policy network upfront to make quick design decisions.
result iDAD can make design decisions in milliseconds, unlike traditional BOED approaches.
Explains differences between WL and folklore-WL formulations in graph neural networks.
problem Understanding the differences between WL and folklore-WL formulations in graph neural networks.
method Visual explanation of differences between WL and folklore-WL formulations.
result Clarifies the differences between WL and folklore-WL formulations.
Paper introduces method to make neural networks symmetrical.
problem Creating symmetrical neural networks for data with inherent symmetries.
method Introduces a method for modifying neural networks to enforce equivariance.
result Group convolutional neural networks are a special case of the introduced framework.
A new technique reduces the size of rRNNs for time series prediction.
problem Minimizing the size of rRNNs for efficient time series prediction.
method Combining Takens-based attractor reconstruction with machine learning for feature extraction.
result Reduced network size by a factor of 15 with improved performance.
Automatically designs neural network structure using matrix conditioning.
problem Choosing an effective size and structure of neural networks for new datasets is time-consuming.
method Adjusts neuron proportions and scales network size using matrix conditioning.
result Small networks achieve high accuracy on various datasets.
Automates X-CNN design with improved accuracy and efficiency.
problem Designing cross-modal convolutional neural networks (X-CNNs) is time-consuming and complex.
method Two approaches: data-driven and iterative, to learn and optimise X-CNN topologies.
result Superior performance and up to 9% accuracy gain compared to hand-designed X-CNNs.
Neural networks enhance linear programming for complex decision-making problems.
problem High-dimensional and combinatorial operations research problems.
method Hybrid solution method combining linear programming and neural networks.
result Neural network value function approximations outperform polynomial approximations in a transportation problem.
Paper proposes a technique to reduce deep neural network parameters without sacrificing accuracy.
problem Designing smaller networks that approximate the operation of larger, more powerful networks.
method Randomized tensor sketching technique applied to convolutional and fully connected layers.
result Smaller networks trained with sketching technique achieve comparable accuracy to original networks.
A new neural network learns optimal designs in one shot.
problem Designing optimal structures in iterative processes.
method Theory-driven machine learning combining student and teacher models.
result Near-optimal designs achieved with less data.
NAS helps find best neural network designs.
problem Designing optimal neural network architectures.
method Optimization algorithms and search spaces.
result Introduction to major advances in NAS for CNNs.