A new deep neural network improves mammogram image processing.
problem Improving mammogram image processing accuracy and efficiency.
method A novel deep neural network architecture with dual-path connections.
result Achieves best mammography segmentation and classification results.
THGFM models dynamic relational systems with cross-type and temporal fusion.
problem Learning on temporal heterogeneous graphs with diverse node and relation types.
method Dual-Path Architecture with Shared-Space and Relational Type-Partitioned Temporal Attention.
result THGFM outperforms baseline models on academic graph benchmarks.
We consider efficient implementations of the generalized lasso dual path algorithm of Tibshirani and Taylor (2011). We first describe a generic approach that covers any penalty matrix D and any (full column rank) matrix X of predictor variables. We then describe fast implementations for the special cases of trend filte…
Adaptive framework predicts stock prices better during volatile periods.
problem Inability of standard prediction models to handle regime-dependent stock market behavior.
method Autoencoder-Gated Dual Node Transformers with Reinforcement Learning Control.
result 0.59% MAPE with adaptive system, compared to 0.80% for baseline.
Improved speech enhancement with MNTFA using time-frequency attention.
problem Speech enhancement with limited model size and memory.
method Designing MNTFA with self-attention modules for long sequences and joint training.
result MNTFA achieves better performance with fewer parameters than DPCRN.
This paper certifies cluster assignments from sum-of-norms clustering algorithms.
problem Certifying the correct cluster assignments from approximate solutions of sum-of-norms clustering.
method Presented a clustering test that identifies and certifies the correct cluster assignment from an approximate solution.
result The correct cluster assignment is guaranteed to be certified by a primal-dual path following algorithm after sufficient iterations.
X-Caps improves medical diagnosis explainability by encoding visual attributes in capsules.
problem Uninterpretable predictions from deep neural networks in healthcare.
method Teaches a novel multi-task capsule network to encode high-level visual attributes and malignancy scores.
result X-Caps outperforms state-of-the-art deep dense 3D CNNs in capturing visually interpretable attributes and malignancy prediction.
Proposes a method to improve neural architectures reproducibly.
problem Lack of reproducibility in Neural Architecture Transformer (NAT).
method Differentiable Neural Architecture Transformation (DNAT).
result DNAT outperforms NAT and is applicable to various models and datasets.
HM-NAS improves neural architecture search by learning optimal architectures.
problem Limited flexibility in architecture candidates due to hand-designed heuristics.
method Incorporates multi-level encoding and hierarchical masking to automatically learn optimal architectures.
result Achieves better architecture search performance and competitive model accuracy.
Simpler neural architecture search method using random architectures and regression.
problem Complex algorithms in neural architecture search.
method Train N random architectures, use them to train a regression model, predict validation accuracies, and select top-K architectures.
result More sample efficient and competitive with complex approaches.
NAT optimizes neural architectures to improve performance without extra cost.
problem Redundant operations in neural architectures consume memory and degrade performance.
method Transformed Markov Decision Process (MDP) and reinforcement learning to replace redundant operations with more efficient ones.
result Transformed architectures outperform original and existing methods on CIFAR-10 and ImageNet datasets.
DNAS disentangles neural architecture search for better interpretability and performance.
problem Lack of interpretability in existing neural architecture search methods.
method DNAS disentangles the hidden representation of the controller into semantically meaningful concepts.
result DNAS achieves state-of-the-art performance and competitive architectures.
Conventional Neural Architecture Search (NAS) aims at finding a single architecture that achieves the best performance, which usually optimizes task related learning objectives such as accuracy. However, a single architecture may not be representative enough for the whole dataset with high diversity and variety. Intuit…
SAEP prunes sub-architectures to reduce search cost while maintaining performance.
problem Redundancy in ensemble sub-architectures leads to high computational cost.
method SAEP leverages diversity to prune sub-architectures, reducing ensemble size.
result SAEP reduces the number of sub-architectures without degrading performance.
Efficiently infers neural architectures for unseen tasks.
problem High cost of neural architecture search.
method Gradient-based framework sharing information across tasks.
result Quick identification of good candidate architectures for new tasks.
City-GAN learns city architecture styles using a custom GAN architecture.
problem Learning architectural styles of cities using standard GAN and CGAN architectures.
method Proposes a custom GAN architecture to improve learning of city architectural styles.
result Demonstrates superior performance of custom GAN architecture for city architectural style learning.
Study reduces NAS search cost by generating multiple complex architectures in one shot.
problem Finding multiple neural architectures with varying complexities efficiently.
method Uses importance sampling to generate and update multiple distributions of architectures.
result Reduces search cost by finding multiple architectures with different complexities in a single search.
SGAS improves neural architecture search by choosing and pruning operations greedily.
problem NAS often fails to generalize in final evaluation.
method Divides search into sub-problems and chooses/prunes candidate operations greedily.
result SGAS finds state-of-the-art architectures with minimal computational cost.
Deep learning models' architectures, including depth and width, are key factors influencing models' performance, such as test accuracy and computation time. This paper solves two problems: given computation time budget, choose an architecture to maximize accuracy, and given accuracy requirement, choose an architecture …
DNArch learns CNN architectures by backpropagation.
problem Discovering optimal CNN architectures.
method Differentiable Neural Architectures (DNArch) learns CNN architectures by backpropagation, controlling kernel sizes, channels, downsampling positions, and depth.
result DNArch finds performant CNN architectures across various tasks.
Differentially-private FNAS protects privacy while collaboratively searching for neural architectures.
problem Collaborative neural architecture search with privacy concerns.
method Federated Neural Architecture Search (FNAS) with differential privacy (DP-FNAS).
result DP-FNAS can search for highly-performant neural architectures while protecting individual parties' privacy.
EENA efficiently searches neural architectures with minimal resources.
problem Lack of direction and high computational cost in neural architecture search.
method EENA uses guided evolution with mutation and crossover operations.
result EENA designs highly effective neural architectures with minimal resources.
NAS favors wide and shallow cell structures, leading to fast convergence but not necessarily better generalization.
problem Understanding and improving the architectures generated by NAS algorithms.
method Empirical and theoretical study of existing NAS algorithms (DARTS, ENAS) and their architectures.
result Existing NAS algorithms favor wide and shallow cell structures, leading to fast convergence but not necessarily better generalization.
Improved neural architecture search through balanced training.
problem Inconsistent ranking of architectures under one-shot training.
method Balanced NAO: balanced training of supernet during search.
result Significant improvements in architecture discovery and performance.
AGNN automates GNN architecture search, achieving best performance.
problem Finding optimal GNN architectures is laborious and requires human expertise.
method AGNN uses reinforcement learning to search for optimal GNN architectures within a predefined space, with a novel parameter sharing strategy.
result AGNN identifies optimal GNN architectures achieving best performance.
PDNAS optimizes GNN architectures for diverse datasets.
problem Inadequate adaptability and combinatorial search space in GNNs.
method Dual architecture search (micro- and macro-architectures) with gradient-based optimization.
result PDNAS finds deeper GNNs with better performance on diverse datasets.
A graph VAE framework optimizes neural architectures in a continuous space.
problem Discovering efficient neural architectures in a discrete space.
method Graph VAE framework with VAE and GNN components, joint learning of predictors and decoders.
result The framework discovers powerful neural architectures with both excellent performance and high computational efficiency.
Bayesian optimization reduces RNN architecture search time.
problem Optimizing RNN architectures for time series prediction.
method Bayesian Optimization with a training-free performance metric.
result BO designs well-performing RNN architectures with reduced optimization time.
GATES improves neural architecture search by modeling operations as information transformation.
problem Improving predictor-based neural architecture search efficiency.
method GATES models operations as information transformation, covering both node and edge cell search spaces.
result GATES boosts sample efficiency and improves predictor performance.
Flat learning curves reveal no progress in ENAS controller.
problem Improving learning speed in neural architecture search.
method Evaluated learning progress of ENAS controller through architecture re-training.
result No observable progress in controller's generated architectures.
The process of designing neural architectures requires expert knowledge and extensive trial and error. While automated architecture search may simplify these requirements, the recurrent neural network (RNN) architectures generated by existing methods are limited in both flexibility and components. We propose a domain-s…
A ranking network improves neural architecture search efficiency.
problem Efficiently search for neural network architectures without extensive training.
method Pairwise ranking loss for a performance predictor trained on task meta-features.
result The ranking network outperforms the performance predictor in architecture search.
SemiNAS reduces NAS cost by predicting accuracy of unlabeled architectures.
problem Costly evaluation of architectures limits NAS efficiency.
method SemiNAS uses unlabeled architectures to train an accuracy predictor.
result SemiNAS achieves comparable accuracy with less data.
This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner. Unlike conventional approaches of applying evolution or reinforcement learning over a discrete and non-differentiable search space, our method is based on the continuous relaxation of the architectu…
New model predicts neural network performance from early training epochs, incorporating architecture impact.
problem Predicting neural network performance from early training epochs, neglecting architecture impact.
method Architecture-aware graph ordinary differential equation model.
result Model outperforms state-of-the-art methods for MLP and CNN learning curves.
GraphNAS uses reinforcement learning to automatically design graph neural network architectures.
problem Designing effective graph neural network architectures requires manual work and domain knowledge.
method GraphNAS generates variable-length strings to describe architectures and trains a recurrent network with reinforcement learning to maximize validation accuracy.
result GraphNAS achieves consistently better performance on various citation and protein networks.
Gradient-based method prunes large models to create transferable architectures.
problem Creating transferable architectures from large models with limited fine-tuning data.
method Gradient-based algorithm for architecture pruning and subset selection.
result Successfully retrain architectures on new tasks with few fine-tuning data.
BNAS improves neural architecture search with a scalable, fast, and efficient approach.
problem Efficiently searching for optimal neural architectures with high performance and low training time.
method Designing a broad scalable architecture (BCNN) with reinforcement learning and parameter sharing, and developing two variants.
result Significantly reduces training time and achieves state-of-the-art performance on CIFAR-10 and ImageNet.
Recent studies on neural architecture search have shown that automatically designed neural networks perform as good as expert-crafted architectures. While most existing works aim at finding architectures that optimize the prediction accuracy, these architectures may have complexity and is therefore not suitable being d…
The paper optimizes dynamic scheduling for ring architectures in deep learning training.
problem Optimizing deep learning training times with ring architectures.
method Formulated a non-convex, non-linear, NP-hard integer programming problem and developed a doubling heuristic.
result Dynamic scheduling can significantly reduce job completion times in ring architectures.
Differentiable NAS method optimizes network architecture and parameters efficiently.
problem Challenging to simultaneously guarantee effectiveness and efficiency in network architecture search.
method Differentiable architecture search with ensemble Gumbel-Softmax estimator.
result End-to-end mechanism for searching network architectures, discovering high-performance architectures efficiently.
A new NAS framework optimizes 3D medical image segmentation architectures.
problem Optimizing neural architectures for high-resolution 3D medical images.
method Stochastic sampling algorithm for scalable gradient-based optimization of neural connectivities and operation types in both encoder and decoder.
result Automatically designed architecture outperforms human-designed U-Net.
BayesNAS uses Bayesian learning to improve neural architecture search efficiency.
problem Improper treatment of zero operations and architecture parameter pruning issues in one-shot NAS methods.
method Employing hierarchical automatic relevance determination (HARD) priors for Bayesian learning to model architecture parameters.
result Found architecture on CIFAR-10 in just 0.2 GPU days using a single GPU.
FedNAS automates federated learning by searching for better architectures.
problem Non-I.I.D. data makes predefined model architectures suboptimal.
method Federated Neural Architecture Search (FedNAS) for collaborative architecture optimization.
result FedNAS searches for better architectures that outperform predefined models.
Deep neural network architectures have traditionally been designed and explored with human expertise in a long-lasting trial-and-error process. This process requires huge amount of time, expertise, and resources. To address this tedious problem, we propose a novel algorithm to optimally find hyperparameters of a deep n…
PARSEC uses a probabilistic approach to reduce memory usage in neural architecture search.
problem Efficiently search over large and complex neural architectures with reduced memory usage.
method Probabilistic sampling to learn a distribution over high-performing architectures, enabling transfer learning.
result Our approach outperforms state-of-the-art methods with significantly less computational cost.
Automatic neural architecture design has shown its potential in discovering powerful neural network architectures. Existing methods, no matter based on reinforcement learning or evolutionary algorithms (EA), conduct architecture search in a discrete space, which is highly inefficient. In this paper, we propose a simple…
Extends NAS to learn both intra-cell and inter-cell architectures for language modeling.
problem Limited NAS systems restrict search to recurrent or convolutional cells.
method Designs a joint learning method to perform intra-cell and inter-cell NAS simultaneously.
result Significantly outperforms a strong baseline on PTB and WikiText data.