New architecture learns class representations from few examples.
problem Few-shot learning with high-quality class representations.
method Conditional embeddings based on target images, flexible network for comparisons.
result Achieves state-of-the-art performance on fine-grained classification task.
This paper compares Grid Search, Random Search, and Genetic Algorithm for NAS.
problem Hyperparameter optimization for neural architecture search.
method Comparison of Grid Search, Random Search, and Genetic Algorithm.
result Genetic Algorithm outperforms Grid Search and Random Search in terms of accuracy and execution time.
Proposes GANs using Capsule Networks for faster image synthesis.
problem Faster image synthesis with fewer training samples and epochs.
method Capsule Networks for image synthesis using GAN architectures.
result Learn data manifold faster and synthesize visually accurate images.
Survey of methods for classifier comparison using precision.
problem Lack of methods for classifier comparison using precision.
method Statistical methods for precision comparison, accounting for inter-precision correlation.
result Methods to test global null hypothesis of model comparison using precision.
A study compares deep learning architectures for autonomous vehicles.
problem Challenges in developing self-driving systems due to lack of systematic comparison of deep learning architectures.
method Controlled comparison of seven deep learning architectures on an indoor racetrack, varying conditions and data types.
result AlexNet and VGG-16 performed best, with AlexNet achieving 100% success rate.
TristouNet improves speaker comparison using neural networks and triplet loss.
problem Speaker comparison and change detection in short speech turns.
method Triplet loss for training neural network to project speech sequences into fixed-dimensional space.
result Significant improvements over state-of-the-art techniques for speaker comparison and change detection.
Study compares nine deep learning architectures for multi-horizon financial forecasting.
problem Evaluating the performance of deep learning architectures for multi-horizon financial forecasting.
method Conducted 918 experiments across cryptocurrency, forex, and equity markets using nine architectures.
result ModernTCN achieves the best mean rank (1.333) with a 75 percent first-place rate.
This study analyzes NAS benchmarks and finds that only a subset of operations is crucial for generating high-performing architectures.
problem NAS benchmarks lack generability and provide skewed performance distributions, leading to unreliable comparisons.
method Empirical analysis of widely used NAS benchmarks (101, 201, TransNAS-Bench-101) focusing on operation importance and generability.
result Only a subset of operations is necessary to generate architectures close to the upper-bound performance range, and convolution layers have the highest impact.
A new method corrects flaws in comparing deep learning architectures.
problem Flaws in comparing deep learning architectures using best single model performance.
method Proposes Boo_n method to correct stochasticity in model performance.
result Corrects flaws in comparing deep learning architectures.
Training logs can improve model comparison precision, but careful covariate selection is key.
problem Improving precision in comparing stochastically trained models.
method Use arm-specific covariate adjustment, where each model is adjusted with statistics from its own runs.
result Simple adjustments based on early training logs often reduce uncertainty in model comparisons.
Graph neural networks struggle with fair evaluation.
problem Evaluation of graph neural networks is flawed.
method Thorough empirical evaluation of four GNN models.
result Different splits of data lead to different model rankings.
Transformers use a unique Hessian structure that differs from classical networks, affecting optimization.
problem Understanding the unique optimization landscape of Transformers.
method Theoretical Hessian analysis of a single self-attention layer in Transformers.
result Transformers have a highly non-linear Hessian structure, distinguishing them from classical networks.
New method improves ABC for Bayesian model comparison.
problem Comparing complex models with observed data.
method Approximate Bayesian Computation with posterior density estimation.
result Efficiently assigns high posterior probabilities to ground-truth models.
Study on deep neural networks for reward modeling with pairwise comparison data.
problem Reward modeling with deep neural networks in non-parametric settings.
method Established a non-asymptotic regret bound for deep reward estimators, introduced a margin-type condition.
result Improved regret bound for deep reward estimators, highlighting the importance of clear human beliefs.
3D CNNs improve brain tumor segmentation using multi-resolution features.
problem Brain tumor segmentation in MR images.
method Three 3D CNN architectures combining fine and coarse features.
result Multi-resolution architectures outperform single-resolution networks.
This work compares NN architectures for spectrum sensing.
problem Choosing the best neural network architecture for spectrum sensing.
method Comparison of fully-connected NN (FC), CNN, RNN, and BiRNN.
result CNN, RNN, and BiRNN achieve similar performance.
Improved cover detection using dominant melody embeddings.
problem Challenging cover detection in large audio databases.
method Neural network architecture for track embeddings, focusing on dominant melody.
result Improved accuracy on small and large datasets, scalable to thousands of tracks.
New method selects neural network architectures without needing data.
problem Choosing efficient deep neural network architectures.
method Developed the deep frame potential to quantify network capacity.
result Deep frame potential correlates with generalization error.
Flexible model predicts sports outcomes over time.
problem Predicting sports outcomes with varying player/team skill over time.
method Probabilistic model using continuous-time Gaussian processes for dynamic parameters, efficient inference algorithm.
result Model outperforms competing approaches in predictive performance and scalability.
Meta-learning approach improves CNN architectures for concrete defect classification.
problem Challenging task of recognizing defects in concrete infrastructure.
method Two reinforcement learning based meta-learning approaches (MetaQNN and NAS) for finding suitable CNN architectures.
result Learned architectures have fewer parameters and better multi-target accuracy.
Automatically designs CNN architectures for medical image segmentation.
problem Manual design of deep network architectures is time-consuming and resource-intensive.
method Policy gradient reinforcement learning with dice index reward function.
result Efficacy demonstrated with low computational cost compared to state-of-the-art networks.
Efficient search methods can outperform random search on challenging tasks.
problem Comparing the performance of efficient and random search methods in neural architecture search.
method Comparison of weight sharing and random search methods on progressively larger search spaces for image classification and detection.
result Efficient search methods can provide substantial gains over random search on large, realistic tasks.
Combines CNN and LSTM for spatio-temporal graph networks.
problem Improving spatio-temporal feature extraction.
method Proposes a new architecture combining CNN and LSTM temporal blocks.
result Empirical comparison shows our model outperforms existing models.
New findings show modern neural networks have finite sample complexity in o-minimal structures.
problem Understanding the learnability of modern neural networks in a broad context.
method Analyzing feedforward neural networks definable in o-minimal structures.
result Modern neural networks, including MLPs, CNNs, GNNs, and transformers, have finite sample complexity in the agnostic PAC setting.
Paper proposes a more robust deep reinforcement learning agent.
problem Creating more robust reinforcement learning agents.
method Modified A3C algorithm with dual input streams.
result Significant reduction in training parameters (30%) with improved robustness.
IENs reduce neural network variance without increasing complexity.
problem Reducing variance in neural networks without increasing model complexity.
method IENs use ensemble parameters during training to reduce variance, removing them during testing.
result IENs reduce network variance by a factor of 1/mL−1, leading to significant error rate decreases. This study evaluates different normalizing flow architectures for MCMC.
problem Lack of systematic comparison of normalizing flow architectures in MCMC.
method Extensive evaluation of various normalizing flow architectures on different MCMC methods and target distributions.
result Contractive residual flows are the best general-purpose models for MCMC.
The paper introduces capacity allocation analysis for neural networks, focusing on spatial capacity.
problem Designing neural network architectures is challenging due to the interplay of intuition, experimentation, and luck.
method Introduces capacity allocation analysis, focusing on spatial capacity allocation in linear settings.
result Quantitative comparison of classical architectures on various synthetic tasks reveals insights into model capacity allocation.
Investigates deep hedging under rough volatility models.
problem Performance of deep hedging framework under non-Markovian conditions.
method Analysis of rough volatility models, use of parsimonious network architectures.
result Parsimonious network architectures can capture non-Markovian time-series.
NAS for financial time series forecasts using chain-structured architectures.
problem Optimizing neural architectures for financial time series forecasting.
method Comparison of three NAS strategies (Bayesian optimization, hyperband, reinforcement learning) on chain-structured search spaces for simple and complex architectures.
result Bayesian optimization and hyperband outperform other strategies, and RNN and 1D CNN perform best among architectures.
Foundation models improve on econometric benchmarks for forecasting volatility, but vary widely across models.
problem Comparing pretrained time series foundation models to econometric benchmarks for volatility forecasting.
method Systematic comparison of nine zero-shot TSFMs against eight econometric specifications on 50 assets across 3 markets and 3 horizons.
result Tiny Time Mixers (TTM) is the only model that consistently beats the Log-HAR benchmark, but performance varies widely across models.
Study compares shallow and deep learning for MS lesion segmentation.
problem Automated segmentation of white matter lesions in early-stage MS patients.
method Training and testing shallow and deep learning architectures on 32 patients.
result Combining shallow and deep architectures improves lesion-wise metrics.
Generative deep models struggle with anomaly detection.
problem Comparing deep generative models to classical methods for anomaly detection.
method Statistical comparison of generative models on various datasets, varying hyperparameters.
result Deep generative models perform poorly when hyperparameters are selected with fewer anomalous samples.
The study compares different neural network architectures for option pricing accuracy and training time.
problem Evaluating the impact of network architectures on option pricing accuracy and training time.
method Empirical investigation of various neural network architectures (plain feed forward, highway, DGM) on option pricing problems.
result Generalized highway network architecture achieves the best performance in terms of mean squared error and training time.
Study evaluates quantum and classical conditional Boltzmann machines for time-series forecasting.
problem Time-series forecasting using quantum and classical conditional Boltzmann machines.
method Developed and compared four conditional energy-based forecasting architectures: Gaussian-Bernoulli CRBM, QCRBM, QQRBM, and QFeatureQRBM. Evaluated using symmetric hyperparameter optimisation.
result No systematic evidence of a quantum advantage in time-series forecasting at the available sample size.
PASHA optimizes model tuning for large datasets with limited resources.
problem Expensive HPO and NAS for large datasets.
method Dynamic resource allocation approach.
result Significantly reduces computational resources while maintaining performance.
MT-CNN outperforms other methods in toxicity prediction.
problem Comparing toxicity prediction methods using a common set of descriptors.
method Constructed a common set of microscopic descriptors based on physical models, evaluated MT-CNN and other methods on four toxicity data sets.
result MT-CNN outperforms state-of-the-art methods in toxicity prediction.
Unified architecture for multi-modal multi-task learning using transformer.
problem Training multiple tasks concurrently with varying modalities.
method Spatio-temporal cache mechanism for multi-modal learning.
result Training multiple tasks together reduces model size by about three times.
Study compares hypergraph and graph-level models for higher-order relational learning.
problem Evaluating effectiveness of hypergraph-level vs. graph-level models in relational learning.
method Systematic evaluation of various hypergraph and graph-level architectures.
result Graph-level models applied to hypergraph expansions outperform hypergraph-level models.
Study compares deep learning and superlearner architectures for classification problems.
problem Assessing performance and computational efficiency of different machine-learning methods.
method Comparative analysis of superlearner and deep learning architectures across various datasets.
result Superlearner outperforms deep neural networks on nonlinear predictor relationship datasets, suggesting faster convergence and interpretability.
CNNs improve signal-background classification in particle physics experiments.
problem Improving accuracy in classifying signal from background in particle physics experiments.
method Extensive convolutional neural architecture search for 2D and 3D image data.
result Achieved high accuracy for signal/background discrimination with CNNs, less parameters than ResNet.
Survey tackles challenges in neural architecture design.
problem Challenges in designing optimal neural architectures.
method New classification perspective based on early NAS algorithms' characteristics, problems, and solutions.
result Comprehensive analysis and comparison of NAS works.
NATS-Bench benchmarks NAS algorithms for architecture topology and size.
problem Incomparable performance of NAS algorithms due to different search spaces and training setups.
method Unified benchmarking platform for architecture topology and size searching.
result Validated benchmark for 15,625 topology and 32,768 size candidates.
This research compares masked diffusion models to autoregressive language models, focusing on architectural differences.
problem Comparing masked diffusion models to autoregressive language models due to architectural differences.
method Equitably compare MDMs within a decoder-only framework, investigating architectural influences.
result Decoder-only MDMs can achieve significant speedups and comparable perplexity with temperature annealing.
UNet++ improves medical image segmentation with deep supervision.
problem Improving accuracy in medical image segmentation.
method Nested U-Net architecture with deep supervision.
result UNet++ achieves significant improvements in IoU scores.
DeepRAM evaluates and selects the best deep learning architecture for DNA/RNA binding specificity prediction.
problem Selecting the best deep learning architecture for predicting DNA/RNA binding specificity.
method Systematic exploration of various deep learning architectures using deepRAM, an end-to-end deep learning tool.
result A k-mer embedding convolutional layer and recurrent layer architecture outperforms other methods.
Translationally equivariant neural networks improve performance and generalization in physics problems.
problem Performance and generalization issues in machine learning applied to physics problems.
method Investigation of translationally equivariant convolutional neural networks for complex scalar field theory on a 2D lattice.
result Translationally equivariant neural networks significantly outperform non-equivariant architectures in various regression and classification tasks.
Equivariant neural networks improve performance and generalization in complex scalar field theory tasks.
problem Improving performance and generalization in neural networks for complex scalar field theory tasks.
method Incorporating translational equivariance into neural network architectures.
result Equivariant neural networks significantly outperform non-equivariant networks in various tasks, including those beyond the training set and across different lattice sizes.