Meta-learning approach improves CNN architectures for concrete defect classification.
problem Challenging task of recognizing defects in concrete infrastructure.
method Two reinforcement learning based meta-learning approaches (MetaQNN and NAS) for finding suitable CNN architectures.
result Learned architectures have fewer parameters and better multi-target accuracy.
Study improves RF sensor robustness for target recognition.
problem Variability in RF target responses makes them vulnerable to attacks.
method Evaluates techniques for building robust classification architectures.
result Improves accuracy in identifying true target characteristics.
This study evaluates different normalizing flow architectures for MCMC.
problem Lack of systematic comparison of normalizing flow architectures in MCMC.
method Extensive evaluation of various normalizing flow architectures on different MCMC methods and target distributions.
result Contractive residual flows are the best general-purpose models for MCMC.
The paper proposes a neural network architecture inspired by Langevin Monte Carlo for sampling from target distributions.
problem Sampling from complex target distributions efficiently.
method A neural network architecture inspired by Langevin Monte Carlo is proposed to map samples from a simple reference distribution to samples from the target.
result The proposed neural network architecture achieves approximation rates in the Wasserstein-2 distance for smooth, log-concave target distributions.
Method transfers knowledge without label overlap, source data, or target architecture consistency.
problem Difficulties in transfer learning due to label mismatch, restricted source data, and specialized target architectures.
method Uses deep generative models in two stages: pseudo pre-training and pseudo semi-supervised learning.
result Outperforms scratch training and knowledge distillation methods.
Machine learning classifies object code for target architecture and endianess.
problem Classifying un-labeled compiled code for analysis.
method Simple byte-value histograms and heuristic features from operands.
result High accuracy in classifying target architecture and endianess.
This work explores how neural network architecture affects robustness to noisy labels.
problem The impact of neural network architecture on robustness to noisy labels.
method Formal framework connecting robustness to architecture alignments, measured by predictive power in representations.
result Network robustness to noisy labels improves when its architecture is more aligned with the target function.
ProxylessNAS directly optimizes neural architectures for large-scale tasks without proxy tasks.
problem Inefficient and costly neural architecture search for large-scale tasks.
method Directly learns neural architectures for large-scale tasks and hardware platforms without proxy tasks.
result Achieves better performance and efficiency than previous methods.
New distance metric for neural architecture search reduces search space complexity.
problem Reducing the complexity of neural architecture search.
method Fisher task distance for measuring task similarity and online neural architecture search.
result Reduced search space complexity for task-specific architectures.
Improved robustness in multi-modal sensor fusion with deep learning.
problem Inconsistency in fusion weights leading to poor performance under sensor failures.
method Proposes deep multi-modal sensor fusion architectures with fusion weight regularization and target learning.
result Proposed architectures outperform existing deep learning methods under sensor failures.
New RL approach transfers policies across related domains.
problem Efficiently transfer policies between different domains in RL.
method Adapts and reuses optimal policies from related source tasks.
result Improves sample efficiency in target domain learning.
New HANNA architecture improves accuracy, latency, and energy efficiency for mobile devices.
problem Designing neural networks optimized for specific hardware devices.
method Differentiable NAS method with energy term added to loss function.
result HANNA achieves significant speedup and energy efficiency over MobileNetv2 and CondenseNet.
Proposes a multi-view architecture for drug-target interaction prediction.
problem Representing compound-target pairs in deep learning models.
method Integrates differentiable and predefined molecular descriptors using an adversarial multi-view architecture.
result Demonstrates potential of the proposed approach on clinically relevant datasets.
Custom solution predicts maritime traffic using AIS data with high accuracy.
problem Accurate predictions on AIS data streams for naval traffic.
method Cell grid architecture based on hash tables, with optional semi-supervised learning.
result High accuracy and scalable performance on AIS data.
VUSFA improves transfer learning for target-driven navigation in AI2THOR.
problem Improving transfer reinforcement learning for complex visual navigation tasks.
method Introducing SFDP and Variational Information Bottlenecks to A3C agent.
result VUSFA achieves state-of-the-art performance and generalizability.
TarMAC targets and coordinates multi-agent communication for cooperative tasks.
problem Coordinating multi-agent reinforcement learning in partially observable environments.
method Targeted multi-round communication approach without supervision.
result Improved performance and sample efficiency in diverse environments.
Policy-gradient method controls multiple non-cohesive targets.
problem Controlling multiple non-cohesive targets in a decentralized manner.
method Proximal Policy Optimization for target selection and driving.
result Effective control of non-cohesive targets without prior dynamics knowledge.
HIDRA optimizes weights for diverse tasks, improving model performance.
problem Optimizing gradient-based optimization strategies with dynamic target variables.
method Meta-learning approach that learns a master neuron to initialize output neurons for any number of target variables.
result Improves model performance and generalizes to tasks with any number of target variables.
IMPACT improves RL training speed without sacrificing sample efficiency.
problem Limited sample efficiency in scalable RL architectures.
method Proposes IMPACT, extending IMPALA with target networks, circular buffers, and truncated importance sampling.
result IMPACT achieves higher rewards and significantly reduces training time compared to IMPALA.
MetaPerturb learns to improve generalization across different tasks and architectures.
problem Improving generalization on unseen data for diverse tasks and architectures.
method MetaPerturb is a meta-learned set-based perturbation function that improves generalization performance across heterogeneous tasks and architectures.
result MetaPerturb significantly outperforms baselines on most tasks and architectures with minimal increase in parameter size and no hyperparameters to tune.
Network recasting transforms network architecture for faster inference.
problem Accelerate inference process through network transformation.
method Block-wise recasting of source blocks in a teacher network to target blocks in a student network.
result Transforms network architecture while preserving accuracy and reducing inference time.
New deep learning architecture learns martingales efficiently.
problem Efficiently learning martingales in financial derivatives pricing.
method High-order weak approximation algorithms of Runge-Kutta type.
result Deep neural networks based on this architecture learn martingales effectively.
We reveal a model rank that predicts successful recovery of target functions at overparameterization.
problem Understanding the mysterious good generalization performance of overparameterized nonlinear models.
method Rank stratification and linear stability theory for general nonlinear models.
result Linearly stable functions are preferred by nonlinear training, and model rank predicts minimal training data size.
FrostNet improves INT8 quantization efficiency in mobile networks.
problem The importance of network architecture for optimal INT8 quantization.
method Quantization-aware training (QAT) with StatAssist and GradBoost, hardware-aware NAS.
result FrostNets achieve higher recognition accuracy with comparable latency when quantized.
Model predicts diverse chemical reactions for target compounds.
problem Making generalizable and diverse retrosynthetic reaction predictions.
method Transformer architecture with novel pre-training methods and a latent variable model.
result Improves performance on USPTO-50k dataset, generating more diverse predictions.
We explore methods of producing adversarial examples on deep generative models such as the variational autoencoder (VAE) and the VAE-GAN. Deep learning architectures are known to be vulnerable to adversarial examples, but previous work has focused on the application of adversarial examples to classification tasks. Deep…
A meta-learning approach for automatic knowledge transfer between networks.
problem Improving performance in small-data real-world problems with heterogeneous architectures and tasks.
method Meta-learning to automatically learn what knowledge to transfer and where in the target network.
result Meta-transfer approach significantly outperforms hand-crafted methods on various datasets and network architectures.
Improved generalization with iterative self-distillation using weighted ground-truth targets.
problem Improving generalization accuracy in neural networks.
method Iterative kernel regression with weighted ground-truth targets and ℓ2 regularization. result Closed-form solution for optimal weighting parameter and efficient estimation.
A new method trains neural networks to sample from complex distributions.
problem Training neural networks to efficiently sample from complex target distributions.
method Amortized Stein Variational Gradient Descent (SVGD) to iteratively adjust network parameters.
result Our method effectively trains neural networks to sample from any target distribution.
A power-law fit to the empirical inference-compute frontier in LOB prediction suggests a scaling-law-style frontier.
problem Limit order book prediction
method Using a suite of models ranging from small decision trees to neural LOB architectures
result A power-law fit to the low- and mid-compute non-MLPLOB frontier extrapolates across multiple orders of magnitude and attains R2=0.941 on the excluded high-compute MLPLOB target frontier. Flexible DSL generates novel RNN architectures for various tasks.
problem Limited flexibility and components in existing RNN architectures.
method Domain-specific language (DSL) for automated architecture search.
result Novel RNN architectures perform well on language modeling and machine translation tasks.
Dynamic Transfer Networks improve NER from clinical notes with less data.
problem Improving named entity recognition from clinical notes with limited data.
method Dynamic Transfer Networks (DTN) using parameter sharing and gating architecture.
result DTN achieves better NER performance with fewer training settings.
SAEP prunes sub-architectures to reduce search cost while maintaining performance.
problem Redundancy in ensemble sub-architectures leads to high computational cost.
method SAEP leverages diversity to prune sub-architectures, reducing ensemble size.
result SAEP reduces the number of sub-architectures without degrading performance.
Neural networks outperform NTK on compositional tasks, revealing a complexity gap.
problem Understanding the performance gap between neural networks and NTK on tasks with compositional structure.
method Characterized Fourier and architectural complexities, and analyzed the minimax rates of the architecture class.
result The NTK estimator is exponentially sub-optimal compared to the minimax floor when complexities decouple.
Hierarchical video model learns actions at coarse and fine levels.
problem Understanding complex video actions at different levels of detail.
method A hierarchical architecture that learns coarse-grained to fine-grained actions.
result Models that learn at multiple levels of granularity perform best.
DNAS disentangles neural architecture search for better interpretability and performance.
problem Lack of interpretability in existing neural architecture search methods.
method DNAS disentangles the hidden representation of the controller into semantically meaningful concepts.
result DNAS achieves state-of-the-art performance and competitive architectures.
This paper proposes using deep neural networks for estimating weaving target frequencies in target tracking.
problem Estimating the unknown weaving frequency of a target for improved miss distance.
method Proposes using deep neural networks instead of Kalman framework for estimating the weaving frequency.
result Deep neural networks outperform multiple model adaptive estimation in terms of accuracy and required measurements.
Efficient neural architecture search by sampling structure and operations.
problem Efficiently searching for optimal neural architectures.
method Decouples structure and operation search, using reinforcement learning with policy vectors.
result Significantly improved efficiency compared to traditional methods.
Paper proposes a technique to reduce deep neural network parameters without sacrificing accuracy.
problem Designing smaller networks that approximate the operation of larger, more powerful networks.
method Randomized tensor sketching technique applied to convolutional and fully connected layers.
result Smaller networks trained with sketching technique achieve comparable accuracy to original networks.
The recently proposed Temporal Ensembling has achieved state-of-the-art results in several semi-supervised learning benchmarks. It maintains an exponential moving average of label predictions on each training example, and penalizes predictions that are inconsistent with this target. However, because the targets change …
High-dimensional geometry makes adversarial examples easier to construct.
problem Adversarial examples in deep neural networks
method Systematic study of input dimensionality
result Adversarial examples become easier to construct as dimensionality increases.
New findings on how convolutional architectures approximate time series data.
problem Understanding the approximation properties of convolutional architectures in time series modeling.
method Mathematical analysis of convolutional architectures applied to time series modeling.
result A new definition of spectrum-based regularity for measuring temporal relationships under convolutional approximation.
Universal audio perturbations fool multiple classification models.
problem Creating audio adversarial perturbations that work across different models.
method Two methods: greedy iterative approach and novel penalty formulation.
result The penalty method produces more successful attacks with limited training data.
New algorithms for deep learning mimic brain's learning but struggle with complex images.
problem Evaluating biologically inspired deep learning algorithms on complex image datasets.
method Implemented and compared various biologically inspired algorithms (TP, FA, DTP) on MNIST, CIFAR-10, and ImageNet.
result Biologically inspired algorithms perform well on MNIST but poorly on CIFAR and ImageNet, suggesting new architectures or algorithms are needed.
Black-box adversarial examples improve ASR system accuracy.
problem Improving ASR system accuracy through targeted adversarial examples.
method Combining genetic algorithms and gradient estimation for black-box attacks.
result Achieved 89.25% targeted attack similarity with 94.6% audio file similarity.
A method for faster neural architecture search using low-fidelity training.
problem Time-consuming evaluations in neural architecture search.
method Bayesian multi-fidelity method with knowledge distillation.
result Training for a few epochs with knowledge distillation leads to better architecture selection.
The study proves Gaussian universality of deep random features learning.
problem Understanding the test error in deep random features learning.
method Proving Gaussian universality of test error in ridge regression and arbitrary convex losses.
result Sharp asymptotic formula for test error in ridge regression setting.
BinConv improves time series forecasting by preserving ordinal information in a classification framework.
problem Lack of ordinal information in existing classification-based time series forecasting methods.
method Cumulative Binary Encoding (CBE) and BinConv architecture.
result BinConv achieves superior performance in time series forecasting compared to existing methods.