Deep networks struggle to learn efficient representations of simple functions.
problem Can deep learning methods find efficient representations of simple functions?
method Trained deep neural networks on the parity function and fast Fourier transform, using gradient-based optimization.
result Deep networks require initialization close to exact solutions to learn efficient representations of simple functions.
TensorLayer simplifies deep learning development for researchers and engineers.
problem Developing deep learning systems is complex and time-consuming.
method Offers rich abstractions for neural networks, model and data management, and parallel workflow.
result Boosts efficiency without sacrificing performance or scalability.
Deep learning uses complex networks for high-dimensional data.
problem Computational inefficiency in training deep learning models.
method Use of hierarchical latent variables, efficient linear algebra, SGD optimization, and batch sampling.
result Efficient training and inference possible with optimized algorithms.
Efficient deep learning for hyperspectral image classification using active learning.
problem Lack of good-quality labeled samples for deep learning in hyperspectral images.
method Weighted incremental dictionary learning for active selection of training samples.
result The proposed algorithm improves deep learning efficiency and effectiveness in hyperspectral image classification.
Improves sample efficiency in evolutionary policy search methods.
problem Worse sample efficiency in evolutionary policy search methods.
method Importance mixing to reuse more samples.
result Improves sample efficiency but still less than deep reinforcement learning.
swTVM optimizes deep learning code for Sunway processors.
problem Efficiently compiling deep learning models for Sunway processors.
method Extends TVM to support Sunway, leveraging architecture features.
result 1.79x improvement on average across benchmarks.
Proposes using gradients as features for efficient deep learning adaptation.
problem Efficient deep representation learning for different tasks.
method Designs a linear model incorporating gradients and activations of a pre-trained network.
result Shows strong results across various tasks and datasets.
The paper identifies overfitting as the main bottleneck in efficient deep reinforcement learning.
problem Improving sample efficiency in deep reinforcement learning.
method Empirical analysis on DMC tasks to identify overfitting as the main issue and developing a hill-climbing method targeting validation TD error.
result Overfitting is the primary bottleneck in sample-efficient deep RL, and regularization techniques can control this.
New protocol benchmarks deep learning methods' data efficiency.
problem Measuring data efficiency of deep learning methods.
method Proposed experimental protocol to benchmark CNNs and HiGSFA.
result HiGSFA outperforms CNNs on smaller datasets.
Proves deep networks can learn hierarchical structures efficiently.
problem Understanding how deep networks learn hierarchical structures in data.
method Random Hierarchy Models, gradient-based methods, layerwise training.
result Proves deep networks can efficiently learn hierarchical structures.
New method improves deep RL efficiency by adaptively setting accuracy requirements.
problem Improving efficiency in deep reinforcement learning.
method Accuracy-based curriculum learning using adaptive selection of accuracy requirements.
result Adaptive accuracy requirements lead to better learning efficiency than random selection.
Increasing input dimensionality improves deep RL performance and sample efficiency.
problem Real-world reinforcement learning applications often lack sufficient training data.
method Proposed an online feature extractor network (OFENet) to improve deep RL performance and sample efficiency.
result RL agents learn more efficiently with high-dimensional input representations than with lower-dimensional state observations.
BatchBALD selects multiple informative points for deep Bayesian active learning, improving data efficiency.
problem Efficient and diverse selection of points for deep Bayesian active learning.
method Develops BatchBALD, a greedy linear-time approximation to mutual information, as an acquisition function.
result Achieves new state-of-the-art performance on benchmarks, improving data efficiency.
Vprop uses RMSprop for efficient Bayesian deep learning.
problem Efficient implementation of Bayesian deep learning methods.
method Gaussian variational inference using RMSprop optimizer.
result Significant reduction in memory requirements for Black-Box Variational Inference.
Deep learning improves portfolio optimization efficiency.
problem Efficient frontier calculation in high-dimensional finance problems.
method Deep neural networks for portfolio optimization with added constraints.
result A new projected feedforward network outperforms classical methods.
New DRL algorithm improves sample efficiency and exploration performance.
problem High sample cost in deep reinforcement learning.
method Combines entropy and bootstrap techniques with Tsallis entropy regularization.
result Demonstrates more efficient and effective exploration on Atari games.
Improved sample efficiency in reinforcement learning with deep Gaussian processes.
problem Efficiently learn to control actions with limited interaction data.
method Deep Gaussian processes that simulate dynamics with depth and prior knowledge.
result Significantly improved early sample efficiency across various tasks, including half-cheetah control.
Deep neural networks are a powerful tool for feature learning and extraction given their ability to model high-level abstractions in highly complex data. One area worth exploring in feature learning and extraction using deep neural networks is efficient neural connectivity formation for faster feature learning and extr…
Combining visual attention with deep reinforcement learning improves sample efficiency.
problem Improving sample efficiency in deep reinforcement learning.
method Visual selective attention mechanism implemented using optical flow and batch normalization.
result Visual selective attention leads to improvements in sample efficiency on Atari games.
Bayesian compression improves deep learning efficiency.
problem Efficiency and compression in deep learning networks.
method Adopting a Bayesian approach with hierarchical priors and posterior uncertainties.
result Achieved state-of-the-art compression rates with competitive performance.
Improved sample efficiency in reinforcement learning with action guidance.
problem Sample inefficiency in deep reinforcement learning with sparse, delayed, and deceptive rewards.
method Integrating a non-expert demonstrator (e.g., MCTS) into asynchronous distributed deep reinforcement learning.
result Our methods learn faster and converge to better policies on a game.
New deep learning architecture learns martingales efficiently.
problem Efficiently learning martingales in financial derivatives pricing.
method High-order weak approximation algorithms of Runge-Kutta type.
result Deep neural networks based on this architecture learn martingales effectively.
New deep learning method solves TSP faster and more efficiently.
problem Approximately solving the Travelling Salesman Problem on 2D Euclidean graphs.
method Uses Graph Convolutional Networks for efficient TSP graph representations and non-autoregressive beam search.
result Significantly reduces optimality gap for large problem instances.
Study develops efficient nested deep hedging method for derivatives pricing.
problem Hedging derivatives in market frictions using multiple options.
method Nested deep hedging approach with efficient learning techniques.
result Reduces arbitrage opportunities and improves hedging risks.
This paper optimizes deep learning systems for high performance and low energy consumption.
problem Achieving ultra-high energy efficiency and performance for deep neural networks.
method Developed an algorithm-hardware co-optimization framework that reduces computational and storage complexity.
result Achieved at least 152X speedup and 71X energy efficiency gain compared to IBM TrueNorth processor.
Design automation optimizes deep learning models for various hardware.
problem Designing efficient deep learning models requires balancing algorithm and hardware.
method Proposes design automation techniques for specialized neural networks, including auto pruning and quantization.
result Learning-based automation achieves superior performance and efficiency compared to human design.
Efficient equivariant MobileNetV2 for medical applications on mobile devices.
problem Limited computational resources for deep learning models in medical applications.
method Design and optimize an equivariant version of MobileNetV2 with model quantization.
result Achieved close-to state-of-the-art performance on medical dataset with improved efficiency.
Deep networks can approximate score functions in high-dimensional graphical models efficiently.
problem Approximation efficiency of score functions by deep neural networks in high-dimensional graphical models like Markov random fields.
method Variational inference denoising algorithms and efficient neural network representation.
result Efficient sample complexity bound for diffusion-based generative modeling when score functions are learned by deep neural networks.
This thesis aims to automate deep neural network design for efficiency and complexity reduction.
problem Manual design of deep neural networks is inefficient and complex.
method Examines and proposes automated approaches to neural network design.
result Creation of less complex models with good performance through automation.
Deep networks can learn functions approximated by shallow networks, but not all functions.
problem The learnability of functions by deep neural networks and the approximation capacity of simpler classes.
method Study the connection between learnability and approximation capacity of functions by deep neural networks and simpler classes.
result A necessary condition for a function to be learnable by deep neural networks is to be approximable by shallow networks.
Method trains deep neural networks on weakly labeled audio data efficiently.
problem Limited training data and lack of temporal labels for audio event detection.
method Multi-instance learning with a new loss function for stacked CNN-RNN.
result Improved performance on low-resource audio datasets.
This paper provides an overview of deep semi-supervised learning methods.
problem Reducing the need for large annotated datasets in deep learning.
method Summarizes dominant semi-supervised approaches in deep learning.
result Provides a comprehensive overview of deep semi-supervised learning.
Paper presents a new method for efficient deep learning with over-complete dictionaries.
problem Learning an over-complete basis for optimal reconstruction without optimization.
method Multiscale Residual Mixture of PCA with a hierarchical approach.
result Exponential decrease of error with depth in a recursive deep approach.
Paper develops efficient variational inference for sparse deep learning with theoretical guarantees.
problem Sparse deep learning's challenge of huge storage consumption and sparse structure recovery.
method Bayesian treatment with spike-and-slab priors and continuous relaxation of Bernoulli distribution for computationally efficient variational inferences.
result Provides variational posterior contraction rate, justifying consistency of the proposed method.
Deep neural networks can efficiently classify Gaussian mixture models with fewer neurons.
problem Classifying Gaussian mixture models efficiently.
method Used deep neural networks with two hidden layers to classify GMMs optimally.
result Deep neural networks require significantly fewer neurons than shallow ones for optimal classification of GMMs.
New method creates coresets for deep neural networks efficiently.
problem Efficiently handling large data streams with limited resources.
method Cardinality-constrained bilevel optimization for deep neural networks.
result Demonstrated efficient generation of coresets for deep neural networks.
Efficiently learns deep factor graphs using Gaussian belief propagation.
problem Learning in deep factor graphs with efficient inference.
method Treats all relevant quantities as random variables, uses belief propagation for inference.
result Efficiently solves training and prediction problems in deep factor graphs with belief propagation.
DAFL learns efficient neural networks without training data.
problem Training data unavailable for deep networks.
method Generative adversarial networks (GANs) to create training samples.
result Achieves high accuracy (92.22%) on CIFAR-10 dataset.
CrossQ improves sample efficiency in deep RL with reduced computation.
problem Improving sample efficiency in deep reinforcement learning.
method Introduces CrossQ, a lightweight algorithm using Batch Normalization and removing target networks.
result Matches or surpasses state-of-the-art methods in sample efficiency with reduced computational cost.
META2 improves taxonomic classification and abundance estimation in metagenomics with deep learning and memory efficiency.
problem Memory constraints and inefficiencies in taxonomic classification and abundance estimation for metagenomics.
method Developed a novel memory-efficient read classification technique combining deep learning and locality-sensitive hashing, and formulated abundance estimation as a Multiple Instance Learning problem.
result Our approach outperforms conventional methods in both single-read taxonomic classification and abundance estimation, especially when memory is limited.
The paper uses deep learning to efficiently price Bermudan swaptions.
problem Pricing Bermudan swaptions efficiently and accurately.
method Combines differential machine learning, Monte Carlo simulation, and joint learning.
result Improves efficiency and accuracy in pricing Bermudan swaptions.
The paper introduces a new reinforcement learning model that improves sample efficiency and performance.
problem Improving sample efficiency and performance in reinforcement learning.
method The paper proposes a transcoder network that includes model learning in the DQN loss function, providing a richer training signal.
result The proposed method leads to superior results on Atari games compared to vanilla DQN.
DNM learns efficient representations of input data using deep neural maps.
problem Learning efficient representations of input data.
method Unsupervised representation learning and visualization using deep convolutional networks and self-organizing maps.
result DNM can learn efficient representations of the input data, reflecting class characteristics.
K-TanH is a fast, efficient deep learning activation function.
problem Improving the speed and efficiency of deep learning models.
method K-TanH is a novel, highly accurate, low-precision integer approximation of TanH.
result K-TanH achieves up to 5x speedup over Intel SVML and is more efficient.
New algorithm reduces Q-value overestimation in deep reinforcement learning.
problem Q-value overestimation in deep reinforcement learning for high-dimensional state spaces.
method Adapting information theory concepts, an intrinsic penalty signal is introduced to encourage reduced Q-value estimates.
result Algorithm outperforms other methods in Atari games, demonstrating efficient and robust learning.
A new correction term improves sample efficiency in deep reinforcement learning.
problem Momentum accumulation in TD learning leads to doubly stale gradients.
method Proposed a correction term to address the issue of doubly stale gradients.
result Improves sample efficiency in policy evaluation.
A new algorithm improves efficiency in selecting examples for deep learning.
problem Efficiently choosing multiple examples to mark up for deep learning on large datasets.
method Large BatchBALD algorithm, approximating BatchBALD with reduced computational complexity.
result Comparable quality in selection while significantly reducing computation time, especially for large batches.
Examines learning efficiency in neural networks and related models.
problem Analyzing efficiency in deep learning models with singular learning coefficients.
method Examined learning coefficients in neural networks and three-layer neural networks with ReLU units.
result Extended results to include Softmax function, providing a broader understanding of learning efficiency.