This work examines aggregation functions in Deep Set learning.
problem The sensitivity of Deep Set networks to aggregation function choices.
method Investigation of alternative aggregation functions, including learnable recurrent ones.
result Learnable aggregations improve performance, reduce hyper-parameter sensitivity, and generalize better.
Deep Sets approximates functions on sets with high-dimensional latent space.
problem Modeling functions of sets (permutation-invariant functions).
method Deep Sets, a method known to be a universal approximator for continuous set functions.
result Deep Sets' universal approximation property is only guaranteed with a sufficiently high-dimensional latent space.
Introduces epistemic deep learning for better uncertainty estimation in neural networks.
problem Uncertainty quantification in deep neural networks.
method Random-set convolutional neural networks with belief function-based loss functions.
result Epistemic approach produces better performance in uncertainty estimation.
Algorithm constructs confidence sets for deep neural networks with PAC guarantees.
problem Ensuring reliable predictions for deep neural networks with high confidence.
method Combines calibrated prediction and learning theory bounds.
result Constructs PAC confidence sets for various deep models.
Deep networks with a wide layer ensure sublevel set connectivity.
problem Ensuring connectivity of sublevel sets in deep learning.
method Analyzing the connectivity of sublevel sets in deep neural networks with a specific layer width.
result A single wide layer of width N+1 suffices to prove connectivity of sublevel sets. Combining Bayesian deep learning and split conformal prediction affects out-of-distribution coverage.
problem Improving out-of-distribution coverage in multiclass image classification.
method Combining Bayesian deep learning with split conformal prediction methods.
result Combining methods can reduce out-of-distribution coverage in some cases.
DMPS learns set data by connecting graph and set learning.
problem Lack of relational learning for set data.
method Deep Message Passing on Sets (DMPS) that connects graph and set learning.
result DMPS achieves competitive or superior results on synthetic and real-world datasets.
Deep learning aids causal inference in complex settings.
problem Estimating heterogeneous treatment effects in non-linear, time-varying, and encoded confounders.
method Intuitive introduction to deep learning and causal inference, focusing on observational data.
result Maximizes accessibility to causal inference through deep learning.
Bayesian deep learning improves deep learning's capabilities across diverse settings.
problem Overlooked metrics, tasks, and data types in deep learning.
method Revisits strengths of Bayesian deep learning and addresses challenges.
result Bayesian deep learning can elevate deep learning's capabilities across diverse settings.
New deep learning model for matching sets of items, preserving exchangeability.
problem Matching two different sets of items while preserving exchangeability.
method Exchangeable deep neural networks architecture and efficient training framework.
result Significant improvements in fashion set recommendation and group re-identification.
This work bridges continual learning, active learning, and open set recognition in deep neural networks.
problem Protecting previously acquired representations from catastrophic forgetting in deep neural networks.
method Surveying the literature and proposing a consolidated view to integrate open set recognition and active learning principles.
result Joint improvement in alleviating catastrophic forgetting, querying data, selecting task orders, and robust open world application.
Develops deep jump learning for continuous treatment OPE.
problem Estimating mean outcomes under new treatment rules using historical data from different rules.
method Adaptive deep discretization of continuous treatment space using deep learning and multi-scale change point detection.
result Validated method through theoretical results, simulations, and real application to Warfarin Dosing.
Paper analyzes and simplifies deep learning framework parallelism for better performance.
problem Optimizing performance in complex machine learning frameworks.
method Analyzed and quantified the impact of key design features on performance, providing guidelines.
result Performance tuning guidelines outperform default settings by 1.29x and 1.34x.
This study compares deep learning with other ML algorithms on credit scoring unbalanced data.
problem Training models on highly unbalanced data is challenging.
method Compared several machine learning algorithms with deep learning on a credit scoring unbalanced dataset.
result Deep learning shows promising performance on imbalanced data with little samples.
A novel deep learning method for chemometric data improves performance over transfer learning.
problem Training deep neural networks from chemometric data with varying input sizes.
method Weight sharing in deep convolutional neural networks trained on multiple data sets of different sizes.
result Superior performance compared to transfer learning, especially when training on medium and small data sets.
Introduces topological deep learning for neural network classification problems.
problem Classifying neural networks using minimal topological structures.
method Formalizes classification problems in a topological setting.
result Demonstrates conditions for the feasibility of classification problems in neural networks.
Challenge encourages reproducible deep learning methods.
problem Inconsistent training behavior in deep learning competitions.
method Evaluate methods based on their training procedures, retrain in controlled settings.
result Guaranteed reproducibility and generalization of methods.
Develops deep neural network techniques for sets as input and output.
problem Bottlenecks in set representation and discontinuity issues in set prediction.
method Techniques for set representation and prediction, addressing unordered nature and relations.
result Improvements in set prediction and representation across various experiments.
We introduce a simple permutation equivariant layer for deep learning with set structure.This type of layer, obtained by parameter-sharing, has a simple implementation and linear-time complexity in the size of each set. We use deep permutation-invariant networks to perform point-could classification and MNIST-digit sum…
ChainerRL is a deep reinforcement learning library for Python.
problem Training and reproducing deep reinforcement learning algorithms.
method Built on Chainer framework, implements various DRL algorithms.
result Fosters reproducible research and replicates benchmark results.
Recently, a number of works have studied clustering strategies that combine classical clustering algorithms and deep learning methods. These approaches follow either a sequential way, where a deep representation is learned using a deep autoencoder before obtaining clusters with k-means, or a simultaneous way, where dee…
AGOP mechanism explains deep neural collapse in neural networks.
problem Explaining the rigid structure of data representations in deep neural networks.
method Introducing AGOP and Deep RFM to demonstrate DNC.
result AGOP mechanism causes deep neural collapse in neural networks.
ECP method improves image classifier uncertainty sets.
problem Generating reliable uncertainty sets for deep classifiers.
method Evidential Conformal Prediction (ECP) based on EDL.
result ECP outperforms state-of-the-art methods in set size and adaptivity.
Deep learning architectures have proved versatile in a number of drug discovery applications, including the modelling of in vitro compound activity. While controlling for prediction confidence is essential to increase the trust, interpretability and usefulness of virtual screening models in drug discovery, techniques t…
O-MedAL optimizes medical image analysis with online active deep learning.
problem Improving accuracy in medical image analysis with limited labeled data.
method Online Active Deep Learning method that queries examples maximizing average distance to training set.
result Significant performance improvements, including 6.30% accuracy boost with 25% labeled data.
Deep MF extracts hierarchical features from large data sets.
problem Mining complex, interleaved features in large data sets.
method Deep matrix factorization models and algorithms.
result Deep MF achieves outstanding performance on unsupervised tasks.
Characterizes deep neural network weight space for adversarial attacks.
problem Poor performance of deep learning models in adversarial examples.
method Characterizes deep neural network solution space using two paradigms.
result Adversarial attacks are less successful against Associative Memory Models.
Survey on deep learning for malware classification, including unknown threats.
problem Classifying and recognizing unknown malware variants.
method Review of deep learning techniques and OSR solutions.
result Deep learning can effectively classify known malware and recognize unknown threats.
This paper introduces Deep Incremental Boosting, a new technique derived from AdaBoost, specifically adapted to work with Deep Learning methods, that reduces the required training time and improves generalisation. We draw inspiration from Transfer of Learning approaches to reduce the start-up time to training each incr…
Deep Sets improve reinforcement learning agent's object-centered navigation and generalization.
problem Improving reinforcement learning agents' ability to generalize to unseen objects and goals.
method Combining object-wise permutation invariant networks (Deep Sets) and gated-attention mechanisms.
result Agent demonstrates strong generalization to out-of-distribution goals in a procedurally-generated 2D world.
This paper surveys gradient-based multi-objective deep learning methods.
problem Balancing multiple conflicting objectives in deep learning models.
method Gradient-based techniques adapted from Multi-Objective Optimization.
result Comprehensive survey of gradient-based multi-objective deep learning algorithms.
Split learning improves deep learning in healthcare by sharing data.
problem Data scarcity in healthcare limits deep learning applications.
method Distributed learning approach for collaborative training of deep neural networks.
result Split learning outperformed non-collaborative methods in both binary and multi-label classification tasks.
ICAL improves deep learning model accuracy and NLL with optimized batch labeling.
problem Deep Bayesian Active Learning for efficient model training.
method ICAL uses HSIC to measure dependency and optimizes batch size scaling.
result Significant improvements in model accuracy and NLL on image datasets.
New model explains deep learning performance at large learning rates.
problem Understanding deep learning performance at different learning rates.
method Developed neural networks with solvable training dynamics.
result Large learning rates lead to convergence to flatter minima.
In this article we review computational aspects of Deep Learning (DL). Deep learning uses network architectures consisting of hierarchical layers of latent variables to construct predictors for high-dimensional input-output models. Training a deep learning architecture is computationally intensive, and efficient linear…
Deep learning autoencoder model clusters unlabeled time series data.
problem Clustering unlabeled time series data.
method Two-stage approach: create labels from time series characteristics, then use autoencoder for clustering.
result 87.5% accuracy in clustering unseen time series data.
Deep reinforcement learning (deep RL) has achieved superior performance in complex sequential tasks by using deep neural networks as function approximators to learn directly from raw input images. However, learning directly from raw images is data inefficient. The agent must learn feature representation of complex stat…
Deep learning aligns GC-MS peaks for biomarker discovery.
problem Aligning retention times of GC-MS peaks across different samples.
method ChromAlignNet, a deep learning model for peak alignment.
result ChromAlignNet outperforms existing methods on complex data sets.
Deep Sets improve reinforcement learning for autonomous driving with variable inputs.
problem Optimal decision making in autonomous driving with varying number of objects.
method Employed Deep Sets for high-level decision making in reinforcement learning.
result Deep Sets outperform other approaches in performance and generalization.
NCP improves deep classifier uncertainty quantification efficiency.
problem Uncertainty quantification for deep classifiers in high-stake applications.
method Neighborhood Conformal Prediction (NCP) algorithm.
result NCP produces smaller prediction sets than traditional CP methods.
With the growth of deep learning, how to describe deep neural networks unifiedly is becoming an important issue. We first formalize neural networks mathematically with their directed graph representations, and prove a generation theorem about the induced networks of connected directed acyclic graphs. Then, we set up a …
The paper extends deep multi-task learning to diverse domains, finding shared functionality.
problem General problem solving from diverse deep learning tasks.
method Decomposes tasks into subproblems, optimizing sharing through stochastic algorithm.
result Joint learning across diverse domains and architectures improves performance.
Deep learning models complex multivariate extremes using geometric shapes.
problem Modeling complex extremal dependencies in high-dimensional data.
method Geometric representation and deep learning for flexible semi-parametric models.
result First approach to modeling limit sets using deep learning for high-dimensional data.
A new measure predicts deep learning model performance.
problem Predicting the generalization error of deep learning models.
method 2sED measure based on effective dimension, layerwise iterative approximation.
result 2sED correlates well with training error and generalization error.
Testing the implementation of deep learning systems and their training routines is crucial to maintain a reliable code base. Modern software development employs processes, such as Continuous Integration, in which changes to the software are frequently integrated and tested. However, testing the training routines requir…
New research shows deep ReLU networks can be learned with polylogarithmic width.
problem Learning deep ReLU networks with limited over-parameterization.
method Using gradient descent, the study establishes learning guarantees for networks with polylogarithmic width.
result Deep ReLU networks can be learned with a polylogarithmic width condition, not just a high degree polynomial.
New insights into continual learning for deep models, showing convergence issues but local linear solutions.
problem Challenges in continual learning for homogeneous deep models.
method Sequential projections onto task margin sets, leveraging nonconvex projection theory.
result Local linear convergence under certain conditions for homogeneous deep networks.
This paper benchmarks batch RL algorithms on Atari, finding DQN and partially-trained policies perform best.
problem Deep RL algorithms fail in batch setting.
method Benchmarked batch RL algorithms on Atari using a single partially-trained policy.
result Many batch RL algorithms underperform DQN and partially-trained policies.