MAPS and MAPS-SE learn from multiple suboptimal experts to improve policies efficiently.
problem Learning from multiple suboptimal experts in reinforcement learning.
method Active policy improvement from multiple black-box oracles.
result MAPS and MAPS-SE achieve sample efficiency and state-wise imitation learning.
The paper proves deep neural networks with analytic activation can approximate any function.
problem Approximating functions with neural networks using analytic activation functions.
method Elementary proofs for real and complex networks, Stone-Weierstrass theorem, Mergelyan's theorem.
result Closure of neural network classes equals space of polynomials for analytic activation.
Self-attention model improves HAR from wearable sensors.
problem Capturing spatio-temporal context from sensor data.
method Proposes a self-attention based neural network model.
result Significant performance improvement over state-of-the-art models.
Improved generalization bounds for CNNs using Rademacher complexity.
problem Establishing non-vacuous generalization bounds for deep learning models.
method Rademacher complexity framework with novel contraction lemmas for high-dimensional mappings.
result Enhanced generalization bounds for a broader class of activation functions.
Traditional human activity recognition (HAR) based on time series adopts sliding window analysis method. This method faces the multi-class window problem which mistakenly labels different classes of sampling points within a window as a class. In this paper, a HAR algorithm based on U-Net is proposed to perform activity…
Visualizes deep neural networks for speech recognition using learned topographic filter maps.
problem Unintuitive internal structure of deep neural networks complicates activation visualization.
method Trains a convolutional speech recognition model with filters arranged in a 2D grid, highlighting similar filters.
result Topographic filter maps visualize artificial neuron activations more intuitively.
Visualizes DNNs using topographic maps for better understanding.
problem Difficulty in understanding how DNNs solve tasks.
method Adapting neuroscience methods to visualize DNN activations.
result Improved transparency and interpretability of DNN-based systems.
This paper classifies typhoon damage features using aerial photography.
problem Typhoon damages and slow recovery times.
method Aerial photography for classification of eight classes including land covers and disaster areas.
result Visualize and explain typhoon disaster features using convolutional activation maps.
Study embeddings between Barron spaces with various activation functions, focusing on RePU.
problem Understanding the influence of activation functions on infinitely wide neural networks.
method Prove embeddings by constructing push-forward maps on measures representing functions.
result Barron spaces with RePU activation have a hierarchical structure similar to Sobolev spaces.
We propose a technique for making Convolutional Neural Network (CNN)-based models more transparent by visualizing input regions that are 'important' for predictions -- or visual explanations. Our approach, called Gradient-weighted Class Activation Mapping (Grad-CAM), uses class-specific gradient information to localize…
Risk-based active learning improves SHM decision-making.
problem Lack of prior labels for structural health monitoring.
method Risk-based active learning approach to guide data labeling.
result Improves decision-maker's performance in SHM.
Active learning method balances bias and variance under class imbalance.
problem Active learning under label shift when class proportions differ.
method Mediated Active Learning under Label Shift (MALLS) using a 'medial distribution'.
result MALLS reduces asymptotic sample complexity under arbitrary label shift.
AReLU uses attention-based rectification to improve neural network performance.
problem Improving neural network performance through better activation functions.
method Integrates attention mechanism with rectified linear unit (ReLU) to learn and scale feature maps.
result AReLU significantly boosts performance of most network architectures with minimal changes.
URNNs are as expressive as general RNNs with ReLU activations.
problem Expressiveness of URNNs compared to general RNNs.
method Input-output equivalence between URNNs and contractive RNNs with ReLU activations.
result URNNs are as expressive as general RNNs with ReLU activations.
Building models, or maps, of robot environments is a highly active research area; however, most existing techniques construct unstructured maps and assume static environments. In this paper, we present an algorithm for learning object models of non-stationary objects found in office-type environments. Our algorithm exp…
Balanced Activation improves object detection performance on long-tailed datasets.
problem Mismatch between training and testing label distributions in object detection.
method Introduces Balanced Activation (Balanced Softmax and Balanced Sigmoid) to address label distribution shift.
result Balanced Activation provides ~3% gain in mAP on LVIS-1.0 compared to state-of-the-art methods.
Improved EXACT strategy reduces GNN memory consumption and runtime.
problem Efficiently training large-scale GNNs with reduced memory usage.
method Block-wise quantization of intermediate activation maps with improved variance minimization.
result Further reduction in memory consumption (>15%) and runtime speedup (5%) with similar performance trade-offs.
First the title could be also understood as ``3-manifolds related by non-zero degree maps" or "Degrees of maps between 3-manifolds" for some aspects in this survey talk. The topology of surfaces was completely understood at the end of 19th century, but maps between surfaces kept to be an active topic in the 20th centur…
Framework tackles class imbalance and noisy labels in active learning.
problem Class imbalance and noisy labels in real-world datasets.
method Uses foundation model priors to select informative samples for active learning.
result Substantial annotation savings (over 50%) with preserved performance and robustness.
Random feature maps improve forecasting of chaotic dynamical systems.
problem Forecasting chaotic dynamical systems with high accuracy.
method Data-driven random feature maps with tanh activation, skip connections, and localization.
result Effective forecasting skill for dynamical systems with dimensions up to 512.
ActiLabel learns activity patterns across diverse sensor devices.
problem Limited adoption of activity recognition models across different domains due to diverse sensor devices.
method Combination of graph model and optimal tiered mapping for learning activity labels.
result Superior performance compared to state-of-the-art methods on public datasets.
We consider active maximum a posteriori (MAP) inference problem for Hidden Markov Models (HMM), where, given an initial MAP estimate of the hidden sequence, we select to label certain states in the sequence to improve the estimation accuracy of the remaining states. We develop an analytical approach to this problem for…
Localization of chest pathologies in chest X-ray images is a challenging task because of their varying sizes and appearances. We propose a novel weakly supervised method to localize chest pathologies using class aware deep multiscale feature learning. Our method leverages intermediate feature maps from CNN layers at di…
Nodes in real world networks often have class labels, or underlying attributes, that are related to the way in which they connect to other nodes. Sometimes this relationship is simple, for instance nodes of the same class are may be more likely to be connected. In other cases, however, this is not true, and the way tha…
Active learning reduces simulation needs for high-fidelity mobility maps.
problem Efficiently training machine learning classifiers for high-fidelity mobility maps.
method Active learning based on PAC learning theory to reduce simulation needs.
result Our sampling algorithm trains neural networks with higher accuracy using less than half the number of simulations.
CBDA improves active learning for semantic segmentation, especially with imbalanced classes.
problem Class imbalance degrades performance in domain adaptive active learning.
method Class Balanced Dynamic Acquisition (CBDA) selects more balanced labels for active learning.
result CBDA increases minority class performance and outperforms baselines by 0.6-2.4 mIoU.
There has been a growing interest in expressivity of deep neural networks. However, most of the existing work about this topic focuses only on the specific activation function such as ReLU or sigmoid. In this paper, we investigate the approximation ability of deep neural networks with a broad class of activation functi…
This paper introduces a novel, generic active learning method for one-class classification. Active learning methods play an important role to reduce the efforts of manual labeling in the field of machine learning. Although many active learning approaches have been proposed during the last years, most of them are restri…
SympNets identify Hamiltonian systems from data using linear, activation, and gradient modules.
problem Identifying Hamiltonian systems from data.
method Composition of linear, activation, and gradient modules; universal approximation theorems.
result SympNets can approximate arbitrary symplectic maps and generalize well to various Hamiltonian systems.
Universal approximation for ODENet and ResNet with a single activation function.
problem Approximating complex dynamical systems with limited vector fields.
method Examined ODENet and ResNet with vector fields composed of a single activation function and affine mapping.
result ODENet and ResNet with restricted vector fields can uniformly approximate those with general vector fields.
Labeling each instance in a large dataset is extremely labor- and time- consuming . One way to alleviate this problem is active learning, which aims to which discover the most valuable instances for labeling to construct a powerful classifier. Considering both informativeness and representativeness provides a promising…
Generalizes neural networks for infinite-dimensional mappings, including PDE solutions.
problem Learning mappings between infinite-dimensional spaces and finite-dimensional approximations.
method Graph kernel network architecture with message passing for kernel integration.
result Competitive performance compared to state-of-the-art solvers for PDEs.
Softmax cross-entropy optimizes mutual information in neural networks.
problem Understanding the relationship between mutual information and classification neural networks.
method Demonstrated that optimizing softmax cross-entropy maximizes mutual information between inputs and labels.
result Softmax cross-entropy can approximate mutual information and highlight relevant image regions.
Establishes upper bounds on generalization error in active learning.
problem Improving query algorithms in active learning.
method Derives upper bounds on generalization error using informativeness and representativeness query strategies.
result Validates the use of regularization techniques to ensure bounds' validity.
Transformers exhibit sparse activation maps, reducing computational load and improving robustness.
problem Sparse activation in Transformer models.
method Extensive experiments on various Transformer architectures and tasks.
result Sparsity in Transformers is a prevalent phenomenon, reducing FLOP count and improving model robustness.
GPMI method interpolates uncertain atrial conduction velocity on non-Euclidean manifolds.
problem Uncertainty in atrial conduction velocity calculations.
method Gaussian Process Manifold Interpolation (GPMI) on human atrial manifolds.
result GPMI accounts for atrial topology and calculates CV uncertainty.
O-MedAL optimizes medical image analysis with online active deep learning.
problem Improving accuracy in medical image analysis with limited labeled data.
method Online Active Deep Learning method that queries examples maximizing average distance to training set.
result Significant performance improvements, including 6.30% accuracy boost with 25% labeled data.
Researchers use operator learning to predict cardiac activation and repolarization times.
problem Computational demands and need for clear, interpretable information in cardiac electrophysiology.
method Exploiting Fourier Neural Operators (FNO) and Kernel Operator Learning (KOL) to learn operator mappings.
result Both FNO and KOL approaches are computationally efficient and robust to hyperparameters.
Study validates saliency maps of GNNs using selective inference.
problem Reliability of GNN saliency maps in graph-structured data.
method Statistical testing framework with selective inference to control Type I error rate.
result Valid p-values for salient subgraphs, ensuring meaningful information. Many neural network architectures rely on the choice of the activation function for each hidden layer. Given the activation function, the neural network is trained over the bias and the weight parameters. The bias catches the center of the activation, and the weights capture the scale. Here we propose to train the netw…
Active learning methods increase classification quality by means of user feedback. An important subcategory is active learning for outlier detection with one-class classifiers. While various methods in this category exist, selecting one for a given application scenario is difficult. This is because existing methods rel…
Study evaluates deep learning models for solar flare prediction with interpretability analysis.
problem Lack of interpretability in deep learning models for solar flare prediction.
method Proximity-based metric for analyzing attribution maps generated by Guided Grad-CAM.
result Models' predictions align with active region characteristics, offering insights into their behavior.
We study the theoretical advantages of active learning over passive learning. Specifically, we prove that, in noise-free classifier learning for VC classes, any passive learning algorithm can be transformed into an active learning algorithm with asymptotically strictly superior label complexity for all nontrivial targe…
The paper studies how neural networks evolve representations, finding a unique fixed point for nonlinear activations.
problem Understanding how neural networks transform input data across layers.
method Theoretical framework for the evolution of the kernel sequence, using mean-field regime and Hermite polynomials.
result For nonlinear activations, the kernel sequence converges globally to a unique fixed point.
Effective feature representation is key to the predictive performance of any algorithm. This paper introduces a meta-procedure, called Non-Euclidean Upgrading (NEU), which learns feature maps that are expressive enough to embed the universal approximation property (UAP) into most model classes while only outputting fea…
New active learning framework for multiclass classification beyond realizability assumption.
problem Active learning in non-realizable settings with convex model classes.
method Surrogate risk minimization, epoch-based fitting, aggregation of models.
result Achieves label and sample complexity comparable to prior work in non-realizable settings.
Improves Active Learning by considering class imbalance and difficulty.
problem Active Learning's focus on individual samples ignores class distribution and difficulty.
method Proposes a method based on Bayes' rule to incorporate class imbalance, using a Variational Auto Encoder (VAE).
result Significantly outperforms state-of-the-art methods on datasets with heavy data imbalance.
Deep belief network improves smartphone activity recognition.
problem Activity recognition on mobile devices.
method Categorization through deep belief network.
result 98.25% correct diagnosis in training data, 93.01% in test data.