We show that a generative random field model, which we call generative ConvNet, can be derived from the commonly used discriminative ConvNet, by assuming a ConvNet for multi-category classification and assuming one of the categories is a base category generated by a reference distribution. If we further assume that the…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,982 papers · 148 categories
Trend · papers per month
48 results for “ConvNets”
TSSC images enhance chaotic signal classification using ConvNets.
problem Classifying chaotic signals accurately and robustly.
method Triad State Space Construction (TSSC) for image encoding, Convolutional Neural Network (ConvNet) for classification.
result TSSC-ConvNet achieves high accuracy and robustness in chaotic signal classification.
Convolutional neural networks (ConvNets) have demonstrated an exceptional capacity to discern visual patterns from digital images and signals. Unfortunately, such powerful ConvNets do not generalize well to arbitrary-shaped manifolds, where data representation does not fit into a tensor-like grid. Hence, many fields of…
A complex-valued convolutional network (convnet) implements the repeated application of the following composition of three operations, recursively applying the composition to an input vector of nonnegative real numbers: (1) convolution with complex-valued vectors followed by (2) taking the absolute value of every entry…
High-dimensional ConvNets detect patterns in 32+ dimensions for geometric registration.
problem Detecting geometric patterns in high-dimensional spaces.
method High-dimensional convolutional networks applied to geometric registration problems.
result High-dimensional ConvNets outperform global pooling approaches in 3D registration and image correspondence.
Graph-structured data such as social networks, functional brain networks, gene regulatory networks, communications networks have brought the interest in generalizing deep learning techniques to graph domains. In this paper, we are interested to design neural networks for graphs with variable length in order to solve le…
Visualizes ConvNets without confounding effects.
problem Misinterpretation of saliency maps due to confounding variables.
method Univariate statistical tests and partial back-propagation to remove confounding effects.
result Visualization of confounder-free saliency maps.
Single-Path NAS designs efficient ConvNets for mobile devices in hours.
problem Designing efficient ConvNets for mobile devices under latency constraints.
method Single-Path NAS, a differentiable method that reduces search cost and inference time.
result Achieves state-of-the-art accuracy on ImageNet with 79ms inference latency.
EAST compresses deep ConvNets for tiny memory nodes.
problem Memory constraints in tiny devices for deep ConvNets.
method Encoding-Aware Sparse Training (EAST) with adaptive group pruning and LZ4 weight encoding.
result EAST achieves deep memory compression with lower sparsity and higher accuracy.
Deep learning with convolutional neural networks for decoding and visualization of EEG pathologycs.LG
We apply convolutional neural networks (ConvNets) to the task of distinguishing pathological from normal EEG recordings in the Temple University Hospital EEG Abnormal Corpus. We use two basic, shallow and deep ConvNet architectures recently shown to decode task-related information from EEG at least as well as establish…
Analog arrays are a promising upcoming hardware technology with the potential to drastically speed up deep learning. Their main advantage is that they compute matrix-vector products in constant time, irrespective of the size of the matrix. However, early convolution layers in ConvNets map very unfavorably onto analog a…
Single-Path NAS designs efficient ConvNets in under 4 hours.
problem Designing efficient ConvNets for mobile devices under time constraints.
method Differentiable NAS method using one single-path ConvNet.
result Achieves 74.96% top-1 accuracy on ImageNet with 79ms latency.
ConvNets and Imagenet have driven the recent success of deep learning for image classification. However, the marked slowdown in performance improvement combined with the lack of robustness of neural networks to adversarial examples and their tendency to exhibit undesirable biases question the reliability of these metho…
The paper addresses over-fitting in deep learning models trained on imbalanced data.
problem Over-fitting to minor classes in deep learning models trained on imbalanced data.
method Investigated feature deviation and proposed class-dependent temperatures (CDT) to compensate for it.
result CDT helps in overcoming feature deviation, improving model performance on test data of minor classes.
This study simplifies ML learning with ConvNet potentials using MCMC, achieving realistic samples without complex hyper-parameters.
problem Training ConvNet potentials for ML learning with realistic samples using MCMC.
method Minimal hyper-parameter framework, noise-initialized MCMC, and correct Langevin noise tuning.
result ConvNet potentials can learn realistic samples with minimal hyper-parameters and correct MCMC tuning.
Convolutional Neural Networks, as most artificial neural networks, are commonly viewed as methods different in essence from kernel-based methods. We provide a systematic translation of Convolutional Neural Networks (ConvNets) into their kernel-based counterparts, Convolutional Kernel Networks (CKNs), and demonstrate th…
Paper shows spectral filters can transfer between different graphs discretizing the same space.
problem Transferability of spectral filters between different graphs.
method Analysis of spectral filters on graphs discretizing the same space.
result Spectral filters have similar effects on graphs discretizing the same space.
This paper proposes a multi-grid method for learning energy-based generative ConvNet models of images. For each grid, we learn an energy-based probabilistic model where the energy function is defined by a bottom-up convolutional neural network (ConvNet or CNN). Learning such a model requires generating synthesized exam…
Recently, there is increasing interest and research on the interpretability of machine learning models, for example how they transform and internally represent EEG signals in Brain-Computer Interface (BCI) applications. This can help to understand the limits of the model and how it may be improved, in addition to possi…
KM method reduces ConvNet parameters to 9% higher accuracy with minimal additional memory.
problem Expensive memory usage for training ConvNets on embedded devices.
method Kernel Modulation (KM) method that adapts all network parameters for each task.
result KM delivers up to 9% higher accuracy than other parameter-efficient methods.
Video sequences contain rich dynamic patterns, such as dynamic texture patterns that exhibit stationarity in the temporal domain, and action patterns that are non-stationary in either spatial or temporal domain. We show that a spatial-temporal generative ConvNet can be used to model and synthesize dynamic patterns. The…
EfficientNet improves ConvNets by scaling depth, width, and resolution effectively.
problem Improving ConvNets' accuracy and efficiency at larger scales.
method Uniformly scaling depth, width, and resolution using a compound coefficient.
result EfficientNet-B7 achieves state-of-the-art 84.3% top-1 accuracy on ImageNet.
ConvNets improve nonstationary covariance estimation for large-scale spatial data.
problem Estimating nonstationary spatial covariance functions on large scales.
method Convolutional Neural Networks (ConvNets) for subregion identification and selection.
result Enhanced accuracy in parameter estimation using ConvNet-based partitioning.
Single-Path NAS reduces NAS search cost to 3 hours, achieving state-of-the-art mobile image classification.
problem Efficiently automate ConvNet design under mobile latency constraints.
method Single-Path NAS, using one single-path ConvNet with shared parameters.
result Achieves state-of-the-art top-1 ImageNet accuracy (75.62%) in 8 epochs (24 TPU-hours).
We consider the task of one-shot learning of visual categories. In this paper we explore a Bayesian procedure for updating a pretrained convnet to classify a novel image category for which data is limited. We decompose this convnet into a fixed feature extractor and softmax classifier. We assume that the target weights…
Methods from convex optimization are widely used as building blocks for deep learning algorithms. However, the reasons for their empirical success are unclear, since modern convolutional networks (convnets), incorporating rectifier units and max-pooling, are neither smooth nor convex. Standard guarantees therefore do n…
In this paper we consider the problem of semi-supervised learning with deep Convolutional Neural Networks (ConvNets). Semi-supervised learning is motivated on the observation that unlabeled data is cheap and can be used to improve the accuracy of classifiers. In this paper we propose an unsupervised regularization term…
We present a novel architecture, the "stacked what-where auto-encoders" (SWWAE), which integrates discriminative and generative pathways and provides a unified approach to supervised, semi-supervised and unsupervised learning without relying on sampling during training. An instantiation of SWWAE uses a convolutional ne…
This paper studies the cooperative training of two generative models for image modeling and synthesis. Both models are parametrized by convolutional neural networks (ConvNets). The first model is a deep energy-based model, whose energy function is defined by a bottom-up ConvNet, which maps the observed image to the ene…
3D ConvNets improved with Project & Excite for medical imaging segmentation.
problem Improving segmentation performance in 3D medical imaging.
method Proposed Project & Excite (PE) modules for 3D F-CNNs, extending 2D recalibration methods.
result Project & Excite modules boost segmentation performance up to 0.3 in Dice Score.
Tricks improve retail product image classification accuracy.
problem Retail Product Image Classification
method Various tricks including a new LCA layer, Instagram-pretrained Convnet, and Maximum Entropy loss.
result Increased accuracy of fine-tuned convnets by a large margin.
Discovering transformations for disentangled representations without prior knowledge of the underlying Lie group.
problem Discovering nonlinear transformations for disentangled representations without prior knowledge of the underlying Lie group.
method Approximating target vectors as matrix-vector products of the form \(\boldsymbol{\widetilde{y}}_i = \boldsymbol{\varphi}(t_i) \boldsymbol{x}_i\) where \(\boldsymbol{\varphi}(t_i)\) belongs to a one-parameter subgroup of \(\mathrm{GL}_n (\mathbb{R})\).
result Learning a Lie algebra from unlabeled data pairs without explicit group definition.
Recently, graph neural networks have been adopted in a wide variety of applications ranging from relational representations to modeling irregular data domains such as point clouds and social graphs. However, the space of graph neural network architectures remains highly fragmented impeding the development of optimized …
Deep neural net improves sleep stage classification from raw EEG.
problem Improving accuracy of sleep stage classification from raw EEG data.
method 34-layer deep residual ConvNet architecture trained on raw single channel EEG data.
result Proposed network outperforms state-of-the-art methods in sleep staging accuracy.
End-to-end DRNs outperform ConvNets in EEG decoding.
problem Improving performance of Deep Riemannian Networks (DRNs) in EEG decoding.
method Wide, end-to-end DRN architecture designed and tested on five public EEG datasets.
result EE(G)-SPDNet outperforms state-of-the-art ConvNets in EEG decoding.
The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Studycs.LG
Study shows how network width affects SGD hyper-parameters and generalization.
problem Understanding how network width impacts SGD hyper-parameters and generalization.
method Generated model families by increasing network width, performed hyper-parameter search.
result Wider networks achieve higher test accuracy and optimal normalized noise scale.
Estimating the state of a dynamical system from a series of noise-corrupted observations is fundamental in many areas of science and engineering. The most well-known method, the Kalman smoother (and the related Kalman filter), relies on assumptions of linearity and Gaussianity that are rarely met in practice. In this p…
The Euclidean scattering transform was introduced nearly a decade ago to improve the mathematical understanding of the success of convolutional neural networks (ConvNets) in image data analysis and other tasks. Inspired by recent interest in geometric deep learning, which aims to generalize ConvNets to manifold and gra…
Convolutional Neural Network is known as ConvNet have been extensively used in many complex machine learning tasks. However, hyperparameters optimization is one of a crucial step in developing ConvNet architectures, since the accuracy and performance are reliant on the hyperparameters. This multilayered architecture pa…
EEG is the most common signal source for noninvasive BCI applications. For such applications, the EEG signal needs to be decoded and translated into appropriate actions. A recently emerging EEG decoding approach is deep learning with Convolutional or Recurrent Neural Networks (CNNs, RNNs) with many different architectu…
Improves posterior approximation speed for Dirichlet process mixture models.
problem Inefficiency of stochastic variational inference in large datasets.
method Uses stochastic gradient ascent with adaptive stepsize optimization.
result Adaptive stepsize improves speed and performance of posterior approximation.
Expressive efficiency refers to the relation between two architectures A and B, whereby any function realized by B could be replicated by A, but there exists functions realized by A, which cannot be replicated by B unless its size grows significantly larger. For example, it is known that deep networks are exponentially…
Computer vision systems for automatic image categorization have become accurate and reliable enough that they can run continuously for days or even years as components of real-world commercial applications. A major open problem in this context, however, is quality control. Good classification performance can only be ex…
U-Nets use belief propagation for efficient image denoising and classification.
problem Efficiently denoise and classify images using generative hierarchical models.
method Interpreted U-Nets as implementing belief propagation in tree-structured models.
result U-Nets can efficiently approximate denoising functions with sample complexity bounds.
Method separates target signal properties from noisy mixtures.
problem Signal recovery from noisy mixtures with specific statistical properties.
method Statistical component separation method using noise samples and matching statistics.
result Method outperforms standard denoising methods in recovering target signal properties.
Stochastic variational inference is an established way to carry out approximate Bayesian inference for deep models. While there have been effective proposals for good initializations for loss minimization in deep learning, far less attention has been devoted to the issue of initialization of stochastic variational infe…
VectorNet predicts car behavior using vectorized HD maps and agent dynamics.
problem Predicting behavior in multi-agent systems with self-driving cars.
method VectorNet uses hierarchical graph neural networks on vectorized representations of HD maps and agent trajectories.
result VectorNet achieves comparable or better performance than state-of-the-art methods while using fewer parameters and less computational power.
We explore the generalization of scattering transforms from traditional (e.g., image or audio) signals to graph data, analogous to the generalization of ConvNets in geometric deep learning, and the utility of extracted graph features in graph data analysis. In particular, we focus on the capacity of these features to r…