ORIGAMI accelerates ML algorithms by splitting compute tasks between in-memory and off-chip accelerators.
problem Memory bandwidth bottleneck in ML processing.
method Heterogeneous in-memory accelerators and off-chip compute platform, pattern-matching for compute patterns, computation-splitting compiler.
result ORIGAMI outperforms state-of-the-art accelerators in performance and energy-efficiency.
Study uses stacked hourglass networks to improve facial landmark detection for medical diagnosis.
problem Improving accuracy of facial landmark detection for medical diagnosis.
method Conducted a study on landmark localisation methods using stacked hourglass networks.
result State-of-the-art stacked hourglass architecture outperforms traditional methods.
Tree-SMU enables strong compositional generalization in neural networks.
problem Zero-shot generalization to novel compositions of concepts.
method Tree Stack Memory Units (Tree-SMU) with Stack Memory Units (SMU).
result Tree-SMU achieves strong empirical results on mathematical reasoning benchmarks.
Stacked LSTM networks improve traffic volume forecasting.
problem Accurate traffic volume prediction for better planning.
method Applying stacked Long Short-Term Memory (LSTM) networks for time series forecasting.
result Stacked LSTM networks enhance the accuracy of traffic volume predictions.
Paper classifies brain signals using eigenvalues for 2D and 3D educational content questions.
problem Classifying brain signals for 2D and 3D educational content questions.
method Eigenvalues of covariance matrix used as features; KNN and SVM classifiers applied.
result No significant difference in learning, memory retention, and recall between 2D and 3D educational content.
In this paper, a taxonomy for memory networks is proposed based on their memory organization. The taxonomy includes all the popular memory networks: vanilla recurrent neural network (RNN), long short term memory (LSTM ), neural stack and neural Turing machine and their variants. The taxonomy puts all these networks und…
A new method increases 3D medical image segmentation accuracy and speed.
problem Training large 3D medical images on GPUs with limited memory.
method Data-swapping method to enlarge GPU memory and avoid patching.
result Improved segmentation accuracy and speed for full-size images.
Study evaluates using multiple slices as input for CNNs in medical image segmentation.
problem Improving segmentation performance in medical images with limited computational resources.
method Compared pseudo-3D and 2D approaches using different CNN architectures and datasets.
result Multi-slice inputs did not significantly improve segmentation performance over 2D or 3D CNNs.
A new 2.5D U-net for 3D segmentation reduces memory constraints.
problem Large storage requirements for 3D convolutions in neural networks.
method Transform volumetric data into sequences of 2D images, apply 2D convolutions, and reconstruct.
result Outperforms existing methods in volumetric segmentation tasks.
Long Short-Term Memory (LSTM) is a well-known method used widely on sequence learning and time series prediction. In this paper we deployed stacked LSTM model in an application of weather forecasting. We propose a 2-layer spatio-temporal stacked LSTM model which consists of independent LSTM models per location in the f…
New neural stack and Turing Machine architectures prove stability and computational power.
problem Designing stable neural network architectures for Turing Machine simulation.
method Introducing neural stack and Turing Machine architectures, proving stability and computational equivalence.
result Differentiable nnTM with bounded neurons can simulate Turing Machine in real-time and is equivalent to UTM.
LatticeNet segments 3D point clouds faster and more efficiently.
problem Challenges in applying CNNs to 3D point cloud data.
method Embeds point cloud geometry into a permutohedral lattice for fast convolutions.
result Achieves state-of-the-art performance in 3D segmentation.
The paper explores different perspectives on rhombile tilings.
problem None explicitly stated, focuses on different viewpoints.
method Four ways of looking at rhombile tilings: cubes, groups, lines, and points.
result Different methods provide insights into rhombile tilings.
Extended LSTM improves language modeling performance.
problem Improving LSTM for large-scale language modeling.
method Exponential gating, modified memory structure, and residual stacking.
result xLSTM outperforms state-of-the-art models in performance and scaling.
New self-shrinkers with multiple ends constructed by stacking planes.
problem Constructing self-shrinkers with any number of ends in 3D space.
method PDE gluing methods and Linearised Doubling methodology.
result Construction of self-shrinkers with 2J+1 ends and genus 2J(m−1). Enhanced LSTM model learns complex temporal dependencies.
problem Modeling long-term dependencies in sequential data.
method Gamma-LSTM with hierarchical memory units and gates.
result Gamma-LSTM outperforms regular and stacked LSTMs in sequence prediction.
Convolutional Neural Networks (CNNs) are extremely computationally demanding, presenting a large barrier to their deployment on resource-constrained devices. Since such systems are where some of their most useful applications lie (e.g. obstacle detection for mobile robots, vision-based medical assistive technology), si…
Worldsheet wraps a 3D mesh sheet onto a single image to synthesize novel views.
problem Synthesizing novel views from a single image with large viewpoint changes.
method Shrink-wrapping a planar mesh sheet onto the input image, consistent with learned depth.
result Worldsheet consistently outperforms prior methods on single-image view synthesis.
Precisely forecasting wind speed is essential for wind power producers and grid operators. However, this task is challenging due to the stochasticity of wind speed. To accurately predict short-term wind speed under uncertainties, this paper proposed a multi-variable stacked LSTMs model (MSLSTM). The proposed method uti…
Stable long-term predictions for fluid flows using neural networks.
problem Predicting complex dynamics of fluid flows with high temporal stability.
method End-to-end trained neural network architecture combining CNN for spatial compression and LSTM for temporal prediction.
result Novel latent space subdivision (LSS) allows stable and controllable long-term predictions.
iPrescribe offers fast online offer recommendations using deep learning.
problem Online offer recommendation in real-time.
method Ensemble of deep learning and machine learning algorithms, optimized streaming technology stack, and efficient LSTM deployment.
result 90th percentile recommendation latency of 38 milliseconds.
The first part of this text is a gentle exposition of some basic constructions and results in the extended prequantum theory of Chern-Simons-type gauge field theories. We explain in some detail how the action functional of ordinary 3d Chern-Simons theory is naturally localized ("extended", "multi-tiered") to a map on t…
STanHop predicts multivariate time series with memory-enhanced capabilities.
problem Predicting multivariate time series with memory-enhanced capabilities.
method Sparse Tandem Hopfield Network (STanHop) with two external memory modules.
result STanHop outperforms dense Hopfield models in memory retrieval error.
Proposes a constant memory iterative inverse model using invertible networks.
problem Memory limitations in iterative learning approaches for inverse problems.
method Invertible networks to avoid storing intermediate activations, constant memory model.
result Trains 400-layer models on 3D MRI volumes, achieving state-of-the-art image reconstruction.
Z-Net improves 3D CT volume segmentation for surgical planning.
problem Discontinuities and class-imbalances in 3D CT volume segmentation.
method Z-Net uses anisotropic spatial separable convolutions to preserve full field-of-view.
result Z-Net achieves up to 12.6% improvement in IoU for CT segmentation.
In this work, we propose a novel recurrent neural network (RNN) architecture. The proposed RNN, gated-feedback RNN (GF-RNN), extends the existing approach of stacking multiple recurrent layers by allowing and controlling signals flowing from upper recurrent layers to lower layers using a global gating unit for each pai…
3D object recognition accuracy can be improved by learning the multi-scale spatial features from 3D spatial geometric representations of objects such as point clouds, 3D models, surfaces, and RGB-D data. Current deep learning approaches learn such features either using structured data representations (voxel grids and o…
Deep learning improves 3D microscopy resolution without matched target images.
problem Anisotropic resolution in volumetric fluorescence microscopy.
method Cycle-consistent generative adversarial network trained on unpaired 2D images.
result Enhanced axial resolution and restored details between imaging planes.
In model-based reinforcement learning, generative and temporal models of environments can be leveraged to boost agent performance, either by tuning the agent's representations during training or via use as part of an explicit planning mechanism. However, their application in practice has been limited to simplistic envi…
This paper evaluates various representations for robotics tasks, improving performance in lifting, stacking, and pushing.
problem Improving data-efficiency in reinforcement learning for robotics with limited data.
method Systematic evaluation of common representations in three robotics tasks: lifting, stacking, and pushing.
result Some representations can perform as well as simulator states as agent inputs, challenging common intuitions.
DA-LSTM adapts LSTM depth to non-uniform data, improving efficiency.
problem Non-uniform information distribution in sequential data cannot be accurately modeled by traditional LSTM.
method Developed DA-LSTM architecture that dynamically adjusts LSTM depth based on information distribution.
result DA-LSTM reduces computation resource usage and convergence time by 41.78% and 46.01% respectively.
The past several years have seen both an explosion in the use of Convolutional Neural Networks (CNNs) and the design of accelerators to make CNN inference practical. In the architecture community, the lion share of effort has targeted CNN inference for image recognition. The closely related problem of video recognition…
Topology-GS improves 3D GS for better structural and feature integrity.
problem Compromised pixel-level and feature-level integrity in 3D GS.
method Incorporates Local Persistent Voronoi Interpolation (LPVI) and PersLoss based on persistent homology.
result Topology-GS outperforms existing methods in PSNR, SSIM, and LPIPS metrics.
PARMESAN learns from memory without parameters for fast, efficient continual learning.
problem Inflexibility in deep learning methods for continual learning.
method Transductive reasoning and memory search for parameter-free learning.
result 3-4 orders of magnitude faster than baselines, comparable performance.
Local-HDP learns independent topics for each 3D object category in real-time.
problem Learning independent topics for each 3D object category in real-time.
method Local-Hierarchical Dirichlet Process (Local-HDP) with online variational inference.
result Local-HDP outperforms other approaches in accuracy, scalability, and memory efficiency.
The rising use of deep learning and other big-data algorithms has led to an increasing demand for hardware platforms that are computationally powerful, yet energy-efficient. Due to the amount of data parallelism in these algorithms, high-performance 3D manycore platforms that incorporate both CPUs and GPUs present a pr…
Paper compares deep learning models for financial series prediction.
problem Predicting financial time series with deep learning methods.
method Compares MLP, 1D CNN, stacked LSTM, attention networks, and weighted attention networks.
result Attention LSTM model produces a 0.76 hit ratio, superior to other methods.
Kymatio simplifies scattering transforms for Python.
problem Signal processing and machine learning applications.
method Wavelet scattering transform implemented in Python.
result Efficient, GPU-accelerated implementation.
We propose a method based on deep learning to perform cardiac segmentation on short axis MRI image stacks iteratively from the top slice (around the base) to the bottom slice (around the apex). At each iteration, a novel variant of U-net is applied to propagate the segmentation of a slice to the adjacent slice below it…
Natural spatiotemporal processes can be highly non-stationary in many ways, e.g. the low-level non-stationarity such as spatial correlations or temporal dependencies of local pixel values; and the high-level variations such as the accumulation, deformation or dissipation of radar echoes in precipitation forecasting. Fr…
Animals execute goal-directed behaviours despite the limited range and scope of their sensors. To cope, they explore environments and store memories maintaining estimates of important information that is not presently available. Recently, progress has been made with artificial intelligence (AI) agents that learn to per…
IAM improves deep RL by selectively storing influential past observations.
problem Training and performance issues with recurrent neural networks in high-dimensional environments.
method Influence-aware memory architecture that restricts input to variables influencing hidden state.
result IAM outperforms standard RNNs in training speed and policy performance.
The paper analyzes how stacking improves model stability.
problem Lack of theoretical insight into how stacking works.
method Stability analysis of learning algorithms, focusing on hypothesis stability.
result The hypothesis stability of stacking is a product of base models and combiner.
Centroid Transformers reduce memory and computation by summarizing inputs into centroids.
problem Efficiently summarize inputs with reduced memory and computation.
method Generalizes self-attention to map N inputs to M centroids (M ≤ N), reducing complexity.
result Centroid Transformers reduce memory and computation while preserving key information.
New model captures long-range patterns in sequences efficiently.
problem Efficiently capturing long-range patterns in sequential data.
method Inspired by wavelet multiresolution analysis, introduces MultiresLayer with multiresolution convolution.
result State-of-the-art performance on sequence classification and autoregressive density estimation tasks.
We review the basic definition of a stack and apply it to the topological and smooth settings. We then address two subtleties of the theory: the correct definition of a ``stack over a stack'' and the distinction between small stacks (which are algebraic objects) and large stacks (which are generalized spaces).
Analyze and predict complex 3D shape deformations using LSTM autoencoders and oriented bounding boxes.
problem Detecting and predicting patterns in sequences of deforming 3D shapes.
method Use LSTM autoencoders to create low-dimensional representations of 3D shapes, incorporating oriented bounding boxes for structural components.
result The method detects patterns in plastic deformation and predicts future states of 3D shapes with improved accuracy.
In this article, we derive many properties of étale stacks in various contexts, and prove that étale stacks may be characterized categorically as those stacks that arise as prolongations of stacks on a site of spaces and local homeomorphisms. Moreover, we show that the bicategory of étale differentiable stacks and loca…