A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Improved EXACT strategy reduces GNN memory consumption and runtime.
FPG uses fractional calculus for efficient reinforcement learning with long-term memory.
The Long-Short-Term-Memory Recurrent Neural Networks (LSTM RNNs) are a popular class of machine learning models for analyzing sequential data. Their training on modern GPUs, however, is limited by the GPU memory capacity. Our profiling results of the LSTM RNN-based Neural Machine Translation (NMT) model reveal that fea…
The well-known Mori-Zwanzig theory tells us that model reduction leads to memory effect. For a long time, modeling the memory effect accurately and efficiently has been an important but nearly impossible task in developing a good reduced model. In this work, we explore a natural analogy between recurrent neural network…
We design and study a Contextual Memory Tree (CMT), a learning memory controller that inserts new memories into an experience store of unbounded size. It is designed to efficiently query for memories from that store, supporting logarithmic time insertion and retrieval operations. Hence CMT can be integrated into existi…
KineticSim: A lightweight, high-performance execution engine for real-time market simulators
SRG improves optimization efficiency with reduced memory and computation overhead.
Study proposes memory-efficient backpropagation for linear layers in neural networks.
Many engineers wish to deploy modern neural networks in memory-limited settings; but the development of flexible methods for reducing memory use is in its infancy, and there is little knowledge of the resulting cost-benefit. We propose structural model distillation for memory reduction using a strategy that produces a …
In this paper, the echo state network (ESN) memory capacity, which represents the amount of input data an ESN can store, is analyzed for a new type of deep ESNs. In particular, two deep ESN architectures are studied. First, a parallel deep ESN is proposed in which multiple reservoirs are connected in parallel allowing …
We propose to execute deep neural networks (DNNs) with dynamic and sparse graph (DSG) structure for compressive memory and accelerative execution during both training and inference. The great success of DNNs motivates the pursuing of lightweight models for the deployment onto embedded devices. However, most of the prev…
Recomputation algorithms collectively refer to a family of methods that aims to reduce the memory consumption of the backpropagation by selectively discarding the intermediate results of the forward propagation and recomputing the discarded results as needed. In this paper, we will propose a novel and efficient recompu…
Global convergence of an online (stochastic) limited memory version of the Broyden-Fletcher- Goldfarb-Shanno (BFGS) quasi-Newton method for solving optimization problems with stochastic objectives that arise in large scale machine learning is established. Lower and upper bounds on the Hessian eigenvalues of the sample …
SliceOut speeds up deep learning training without sacrificing accuracy.
Enhanced Hopfield model boosts memory retrieval capacity.
Improved biclustering algorithm reduces memory usage and runtime.
PCA reduces language model embeddings to improve speed and memory efficiency.
A method to reduce memory usage in NAS by pruning the search space.
SDA method reduces memory and time for assimilating noisy geophysical data.
Paper proposes efficient BNN inference flow to reduce computation and memory costs.
Memory-efficient algorithm reduces variance in off-policy RL.
A new text classification method using separable convolution reduces memory consumption.
This research uncovers high-performing subnetworks in deep GNNs without training.
KineticSim accelerates financial market simulations 3406x over CPU.
We present DRLViz, a visual analytics interface to interpret the internal memory of an agent (e.g. a robot) trained using deep reinforcement learning. This memory is composed of large temporal vectors updated when the agent moves in an environment and is not trivial to understand due to the number of dimensions, depend…
Serenity optimizes neural network execution for edge devices by scheduling with optimal memory footprint.
New algorithm learns optimal policies with minimal memory and time.
This paper reviews methods to create compact neural networks for IoT applications.
In this paper we study a family of variance reduction methods with randomized batch size---at each step, the algorithm first randomly chooses the batch size and then selects a batch of samples to conduct a variance-reduced stochastic update. We give the linear convergence rate for this framework for composite functions…
A new method reduces memory usage for deep neural networks ensembles.
New method for multi-modal depth prediction challenges.
A new framework for efficient sequence maps using Bayesian filtering and covariance.
ADHD is being recognized as a diagnosis which persists into adulthood impacting economic, occupational, and educational outcomes. There is an increased need to accurately diagnose and recommend interventions for this population. One consideration is the development and implementation of reliable and valid outcome measu…
Variance reduction has been commonly used in stochastic optimization. It relies crucially on the assumption that the data set is finite. However, when the data are imputed with random noise as in data augmentation, the perturbed data set be- comes essentially infinite. Recently, the stochastic MISO (S-MISO) algorithm i…
Variance reduction (VR) methods boost the performance of stochastic gradient descent (SGD) by enabling the use of larger, constant stepsizes and preserving linear convergence rates. However, current variance reduced SGD methods require either high memory usage or an exact gradient computation (using the entire dataset)…
We present CYCLADES, a general framework for parallelizing stochastic optimization algorithms in a shared memory setting. CYCLADES is asynchronous during shared model updates, and requires no memory locking mechanisms, similar to HOGWILD!-type algorithms. Unlike HOGWILD!, CYCLADES introduces no conflicts during the par…
Recent progress in deep learning is revolutionizing the healthcare domain including providing solutions to medication recommendations, especially recommending medication combination for patients with complex health conditions. Existing approaches either do not customize based on patient health history, or ignore existi…
EnLSTM network improves log generation from small datasets.
Improves energy efficiency of neuromorphic hardware by optimizing memory organization and encoding schemes.
This paper proposes a hardware-oriented dropout algorithm, which is efficient for field programmable gate array (FPGA) implementation. In deep neural networks (DNNs), overfitting occurs when networks are overtrained and adapt too well to training data. Consequently, they fail in predicting unseen data used as test data…
Paper reduces recommender system model size by 90%.
Efficient tensor kernel method reduces memory usage and computational cost for sparse regression.
We propose a max-pooling based loss function for training Long Short-Term Memory (LSTM) networks for small-footprint keyword spotting (KWS), with low CPU, memory, and latency requirements. The max-pooling loss training can be further guided by initializing with a cross-entropy loss trained network. A posterior smoothin…
Machine learning-based applications are increasingly prevalent in IoT devices. The power and storage constraints of these devices make it particularly challenging to run modern neural networks, limiting the number of new applications that can be deployed on an IoT system. A number of compression techniques have been pr…
Paper improves efficiency in matrix computations for Gaussian processes.
A method to reduce Hessian matrix calculation cost in gradient-based meta-learning.
TERA method speeds up derivative Gaussian processes in high dimensions.
In this work, we revisit fast dimension reduction approaches, as with random projections and random sampling. Our goal is to summarize the data to decrease computational costs and memory footprint of subsequent analysis. Such dimension reduction can be very efficient when the signals of interest have a strong structure…