arXiv research

Paper tackles online optimization with memory and competitive control.

New memory models improve LSSVM's generalization and reduce time cost.

The multimodal web elements such as text and images are associated with inherent memory costs to store and transfer over the Internet. With the limited network connectivity in developing countries, webpage rendering gets delayed in the presence of high-memory demanding elements such as images (relative to text). To ove…

Tail averaging consists in averaging the last examples in a stream. Common techniques either have a memory requirement which grows with the number of samples to average, are not available at every timestep or do not accomodate growing windows. We propose two techniques with a low constant memory cost that perform tail …

Convolutional Neural Networks (CNN) are being actively explored for safety-critical applications such as autonomous vehicles and aerospace, where it is essential to ensure the reliability of inference results in the presence of possible memory faults. Traditional methods such as error correction codes (ECC) and Triple …

Deep Neural Networks(DNNs) require huge GPU memory when training on modern image/video databases. Unfortunately, the GPU memory is physically finite, which limits the image resolutions and batch sizes that could be used in training for better DNN performance. Unlike solutions that require physically upgrade GPUs, the G…

Neural SDEs improve time series generation efficiency.

New algorithm reduces decision switching in dynamic environments.

Deep RNNs compute American option prices and deltas efficiently.

State-of-the-art convolutional neural networks (CNNs) used in vision applications have large models with numerous weights. Training these models is very compute- and memory-resource intensive. Much research has been done on pruning or compressing these models to reduce the cost of inference, but little work has address…

We study the power of different types of adaptive (nonoblivious) adversaries in the setting of prediction with expert advice, under both full-information and bandit feedback. We measure the player's performance using a new notion of regret, also known as policy regret, which better captures the adversary's adaptiveness…

New algorithm reduces costs and latency for large language model inference.

We study distribution testing with communication and memory constraints in the following computational models: (1) The {\em one-pass streaming model} where the goal is to minimize the sample complexity of the protocol subject to a memory constraint, and (2) A {\em distributed model} where the data samples reside at mul…

Improves energy efficiency of neuromorphic hardware by optimizing memory organization and encoding schemes.

New algorithm reduces dynamic regret in time-varying movement costs.

This work analyzes and optimizes memory and compute costs of learned optimizers.

Investor optimizes investment timing with future knowledge, overcoming transaction costs.

Resource-efficient oblique trees reduce neural signal classification costs.

In neural architecture search (NAS), the space of neural network architectures is automatically explored to maximize predictive accuracy for a given task. Despite the success of recent approaches, most existing methods cannot be directly applied to large scale problems because of their prohibitive computational complex…

TOMA generates abstract graphs for RL, reducing memory and computation costs.

GPU (graphics processing unit) has been used for many data-intensive applications. Among them, deep learning systems are one of the most important consumer systems for GPU nowadays. As deep learning applications impose deeper and larger models in order to achieve higher accuracy, memory management becomes an important …

New ODE solvers improve training efficiency and accuracy.

Efficiently infers latent SDEs with scalable memory and time costs.

BatchEnsemble reduces ensemble costs by 3X in training and testing.

Learning to remember long sequences remains a challenging task for recurrent neural networks. Register memory and attention mechanisms were both proposed to resolve the issue with either high computational cost to retain memory differentiability, or by discounting the RNN representation learning towards encoding shorte…

Optimal linear contracts are possible even with memory in Gaussian settings.

Study reduces memory needs for active learning with enriched queries.

Recurrent neural networks (RNNs) provide state-of-the-art performance in processing sequential data but are memory intensive to train, limiting the flexibility of RNN models which can be trained. Reversible RNNs---RNNs for which the hidden-to-hidden transition can be reversed---offer a path to reduce the memory require…

New algorithm for nonstationary GLBs reduces computation and memory costs.

Many engineers wish to deploy modern neural networks in memory-limited settings; but the development of flexible methods for reducing memory use is in its infancy, and there is little knowledge of the resulting cost-benefit. We propose structural model distillation for memory reduction using a strategy that produces a …

Efficient tensor kernel method reduces memory usage and computational cost for sparse regression.

Spot option prices, forwards and options on forwards relevant for the commodity markets are computed when the underlying process S is modelled as an exponential of a process ξ with memory as e.g. a Lévy semi-stationary process. Moreover a risk premium \r{ho} representing storage costs, illiquidity, convenience yield or…

Online method learns sparse models efficiently in large scale settings.

A method to reduce Hessian matrix calculation cost in gradient-based meta-learning.

A new energy-efficient pruning method for federated learning.

FROSH and DFROSH speed up online sketching hashing for big data.

A new algorithm reduces memory and computational needs for reinforcement learning.

Bayesian neural networks can be partially stochastic without losing predictive power.

We formalize the problem of trading-off DNN training time and memory requirements as the tensor rematerialization optimization problem, a generalization of prior checkpointing strategies. We introduce Checkmate, a system that solves for optimal rematerialization schedules in reasonable times (under an hour) using off-t…

Sequential learning of tasks using gradient descent leads to an unremitting decline in the accuracy of tasks for which training data is no longer available, termed catastrophic forgetting. Generative models have been explored as a means to approximate the distribution of old tasks and bypass storage of real data. Here …

Distributed learning with random features and gradient descent improves performance and reduces memory usage.

SmartExchange trades memory for computation in neural networks.

PSI-LinUCB improves scalability for large recommender systems.

New algorithm controls linear systems with bandit feedback, achieving optimal regret.

As more data are produced each day, and faster, data stream mining is growing in importance, making clear the need for algorithms able to fast process these data. Data stream mining algorithms are meant to be solutions to extract knowledge online, specially tailored from continuous data problem. Many of the current alg…

CICLAD efficiently mines frequent closed itemsets from data streams with minimal memory usage.

SAGE improves memory efficiency by selectively adding, merging, or ignoring new facts.

Designing deep learning models for highly-constrained hardware would allow imbuing many edge devices with intelligence. Microcontrollers (MCUs) are an attractive platform for building smart devices due to their low cost, wide availability, and modest power usage. However, they lack the computational resources to run ne…