arXiv research

New algorithms for constrained online optimization with memory and predictions.

New constraints on space and adaptivity in bandits force more batches and memory use.

Memory-limited learning tackles adversarial bandits with reduced storage.

We study distribution testing with communication and memory constraints in the following computational models: (1) The {\em one-pass streaming model} where the goal is to minimize the sample complexity of the protocol subject to a memory constraint, and (2) A {\em distributed model} where the data samples reside at mul…

Survey on statistical inference under memory constraints.

SMGD trains low-bit neural networks with memory constraints.

Subbagging estimation for big data reduces memory usage while maintaining statistical consistency.

Improved deep learning model deployment on tiny MCUs with mixed-precision quantization.

Severe constraints on memory and computation characterizing the Internet-of-Things (IoT) units may prevent the execution of Deep Learning (DL)-based solutions, which typically demand large memory and high processing load. In order to support a real-time execution of the considered DL model at the IoT unit level, DL sol…

We note that known methods achieving the optimal oracle complexity for first order convex optimization require quadratic memory, and ask whether this is necessary, and more broadly seek to characterize the minimax number of first order queries required to optimize a convex Lipschitz function subject to a memory constra…

Many machine learning approaches are characterized by information constraints on how they interact with the training data. These include memory and sequential access constraints (e.g. fast first-order methods to solve stochastic optimization problems); communication constraints (e.g. distributed learning); partial acce…

New algorithm reduces costs and latency for large language model inference.

The implementation of Deep Convolutional Neural Networks (ConvNets) on tiny end-nodes with limited non-volatile memory space calls for smart compression strategies capable of shrinking the footprint yet preserving predictive accuracy. There exist a number of strategies for this purpose, from those that play with the to…

Memory-constrained algorithms need superlinear memory for efficient convex optimization.

POET enables large neural network training on tiny devices with reduced energy.

HabitatAgent offers a multi-agent system for transparent housing consultation.

This paper presents a novel end-to-end methodology for enabling the deployment of low-error deep networks on microcontrollers. To fit the memory and computational limitations of resource-constrained edge-devices, we exploit mixed low-bitwidth compression, featuring 8, 4 or 2-bit uniform quantization, and we model the i…

New memory models improve LSSVM's generalization and reduce time cost.

NeuZip compresses neural network weights to save memory during training and inference.

We assume that we are given a time series of data from a dynamical system and our task is to learn the flow map of the dynamical system. We present a collection of results on how to enforce constraints coming from the dynamical system in order to accelerate the training of deep neural networks to represent the flow map…

We propose two optimization techniques to minimize memory usage and computation while meeting system timing constraints for real-time classification in wearable systems. Our method derives a hierarchical classifier structure for Support Vector Machine (SVM) in order to reduce the amount of computations, based on the pr…

This paper addresses the problem of scalable optimization for L1-regularized conditional Gaussian graphical models. Conditional Gaussian graphical models generalize the well-known Gaussian graphical models to conditional distributions to model the output network influenced by conditioning input variables. While highly …

Recomputation algorithms collectively refer to a family of methods that aims to reduce the memory consumption of the backpropagation by selectively discarding the intermediate results of the forward propagation and recomputing the discarded results as needed. In this paper, we will propose a novel and efficient recompu…

XLA compiler extension improves memory efficiency for machine learning.

FactorMiner discovers financial alpha factors with low redundancy.

PipeDream-2BW accelerates large model training by 20x with minimal memory usage.

We study the problem of identifying correlations in multivariate data, under information constraints: Either on the amount of memory that can be used by the algorithm, or the amount of communication when the data is distributed across several machines. We prove a tight trade-off between the memory/communication complex…

Recent work on Bayesian optimization has shown its effectiveness in global optimization of difficult black-box objective functions. Many real-world optimization problems of interest also have constraints which are unknown a priori. In this paper, we study Bayesian optimization for constrained problems in the general ca…

HardCoRe-NAS finds fitting neural networks adhering to hard resource constraints.

New model captures long-term memory effects in epidemic dynamics.

Novel algorithm speeds up log-determinant estimation for large matrices.

SPARC improves continual learning with minimal memory and computational overhead.

Paper proposes efficient network pruning method for deep neural networks.

Group fairness is an important concern for machine learning researchers, developers, and regulators. However, the strictness to which models must be constrained to be considered fair is still under debate. The focus of this work is on constraining the expected outcome of subpopulations in kernel regression and, in part…

This paper studies the inference problem in quantile regression (QR) for a large sample size $n$ but under a limited memory constraint, where the memory can only store a small batch of data of size $m$. A natural method is the naïve divide-and-conquer approach, which splits data into batches of size $m$, computes the l…

A framework for analyzing financial systems under scenario constraints.

Tensor models decode human perception and memory using SPO triples.

We consider the problem of performing linear regression over a stream of $d$-dimensional examples, and show that any algorithm that uses a subquadratic amount of memory exhibits a slower rate of convergence than can be achieved without memory constraints. Specifically, consider a sequence of labeled examples $(a_1,b_1)…

Serenity optimizes neural network execution for edge devices by scheduling with optimal memory footprint.

This paper optimizes SMPC for neural network inference, reducing memory and time.

A new RL model ensures safe learning in uncertain environments.

PySAD offers a unified Python framework for efficient streaming anomaly detection.

New algorithms for streaming bandits with limited memory.

This paper reviews methods to create compact neural networks for IoT applications.

Sharp limits found for storing and retrieving input-output associations in linear associative memories.

Efficiently preserves old class knowledge in memory-limited settings.

PLUMAGE improves large model training efficiency and stability.

The ability to look multiple times through a series of pose-adjusted glimpses is fundamental to human vision. This critical faculty allows us to understand highly complex visual scenes. Short term memory plays an integral role in aggregating the information obtained from these glimpses and informing our interpretation …