Study asks if memory constraints affect optimal convex optimization methods.
problem Characterize the minimax number of queries for convex optimization with memory constraints.
method Analyze first order methods under memory limitations.
result Optimal oracle complexity may be achievable with limited memory.
New algorithms for constrained online optimization with memory and predictions.
problem Control of constrained dynamical systems and scheduling with reconfiguration budgets.
method Proposed algorithms achieving sublinear regret and constraint violation under time-varying constraints, both with and without predictions.
result First algorithms achieving sublinear regret and constraint violation in constrained online optimization with memory.
Efficiently tests discrete distributions with limited memory and communication.
problem Testing discrete distributions with constraints on memory and communication.
method Developed efficient algorithms for uniformity/identity and closeness testing in streaming and distributed models.
result Nearly-tight lower bounds on sample complexity and communication cost for uniformity testing.
New constraints on space and adaptivity in bandits force more batches and memory use.
problem Simultaneous space and adaptivity constraints in stochastic bandits.
method Proved lower bounds and constructed an algorithm with near-minimax regret.
result Near-minimax regret requires more batches and memory than previously thought.
Memory-limited learning tackles adversarial bandits with reduced storage.
problem Adversarial bandit problem with limited memory storage.
method Hierarchical learning policy with sublinear memory requirement.
result Established sublinear regret bounds for weak and shifting regrets.
Study shows memory and communication constraints impact correlation detection in data.
problem Detecting correlations with limited memory and communication resources.
method Proves a tight trade-off between memory/communication complexity and sample complexity.
result Optimal sample complexity requires quadratic memory/communication complexity in dimension.
Survey on statistical inference under memory constraints.
problem Effect of memory limitations on statistical inference performance.
method Review of state-of-the-art in several canonical problems.
result Identification of fundamental building blocks and useful techniques.
Paper designs distributed CNN execution for IoT units.
problem Memory and processing constraints in IoT units.
method Optimization problem to minimize latency, considering memory and processing constraints.
result Design supports multiple sources and CNNs in a single IoT system.
Memory constraints slow linear regression convergence.
problem Linear regression with limited memory.
method Analyzing memory and sample requirements for regression.
result Subquadratic memory algorithms converge slower than unrestricted methods.
SMGD trains low-bit neural networks with memory constraints.
problem Training large neural networks with limited memory.
method Stochastic Markov Gradient Descent (SMGD).
result Encouraging numerical results and theoretical guarantees.
Efficiently estimates quantile regression under memory constraints.
problem Inference in quantile regression with limited memory.
method Divide-and-conquer with iterative refinement.
result Asymptotic normality and efficiency of the estimator.
Low-precision quantization improves kernel approximation under memory constraints.
problem Training kernel approximation methods efficiently with limited memory.
method Low-precision quantization of random Fourier features (LP-RFFs).
result LP-RFFs can match the performance of full-precision RFFs and Nyström method with significantly less memory.
Tree-based method selects features from high-dimensional datasets with memory constraints.
problem Feature selection in high-dimensional datasets with limited memory.
method Randomized trees on subsamples of variables, mixing relevant and randomly selected variables.
result The method provides theoretical analysis and convergence speed under various scenarios.
Subbagging estimation for big data reduces memory usage while maintaining statistical consistency.
problem Memory constraints in analyzing massive datasets.
method Randomly subsample the data, aggregate estimators from subsamples, and use incomplete U-statistics theory.
result Subbagging estimator achieves N \sqrt{N} N -consistency and asymptotic normality under certain conditions. Improved deep learning model deployment on tiny MCUs with mixed-precision quantization.
problem Memory limitations prevent accurate deployment of DNN models on tiny MCUs.
method Automated mixed-precision quantization using Reinforcement Learning for MCU constraints.
result Mixed-precision models achieve high accuracy with uniform quantization policies.
New algorithm reduces costs and latency for large language model inference.
problem Optimizing inference costs and latency for large language models with GPU constraints.
method Formulated as an online scheduling problem with endogenous memory growth, introduced fluid model and WAIT algorithms.
result Reduced costs and latency, especially in near-overloaded and overloaded regimes.
Many machine learning approaches are characterized by information constraints on how they interact with the training data. These include memory and sequential access constraints (e.g. fast first-order methods to solve stochastic optimization problems); communication constraints (e.g. distributed learning); partial acce…
EAST compresses deep ConvNets for tiny memory nodes.
problem Memory constraints in tiny devices for deep ConvNets.
method Encoding-Aware Sparse Training (EAST) with adaptive group pruning and LZ4 weight encoding.
result EAST achieves deep memory compression with lower sparsity and higher accuracy.
DeepDrum generates drum rhythms under musical constraints.
problem Generating rhythms that adhere to musical style and constraints.
method Adaptive Neural Network with Conditional Layers containing musical parameters and instrumentation.
result DeepDrum effectively generates rhythms that resemble learned styles while conforming to given constraints.
Memory-constrained algorithms need superlinear memory for efficient convex optimization.
problem Efficiently minimizing convex functions with limited memory.
method Analyzing first-order algorithms with superlinear memory constraints.
result Superlinear memory is necessary for optimal performance in convex optimization.
POET enables large neural network training on tiny devices with reduced energy.
problem Training large neural networks on memory-limited edge devices.
method Jointly optimizes rematerialization and paging for memory reduction, formulating an MILP for energy-efficient training.
result POET trains ResNet-18 and BERT within Cortex-M memory constraints, outperforming current methods in energy efficiency.
A new algorithm for streaming submodular maximization with memory constraints.
problem Maximizing a submodular function in a stream with memory removal.
method STAR-T algorithm based on partitioning and thresholding.
result A simple greedy algorithm achieves constant-factor approximation after memory removal.
HabitatAgent offers a multi-agent system for transparent housing consultation.
problem Opaque reasoning and brittle multi-constraint handling in housing recommendation systems.
method HabitatAgent is a multi-agent architecture with specialized roles for memory, retrieval, generation, and validation.
result HabitatAgent achieves 95% accuracy in real user consultation scenarios, significantly outperforming a strong baseline.
This paper enables deep network inference on microcontrollers with improved accuracy and reduced memory usage.
problem Deploying deep networks on resource-constrained edge-devices with low memory and computational constraints.
method Mixed low-bitwidth compression, rule-based iterative procedure for bit precision determination, quantization-aware retraining, and integer-only model conversion.
result Improved Top1 accuracy of 68% on a 2MB FLASH memory STM32H7 microcontroller, 8% higher than 8-bit implementations.
New memory models improve LSSVM's generalization and reduce time cost.
problem Improving LSSVM's generalization performance and reducing training time.
method Introducing MIMM and WIMM models that modify LSSVM's memory influence mechanism.
result MIMM and WIMM models outperform LSSVM in generalization and time efficiency.
A modified VDCNN model reduces size and latency for mobile platforms.
problem Memory and processing constraints on mobile platforms.
method Temporal Depthwise Separable Convolutions and Global Average Pooling.
result The squeezed model (SVDCNN) is 10x-20x smaller with minimal accuracy loss.
NeuZip compresses neural network weights to save memory during training and inference.
problem Memory constraints in neural network training and inference.
method Entropy-based dynamic weight compression.
result Significant reduction in memory usage without performance loss.
Optimizes memory and computation in wearable systems.
problem Minimizing resource usage in real-time classification for wearable systems.
method Hierarchical SVM classifier structure and memory optimization techniques.
result Up to 56% reduction in memory storage for activity recognition.
A new method reduces memory usage in neural networks by 36-81%.
problem Minimizing memory usage in neural networks during backpropagation.
method Formalized as a graph theory problem, uses dynamic programming to minimize computational overhead.
result Reduces peak memory consumption by 36-81% on various networks.
This paper addresses the problem of scalable optimization for L1-regularized conditional Gaussian graphical models. Conditional Gaussian graphical models generalize the well-known Gaussian graphical models to conditional distributions to model the output network influenced by conditioning input variables. While highly …
FactorMiner discovers financial alpha factors with low redundancy.
problem Finding novel financial alpha factors in a vast search space.
method Modular Skill Architecture and Experience Memory to distill and guide exploration.
result FactorMiner constructs a diverse library of high-quality factors with competitive performance.
XLA compiler extension improves memory efficiency for machine learning.
problem Memory constraints limit the scalability of memory-intensive machine learning algorithms.
method Developed an XLA compiler extension that adjusts algorithm data-flow representation to fit memory limits.
result k-nearest neighbour and sparse Gaussian process regression can be run at larger scales.
The paper presents methods to enforce constraints for time series prediction in various learning modes.
problem Learning the flow map of dynamical systems from time series data.
method Enforces constraints during training for supervised, unsupervised, and reinforcement learning.
result Accelerates training and increases predictive ability of deep neural networks.
PipeDream-2BW accelerates large model training by 20x with minimal memory usage.
problem Training large models requires memory beyond single accelerator capacity.
method Pipeline parallelism, weight gradient coalescing, double buffering.
result Accelerates large model training by up to 20x.
Recent work on Bayesian optimization has shown its effectiveness in global optimization of difficult black-box objective functions. Many real-world optimization problems of interest also have constraints which are unknown a priori. In this paper, we study Bayesian optimization for constrained problems in the general ca…
HardCoRe-NAS finds fitting neural networks adhering to hard resource constraints.
problem Finding fitting neural networks that adhere to hard resource constraints.
method Accurate formulation of resource requirement and scalable search method.
result HardCoRe-NAS generates state-of-the-art architectures strictly satisfying hard resource constraints.
Quantized neural networks reduce model size and energy consumption.
problem Memory and energy constraints in mobile devices.
method Using integer or binary representations to store weights instead of 32-bit floats.
result Quantization can reduce model size and energy consumption without significantly compromising performance.
New model captures long-term memory effects in epidemic dynamics.
problem Identifying memory effects in disease progression and recovery.
method Physics-informed neural networks (PINN) with fractional SEIRD model.
result Fractional memory order α α α improves predictive performance over classical models. Novel algorithm speeds up log-determinant estimation for large matrices.
problem Efficiently estimating log-determinants of large positive definite matrices under memory constraints.
method Hierarchical algorithm based on block-wise computation of LDL decomposition.
result Accurate estimation of NTK log-determinants from a tiny fraction of the full dataset.
SPARC improves continual learning with minimal memory and computational overhead.
problem Efficient continual learning for deep neural networks.
method Combines task-specific working memories and task-agnostic semantic memory.
result Significantly reduces parameter usage (6% of full-model surrogates) while maintaining performance.
Paper proposes efficient network pruning method for deep neural networks.
problem High computational and memory cost of deep neural networks.
method Annealing and direct sparsity control for channel-level pruning.
result Proposed method achieves better or competitive performance compared to other methods.
A framework for analyzing financial systems under scenario constraints.
problem Quantifying worst-case and best-case performance in financial systems.
method Quantitative automata-based framework integrating event history automata and weighted finance finite automata.
result Exact calculation of upper and lower payoff bounds with interpretable witness event histories.
New algorithms for sampling DPPs reduce computational cost.
problem Sampling DPPs efficiently with reduced computational cost.
method Three-phase algorithm with eigendecomposition, eigenvector sampling, and orthogonalisation. Dual variant for memory constraints.
result Algorithm reduces computational complexity from O ( N μ 3 ) \mathcal{O}(Nμ^3) O ( N μ 3 ) to O ( N μ 2 ) \mathcal{O}(Nμ^2) O ( N μ 2 ) . Tensor models decode human perception and memory using SPO triples.
problem Understanding implicit and explicit perception and memory in the brain.
method Tensor models with SPO triples, dual representations, and four layers.
result Semantic memory is crucial for explicit perception and declarative memories.
New method processes large images with sampling and attention.
problem Computational and memory constraints in processing large images.
method Fully differentiable end-to-end trainable model that samples and processes only a fraction of the input image.
result Sampling from attention distribution results in an unbiased estimator of the full model with minimal variance.
Serenity optimizes neural network execution for edge devices by scheduling with optimal memory footprint.
problem Order of nodes in irregular neural networks affects memory footprint, complicating execution under resource constraints.
method Memory-aware compiler using dynamic programming and graph rewriting to find optimal schedules.
result Achieves optimal peak memory and further improves it with graph rewriting, reducing memory usage by 1.68x-1.86x compared to TensorFlow Lite.
This work addresses fairness constraints for multiple subpopulations in machine learning models.
problem Fairness constraints for multiple subpopulations in machine learning models.
method Constraining the expected outcome of subpopulations in kernel regression and decision tree regression, specifically random forests and boosted trees.
result The proposed solution does not affect the computational or memory complexity of decision trees and can be easily integrated post training.
This paper optimizes SMPC for neural network inference, reducing memory and time.
problem Memory and time constraints in secure neural network inference.
method Implemented ABY2.0 protocol, optimized memory usage, and used a helper node.
result MNIST inference reduced from 8.03 GB RAM and 200s to 0.2 GB RAM and 32s.