OpTorch optimizes deep learning for resource-limited environments.
problem Resource constraints in deep learning training.
method Optimized deep learning pipelines in training time and memory.
result Achieved similar accuracy to existing libraries with reduced memory usage.
Efficiently discovers Bayesian network structure with reduced memory usage.
problem Bayesian network structure discovery is NP-hard and memory-intensive.
method Progressively leveled scoring approach with hierarchical computation.
result Achieved processing of a 28-variable Bayesian network using only memory.
Count-sketches reduce memory usage for deep learning models without sacrificing performance.
problem Memory constraints in deep learning models as they grow larger.
method Maintain a linear sketch to compress auxiliary variables in optimization methods.
result Compressed optimization methods maintain the same performance as full-sized baselines while using significantly less memory.
Optimizes memory and computation in wearable systems.
problem Minimizing resource usage in real-time classification for wearable systems.
method Hierarchical SVM classifier structure and memory optimization techniques.
result Up to 56% reduction in memory storage for activity recognition.
New method optimizes memory usage in neural networks, improving sequential learning.
problem Current memory models in neural networks waste memory and computation.
method Formulated an optimization problem to maximize information storage, introduced Cached Uniform Writing.
result Proved Cached Uniform Writing optimizes memory usage and outperforms other methods.
New memory-query tradeoffs for convex optimization algorithms.
problem Optimizing memory usage for convex optimization algorithms.
method Analyzing randomized first-order algorithms for minimizing convex functions.
result Cutting plane methods are optimal in terms of memory and query complexity.
Flora uses random projections to achieve high-rank updates with low memory usage.
problem Excessive memory usage in large neural networks during training.
method Flora approximates LoRA using random projections to enable high-rank updates with sublinear space complexity.
result Flora achieves high-rank updates with significantly reduced memory usage compared to LoRA.
A new method reduces memory usage for large models by three orders of magnitude.
problem Memory limitations in training large models.
method Extreme tensoring for adaptive preconditioning.
result Significant reduction in memory usage without performance degradation.
RIn-Close_CVC2 improves biclustering efficiency by reducing memory usage.
problem Mining maximal biclusters in numerical datasets efficiently and without redundancy.
method Proposes RIn-Close_CVC2, a new version of RIn-Close_CVC that eliminates redundant biclusters without a symbol table.
result RIn-Close_CVC2 reduces memory usage and improves runtime compared to RIn-Close_CVC.
New method reduces memory usage for adaptive optimization.
problem Significant memory overhead in adaptive optimizers like Adagrad and Adam.
method Memory-efficient adaptive optimization method that retains per-parameter adaptivity.
result Significantly larger models and batch sizes possible with up to 2-fold speedups.
PES method reduces bias in gradient estimation for unrolled graphs.
problem High variance and bias in gradient estimation for unrolled computation graphs.
method Divide graph into unrolls, apply ES update, accumulate correction terms.
result PES provides unbiased, low-variance gradient estimates.
Paper optimizes neural network layers to reduce energy usage without sacrificing accuracy.
problem Energy consumption in deep neural networks during inference.
method Layerwise noise maximization to optimize reliability of memory elements.
result Reduces memory energy consumption by 3.3 times at equal accuracy.
Checkmate optimizes DNN training time and memory usage.
problem Optimizing DNN training time and memory usage.
method Formalizes tensor rematerialization as an optimization problem, uses MILP solvers or approximation algorithms to find optimal schedules, and accelerates training with hardware-aware cost models.
result Reduces training cost and enables training with larger input sizes.
Efficiently trains GCNs with reduced time and memory usage.
problem Hard training of GCNs over large graph datasets.
method Layer-wise and learned efficient training framework (L2-GCN). result Significantly reduces training time and memory usage.
StatQAT optimizes quantization for deep networks, reducing computational cost and memory usage.
problem Optimal quantization parameters selection for deep neural networks with diverse data distributions.
method Statistical error analysis framework for uniform and floating-point quantization, iterative and analytic quantizers designed for arbitrary and Gaussian-like distributions.
result Improved accuracy and stability in training low-precision neural networks.
Mixed dimension embeddings reduce memory usage in recommendation systems.
problem Space-intensive embedding representations in recommendation systems.
method Mixed dimension embeddings where vector dimension scales with query frequency.
result Significant reduction in memory usage with minimal performance loss.
Paper proposes efficient optimizers for large language models with fast convergence and low memory usage.
problem Designing efficient optimizers for large language models with low-memory requirements and fast convergence.
method Structured Fisher information matrix approximation and low-rank extension framework.
result New optimizers (RACS and Alice) achieve better convergence and lower memory usage than existing methods.
FLRML efficiently handles large-scale, high-dimensional data for metric learning.
problem Metric learning for large-scale, high-dimensional datasets is computationally expensive and memory-intensive.
method FLRML reformulates low-rank metric learning as an unconstrained optimization problem on the Stiefel manifold, enabling efficient mini-batch processing.
result FLRML achieves high accuracy with significantly reduced computational and memory costs.
Deep learning detects inaccurate smart meters for resource savings.
problem Detecting and replacing inaccurate smart meters to save resources.
method Developed a deep-learning method using LSTM and CNN to predict electricity usage trajectories.
result High accuracy in detecting inaccurate meters for practical usage.
PSI-LinUCB improves scalability for large recommender systems.
problem Efficiently training and inferring for large action spaces in recommender systems.
method Represent inverse design matrix as diagonal + low-rank correction, derive stable rank-1 and batched updates, use projector-splitting integrator.
result Demonstrated effectiveness on recommender system datasets, achieving scalable training and inference.
AlphaGrad optimizes memory usage in RL algorithms by normalizing gradients.
problem Memory overhead and hyperparameter complexity in adaptive optimizers.
method Tensor-wise L2 normalization followed by a smooth hyperbolic tangent transformation controlled by a single parameter.
result AlphaGrad provides enhanced training stability and competitive performance in various RL algorithms.
Clapping reduces memory usage in distributed optimization by reusing data samples.
problem Significant communication overhead and impractical memory overhead in pipeline-parallel distributed optimization.
method Lazy sampling strategy to reuse data samples across steps, supporting convergence without unbiased gradient assumptions.
result Clapping achieves convergence in few-epoch or online training regimes without sample-size memory overhead.
LiteMORT reduces memory usage for GBDT models by 30% with improved accuracy.
problem Memory inefficiency in GBDT models, especially with adaptive compact distributions.
method Share memory, implicit merging, adaptive histogram resizing.
result Significant reduction in memory usage with improved accuracy.
This paper optimizes SMPC for neural network inference, reducing memory and time.
problem Memory and time constraints in secure neural network inference.
method Implemented ABY2.0 protocol, optimized memory usage, and used a helper node.
result MNIST inference reduced from 8.03 GB RAM and 200s to 0.2 GB RAM and 32s.
A new clustering method reduces time and memory usage for massive datasets.
problem Prohibitive computational cost and memory usage of clustering algorithms for massive datasets.
method Iterative hybridized threshold clustering (IHTC) that reduces data points into prototypes and applies clustering algorithms on them.
result IHTC reduces run time and memory usage of k-means and HAC while preserving their performance. A new algorithm reduces memory usage for deep learning models.
problem Training deep learning models requires significant memory.
method Dynamic Tensor Rematerialization (DTR) is a greedy online algorithm that dynamically plans recomputations.
result DTR achieves comparable performance to optimal static checkpointing with only a small memory budget.
The study reduces neural network training memory by up to 60.7x on CIFAR-10.
problem Reducing memory requirements for neural network training.
method Profiled memory usage, evaluated four techniques: sparsity, low precision, microbatching, gradient checkpointing.
result Appropriate combinations of techniques can reduce memory usage up to 60.7x on CIFAR-10.
Researchers save memory on MCUs by reordering neural network operators.
problem Memory constraints on microcontrollers for neural network inference.
method Operator reordering to save memory, orthogonal to other compression methods.
result Reduced memory footprint of a CNN to deploy on an MCU with 512KB SRAM.
CICLAD efficiently mines frequent closed itemsets from data streams with minimal memory usage.
problem Mining frequent closed itemsets from data streams is resource-intensive.
method CICLAD is an intersection-based sliding-window FCI miner that optimizes memory usage while maintaining performance.
result CICLAD achieves significantly lower memory footprint compared to existing methods.
A new backprop method reduces memory usage for deep neural networks.
problem Memory-intensive backpropagation in deep neural networks.
method Uses approximations of activations in buffers to reduce memory usage.
result Performance close to exact training with 4-bit precision activations.
This paper addresses the problem of scalable optimization for L1-regularized conditional Gaussian graphical models. Conditional Gaussian graphical models generalize the well-known Gaussian graphical models to conditional distributions to model the output network influenced by conditioning input variables. While highly …
Efficient DSP features reduce speech recognition memory usage.
problem Limited memory on DSPs for speech recognition.
method Developed efficient bottleneck features (BNFs) for DSPs.
result Reduced speech recognition memory usage by 10x with minimal accuracy loss.
Efficient sampling reduces memory usage for Minimax distance analysis.
problem Quadratic memory requirement for existing Minimax distance methods.
method Proposes a novel sampling technique with linear space complexity.
result Demonstrates significant reduction in memory usage for Minimax distances.
A new algorithm reduces memory usage for long token attention in streaming applications.
problem Memory inefficiency in computing attention for long documents.
method One-pass streaming algorithm using sublinear space storage.
result Super-efficient memory usage for long token attention.
PARSEC uses a probabilistic approach to reduce memory usage in neural architecture search.
problem Efficiently search over large and complex neural architectures with reduced memory usage.
method Probabilistic sampling to learn a distribution over high-performing architectures, enabling transfer learning.
result Our approach outperforms state-of-the-art methods with significantly less computational cost.
New algorithms solve convex optimization problems with limited memory.
problem Solving convex optimization problems with constrained memory.
method Recursive cutting-plane algorithms dividing variables into blocks.
result Achieves optimal memory usage and oracle complexity in certain regimes.
Safe screening rule improves Group SLOPE efficiency.
problem Efficiently selecting groups of predictors in high-dimensional sparse learning.
method Safe screening rule for Group SLOPE, addressing block non-separable group effects.
result Significant computational efficiency gains without sacrificing accuracy.
Bayesian histograms achieve optimal distribution estimation with minimal memory usage.
problem Efficiently estimating distributions with minimal memory footprint.
method Bayesian histograms for distribution estimation under Wasserstein distance.
result Bayesian histograms require fewer bins to achieve minimax optimality, reducing memory usage by a polynomial factor.
A new algorithm speeds up CP decomposition for large tensors.
problem Efficiently processing large-scale tensors in real-time.
method Randomized online CP decomposition (ROCP) algorithm.
result ROCP reduces computing time and memory usage significantly.
RSmote improves PINNs accuracy with less memory usage.
problem Imbalanced learning in Physics-Informed Neural Networks (PINNs).
method Residual-based Smote (RSmote) for local adaptive sampling.
result RSmote achieves or exceeds accuracy of state-of-the-art methods while reducing memory usage.
Innovative neural networks reduce memory usage for efficient, accurate segmentation.
problem Efficiently segmenting large graphs with limited memory.
method Iterative neural networks with loops and multiple outputs.
result State-of-the-art semantic segmentation results on demanding datasets.
A new method reduces memory usage for deep neural networks ensembles.
problem Reducing memory footprint of ensemble methods for deep learning.
method Extract multiple sub-networks from a single neural network through end-to-end optimization.
result Achieves higher or comparable accuracy with significantly less memory usage.
Improved biclustering algorithm reduces memory usage and runtime.
problem Efficiently enumerating maximal biclusters in numerical datasets.
method Online partitioning to guide biclustering results.
result RIn-Close_CVC3 reduces memory usage and runtime, handles missing values.
Two GPU memory management approaches reduce deep learning model memory usage.
problem Limited GPU memory in deep learning systems.
method Two orthogonal approaches exploiting iterative training algorithm to optimize memory usage.
result Up to 34.2% reduction in memory usage without communication overhead.
New algorithm reduces memory-regret trade-off for bandits.
problem Optimizing memory usage in stochastic multi-armed bandits.
method Designing an algorithm with specific regret bound and proving lower bound.
result Optimal regret characterization in terms of memory and rounds.
A new algorithm for optimizing huge-scale black-box problems with reduced memory usage.
problem Optimizing huge-scale black-box problems with limited vector operations.
method ZO-BCD algorithm for zeroth-order optimization with reduced memory footprint.
result ZO-BCD achieves state-of-the-art adversarial attack success rate of 97.9%.
This study evaluates and compares novelty detection algorithms for discrete sequences.
problem Identifying anomalies in temporal data.
method Experimental comparison of state-of-the-art novelty detection methods on various public and industrial datasets.
result Recommendations for efficient and appropriate methods based on extensive experiments and scalability tests.
Reinforcement learning optimizes neural network execution costs.
problem Minimizing execution cost of neural network computation graphs.
method Offline training of a reinforcement learning optimizer for generalization to unseen graphs.
result Significant improvements in execution time and memory usage compared to baselines.