We present Memory Augmented Policy Optimization (MAPO), a simple and novel way to leverage a memory buffer of promising trajectories to reduce the variance of policy gradient estimate. MAPO is applicable to deterministic environments with discrete actions, such as structured prediction and combinatorial optimization ta…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “memory buffer”
This work improves reinforcement learning with sparse rewards by following diverse past trajectories.
problem Challenges in reinforcement learning with sparse rewards and myopic behavior.
method Proposes a trajectory-conditioned policy to learn from a memory buffer of diverse past trajectories.
result Significantly outperforms existing methods on complex tasks with local optima.
Secure aggregation for buffered asynchronous federated learning without TEEs.
problem Privacy and convergence in buffered asynchronous federated learning.
method Developed a new protocol (BASecAgg) that ensures privacy without TEEs by carefully designing masks.
result BASecAgg achieves similar convergence guarantees as FedBuff without TEEs.
PipeDream-2BW accelerates large model training by 20x with minimal memory usage.
problem Training large models requires memory beyond single accelerator capacity.
method Pipeline parallelism, weight gradient coalescing, double buffering.
result Accelerates large model training by up to 20x.
Efficiently preserves old class knowledge in memory-limited settings.
problem Catastrophic forgetting in class-incremental learning.
method Memory-efficient exemplar preserving scheme and domain-compatible feature extractors.
result Low-fidelity exemplar samples can replace high-fidelity ones with less memory cost.
Tensor models decode human perception and memory using SPO triples.
problem Understanding implicit and explicit perception and memory in the brain.
method Tensor models with SPO triples, dual representations, and four layers.
result Semantic memory is crucial for explicit perception and declarative memories.
MER algorithm speeds up VI solving with Markovian data.
problem Solving stochastic variational inequalities with Markovian data.
method MER algorithm using multi-scale sampling from a Markovian buffer.
result Achieves faster convergence without knowing Markov chain mixing time.
Efficiently combines autoregressive and set-based models for joint distributions.
problem Joint distributions over multiple predictions from set-based models.
method Causal autoregressive buffer that caches context and captures dependencies.
result Up to 20x faster joint sampling and density evaluation, up to 7x lower memory usage.
We propose a streaming submodular maximization algorithm "stream clipper" that performs as well as the offline greedy algorithm on document/video summarization in practice. It adds elements from a stream either to a solution set or to an extra buffer based on two adaptive thresholds, and improves by a final…
New method improves ABI for sequential data, reducing forgetting and improving accuracy.
problem Performance degradation of ABI under model misspecification and distribution shifts.
method Decouples simulation-based pre-training from unsupervised SC fine-tuning, using memory buffer and elastic weight consolidation.
result Significant mitigation of forgetting and improved posterior estimates compared to standard simulation-based training.
Experience replay is a key technique behind many recent advances in deep reinforcement learning. Allowing the agent to learn from earlier memories can speed up learning and break undesirable temporal correlations. Despite its wide-spread application, very little is understood about the properties of experience replay. …
Optimizer memory affects learning rate sensitivity in shuffle order, impacting fine-tuning noise.
problem Optimizer memory affects the learning rate sensitivity in shuffle order, leading to fine-tuning noise.
method Isolated the mechanism of fixed-clock optimizer memory affecting the learning rate sensitivity in shuffle order, deriving a fit-free way to size the noise.
result Fixed-clock optimizers like AdamW produce a larger first-order noise channel compared to memoryless optimizers, affecting fine-tuning comparisons.
A new memory replay mechanism improves reinforcement learning stability and speed.
problem Forgetting in reinforcement learning with continuous control.
method Augmented Memory Replay (AMR) that optimizes the replay of past experiences.
result AMR enhances stability and convergence speed of learning algorithms.
Training convolutional neural network models is memory intensive since back-propagation requires storing activations of all intermediate layers. This presents a practical concern when seeking to deploy very deep architectures in production, especially when models need to be frequently re-trained on updated datasets. In…
Both the human brain and artificial learning agents operating in real-world or comparably complex environments are faced with the challenge of online model selection. In principle this challenge can be overcome: hierarchical Bayesian inference provides a principled method for model selection and it converges on the sam…
We introduce a Markovian single point process model, with random intensity regulated through a buffer mechanism and a self-exciting effect controlling the arrival stream to the buffer. The model applies the principle of the Hawkes process in which point process jumps generate a shot-noise intensity field. Unlike the Ha…
SDCOR clusters massive datasets efficiently, detecting outliers with low memory usage.
problem Local outlier detection in large-scale datasets.
method Chunk-based density clustering with incremental updates.
result SDCOR achieves lower linear time complexity and better efficiency than traditional methods.
New method improves continual learning by anchoring past knowledge.
problem Catastrophic forgetting in continual learning.
method Bilevel optimization to update current task knowledge while keeping past task predictions.
result Improves accuracy and forgetting metrics compared to experience replay.
Streaming algorithms are generally judged by the quality of their solution, memory footprint, and computational complexity. In this paper, we study the problem of maximizing a monotone submodular function in the streaming setting with a cardinality constraint . We first propose Sieve-Streaming++, which requires just…
Paper proposes a new method to select memory data for online class-incremental learning.
problem Selecting which buffered images to replay for online class-incremental learning.
method Adversarial Shapley value scoring method to preserve latent decision boundaries.
result Proposed ASER method provides competitive or improved performance compared to state-of-the-art methods.
New algorithm reduces costs and latency for large language model inference.
problem Optimizing inference costs and latency for large language models with GPU constraints.
method Formulated as an online scheduling problem with endogenous memory growth, introduced fluid model and WAIT algorithms.
result Reduced costs and latency, especially in near-overloaded and overloaded regimes.
LiDER refreshes past experiences in RL by dreaming about them.
problem Improving data efficiency in off-policy RL algorithms.
method Refreshing past experiences in a replay buffer using the current policy.
result LiDER consistently improves performance in Atari games.
A new buffer system improves continual learning in RL agents by adapting to changing environments.
problem Improving RL agents' ability to learn from changing environments over time.
method Multi-timescale replay buffer combined with invariant risk minimization.
result The method shows improvement over baselines in continual learning settings.
Paper improves online time series forecasting by combining natural gradient and robust t-distribution.
problem Online time series forecasting challenges in rapidly adapting to evolving data.
method Reframed neural network optimization as a parameter filtering problem, using natural gradient and Student's t likelihood.
result Natural Score-driven Replay (NatSR) achieves stronger forecasting performance than state-of-the-art methods.
A federated model learns shared archetypes from heterogeneous clients in continual learning.
problem Federated learning struggles with client heterogeneity and streaming distribution shifts.
method Clients encode their data as low-rank Hebbian operators, which are sent to a central server for aggregation and factorization into global archetypes.
result Improved global archetype reconstruction and associative retrieval in heterogeneous clients, drift, and novelty settings.
This study examines how risky investments affect insurance capital valuation.
problem Standard cost-of-capital assumptions do not account for risky investments.
method Analyzed effects of allowing buffer capital investments in risky assets.
result Decomposition of buffer capital contributions varies with riskiness.
Neural network tackles continual learning with neuromodulation and local error signals.
problem Catastrophic forgetting in continuous learning.
method Biologically-inspired neural architecture with local learning and neuromodulation, combined with transfer metalearning.
result Superior performance in continual learning tasks compared to other approaches.
BUZz defends images from adversarial attacks using simple transformations.
problem Adversarial attacks on deep neural networks for image classification.
method Combination of deep neural networks and simple image transformations.
result Achieves significant improvement over state-of-the-art defenses with a modest drop in clean accuracy.
Random projections of labels enable feedforward training of deep neural networks without feedback.
problem Biological plausibility and low-cost adaptive smart sensors constrained by backpropagation.
method Fixed random projections of targets for feedforward training of hidden layers.
result DRTP algorithm provides a tradeoff between accuracy and computational cost suitable for edge computing.
Regulator allocates buffers to prevent financial contagion in networks with common assets.
problem Containment of default contagion in financial networks with common asset exposures.
method Allocates nonnegative buffer vectors under linear budget constraints to maximize default or insolvency resilience margins or minimize worst-case systemic losses.
result Exact synthesis results for buffer allocation under and uncertainty sets, showing significant gains over uniform and exposure-proportional allocations.
SELD-TCN improves sound event localization and detection efficiency.
problem Efficient sound event localization and detection on embedded hardware.
method Developed a novel temporal convolutional network (TCN) architecture.
result SELD-TCN outperforms state-of-the-art SELDnet on four datasets.
Robotic navigation improves with RL and ultrasound images.
problem Improving accuracy in robotic navigation for ultrasound-guided procedures.
method Combines deep Q-networks with ultrasound images and binary classifier.
result Significantly improved navigation accuracy compared to pure RL and SL.
In a typical online learning scenario, a learner is required to process a large data stream using a small memory buffer. Such a requirement is usually in conflict with a learner's primary pursuit of prediction accuracy. To address this dilemma, we introduce a novel Bayesian online classi cation algorithm, called the Vi…
A new method for generating replay samples on the fly, optimizing for not forgetting.
problem Addressing the issue of forgetting in neural networks.
method Generates auxiliary samples on the fly using the model's implicit memory, specialized to each real training batch.
result Optimizing for not forgetting leads to more efficient and scalable generation of specialized samples.
The paper proposes a method to learn from both simulation and real-world data.
problem Training autonomous systems in simulation and applying them to real-world environments.
method Balancing samples from simulation and real-world data using a replay buffer.
result The method achieves better performance in real-world tasks compared to training only in simulation.
Hierarchical GANs reduce anomaly detection costs.
problem Balancing anomaly detection accuracy and sampling costs.
method Hierarchical GANs for nonuniform sampling and buffer zones.
result Proposed GAN-based detector outperforms baseline in detection delay and average cost of error.
In this study, we investigate the limits of the current state of the art AI system for detecting buffer overflows and compare it with current static analysis tools. To do so, we developed a code generator, s-bAbI, capable of producing an arbitrarily large number of code samples of controlled complexity. We found that t…
State space models (SSMs) provide a flexible framework for modeling complex time series via a latent stochastic process. Inference for nonlinear, non-Gaussian SSMs is often tackled with particle methods that do not scale well to long time series. The challenge is two-fold: not only do computations scale linearly with t…
Mean Reversion Pays, but Costsq-fin.TR
A mean-reverting financial instrument is optimally traded by buying it when it is sufficiently below the estimated `mean level' and selling it when it is above. In the presence of linear transaction costs, a large amount of value is paid away crossing bid-offers unless one devises a `buffer' through which the price mus…
In Reinforcement Learning (RL), an agent explores the environment and collects trajectories into the memory buffer for later learning. However, the collected trajectories can easily be imbalanced with respect to the achieved goal states. The problem of learning from imbalanced data is a well-known problem in supervised…
Multi-period Mean-Buffered Probability of Exceedance in Defined Contribution Portfolio Optimizationq-fin.PM
Investigates multi-period portfolio optimization for DC plans using buffered Probability of Exceedance.
problem Optimizing long-term Defined Contribution plans with realistic constraints and dynamic dynamics.
method Formulates and solves bilevel optimization problems for pre-commitment and time-consistent Mean-bPoE and Mean-CVaR portfolio optimization.
result Time-consistent Mean-bPoE strategies maintain investor preferences for minimum terminal wealth, unlike Mean-CVaR.
A continual learning agent learns online with a non-stationary and never-ending stream of data. The key to such learning process is to overcome the catastrophic forgetting of previously seen data, which is a well known problem of neural networks. To prevent forgetting, a replay buffer is usually employed to store the p…
DAC enhances exploration in reinforcement learning with entropy regularization.
problem Improving exploration efficiency in reinforcement learning.
method Sample-aware entropy regularization using replay buffer action distributions.
result DAC significantly outperforms existing algorithms in reinforcement learning tasks.
A simple technique improves continual learning by 50% on image datasets.
problem Challenges in training neural networks on a stream of shifting data.
method Experience Replay (ER) with five tricks to mitigate its shortcomings.
result ER, enhanced with tricks, achieves significant accuracy gains.
Neural approximate computing gains enormous energy-efficiency at the cost of tolerable quality-loss. A neural approximator can map the input data to output while a classifier determines whether the input data are safe to approximate with quality guarantee. However, existing works cannot maximize the invocation of the a…
Flexible VHDL design for multiple neural networks on FPGAs.
problem Inflexible neural network designs for FPGAs.
method Proposes a flexible VHDL structure with multiple processor groups.
result Allows training and testing of multiple neural networks on multiple FPGAs.
Continual learning is the problem of learning new tasks or knowledge while protecting old knowledge and ideally generalizing from old experience to learn new tasks faster. Neural networks trained by stochastic gradient descent often degrade on old tasks when trained successively on new tasks with different data distrib…
Study improves neural network performance in sequential learning for image classification.
problem Improving neural network performance in sequential learning for image classification.
method Evaluation of approaches for computing prequential description lengths, proposing forward-calibration and replay-streams.
result Improved description lengths for image classification datasets, outperforming previous results.