arXiv research

Global Memory Augmentation (GMAT) improves Transformer performance on long documents.

Efficiently discovers Bayesian network structure with reduced memory usage.

Memory-Augmented Meta-Optimization improves cold-start recommendation.

V-HMN integrates memory mechanisms for improved image recognition.

A new L-BFGS method tackles large-scale optimization with fewer evaluations.

We investigate the robustness properties of image recognition models equipped with two features inspired by human vision, an explicit episodic memory and a shape bias, at the ImageNet scale. As reported in previous work, we show that an explicit episodic memory improves the robustness of image recognition models agains…

HawkesLLM models text generation with temporal influence, improving semantic alignment under limited memory.

Global convergence of an online (stochastic) limited memory version of the Broyden-Fletcher- Goldfarb-Shanno (BFGS) quasi-Newton method for solving optimization problems with stochastic objectives that arise in large scale machine learning is established. Lower and upper bounds on the Hessian eigenvalues of the sample …

CICLAD efficiently mines frequent closed itemsets from data streams with minimal memory usage.

KineticSim accelerates financial market simulations 3406x over CPU.

KineticSim: A lightweight, high-performance execution engine for real-time market simulators

We analyse perception and memory, using mathematical models for knowledge graphs and tensors, to gain insights into the corresponding functionalities of the human mind. Our discussion is based on the concept of propositional sentences consisting of \textit{subject-predicate-object} (SPO) triples for expressing elementa…

ECPv2 optimizes Lipschitz functions efficiently and scalably.

Large language models based on transformers have achieved great empirical successes. However, as they are deployed more widely, there is a growing need to better understand their internal mechanisms in order to make them more reliable. These models appear to store vast amounts of knowledge from their training data, and…

N-BEATS(P) efficiently forecasts millions of time series with reduced memory and time.

Hydra boosts efficiency for long-context reasoning in resource-constrained settings.

New cooperative dynamics enhances retrieval performance in neural networks.

New approach reduces malware detection memory requirements and speeds up training.

A federated model learns shared archetypes from heterogeneous clients in continual learning.

The paper examines how long-memory dynamics, rough-volatility, and persistence affect equity volatility forecasting.

Deep models store facts in geometric embeddings, not just associative memory.

Recently several methods were proposed for sparse optimization which make careful use of second-order information [10, 28, 16, 3] to improve local convergence rates. These methods construct a composite quadratic approximation using Hessian information, optimize this approximation using a first-order method, such as coo…

Compact Recurrent Transformer (CRT) improves Transformer efficiency for long sequences.

We present ARU, an Adaptive Recurrent Unit for streaming adaptation of deep globally trained time-series forecasting models. The ARU combines the advantages of learning complex data transformations across multiple time series from deep global models, with per-series localization offered by closed-form linear models. Un…

New method certifies global robustness of neural networks efficiently.

Multivariate time series (MTS) forecasting is widely used in various domains, such as meteorology and traffic. Due to limitations on data collection, transmission, and storage, real-world MTS data usually contains missing values, making it infeasible to apply existing MTS forecasting models such as linear regression an…

We introduce "TriMap"; a dimensionality reduction technique based on triplet constraints, which preserves the global structure of the data better than the other commonly used methods such as t-SNE, LargeVis, and UMAP. To quantify the global accuracy of the embedding, we introduce a score that roughly reflects the relat…

LSTMs outperform DFM in nowcasting COVID-19 economic variables.

Land use classification of low resolution spatial imagery is one of the most extensively researched fields in remote sensing. Despite significant advancements in satellite technology, high resolution imagery lacks global coverage and can be prohibitively expensive to procure for extended time periods. Accurately classi…

The paper connects PSO and CBO methods using stochastic modeling and mean-field limits.

This work tackles exploding inverses in INNs, revealing and mitigating their numerical non-invertibility.

InstantEmbedding efficiently generates node representations with less computation and memory.

Tensor networks improve anomaly detection efficiency.

Numerical optimization is an important tool in the field of computational physics in general and in nano-optics in specific. It has attracted attention with the increase in complexity of structures that can be realized with nowadays nano-fabrication technologies for which a rational design is no longer feasible. Also, …

DeepPPMNet forecasts EMS demand and performs causal analyses for policy-making.

Global methods outperform local in forecasting groups of time series, even in heterogeneous datasets.

Study on natural actor-critic for POMDPs with finite memory.

A bridge between continuous signals and discrete Ising spins for associative memory.

A new algorithm speeds up convex clustering.

A modern Hopfield network improves deep learning with better memory and attention.

Spatio-temporal data compression method reduces memory usage.

Transformers are powerful sequence models, but require time and memory that grows quadratically with the sequence length. In this paper we introduce sparse factorizations of the attention matrix which reduce this to $O(n \sqrt{n})$. We also introduce a) a variation on architecture and initialization to train deeper net…

New framework learns policies for partially observable systems.

Research uses SWT and BDLSTM to forecast stock and oil prices amid COVID-19.

HAMN combines CF models to improve drug repositioning.

SogCLR uses small batch sizes for global contrastive learning, achieving similar performance to SimCLR.

Point-of-Interest (POI) recommender systems play a vital role in people's lives by recommending unexplored POIs to users and have drawn extensive attention from both academia and industry. Despite their value, however, they still suffer from the challenges of capturing complicated user preferences and fine-grained user…

Proposes a novel model for healthcare and SME credit risk prediction.