A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
The paper explores how sinks and diagonal patterns prevent attention oversmoothing.
Although deep convolutional neural networks achieve state-of-the-art performance across nearly all image classification tasks, their decisions are difficult to interpret. One approach that offers some level of interpretability by design is \textit{hard attention}, which uses only relevant portions of the image. However…
Hard visual attention is a promising approach to reduce the computational burden of modern computer vision methodologies. Hard attention mechanisms are typically non-differentiable. They can be trained with reinforcement learning but the high-variance training this entails hinders more widespread application. We show h…
There has recently been significant interest in hard attention models for tasks such as object recognition, visual captioning and speech recognition. Hard attention can offer benefits over soft attention such as decreased computational cost, but training hard attention models can be difficult because of the discrete la…
This paper studies deep learning methodologies for portfolio optimization in the US equities market. We present a novel residual switching network that can automatically sense changes in market regimes and switch between momentum and reversal predictors accordingly. The residual switching network architecture combines …
This technical note extends recent results on the computational complexity of globally minimizing the error of piecewise-affine models to the related problem of minimizing the error of switching linear regression models. In particular, we show that, on the one hand the problem is NP-hard, but on the other hand, it admi…
Attention operators have been widely applied in various fields, including computer vision, natural language processing, and network embedding learning. Attention operators on graph data enables learnable weights when aggregating information from neighboring nodes. However, graph attention operators (GAOs) consume exces…
This work extends identifiability analysis to sequential latent variable models, focusing on Switching Dynamical Systems.
Neural attention has become central to many state-of-the-art models in natural language processing and related domains. Attention networks are an easy-to-train and effective method for softly simulating alignment; however, the approach does not marginalize over latent alignments in a probabilistic sense. This property …
Annealed Entropic Allocation improves ranking and selection by mitigating hard switching and improving finite-budget discrimination.
As a metric to measure the performance of an online method, dynamic regret with switching cost has drawn much attention for online decision making problems. Although the sublinear regret has been provided in many previous researches, we still have little knowledge about the relation between the dynamic regret and the s…
Breaks down complex nonlinear dynamics into simpler components.
Estimates hybrid dynamical systems with polynomial expansions and Markovian switching.
New framework optimizes deep learning training by deferring large batch sizes to late stages.
Catastrophic forgetting occurs when a neural network loses the information learned in a previous task after training on subsequent tasks. This problem remains a hurdle for artificial intelligence systems with sequential learning capabilities. In this paper, we propose a task-based hard attention mechanism that preserve…
Memory networks are neural networks with an explicit memory component that can be both read and written to by the network. The memory is often addressed in a soft way using a softmax function, making end-to-end training with backpropagation possible. However, this is not computationally scalable for applications which …
Proposes an EM algorithm for high-dimensional Markov-switching VAR models.
Near-logarithmic regret per switch achieved for mixable/exp-concave losses.
Sequence-to-sequence text-to-speech (TTS) is dominated by soft-attention-based methods. Recently, hard-attention-based methods have been proposed to prevent fatal alignment errors, but their sampling method of discrete alignment is poorly investigated. This research investigates various combinations of sampling methods…
The paper explores when to prioritize easy or hard samples in learning tasks.
Adaptive framework improves NB accuracy by fusing two index categories.
Motivated by recommendation problems in music streaming platforms, we propose a nonstationary stochastic bandit model in which the expected reward of an arm depends on the number of rounds that have passed since the arm was last pulled. After proving that finding an optimal policy is NP-hard even when all model paramet…
In this paper we take a problem of unsupervised nodes clustering on graphs and show how recent advances in attention models can be applied successfully in a "hard" regime of the problem. We propose an unsupervised algorithm that encodes Bethe Hessian embeddings by optimizing soft modularity loss and argue that our mode…
A switchable deep beamformer enables versatile image processing.
Detects corruption in agentic models during execution.
End-to-end text-to-speech (TTS) synthesis is a method that directly converts input text to output acoustic features using a single network. A recent advance of end-to-end TTS is due to a key technique called attention mechanisms, and all successful methods proposed so far have been based on soft attention mechanisms. H…
Large-scale non-convex sparsity-constrained problems have recently gained extensive attention. Most existing deterministic optimization methods (e.g., GraSP) are not suitable for large-scale and high-dimensional problems, and thus stochastic optimization methods with hard thresholding (e.g., SVRGHT) become more attract…
We develop a robust RL algorithm for off-dynamics environments with improved suboptimality bounds and computational efficiency.
New algorithm reduces switching costs in multinomial logit bandit problems.
This paper analyzes how periodic and soft target updates stabilize linear Q-learning.
SAttention improves long sequence attention with smoothed skeleton sketching.
Meta-learning strategy improves few-shot classification performance.
Proposes a new sampling policy for ranking and selection problems.
New polynomial invariants derived from birack and switch structures.
In this paper, we study optimal switching problems under ambiguity. To characterize the optimal switching under ambiguity in the finite horizon, we use multidimensional reflected backward stochastic differential equations (multidimensional RBSDEs) and show that a value function of the optimal switching under ambiguity …
The study examines higher-order modern portfolio theory with complex critical points and feasible portfolio variety.
We introduce an adversarial method for producing high-recall explanations of neural text classifier decisions. Building on an existing architecture for extractive explanations via hard attention, we add an adversarial layer which scans the residual of the attention for remaining predictive signal. Motivated by the impo…
The problem of optimal switching between nonlinear autonomous subsystems is investigated in this study where the objective is not only bringing the states to close to the desired point, but also adjusting the switching pattern, in the sense of penalizing switching occurrences and assigning different preferences to util…
Code-switching, the alternation of languages within a conversation or utterance, is a common communicative phenomenon that occurs in multilingual communities across the world. This survey reviews computational approaches for code-switched Speech and Natural Language Processing. We motivate why processing code-switched …
Transformers struggle to learn Markovian dynamics, showing NP-hard optimization challenges.
New algorithms improve sampling from complex distributions.
HardNet adds hard constraints to neural networks without sacrificing performance.
In this work, we consider the optimal portfolio selection problem under hard constraints on trading volume amounts when the dynamics of the risky asset returns are governed by a discrete-time approximation of the Markov-modulated geometric Brownian motion. The states of Markov chain are interpreted as the states of an …
Squirrel switches between optimizers for better performance.
Optimal algorithms for mixable losses in dynamic environments with reduced redundancy.
Study approximates financial market with discrete-time models.
Black-box neural network models are widely used in industry and science, yet are hard to understand and interpret. Recently, the attention mechanism was introduced, offering insights into the inner workings of neural language models. This paper explores the use of attention-based neural networks mechanism for estimatin…
Optimizes control of hybrid systems with multiple switching processes.