A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
RNNs can store information in parameters and input history, with capacity and trainability being key factors.
problem Understanding and optimizing the capacity and trainability of RNNs.
method Experimental analysis of various RNN architectures, including comparison of training difficulty and per-task parameter capacity.
result RNNs can store an amount of task information linear in the number of parameters (approximately 5 bits per parameter) and can store one real number from input history per hidden unit.
Neurons in cortical circuits exhibit coordinated spiking activity, and can produce correlated synchronous spikes during behavior and cognition. We recently developed a method for estimating the dynamics of correlated ensemble activity by combining a model of simultaneous neuronal interactions (e.g., a spin-glass model)…
RNNs are suboptimal at compressing past sensory inputs for future prediction.
problem RNNs do not optimally compress past sensory inputs for future prediction.
method Investigated RNNs trained with maximum likelihood and found they extract unnecessary information. Injected noise into hidden states to improve performance.
result Injecting noise into RNN hidden states improves predictive information, sample quality, likelihood, and classification performance.
Pseudorandom inputs in diffusion models affect generation quality.
problem Pseudorandom inputs in diffusion models can be learned and affect model performance.
method Used a small multilayer perceptron to predict next values in pseudorandom orbits and a diffusion probe to replace real images with random tensors.
result Pseudorandom inputs can produce markedly different diffusion losses and generation quality.
The paper explains why estimating a history-dependent policy can reduce MSE in reinforcement learning.
problem Understanding why history-dependent policies can improve MSE in off-policy evaluation.
method The paper derives a bias-variance decomposition of MSE for various OPE estimators, showing how history-dependent policies can decrease variance and increase bias.
result History-dependent policies can decrease the variance of importance sampling estimators, leading to lower MSE.
It is shown how the generating functional method of De Dominicis can be used to solve the dynamics of the original version of the minority game (MG), in which agents observe real as opposed to fake market histories. Here one again finds exact closed equations for correlation and response functions, but now these are de…
Neural Assistant integrates knowledge reasoning and dialogue generation in a single model.
problem Challenges in task-oriented dialog systems, including multi-turn language understanding and generation, knowledge retrieval and reasoning, and action prediction.
method Develops a single neural network model that jointly predicts text responses and actions from conversation history and external knowledge.
result The model learns to reason on external knowledge with weak supervision, improving factual accuracy and language generation performance.
We present in this chapter (Chapter II) the history of ideas which lead up to the development of modern knot theory. We are more detailed when pre-XX century history is reported. With more recent times we are more selective, stressing developments related to Jones type invariants of links. In the Appendix, A.Przybyszew…
We give an explicit definition of decentralization and show you that decentralization is almost impossible for the current stage and Bitcoin is the first truly noncentralized currency in the currency history. We propose a new framework of noncentralized cryptocurrency system with an assumption of the existence of a wea…
Epsilon-machines are minimal, unifilar presentations of stationary stochastic processes. They were originally defined in the history machine sense, as hidden Markov models whose states are the equivalence classes of infinite pasts with the same probability distribution over futures. In analyzing synchronization, though…
A dangerously brief history of the developments of the main ideas in economics, as observed by a physicist, is given. This was published in 'Econophysics of Stock and Other Markets', Eds. A. Chatterjee, B. K. Chakrabarti, New Economic Windows Series, Springer, Milan, 2006, pp~219-224.
Marden's Tameness Conjecture predicts that every hyperbolic 3-manifold with finitely generated fundamental group is homeomorphic to the interior of a compact 3-manifold. It was recently established by Agol and Calegari-Gabai. We will survey the history of work on this conjecture and discuss its many applications.