A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Neural Power Unit (NPU) learns arbitrary power functions on real numbers.
Paper tackles division difficulty, proposing new methods to improve accuracy.
New KWS neural networks improve accuracy and power efficiency.
Neural process model improves real-time condition monitoring signal prediction.
A study on power market price forecasting by deep learning is presented. As one of the most successful deep learning frameworks, the LSTM (Long short-term memory) neural network is utilized. The hourly prices data from the New England and PJM day-ahead markets are used in this study. First, a LSTM network is formulated…
Foreign exchange markets show that currency units (= accounting or nominal price units) are variables. Technical and economic progress evidence that the consumer baskets (= purchasing power units or real price units) are also variables. In contrast, all physical measurement units are constants and either defined in the…
We show that there is a simple (approximately radial) function on , expressible by a small 3-layer feedforward neural networks, which cannot be approximated by any 2-layer network, to more than a certain constant accuracy, unless its width is exponential in the dimension. The result holds for virtually all kn…
Conditional restricted Boltzmann machines are undirected stochastic neural networks with a layer of input and output units connected bipartitely to a layer of hidden units. These networks define models of conditional probability distributions on the states of the output units given the states of the input units, parame…
Diverse fault types, fast re-closures, and complicated transient states after a fault event make real-time fault location in power grids challenging. Existing localization techniques in this area rely on simplistic assumptions, such as static loads, or require much higher sampling rates or total measurement availabilit…
There is growing interest in being able to run neural networks on sensors, wearables and internet-of-things (IoT) devices. However, the computational demands of neural networks make them difficult to deploy on resource-constrained edge devices. To meet this need, our work introduces a new recurrent unit architecture th…
Existing Bayesian treatments of neural networks are typically characterized by weak prior and approximate posterior distributions according to which all the weights are drawn independently. Here, we consider a richer prior distribution in which units in the network are represented by latent variables, and the weights b…
Recent technological advances have proliferated the available computing power, memory, and speed of modern Central Processing Units (CPUs), Graphics Processing Units (GPUs), and Field Programmable Gate Arrays (FPGAs). Consequently, the performance and complexity of Artificial Neural Networks (ANNs) is burgeoning. While…
Study designs neural networks for fault localization, state estimation, and optimal PMU placement in power systems.
Unified theory for neural scaling laws in hierarchically compositional data.
Recurrent Neural Networks (RNNs) with sophisticated units that implement a gating mechanism have emerged as powerful technique for modeling sequential signals such as speech or electroencephalography (EEG). The latter is the focus on this paper. A significant big data resource, known as the TUH EEG Corpus (TUEEG), has …
Study shows shallow ReLU networks struggle with high-dimensional Lipschitz functions.
New method reduces overfitting in deep neural networks by measuring and regulating hidden unit diversity.
The growing number of low-power smart devices in the Internet of Things is coupled with the concept of "Edge Computing", that is moving some of the intelligence, especially machine learning, towards the edge of the network. Enabling machine learning algorithms to run on resource-constrained hardware, typically on low-p…
Multivariate splines linked to infinitely-wide neural networks with improved numerical performance.
Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mecha…
We introduce dropout compaction, a novel method for training feed-forward neural networks which realizes the performance gains of training a large model with dropout regularization, yet extracts a compact neural network for run-time efficiency. In the proposed method, we introduce a sparsity-inducing prior on the per u…
This work analyzes neural scaling laws using power-law data spectra and derives analytical expressions for generalization error.
PoET-BiN reduces power consumption in neural networks on embedded devices.
On-device inference of machine learning models for mobile phones is desirable due to its lower latency and increased privacy. Running such a compute-intensive task solely on the mobile CPU, however, can be difficult due to limited computing power, thermal constraints, and energy consumption. App developers and research…
In this paper we propose a synergistic melting of neural networks and decision trees (DT) we call neural decision trees (NDT). NDT is an architecture a la decision tree where each splitting node is an independent multilayer perceptron allowing oblique decision functions or arbritrary nonlinear decision function if more…
Novel method combines neural network features with survival models for ICU infections.
This paper shows neural networks can solve complex graph problems efficiently.
In 1974, Thurston proved that, up to isotopy, every automorphism of closed orientable surface is either periodic, reducible, or pseudo-Anosov. The latter case has lead to a rich theory with applications ranging from dynamical systems to low dimensional topology. Associated with every pseudo-Anosov map is a real number …
DeepBayes uses neural networks to efficiently estimate parameters in complex dynamical models.
Computer vision performances have been significantly improved in recent years by Convolutional Neural Networks(CNN). Currently, applications using CNN algorithms are deployed mainly on general purpose hardwares, such as CPUs, GPUs or FPGAs. However, power consumption, speed, accuracy, memory footprint, and die size sho…
The standard LSTM recurrent neural networks while very powerful in long-range dependency sequence applications have highly complex structure and relatively large (adaptive) parameters. In this work, we present empirical comparison between the standard LSTM recurrent neural network architecture and three new parameter-r…
GraPhyR uses GNNs to optimize power grid reconfiguration in real-time.
We address the efficiency issue for the construction of a deep graph neural network (GNN). The approach exploits the idea of representing each input graph as a fixed point of a dynamical system (implemented through a recurrent neural network), and leverages a deep architectural organization of the recurrent units. Effi…
Neural networks are a powerful class of functions that can be trained with simple gradient descent to achieve state-of-the-art performance on a variety of applications. Despite their practical success, there is a paucity of results that provide theoretical guarantees on why they are so effective. Lying in the center of…
The concepts of unitary evolution matrices and associative memory have boosted the field of Recurrent Neural Networks (RNN) to state-of-the-art performance in a variety of sequential tasks. However, RNN still have a limited capacity to manipulate long-term memory. To bypass this weakness the most successful application…
It is a known fact that training recurrent neural networks for tasks that have long term dependencies is challenging. One of the main reasons is the vanishing or exploding gradient problem, which prevents gradient information from propagating to early layers. In this paper we propose a simple recurrent architecture, th…
Proposes CoDEAL for estimating heterogeneous treatment effects in panel data models.
The increasing penetration level of energy generation from renewable sources is demanding for more accurate and reliable forecasting tools to support classic power grid operations (e.g., unit commitment, electricity market clearing or maintenance planning). For this purpose, many physical models have been employed, and…
Contemporary power grids are being challenged by rapid voltage fluctuations that are caused by large-scale deployment of renewable generation, electric vehicles, and demand response programs. In this context, monitoring the grid's operating conditions in real time becomes increasingly critical. With the emergent large …
Study on hidden units in finite Bayesian neural networks and their tail properties.
Study examines dependence properties of Bayesian neural network units in finite-width networks.
Study approximates nonlinear functionals using deep ReLU networks.
Every design choice will have different effects on different units. However traditional A/B tests are often underpowered to identify these heterogeneous effects. This is especially true when the set of unit-level attributes is high-dimensional and our priors are weak about which particular covariates are important. How…
Rectified Linear Units (ReLU) have become the main model for the neural units in current deep learning systems. This choice has been originally suggested as a way to compensate for the so called vanishing gradient problem which can undercut stochastic gradient descent (SGD) learning in networks composed of multiple lay…
Many state-of-the-art results obtained with deep networks are achieved with the largest models that could be trained, and if more computation power was available, we might be able to exploit much larger datasets in order to improve generalization ability. Whereas in learning algorithms such as decision trees the ratio …
We present the data on wealth and income distributions in the United Kingdom, as well as on the income distributions in the individual states of the USA. In all of these data, we find that the great majority of population is described by an exponential distribution, whereas the high-end tail follows a power law. The di…
AReLU uses attention-based rectification to improve neural network performance.
Neural Network is a powerful Machine Learning tool that shows outstanding performance in Computer Vision, Natural Language Processing, and Artificial Intelligence. In particular, recently proposed ResNet architecture and its modifications produce state-of-the-art results in image classification problems. ResNet and mos…