A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Region-specific linear models are widely used in practical applications because of their non-linear but highly interpretable model representations. One of the key challenges in their use is non-convexity in simultaneous optimization of regions and region-specific models. This paper proposes novel convex region-specific…
The study estimates the expressiveness of GCNs with bounds on the number of linear regions.
Method constructs confidence regions for linear models with arbitrary predictors.
A deep neural network (DNN) with piecewise linear activations can partition the input space into numerous small linear regions, where different linear functions are fitted. It is believed that the number of these regions represents the expressivity of the DNN. This paper provides a novel and meticulous perspective to l…
We investigate the complexity of deep neural networks (DNN) that represent piecewise linear (PWL) functions. In particular, we study the number of linear regions, i.e. pieces, that a PWL function represented by a DNN can attain, both theoretically and empirically. We present (i) tighter upper and lower bounds for the m…
This work generalizes bounds on the number of linear regions in CPWL NNs.
It is well-known that the expressivity of a neural network depends on its architecture, with deeper networks expressing more complex functions. In the case of networks that compute piecewise linear functions, such as those with ReLU activation, the number of distinct linear regions is a natural measure of expressivity.…
Estimates the upper bound of linear regions in spheres centered at specific data points in ReLU neural networks.
We can compare the expressiveness of neural networks that use rectified linear units (ReLUs) by the number of linear regions, which reflect the number of pieces of the piecewise linear functions modeled by such networks. However, enumerating these regions is prohibitive and the known analytical bounds are identical for…
Paper studies the expressivity of Convolutional Neural Networks (CNNs).
fMRI semantic category understanding using linguistic encoding models attempts to learn a forward mapping that relates stimuli to the corresponding brain activation. State-of-the-art encoding models use a single global model (linear or non-linear) to predict brain activation given the stimulus. However, the critical as…
Study on the complexity of 1D ReLU neural networks, proving growth in linear regions.
The SPS method constructs confidence regions for true parameters with optimal sample complexity.
A new method to measure neural network expressiveness using tighter upper bounds.
Introduces CCR for constructing confidence regions from conformal predictions.
One-hot CNN (convolutional neural network) has been shown to be effective for text categorization (Johnson & Zhang, 2015). We view it as a special case of a general framework which jointly trains a linear model with a non-linear feature generator consisting of `text region embedding + pooling'. Under this framework, we…
A new complexity measure for neural networks improves upon classical methods.
A new method, Residual-Permuted Sums, improves confidence region construction for linear regression models.
The developments of deep neural networks (DNN) in recent years have ushered a brand new era of artificial intelligence. DNNs are proved to be excellent in solving very complex problems, e.g., visual recognition and text understanding, to the extent of competing with or even surpassing people. Despite inspiring and enco…
fMRI semantic category understanding using linguistic encoding models attempt to learn a forward mapping that relates stimuli to the corresponding brain activation. Classical encoding models use linear multi-variate methods to predict the brain activation (all voxels) given the stimulus. However, these methods essentia…
Deep networks realize complex mappings that are often understood by their locally linear behavior at or around points of interest. For example, we use the derivative of the mapping with respect to its inputs for sensitivity analysis, or to explain (obtain coordinate relevance for) a prediction. One key challenge is tha…
For an arrangement of pseudolines in the real projective plane let us denote by the number of vertices incident to lines. We obtain a linear on inequality similar to the Hirzebruch one, but with an elementary proof. We present an algorithm for producing lower bounds of the number of regions basing o…
This work aims to study the Portuguese regional agglomeration process, using the linear form the New Economic Geography models that emphasize the importance of spatial factors (distance, costs of transport and communication) in explaining of the concentration of economic activity in certain locations. In a theoretical …
In this paper we deal with the classical question of existence of polynomial in momenta integrals for geodesic flows on the 2-torus. For the quasi-linear system on coefficients of the polynomial integral we consider the region (so called elliptic regions) where there are complex-conjugate eigenvalues. We show that for …
There are few papers about the consumption pattern of the Portuguese wine, using econometrics techniques. This work, pretend to analyze the consumers behavior of the wine produced in Portugal, determining the demand equation with panel data methods. There were used statistical data available in the Alentejo Regional Wi…
To help understand the underlying mechanisms of neural networks (NNs), several groups have, in recent years, studied the number of linear regions of piecewise linear functions generated by deep neural networks (DNN). In particular, they showed that can grow exponentially with the number of network paramet…
In this paper an approach based on expectation maximization (EM) clustering to find the climate regions and a support vector machine to build a predictive model for each of these regions is proposed. To minimize the biases in the estimations a ten cross fold validation is adopted both for obtaining clusters and buildin…
Gradient estimate for linearized translator equation in R^4.
We conjecture explicit evolution formulas for Khovanov polynomials for pretzel knots in some regions in the windings space. Our description is exhaustive for genera 1 and 2. As previously observed, evolution at T != -1 is not fully smooth: it switches abruptly at the boundaries between different regions. We reveal that…
DDGroup identifies subgroups with uniform linear relationships.
We consider deep feedforward neural networks with rectified linear units from a signal processing perspective. In this view, such representations mark the transition from using a single (data-driven) linear representation to utilizing a large collection of affine linear representations tailored to particular regions of…
In this paper, a regional knot invariant is constructed. Like the Wirtinger presentation of a knot group, each planar region contributes a generator, and each crossing contributes a relation. The invariant is call a tridle of the link. As in the quandle theory, one can define Alexander quandle and get Alexander polynom…
We investigate the properties of multidimensional probability distributions in the context of latent space prior distributions of implicit generative models. Our work revolves around the phenomena arising while decoding linear interpolations between two random latent vectors -- regions of latent space in close proximit…
We reconsider the problem of optimal trading in the presence of linear and quadratic costs, for arbitrary linear costs but in the limit where quadratic costs are small. Using matched asymptotic expansion techniques, we find that the trading speed vanishes inside a band that is narrower than in the absence of quadratic …
We present a framework to derive upper bounds on the number of regions that feed-forward neural networks with ReLU activation functions are affine linear on. It is based on an inductive analysis that keeps track of the number of such regions per dimensionality of their images within the layers. More precisely, the info…
Construction of tight confidence regions and intervals is central to statistical inference and decision making. This paper develops new theory showing minimum average volume confidence regions for categorical data. More precisely, consider an empirical distribution generated from iid real…
New GP model estimates piecewise continuous functions.
Scalable method for regionalizing and extracting temporal patterns from time series data.
A new method detects interactions in machine learning models.
In this global study of solutions to the linear wave equation on Schwarzschild de Sitter spacetimes we attend to the cosmological region of spacetime which is bounded in the past by cosmological horizons and to the future by a spacelike hypersurface at infinity. We prove an energy estimate capturing the expansion of th…
We present a sparse knowledge gradient (SpKG) algorithm for adaptively selecting the targeted regions within a large RNA molecule to identify which regions are most amenable to interactions with other molecules. Experimentally, such regions can be inferred from fluorescence measurements obtained by binding a complement…
Previous work has questioned the conditions under which the decision regions of a neural network are connected and further showed the implications of the corresponding theory to the problem of adversarial manipulation of classifiers. It has been proven that for a class of activation functions including leaky ReLU, neur…
The location-based social network, Foursquare, reflects the human activities of a city. The mobility dynamics inferred from Foursquare helps us understanding urban social events like crime In this paper, we propose a directed graph from the aggregated movement between regions using Foursquare data. We derive region ris…
New insights into how neural networks classify data.
The paper examines stability of ReLU networks in tangent space and activation regions.
We present a new, unifying approach following some recent developments on the complexity of neural networks with piecewise linear activations. We treat neural network layers with piecewise linear activations as tropical polynomials, which generalize polynomials in the so-called or tropical algebra, with pos…
We study algebraic varieties of ReLU networks to understand their representable functions.
In exchange for large quantities of data and processing power, deep neural networks have yielded models that provide state of the art predication capabilities in many fields. However, a lack of strong guarantees on their behaviour have raised concerns over their use in safety-critical applications. A first step to unde…