We present a new active learning algorithm that adaptively partitions the input space into a finite number of regions, and subsequently seeks a distinct predictor for each region, both phases actively requesting labels. We prove theoretical guarantees for both the generalization error and the label complexity of our al…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “activation regions”
fMRI semantic category understanding using linguistic encoding models attempt to learn a forward mapping that relates stimuli to the corresponding brain activation. Classical encoding models use linear multi-variate methods to predict the brain activation (all voxels) given the stimulus. However, these methods essentia…
Improved bounds on neural network regions using activation histograms.
problem Bounding the number of affine regions in ReLU networks.
method Analysis of algebraic topology problem, extension of framework to subnetwork composition.
result Slightly tighter bounds and insights into parameter initialization.
Hidden Markov Models detect hand gestures from wearable sEMG signals.
problem Detecting activity regions in continuous sEMG signals.
method Hidden Markov Models applied to sEMG signals for gesture recognition.
result Average accuracy of 96.25% for activity onsets and 87.5% for activity terminations.
fMRI semantic category understanding using linguistic encoding models attempts to learn a forward mapping that relates stimuli to the corresponding brain activation. State-of-the-art encoding models use a single global model (linear or non-linear) to predict brain activation given the stimulus. However, the critical as…
New method avoids failures in physics-constrained systems using active learning.
problem Handling fatal failures in systems governed by physics constraints.
method Develops a novel active learning method that considers implicit physics constraints.
result Achieves zero-failure in composite fuselage assembly process without explicit failure regions.
This work generalizes bounds on the number of linear regions in CPWL NNs.
problem Determining the number of linear regions in CPWL neural networks is challenging.
method Generalized bounds on the maximal number of linear regions for arbitrary CPWL activation functions.
result Depth significantly increases the number of linear regions, but not exponentially.
Deep ReLU networks with extra parameters have mostly good loss landscapes.
problem Finding good local minima in the loss landscape of deep neural networks.
method Analyzing shallow and deep ReLU networks with extra parameters on a generic dataset.
result Most activation patterns correspond to regions with no bad local minima.
This paper proposes an improved active learning method using classification trees.
problem Reducing the size of training sets while maintaining high accuracy in supervised learning.
method A wrapper active learning method using a classification tree to sub-sample from low-entropy regions.
result The proposed method constructs accurate classification models even with severely restricted labeled data.
We show that for neural network functions that have width less or equal to the input dimension all connected components of decision regions are unbounded. The result holds for continuous and strictly monotonic activation functions as well as for the ReLU activation function. This complements recent results on approxima…
New algorithm improves graph-based active learning by identifying unexplored regions.
problem Improving graph-based active learning by identifying unexplored regions.
method Poisson Reweighted Laplacian Uncertainty Sampling (PWLL) with a diagonal perturbation.
result PWLL effectively identifies unexplored regions in graph-based data.
The DAO Report led to a significant shift of ICO activity to Europe.
problem The impact of U.S. regulatory changes on global ICO activity.
method Analysis of a global dataset of ICOs from 2014 to 2021, focusing on the DAO Report's effects.
result A substantial and persistent reallocation of ICO activity to Europe following the DAO Report.
Study examines money flow network among firms' accounts in a Japanese region.
problem Understanding the relationship between money flow and economic activities of firms.
method Employed exhaustive bank transfer data, network statistics, Hodge decomposition, and non-negative matrix factorization.
result Identified a 'walnut' structure with core and upstream/downstream components, correlated with economic activities.
Distributed securities exchanges may become de facto fragmented if they span geographical regions with asymmetric computer infrastructure. First, we build an economic model of a decentralized exchange with two miner clusters, standing in for compact areas of economic activity (e.g., cities). "Local" miners in the area …
The uninformative ordering of artificial neurons in Deep Neural Networks complicates visualizing activations in deeper layers. This is one reason why the internal structure of such models is very unintuitive. In neuroscience, activity of real brains can be visualized by highlighting active regions. Inspired by those te…
Estimates the upper bound of linear regions in spheres centered at specific data points in ReLU neural networks.
problem Bounding the number of linear regions in specific areas of neural networks using ReLU activations.
method Developed a method to estimate the upper bound of linear regions in any sphere within the input space of a ReLU neural network.
result The boundaries of linear regions move away from training data points during training, and spheres centered at these points contain more regions than arbitrary points.
The study estimates the expressiveness of GCNs with bounds on the number of linear regions.
problem Characterizing the expressiveness of graph convolutional networks (GCNs).
method Estimates the number of linear regions for one-layer and multi-layer GCNs.
result GCNs with multiple layers have exponentially more expressivity per parameter than one-layer GCNs.
Previous work has questioned the conditions under which the decision regions of a neural network are connected and further showed the implications of the corresponding theory to the problem of adversarial manipulation of classifiers. It has been proven that for a class of activation functions including leaky ReLU, neur…
We propose a methodological framework to study the dynamics of inter-regional investment flow in Europe from a Complex Networks perspective, an approach with recent proven success in many fields including economics. In this work we study the network of investment stocks in Europe at two different levels: first, we comp…
An activation boundary for a neuron refers to a separating hyperplane that determines whether the neuron is activated or deactivated. It has been long considered in neural networks that the activations of neurons, rather than their exact output values, play the most important role in forming classification friendly par…
OLALA automates document layout annotation by selecting ambiguous regions for labeling.
problem Efficiently annotating complex document layouts with limited resources.
method Object-Level Active Learning framework that selects ambiguous regions for labeling and uses semi-automatic correction.
result OLALA significantly boosts model performance and improves annotation efficiency.
The paper examines stability of ReLU networks in tangent space and activation regions.
problem Stability and sensitivity of ReLU networks to small changes.
method Tangent sensitivity measure for ReLU networks, focusing on stability induced by individual examples.
result Tangent sensitivity correlates with the distribution of activation regions and generalization gap.
Neural networks encode inputs deterministically and categorically, behaving like hash encoders.
problem Understanding the encoding properties of neural networks.
method Analyzed the input space partitioned by ReLU-like activations in neural networks.
result Neural networks can be represented by unique activation patterns, similar to hash encoders.
Deep active learning improves solvability prediction in power systems.
problem Inconclusive conservatism in traditional solvability region analysis methods.
method Proposes a deep active learning framework to reduce labeled dataset size.
result Significantly reduces the size of labeled dataset for training.
We can compare the expressiveness of neural networks that use rectified linear units (ReLUs) by the number of linear regions, which reflect the number of pieces of the piecewise linear functions modeled by such networks. However, enumerating these regions is prohibitive and the known analytical bounds are identical for…
Autonomous systems can be used to search for sparse signals in a large space; e.g., aerial robots can be deployed to localize threats, detect gas leaks, or respond to distress calls. Intuitively, search algorithms may increase efficiency by collecting aggregate measurements summarizing large contiguous regions. However…
We introduce a simple and effective method for regularizing large convolutional neural networks. We replace the conventional deterministic pooling operations with a stochastic procedure, randomly picking the activation within each pooling region according to a multinomial distribution, given by the activities within th…
DAMI uses interpretable regions to select informative samples for deep learning models.
problem Efficiently identifying informative samples for deep learning models with minimal annotation cost.
method Inspired by piece-wise linear interpretability in DNN, DAMI selects samples on different linearly separable regions.
result DAMI outperforms state-of-the-art approaches in tabular data.
A new VAD method uses respiration patterns from video to detect speech.
problem Improving VAD performance in noisy audio recordings.
method Extract respiration patterns from video, use neural models to detect speech.
result Efficacy demonstrated through experiments on real acoustic environments.
A new framework for verifying robustness of neural networks.
problem Verifying the robustness of neural networks against adversarial attacks.
method LayerCert framework exploiting the nested hyperplane arrangement structure of ReLU networks.
result LayerCert reduces the number and size of convex programs needed for robustness verification.
The developments of deep neural networks (DNN) in recent years have ushered a brand new era of artificial intelligence. DNNs are proved to be excellent in solving very complex problems, e.g., visual recognition and text understanding, to the extent of competing with or even surpassing people. Despite inspiring and enco…
Functional connectivity refers to the temporal statistical relationship between spatially distinct brain regions and is usually inferred from the time series coherence/correlation in brain activity between regions of interest. In human functional brain networks, the network structure is often inferred from functional m…
We present a new, unifying approach following some recent developments on the complexity of neural networks with piecewise linear activations. We treat neural network layers with piecewise linear activations as tropical polynomials, which generalize polynomials in the so-called or tropical algebra, with pos…
Neural network accuracy improves with denser training samples.
problem Improving neural network accuracy on unseen test samples.
method Bounding empirical training error smoothed across activation regions and using it to discard high-risk test samples.
result Discarding high-risk test samples based on error bounds improves prediction accuracy by up to 20%.
We study algebraic varieties of ReLU networks to understand their representable functions.
problem Understanding the functions that ReLU neural networks can represent.
method We introduce algebraic varieties associated with ReLU networks and derive polynomial equations to characterize representable functions.
result Conditions under which ReLU networks attain their expected dimension, providing insight into their structural properties.
Paper proposes LANN to measure model complexity of neural networks with curve activation functions.
problem Measuring model complexity of neural networks with curve activation functions.
method Proposes LANN, a piecewise linear framework to approximate curve activation functions, and derives complexity measure based on the number of linear regions.
result Demonstrates positive correlation between overfitting and model complexity during training.
A deep neural network (DNN) with piecewise linear activations can partition the input space into numerous small linear regions, where different linear functions are fitted. It is believed that the number of these regions represents the expressivity of the DNN. This paper provides a novel and meticulous perspective to l…
Maxout networks show similar complexity issues as ReLU networks.
problem Understanding the complexity of maxout networks and decision boundaries.
method Analyzing the parameter space and decision boundaries, obtaining lower bounds, and investigating initialization procedures.
result Maxout networks exhibit a wide range of complexity, similar to ReLU networks.
Proposes efficient data acquisition for personalized treatment effects from observational data.
problem Efficiently acquiring outcomes for personalized treatment effects in observational studies.
method Introduces causal, Bayesian acquisition functions to select points with overlapping support.
result Demonstrates improved sample efficiency and accuracy in learning personalized treatment effects.
One of the founding paradigms of machine learning is that a small number of variables is often sufficient to describe high-dimensional data. The minimum number of variables required is called the intrinsic dimension (ID) of the data. Contrary to common intuition, there are cases where the ID varies within the same data…
Active Learning for Approximation of Expensive Functions with Normal Distributed Output Uncertaintycs.LG
When approximating a black-box function, sampling with active learning focussing on regions with non-linear responses tends to improve accuracy. We present the FLOLA-Voronoi method introduced previously for deterministic responses, and theoretically derive the impact of output uncertainty. The algorithm automatically p…
Multi-Region Neural Representation: A novel model for decoding visual stimuli in human brainsstat.ML
Multivariate Pattern (MVP) classification holds enormous potential for decoding visual stimuli in the human brain by employing task-based fMRI data sets. There is a wide range of challenges in the MVP techniques, i.e. decreasing noise and sparsity, defining effective regions of interest (ROIs), visualizing results, and…
Researchers in functional neuroimaging mostly use activation coordinates to formulate their hypotheses. Instead, we propose to use the full statistical images to define regions of interest (ROIs). This paper presents two machine learning approaches, transfer learning and selection transfer, that are compared upon their…
The paper introduces CoCoCat bonds for multi-region natural catastrophes, accounting for complex dependencies.
problem Valuation of multi-region contingent convertible bonds under complex dependencies.
method Developed a model accounting for inter-regional dependencies using change-of-measure techniques.
result Significant impact of inter-regional dependencies on CoCoCat bond pricing.
The location-based social network, Foursquare, reflects the human activities of a city. The mobility dynamics inferred from Foursquare helps us understanding urban social events like crime In this paper, we propose a directed graph from the aggregated movement between regions using Foursquare data. We derive region ris…
We present an approach for polarimetric Synthetic Aperture Radar (SAR) image region boundary detection based on the use of B-Spline active contours and a new model for polarimetric SAR data: the GHP distribution. In order to detect the boundary of a region, initial B-Spline curves are specified, either automatically or…
Post-ADC inference corrects bias in statistical inference after active data collection.
problem Bias in inference after active data collection.
method Post-ADC inference framework that corrects bias from both ADC process and data-driven target construction.
result Valid inference for data collected by SMBO methods like GP-UCB and TPE.
New algorithm improves active learning in agnostic pool-based classification.
problem Efficient active learning in the agnostic setting with minimized sample complexity.
method Solves an experimental design problem to determine a distribution over examples for label requests.
result Achieves sample complexity bounds never worse than best disagreement coefficient-based bounds, sometimes significantly smaller.