An activation boundary for a neuron refers to a separating hyperplane that determines whether the neuron is activated or deactivated. It has been long considered in neural networks that the activations of neurons, rather than their exact output values, play the most important role in forming classification friendly par…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “activation boundaries”
We propose a new active learning strategy designed for deep neural networks. The goal is to minimize the number of data annotation queried from an oracle during training. Previous active learning strategies scalable for deep networks were mostly based on uncertain sample selection. In this work, we focus on examples ly…
Study fractal and regular geometry in deep neural networks.
problem Investigate geometric properties of neural networks.
method Analyze boundary volumes of excursion sets for different activations.
result Hausdorff dimension increases with depth for non-regular activations.
We present the first adaptive strategy for active learning in the setting of classification with smooth decision boundary. The problem of adaptivity (to unknown distributional parameters) has remained opened since the seminal work of Castro and Nowak (2007), which first established (active learning) rates for this sett…
Active learning (AL) repeatedly trains the classifier with the minimum labeling budget to improve the current classification model. The training process is usually supervised by an uncertainty evaluation strategy. However, the uncertainty evaluation always suffers from performance degeneration when the initial labeled …
The paper examines topological features of ReLU networks and their relation to decision boundaries and training loss.
problem Understanding the topological structure of ReLU neural network activation patterns.
method Polytope decomposition of feature space, Fiedler partition of dual graph, homology computation of cellular decomposition.
result The Fiedler partition of the dual graph correlates with decision boundaries in binary classification tasks, and similar patterns in training loss and polyhedral cell-count emerge in regression tasks.
A Model-Free Sampling Method for Estimating Basins of Attraction Using Hybrid Active Learning (HAL)cs.LG
New method finds basins of attraction without needing system models.
problem Determining basins of attraction (BoA) for nonlinear systems without prior knowledge.
method Hybrid Active Learning (HAL) method combining AST, AL, and DBS.
result Efficiently finds and labels boundary of BoA without model knowledge.
This paper proposes LDM for efficient deep active learning.
problem Improving model performance by selecting informative unlabeled data.
method Least Disagree Metric (LDM) and its estimator for deep learning models.
result LDM-based active learning achieves state-of-the-art performance.
Internet of things (IoT) applications have become increasingly popular in recent years, with applications ranging from building energy monitoring to personal health tracking and activity recognition. In order to leverage these data, automatic knowledge extraction - whereby we map from observations to interpretable stat…
Novel approach analyzes ReLU networks' training dynamics and proposes GmP for improved optimization.
problem Stochastic optimization instability in ReLU networks impedes convergence and generalization.
method Characteristic activation boundaries analysis and Geometric Parameterization (GmP) technique.
result GmP resolves instability, leading to better optimization, convergence, and generalization.
New neural network approach solves Poisson equations efficiently.
problem Approximating solutions to Poisson equations with Dirichlet boundary conditions.
method Using shallow ReLU-networks to solve Laplace operator equations.
result Neural networks can approximate solutions to the Laplace operator with Dirichlet boundary conditions efficiently.
A fundamental problem in computer vision is boundary estimation, where the goal is to delineate the boundary of objects in an image. In this paper, we propose a method which jointly incorporates geometric and topological information within an image to simultaneously estimate boundaries for objects within images with mo…
New study analyzes security of neural network data reconstruction attacks.
problem Data reconstruction attacks pose a threat to private training data.
method Analyzes security boundary of data reconstruction attacks via neuron exclusivity state.
result Characterizes insecure/secure boundary of data reconstruction attacks.
Generating labeled training datasets has become a major bottleneck in Machine Learning (ML) pipelines. Active ML aims to address this issue by designing learning algorithms that automatically and adaptively select the most informative examples for labeling so that human time is not wasted labeling irrelevant, redundant…
Maxout networks show similar complexity issues as ReLU networks.
problem Understanding the complexity of maxout networks and decision boundaries.
method Analyzing the parameter space and decision boundaries, obtaining lower bounds, and investigating initialization procedures.
result Maxout networks exhibit a wide range of complexity, similar to ReLU networks.
Develops active intervals for geodesics in Teichmüller space.
problem Understanding geodesics in Teichmüller space with no backtracking.
method Defines active intervals for subsurfaces along geodesics in Thurston metric.
result Active intervals represent reparametrized quasi-geodesics in curve graphs with bounded movement outside.
The Dirichlet-to-Neumann map, the boundary Laplacian, and Hörmander's rediscovered manuscriptmath.SP
Recent research connects Hörmander's old work to modern boundary Laplacian analysis.
problem How close is the Dirichlet-to-Neumann map to the boundary Laplacian?
method Investigates techniques from Hörmander's 1950s manuscript to solve modern boundary Laplacian problems.
result Obtained results for DtN maps on non-smooth boundaries, Helmholtz equation, and differential forms.
In this survey, we discuss some recent results on free boundary minimal surfaces in the Euclidean unit-ball. The subject has been a very active field of research in the past few years due to the seminal work of Fraser and Schoen on the extremal Steklov eigenvalue problem. We review several different techniques of const…
Dirichlet Active Learningstat.ML
DiAL uses Bayesian Dirichlet random fields for active learning with sparse labels.
problem Active learning with limited labeled data.
method Bayesian Dirichlet random field for feature-conditional class probabilities, calibrating with graph Laplacian.
result Competitive performance in low-label rate graph learning tasks.
There is a large body of work on convergence rates either in passive or active learning. Here we first outline some of the main results that have been obtained, more specifically in a nonparametric setting under assumptions about the smoothness of the regression function (or the boundary between classes) and the margin…
Common nonlinear activation functions used in neural networks can cause training difficulties due to the saturation behavior of the activation function, which may hide dependencies that are not visible to vanilla-SGD (using first order gradients only). Gating mechanisms that use softly saturating activation functions t…
In this paper we present a new method for motion tracking of tumors in liver ultrasound image sequences. Our algorithm has two main steps. In the first step, we apply mean shift algorithm with multiple features to estimate the center of the target in each frame. Target in the first frame is defined using an ellipse. Ed…
The paper explores transferability of adversarial examples between convex and 01 loss models, finding non-transferability due to different decision boundaries caused by outliers.
problem Transferability of adversarial examples between convex and 01 loss models.
method Empirical study of transferability between linear 01 loss and convex (hinge) loss models, and between neural networks with different activation functions.
result Adversarial examples are non-transferable between convex and 01 loss models due to different decision boundaries caused by outliers.
A new criterion for deep active learning selects minimal labeled data points.
problem Efficiently select minimal labeled data points for deep neural networks.
method Diffuses label information over a graph of data representations to switch between exploration and refinement.
result The diffusion-based criterion outperforms existing methods in deep active learning.
Deep learning models have been the subject of study from various perspectives, for example, their training process, interpretation, generalization error, robustness to adversarial attacks, etc. A trained model is defined by its decision boundaries, and therefore, many of the studies about deep learning models speculate…
Recent years have witnessed amazing outcomes from "Big Models" trained by "Big Data". Most popular algorithms for model training are iterative. Due to the surging volumes of data, we can usually afford to process only a fraction of the training data in each iteration. Typically, the data are either uniformly sampled or…
We present an approach for polarimetric Synthetic Aperture Radar (SAR) image region boundary detection based on the use of B-Spline active contours and a new model for polarimetric SAR data: the GHP distribution. In order to detect the boundary of a region, initial B-Spline curves are specified, either automatically or…
An open question in systems and computational neuroscience is how neural circuits accumulate evidence towards a decision. Fitting models of decision-making theory to neural activity helps answer this question, but current approaches limit the number of these models that we can fit to neural data. Here we propose a unif…
This paper develops mfEGRA, a multifidelity active learning method using data-driven adaptively refined surrogates for failure boundary location in reliability analysis. This work addresses the issue of prohibitive cost of reliability analysis using Monte Carlo sampling for expensive-to-evaluate high-fidelity models by…
Deep generative neural networks (DGNNs) have achieved realistic and high-quality data generation. In particular, the adversarial training scheme has been applied to many DGNNs and has exhibited powerful performance. Despite of recent advances in generative networks, identifying the image generation mechanism still rema…
Piecewise linear activations create many spurious local minima in neural networks.
problem Understanding the loss surface of neural networks with piecewise linear activations.
method Proved the existence of infinite spurious local minima and partitioned the loss surface into smooth cells.
result Piecewise linear activations create many spurious local minima that are invariant under a continuous path.
In manifold learning, algorithms based on graph Laplacians constructed from data have received considerable attention both in practical applications and theoretical analysis. In particular, the convergence of graph Laplacians obtained from sampled data to certain continuous operators has become an active research topic…
BEKAN uses RBFs and evolutionary methods to solve PDEs with boundary conditions.
problem Enforcing boundary conditions in neural networks for PDE solutions.
method Boundary condition-guaranteed evolutionary Kolmogorov-Arnold Network (BEKAN) with radial basis functions (RBFs). Incorporates Dirichlet, periodic, and Neumann conditions.
result BEKAN outperforms MLP and B-splines KAN in solving PDEs with boundary conditions.
This work tackles the problem of characterizing and understanding the decision boundaries of neural networks with piecewise linear non-linearity activations. We use tropical geometry, a new development in the area of algebraic geometry, to characterize the decision boundaries of a simple network of the form (Affine, Re…
Novel metric space magnitude and weighting vectors improve machine learning tasks.
problem Improving machine learning algorithms using novel metric space concepts.
method Metric space magnitude and weighting vectors for better machine learning.
result The weighting vector effectively detects boundaries and improves classic machine learning tasks.
Study decision boundaries using heat diffusion and probabilistic techniques.
problem Understanding the geometry of decision boundaries in machine learning.
method Using Brownian motion and probabilistic techniques to analyze decision boundaries.
result Decision boundaries exhibit persistent 'wiggly and fuzzy' regions, even under adversarial attacks.
Derives variance kernel for reaction boundary in financial models.
problem Separating components in financial volatility models.
method Operational-time variance kernel, damped Abel response kernel, closed asymptotic form.
result Operational variance has a closed asymptotic form involving various parameters.
Derives operational-time variance kernel for reaction boundaries in financial markets.
problem Separating components in volatility models to better understand market dynamics.
method Derives a variance kernel for a latent-order-book reaction boundary, separating structural boundary cumulant, clock projection, and pricing-measure choice.
result Operational variance has a closed asymptotic form for long-memory forcing, with effective signed-forcing intensity and resilience.
In order to classify linearly non-separable data, neurons are typically organized into multi-layer neural networks that are equipped with at least one hidden layer. Inspired by some recent discoveries in neuroscience, we propose a new model of artificial neuron along with a novel activation function enabling the learni…
We provide a theoretical explanation of the role of the number of nodes at each layer in deep neural networks. We prove that the largest variation of a deep neural network with ReLU activation function arises when the layer with the fewest nodes changes its activation pattern. An important implication is that deep neur…
Authors compute anomalies for fermions in 3, 4, and 5 dimensions.
problem Quantum anomalies in fermions with boundaries in odd dimensions.
method Explicit computation of anomalies for fermions in dimensions 3, 4, and 5.
result New boundary terms for chiral anomaly in 4 dimensions.
Paper proposes a new uncertainty measure for active learning in neural networks.
problem Efficiently selecting informative data points in limited labeled data scenarios.
method BalEntAcq, a new uncertainty measure based on balanced entropy, approximated by Beta distributions.
result BalEntAcq outperforms existing uncertainty measures in active learning.
We propose a sparse-coding framework for activity recognition in ubiquitous and mobile computing that alleviates two fundamental problems of current supervised learning approaches. (i) It automatically derives a compact, sparse and meaningful feature representation of sensor data that does not rely on prior expert know…
It has been widely assumed that a neural network cannot be recovered from its outputs, as the network depends on its parameters in a highly nonlinear way. Here, we prove that in fact it is often possible to identify the architecture, weights, and biases of an unknown deep ReLU network by observing only its output. Ever…
Given a collection of entities (or nodes) in a network and our intermittent observations of activities from each entity, an important problem is to learn the hidden edges depicting directional relationships among these entities. Here, we study causal relationships (excitations) that are realized by a multivariate Hawke…
A game-theoretic analysis of DEX competition through dynamic trading fees.
problem Competition between decentralized exchanges (DEXs) and their impact on trading fees and slippage.
method Characterization of an approximate Nash equilibrium via coupled system of partial differential equations and closed-form expressions for equilibrium fees.
result The equilibrium trading fees shift from the oracle price to a weighted average of the oracle and competitors' exchange rates under competition.
Optimal relaxed portfolio strategies for growth rate maximization problems with transaction costsq-fin.PM
In this paper we investigate a new class of growth rate maximization problems based on impulse control strategies such that the average number of trades per time unit does not exceed a fixed level. Moreover, we include proportional transaction costs to make the portfolio problem more realistic. We provide a Verificatio…
Estimates funding impact from an algorithmic relief rule, finding little effect on hospital activities.
problem Evaluating the impact of algorithmic policy decisions.
method Developed a treatment-effect estimator using algorithmic decisions as instruments.
result Funding from an algorithmic relief rule had little effect on COVID-19-related hospital activities.