The paper explores the geometry and topology of DNN decision boundaries.
problem Understanding the geometric and topological properties of DNN decision boundaries.
method Differential geometry and the Gauss-Bonnet-Chern theorem.
result Computed the Euler characteristics of compact decision boundaries.
This work connects the Hessian to the decision boundary complexity in neural networks.
problem Understanding the decision boundary complexity in high-dimensional input space.
method Characterizing the decision boundary using the Hessian top eigenvectors and analyzing the number of outliers.
result The number of outliers in the Hessian spectrum is proportional to the complexity of the decision boundary.
DeepDIG generates samples near decision boundaries of deep neural networks for better understanding.
problem Limited knowledge of how deep neural networks make decisions.
method Adversarial example generation to create samples near decision boundaries.
result Characterized decision boundaries of various deep neural network models.
Many recent works on knowledge distillation have provided ways to transfer the knowledge of a trained network for improving the learning process of a new one, but finding a good technique for knowledge distillation is still an open problem. In this paper, we provide a new perspective based on a decision boundary, which…
Deep learning models have been the subject of study from various perspectives, for example, their training process, interpretation, generalization error, robustness to adversarial attacks, etc. A trained model is defined by its decision boundaries, and therefore, many of the studies about deep learning models speculate…
Study decision boundaries using heat diffusion and probabilistic techniques.
problem Understanding the geometry of decision boundaries in machine learning.
method Using Brownian motion and probabilistic techniques to analyze decision boundaries.
result Decision boundaries exhibit persistent 'wiggly and fuzzy' regions, even under adversarial attacks.
This work uses tropical geometry to understand neural network decision boundaries.
problem Characterizing neural network decision boundaries with piecewise linear activations.
method Tropical geometry applied to a simple neural network model.
result Decision boundaries are a subset of a tropical hypersurface related to a polytope formed by zonotopes.
New method measures generalizability of deep neural networks based on decision boundary complexity.
problem Lack of generalization methods for deep neural networks.
method Created Decision Boundary Complexity (DBC) score to measure DNN complexity.
result Simpler decision boundaries lead to better generalizability, supporting Occam's Razor.
The paper shows how neural networks with less decision boundary variability generalize better.
problem Improving neural network generalizability by reducing decision boundary variability.
method Introduces new measures (algorithm DB variability and (ε,η)-data DB variability) to quantify decision boundary variability and proves theoretical bounds on generalizability. result Neural networks with lower decision boundary variability have better generalizability, as shown by extensive experiments and theoretical bounds.
Measures neural network decision boundary volume to predict model performance.
problem Understanding the geometry of deep learning models for better performance.
method Local surface volumes to measure decision boundary, applying Weyl's tube formula.
result Smaller surface volume correlates with higher classification accuracy.
Deep learning models generalize by extending decision boundaries outside the convex hull of training data.
problem Understanding how deep learning models generalize beyond their training data.
method Investigation of decision boundaries inside and outside the convex hull of training sets, using various neural network architectures and training regimes.
result Over-parameterization is necessary for deep learning models to extend decision boundaries outside the convex hull of their training data.
Study reveals how features influence deep network decision boundaries.
problem Understanding the role of features in neural network decision boundaries.
method Adopted adversarial robustness tools to measure changes in CNN decision boundaries.
result Neural networks exhibit high invariance to non-discriminative features and are sensitive to small perturbations of training samples.
Deep neural networks' decision boundaries move closer to natural images during training.
problem Limited understanding of deep neural networks' decision boundaries and regions.
method Examined the minimum distance of data points to the decision boundary over training.
result The decision boundary moves closer to natural images during training, even in late epochs.
New method visualizes decision boundaries of classification models.
problem Difficulty in understanding how classification models interpret data.
method Hybrid supervised-unsupervised technique for visualizing decision boundaries.
result Provides interpretable maps for qualitative and quantitative analysis.
We propose the labeled Čech complex, the plain labeled Vietoris-Rips complex, and the locally scaled labeled Vietoris-Rips complex to perform persistent homology inference of decision boundaries in classification tasks. We provide theoretical conditions and analysis for recovering the homology of a decision boundary fr…
The goal of this paper is to analyze the geometric properties of deep neural network classifiers in the input space. We specifically study the topology of classification regions created by deep networks, as well as their associated decision boundary. Through a systematic empirical investigation, we show that state-of-t…
Proposes a method to speed up model selection for classification.
problem Time-consuming model selection process and lack of dataset-specific insights.
method Relaxes decision boundaries of neural decision trees to find equivalent or better models.
result Reduces the scope of exploration needed for model selection.
Deep neural networks have been widely deployed in various machine learning tasks. However, recent works have demonstrated that they are vulnerable to adversarial examples: carefully crafted small perturbations to cause misclassification by the network. In this work, we propose a novel defense mechanism called Boundary …
Paper proposes a new classifier for hyperbolic spaces using horospherical boundaries.
problem Optimization of large margin classifiers in hyperbolic spaces.
method Horospherical decision boundaries for geodesically convex optimization.
result Geodesically convex optimization leads to globally optimal solutions.
Convex polytope trees expand decision trees with interpretable boundaries.
problem High accuracy often requires many nodes in decision trees, reducing interpretability.
method CPT uses logical disjunction of weighted linear decision-makers, geometrically a convex polytope.
result CPT achieves high accuracy with fewer nodes compared to existing methods.
Decision trees are a popular technique in statistical data classification. They recursively partition the feature space into disjoint sub-regions until each sub-region becomes homogeneous with respect to a particular class. The basic Classification and Regression Tree (CART) algorithm partitions the feature space using…
Mixup reduces the sample complexity of finding optimal decision boundaries for more separable data.
problem Finding optimal decision boundaries in separable data distributions.
method Mixup technique applied to binary linear classification problems.
result Mixup significantly reduces the sample complexity for more separable data.
Capacity control, the bias/variance dilemma, and learning unknown functions from data, are all concerned with identifying effective and consistent fits of unknown geometric loci to random data points. A geometric locus is a curve or surface formed by points, all of which possess some uniform property. A geometric locus…
We use flip points to explain and audit deep learning models, revealing decision boundaries and improving model performance.
problem Lack of interpretability in deep learning models hinders their use in important applications.
method Flip points are used to analyze decision boundaries of deep learning models with continuous output scores.
result Flip points reveal the least changes in input that would alter a model's classification, enabling better understanding and improvement of model behavior.
Paper extends transfer learning for decision rules, improving treatment rule estimation.
problem Estimating optimal individualized treatment rules under changing conditions.
method Bayes decision rules and low-dimensional empirical risk minimization.
result Consistent estimators and risk bounds established under mild conditions.
New attacks reveal membership in label-only ML models.
problem Vulnerability of ML models to membership inference attacks.
method Developed decision-based membership inference attacks.
result Label-only exposures are vulnerable to membership leakage.
The goal of a decision-based adversarial attack on a trained model is to generate adversarial examples based solely on observing output labels returned by the targeted model. We develop HopSkipJumpAttack, a family of algorithms based on a novel estimate of the gradient direction using binary information at the decision…
Local surrogate models, to approximate the local decision boundary of a black-box classifier, constitute one approach to generate explanations for the rationale behind an individual prediction made by the back-box. This paper highlights the importance of defining the right locality, the neighborhood on which a local su…
A set of introductory notes on the subject of data classification using a linear classifier and least-squares cost function, and the negative effect of the presence of outliers on the decision boundary of the linear discriminant. We also show how a simple scaling could make the outlier less significant, thereby obtaini…
Deep neural networks achieve optimal learning rates for high-dimensional classification.
problem Learning classification functions from noisy data with smooth boundaries.
method Empirical risk minimization over deep neural networks for locally Barron-regular decision boundaries.
result Optimal estimation rates are independent of dimension and can be achieved by deep neural networks.
New method reduces model bias and variance by adjusting training sample weights based on label uncertainty.
problem Tradeoff between model bias and variance in classification models.
method Estimate label uncertainty, adjust training sample weights, and fine-tune decision boundary.
result Improves model performance and reduces variance in physical activity recognition.
Local decision boundary approximation improves model explanations for complex models.
problem Challenges in explaining complex, opaque machine learning models.
method Train a variational autoencoder to learn a latent space and map it to meaningful attributes. Use these attributes to approximate the local decision boundary and explain model predictions.
result Can recover latent attributes that determine class decisions in a new benchmark data set.
Unified framework improves cross-corpus EEG emotion recognition by aligning prototypes and refining decision boundaries.
problem Cross-corpus EEG emotion recognition suffers from performance degradation due to physiological variability and device inconsistencies.
method Prototype-driven Adversarial Alignment (PAA) framework with three configurations: local, contrastive, and boundary-aware.
result State-of-the-art performance improvements across four cross-corpus evaluation protocols.
RAPID efficiently samples SVDD subsets for better anomaly detection.
problem Efficiently sampling SVDD subsets for large datasets.
method Formulated as an optimization problem, RAPID selects samples that approximate the full SVDD decision boundary.
result RAPID outperforms competitors in classification accuracy, sample size, and runtime.
In practical work with American put options, it is important to be able to know when to exercise the option, and when not to do so. In computer simulation based on the standard theory of geometric Brownian motion for simulating stock price movements, this problem is fairly easy to handle for options with a short lifesp…
The paper identifies potential adversarial samples near decision boundaries of neural networks.
problem Vulnerability of deep neural networks to small perturbations of inputs.
method Developed a method to explore near decision boundaries of trained classifiers to identify potential adversarial samples.
result Potential adversarial samples represent only 61% of the test data but cover more than 82% of adversarial samples produced by iFGSM and 92% of those by DeepFool on CIFAR10.
SBAMDT uses adaptive soft splits to model complex decision boundaries.
problem Limited ability of standard decision trees to capture complex decision boundaries.
method Probabilistic additive decision tree model with adaptive soft multivariate splits.
result Demonstrated improved predictive performance on synthetic and real datasets.
Deep neural networks are vulnerable to adversarial attacks and hard to interpret because of their black-box nature. The recently proposed invertible network is able to accurately reconstruct the inputs to a layer from its outputs, thus has the potential to unravel the black-box model. An invertible network classifier c…
XVA is a material component of a trade valuation and hence it must impact the decision to exercise options within a given netting set. This is true for both unsecured trades and secured / cleared trades where KVA and MVA play a material role even if CVA and FVA do not. However, this effect has frequently been ignored i…
Decision Machines embeds decision trees into vector spaces for improved optimization.
problem Overfitting and difficulty in finding optimal decision tree structure.
method Embedding Boolean tests into a binary vector space and representing tree structure as matrices.
result Optimized decision trees with enhanced predictive power.
The paper introduces boundary thickness as a measure for improving model robustness.
problem Improving the robustness of machine learning models to adversarial and non-adversarial corruptions.
method Introducing boundary thickness as a measure and showing how various procedures can increase it.
result Thicker decision boundaries lead to improved robustness against adversarial and out-of-distribution transforms.
Deep networks have recently been shown to be vulnerable to universal perturbations: there exist very small image-agnostic perturbations that cause most natural images to be misclassified by such classifiers. In this paper, we propose the first quantitative analysis of the robustness of classifiers to universal perturba…
Paper develops a fractal dimension-based generalization measure.
problem Developing a robust generalization measure for machine learning models.
method Analyzes decision boundaries using fractal dimension concept.
result Developed a generalization measure based on fractal dimension.
We propose a new active learning strategy designed for deep neural networks. The goal is to minimize the number of data annotation queried from an oracle during training. Previous active learning strategies scalable for deep networks were mostly based on uncertain sample selection. In this work, we focus on examples ly…
Unified framework models neural decision-making, improving accuracy.
problem Limitations in modeling neural activity during decision-making.
method Unifying framework based on state-space models with scalable inference.
result Two-dimensional accumulator better captures neural responses.
ACE improves counterfactual explanations with fewer model queries.
problem Inefficient sampling for counterfactual explanations in machine learning models.
method Adaptive sampling combining Bayesian estimation and stochastic optimization.
result ACE achieves superior evaluation efficiency compared to state-of-the-art methods.
A new classifier improves one-class predictions on unevenly sampled data.
problem Non-uniformly sampled data affects one-class classifier performance.
method Dynamic decision boundary based on minimum spanning tree.
result Proves effectiveness and robustness compared to state-of-the-art classifiers.
A new method for generating counterfactual explanations in high-dimensional datasets.
problem Creating realistic counterfactual explanations in complex, high-dimensional data.
method A discretized approach using binary search and boundary approximation.
result Our method reduces the distance between counterfactuals by 5% to 50% in terms of the L2 norm.