The paper explores the geometry and topology of DNN decision boundaries.
problem Understanding the geometric and topological properties of DNN decision boundaries.
method Differential geometry and the Gauss-Bonnet-Chern theorem.
result Computed the Euler characteristics of compact decision boundaries.
Analyzes how learning algorithms affect and are affected by data manipulation.
problem Characterizing the closed-loop behavior of learning algorithms in the presence of decision-dependent data.
method Analyzes repeated risk minimization as perturbed gradient flows of performative risk minimization, considering multiple local minimizers.
result Characterizes the region of attraction for various equilibria and introduces performative alignment.
We show linear XOR classification is possible and propose equality separation for anomaly detection.
problem Linearly separating XOR data.
method Equality separation, adapting SVM objective for data within/outside margin.
result Equality separation can detect both seen and unseen anomalies.
Previous work has questioned the conditions under which the decision regions of a neural network are connected and further showed the implications of the corresponding theory to the problem of adversarial manipulation of classifiers. It has been proven that for a class of activation functions including leaky ReLU, neur…
We show that for neural network functions that have width less or equal to the input dimension all connected components of decision regions are unbounded. The result holds for continuous and strictly monotonic activation functions as well as for the ReLU activation function. This complements recent results on approxima…
Proposes CPO framework for robust decision-making with explainable uncertainty regions.
problem Overly conservative uncertainty regions in data-driven optimization lead to suboptimal decisions.
method Conformal-Predict-Then-Optimize (CPO) framework using conditional generative models and visual summaries.
result Demonstrates improved robustness and explainability in decision-making.
In the recent literature the important role of depth in deep learning has been emphasized. In this paper we argue that sufficient width of a feedforward network is equally important by answering the simple question under which conditions the decision regions of a neural network are connected. It turns out that for a cl…
The lack of interpretability remains a barrier to the adoption of deep neural networks. Recently, tree regularization has been proposed to encourage deep neural networks to resemble compact, axis-aligned decision trees without significant compromises in accuracy. However, it may be unreasonable to expect that a single …
Attribution methods provide insights into the decision-making of machine learning models like artificial neural networks. For a given input sample, they assign a relevance score to each individual input variable, such as the pixels of an image. In this work we adapt the information bottleneck concept for attribution. B…
Paper defines ε-Safe Decision Regions for exponential family distributions and approximates them for unbalanced data.
problem Need probabilistic guarantees for reliable predictions in machine learning.
method Formalizes ε-Safe Decision Regions, proves their form for exponential family distributions, and develops Multi Cost SVM for unbalanced data.
result Formal definition and analytical determination of ε-Safe Decision Regions for exponential family distributions.
New insights into how neural networks classify data.
problem Understanding the topological structure of decision regions in ReLU networks.
method Defining generic and transversal ReLU networks, and using linear complexes to identify obstructions.
result Generic, transversal ReLU networks have at most one bounded connected component in their decision regions.
CREDO assesses decision optimality under uncertainty without assuming a model.
problem Uncertainty in decision-making without reliable quantification of optimality.
method CREDO uses the inverse feasible region and conformal prediction balls to estimate decision optimality probability.
result CREDO provides accurate, efficient, and reliable evaluations of decision optimality.
End-to-end pipeline for data-driven decision making in mixed-integer optimization.
problem Data-driven decision making in mixed-integer optimization with uncertainty.
method Exploiting mixed-integer optimization-representability of machine learning methods, characterizing decision trust regions, and ensembling multiple models.
result Framework generates high-quality prescriptions and controls model robustness.
The goal of this paper is to analyze the geometric properties of deep neural network classifiers in the input space. We specifically study the topology of classification regions created by deep networks, as well as their associated decision boundary. Through a systematic empirical investigation, we show that state-of-t…
Study on inventory management under uncertainty using smooth ambiguity preference.
problem Managing inventory under Knightian uncertainty with smooth ambiguity preference.
method Demonstrates continuous-time smooth ambiguity as the infinitesimal limit of Kalman-Bucy filtering with recursive robust utility. Solves forward-backward stochastic differential equations with quadratic growth to determine cost function. Derives value function and optimal control policy using variational inequalities and viscosity solutions. Transforms problem into two-dimensional singular control.
result Ambiguity drives decision-makers to act earlier, reducing the continuation region.
Regular shrinkers describe blow-up limits of a finite-time singularity of the motion by curvature of planar network of curves. This follows from Huisken's monotonicity formula. In this paper, we show that there is only one regular shrinker with 2 closed regions. This regular shrinker is the Cisgeminate eye. Moreover, w…
DFFL tackles federated learning with heterogeneous objectives and constraints.
problem Federated learning with clients having different objectives and feasible regions.
method Derived heterogeneity bounds for cost-vector distances and support-function/shape-distance terms. Lifted pointwise bounds to local-versus-federated excess-risk comparison.
result Federation is beneficial when the statistical advantage of pooling exceeds a client-specific heterogeneity penalty.
We introduce a local move on a link diagram named a region freeze crossing change which is close to a region crossing change, but not the same. We study similarity and difference between region crossing change and region freeze crossing change.
Stochastic gradient descent approximates Gaussian process posteriors efficiently.
problem Efficiently sampling from Gaussian process posteriors with limited computational resources.
method Developed stochastic gradient optimization objectives for sampling from Gaussian process posteriors.
result Stochastic gradient descent produces accurate predictive distributions, even in non-convergent cases.
This paper proposes a new method to improve adversarial robustness of DNNs by focusing on semantic information.
problem Vulnerability of deep neural networks to adversarial attacks.
method Region adversarial training (RAT) that generates a single adversarial perturbation carrying semantic information.
result RAT greatly improves adversarial robustness with a small dataset and defends against FGSM attacks.
A central challenge in reinforcement learning is discovering effective policies for tasks where rewards are sparsely distributed. We postulate that in the absence of useful reward signals, an effective exploration strategy should seek out {\it decision states}. These states lie at critical junctions in the state space …
In data-limited settings, stochastic policies can outperform deterministic ones in bandit problems.
problem Making reliable decisions with limited data in bandit problems.
method Designing TRUST, an algorithm that uses localization laws and relative pessimism.
result TRUST achieves comparable sample complexity to LCB on minimax problems but is significantly lower on few-sample problems.
Decision Machines embeds decision trees into vector spaces for improved optimization.
problem Overfitting and difficulty in finding optimal decision tree structure.
method Embedding Boolean tests into a binary vector space and representing tree structure as matrices.
result Optimized decision trees with enhanced predictive power.
A deep neural network (DNN) with piecewise linear activations can partition the input space into numerous small linear regions, where different linear functions are fitted. It is believed that the number of these regions represents the expressivity of the DNN. This paper provides a novel and meticulous perspective to l…
No closed timelike geodesics in Kerr spacetimes, proving absence of closed causal geodesics.
problem Proving the nonexistence of closed timelike geodesics in Kerr spacetimes.
method Analyzing the Kerr-star spacetime, excluding closed null geodesics and proving the nonexistence of closed timelike geodesics.
result No closed timelike geodesics in Kerr spacetimes.
Generic smooth boundaries for isoperimetric regions in 8D manifolds.
problem Understanding boundaries of isoperimetric regions in high-dimensional spaces.
method Generic regularity results for isoperimetric regions in closed Riemannian manifolds of dimension eight.
result Smooth nondegenerate boundaries for isoperimetric regions for generic metrics and volumes.
The paper develops optimal confidence regions for categorical data.
problem Constructing tight confidence regions for categorical data.
method Develops new theory for minimum average volume confidence regions.
result Shows optimality of the regions for categorical data and its implications for machine learning.
Decision trees are a popular technique in statistical data classification. They recursively partition the feature space into disjoint sub-regions until each sub-region becomes homogeneous with respect to a particular class. The basic Classification and Regression Tree (CART) algorithm partitions the feature space using…
Deep reinforcement learning (DeepRL) agents surpass human-level performance in many tasks. However, the direct mapping from states to actions makes it hard to interpret the rationale behind the decision-making of the agents. In contrast to previous a-posteriori methods for visualizing DeepRL policies, in this work, we …
Accurate real-time monitoring systems of influenza outbreaks help public health officials make informed decisions that may help save lives. We show that information extracted from cloud-based electronic health records databases, in combination with machine learning techniques and historical epidemiological information,…
In a compact orbifold, for small prescribed volume, an isoperimetric region is close to a small metric ball; in a Euclidean orbifold, it is a small metric ball.
Constructs isoperimetric regions from separating hypersurfaces.
problem Finding isoperimetric regions in closed manifolds.
method Constructs isoperimetric regions from separating hypersurfaces.
result Yields isoperimetric boundaries with diverse topological types and singular sets.
New rule reduces exploration regret to logarithmic, improving bad episode handling.
problem Improving exploration regret in average reward MDPs.
method Replacing Doubling Trick with Vanishing Multiplicative rule in EVI-based algorithms.
result Regret is logarithmic under the new rule, significantly better than linear.
For gravitational collapse, we observe a correspondence between region close to past null infinity and region close to central singularity. In line with this philosophy, we construct a new ansatz, with which we first present a 40-page self-contained proof of trapped surface formation in a far-from-center region. A syst…
In this paper we propose a synergistic melting of neural networks and decision trees (DT) we call neural decision trees (NDT). NDT is an architecture a la decision tree where each splitting node is an independent multilayer perceptron allowing oblique decision functions or arbritrary nonlinear decision function if more…
Study on bounds of knot untangling for specific types of knots.
problem Determining upper limits for knot untangling.
method Defined warping degree, examined diagrams combinatorially.
result Upper bounds for unknotting and region unknotting numbers.
Proposes sparse local and regional counterfactual rules for robust recourses.
problem Challenges in counterfactual explanations, especially stability, synthesis, and implementation.
method Probabilistic framework using Random Forest to derive sparse local and regional counterfactual rules.
result Effective recourses derived from high-density regions, providing sparse and robust counterfactual rules.
Agents collaboratively learn optimal policies in MDPs with limited capabilities.
problem Learning optimal policies in MDPs with heterogeneous agents and limited communication.
method Introduced concepts of leakage probabilities and proposed Federated-Q protocol (FedQ) for collaborative learning.
result FedQ protocol effectively aggregates knowledge and modifies learning problems for further training.
Paper shows regions close to negatively curved metrics are minimal fillings and rigid.
problem Boundary rigidity and minimality of metrics near negatively curved ones.
method Generalizes previous work on filling volume minimality and boundary rigidity for almost hyperbolic metrics.
result Regions with metrics close to a negatively curved symmetric metric are strict minimal fillings and boundary rigid.
Decision trees and shallow neural networks have different geometric complexities, impacting their interpretability and accuracy.
problem The geometric simplicity of decision boundaries in decision trees conflicts with the approximation capabilities of shallow neural networks.
method Analysis of the Radon total variation (RTV) seminorm to compare geometric complexity of decision regions and neural network approximations.
result Smooth barrier scores can approximate decision regions with finite RTV, but their performance depends on the tube-mass condition near the decision boundary.
Entropy-regularized NPG methods converge linearly in discounted MDPs.
problem Theoretical limitations of NPG methods in reinforcement learning.
method Entropy regularization in conjunction with NPG methods for discounted MDPs.
result Entropy-regularized NPG methods converge linearly in discounted MDPs.
A new model predicts spatio-temporal data using adaptive decision trees and point processes.
problem Predicting spatio-temporal data with real-life applications.
method Hawkes process, adaptive decision tree, joint optimization algorithm.
result Significant improvement in predictions compared to standard methods.
A smart method predicts and optimizes decisions online with resource constraints.
problem Online decision-making with resource constraints.
method Combines prediction and optimization with dual update using mirror descent.
result Regret bounds and convergence rates for general convex feasible regions.
The paper finds singular isoperimetric regions in high-dimensional spaces.
problem Existence of singular isoperimetric regions in high-dimensional manifolds.
method Construction of specific manifolds with singular isoperimetric regions.
result Construction of manifolds with singular isoperimetric regions.
OneFlow detects anomalies by finding a minimal volume region, outperforming other methods.
problem Anomaly detection in data with complex outlier structures.
method Flow-based one-class classifier that uses a minimal volume region to define outliers.
result OneFlow outperforms other methods in real-world anomaly detection tasks.
Conformal Prediction Regions match Imprecise Highest Density Regions under consonance.
problem Matching conformal prediction regions with highest density regions.
method Using consonance and the Imprecise Probability theory of clouds.
result Imprecise Highest Density Regions are equivalent to Conformal Prediction Regions under consonance.
We prove a comparison theorem for the isoperimetric profiles of simple closed curves evolving by the normalized curve shortening flow: If the isoperimetric profile of the region enclosed by the initial curve is greater than that of some `model' convex region with exactly four vertices and with reflection symmetry in bo…
Q-learning adapted to find near-equivalent treatment strategies.
problem Finding optimal treatment sequences in dynamic treatment regimes.
method Introducing a worst-value tolerance criterion to find sets of near-equivalent policies.
result Constructs families of near-equivalent treatment strategies.