Neural networks can represent complex piecewise functions efficiently.
problem Representing continuous piecewise affine functions with neural networks.
method Two hidden layers with ReLU activation, O(p) neurons for p pieces. result CPA functions can be represented by a neural network with linear size.
BN refines local partition geometry in piecewise-affine networks during training.
problem Understanding the effect of BN on the function realized during training in piecewise-affine networks.
method Analyzing the geometry of switching hyperplanes and affine-region partition conditioned on a mini-batch.
result BN increases expected local partition refinement in ReLU and piecewise-affine networks.
Paper presents ABGD for efficient piecewise linear regression in high dimensions.
problem Efficiently solving piecewise linear regression in high-dimensional spaces.
method Parametrizes piecewise linear functions as difference of max-affine functions, using ABGD algorithm.
result ABGD converges linearly to an ε-accurate estimate with optimal sample complexity.
In exchange for large quantities of data and processing power, deep neural networks have yielded models that provide state of the art predication capabilities in many fields. However, a lack of strong guarantees on their behaviour have raised concerns over their use in safety-critical applications. A first step to unde…
Method identifies latent variables from high-dimensional data with piecewise affine mixing.
problem Identifying latent variables from high-dimensional observations with dependencies and piecewise affine transformations.
method Proposes a two-stage method with sparsity and Gaussianity regularization.
result Effectively recovers ground-truth latent variables from synthetic and image data.
We use partial actions, as formalized by Exel, to construct various commensurating actions. We use this in the context of groups piecewise preserving a geometric structure, and we interpret the transfixing property of these commensurating actions as the existence of a model for which the group acts preserving the geome…
Max-affine regression method converges linearly using GD and SGD.
problem Regression of max-affine models in signal processing and statistics.
method Gradient descent and mini-batch stochastic gradient descent analysis.
result GD and SGD converge linearly to a neighborhood of the ground truth under sub-Gaussian assumptions.
New method uses DC functions for piecewise linear regression.
problem Regression with piecewise linear constraints.
method Estimates piecewise linear convex functions using a difference of convex functions.
result Method achieves close to minimax statistical risk and comparable performance to existing methods.
The paper designs neural networks with assurance for controlling nonlinear systems.
problem Designing neural networks with assurance for nonlinear system control.
method Bounding the number of affine functions needed for a CPWA function, connecting it to a TLL NN architecture.
result The TLL NN architecture is parameterized by the number of affine functions in the CPWA function it realizes.
PAR provides a flexible framework for quantization in optimization problems.
problem Challenges in optimization problems over discrete or quantized variables.
method Piecewise-affine regularization (PAR) for modeling and computational optimization.
result PAR-regularized loss functions exhibit high quantization at critical points in the overparameterized regime.
Paper develops algorithms for PWA systems with polynomial regret.
problem Learning in piecewise affine systems due to discontinuities.
method Smoothed online learning framework applied to PWA systems.
result First algorithms with polynomial regret in PWA systems.
PARC uses piecewise linear predictors for regression and classification.
problem Multivariate regression and classification problems.
method Alternates between ridge and softmax regression, and cluster assignment based on accuracy and separability.
result Converges to a local minimum in a finite number of steps.
We study the coarse geometry of the moduli space of dilation tori with two singularities and the dynamical properties of the action of the Teichmuller flow on this moduli space. This leads to a proof that the vertical foliation of a dilation torus is almost always Morse-Smale. As a corollary, we get that the generic pi…
Nonlinearity is crucial to the performance of a deep (neural) network (DN). To date there has been little progress understanding the menagerie of available nonlinearities, but recently progress has been made on understanding the rôle played by piecewise affine and convex nonlinearities like the ReLU and absolute value …
Quantum Monte Carlo speeds up option pricing for complex payoff functions.
problem Efficiently pricing options with complex payoff functions using quantum computing.
method Developed a quantum Monte Carlo algorithm for multidimensional Black-Scholes PDEs.
result Proved polynomial computational complexity and speed-up over classical methods.
Study growth patterns in random networks using i.i.d. perturbations.
problem Understanding the growth of affine regions in random piecewise-linear networks.
method Analyzes a random compositional model with i.i.d. perturbations of the tent map, proving submultiplicative pressure and using finite-state defect process for upper-tail lower bounds.
result Proves the existence of a submultiplicative pressure for \(N_n\) and gives exponential upper bounds for \(n^{-1}\log N_n\).
Efficiently finds sparse solutions to max-plus equations for convex regression.
problem Finding sparse solutions to max-plus equations for convex multivariate regression.
method Polynomial-time algorithm for sparse approximate solutions.
result Optimal piecewise-linear fitting with minimum number of regions.
Paper proposes a new method for SP with covariates using PADR and ERM.
problem Stochastic programming with covariate information.
method Empirical risk minimization (ERM) with nonconvex piecewise affine decision rules (PADR).
result The method provides theoretical consistency and computational tractability for nonconvex SP problems.
BNN-DP improves robustness analysis of Bayesian Neural Networks.
problem Ensuring robustness of Bayesian Neural Networks against adversarial attacks.
method Dynamic Programming applied to Bayesian Neural Networks as stochastic dynamical systems.
result BNN-DP provides tighter and more efficient bounds on prediction ranges compared to existing methods.
This paper considers affine analogues of the isoperimetric inequality in the sense of piecewise linear topology. Given a closed polygon P embedded in R^d having n edges, we give upper and lower bounds for the minimal number of triangles needed to forma triangulated embedded orientable surface in R^d having P as its geo…
Normalizing flows attempt to model an arbitrary probability distribution through a set of invertible mappings. These transformations are required to achieve a tractable Jacobian determinant that can be used in high-dimensional scenarios. The first normalizing flow designs used coupling layer mappings built upon affine …
Batch normalization improves deep networks by aligning their decision boundaries with data.
problem Improving the performance and generalization of deep networks.
method Theoretical analysis of batch normalization as a function approximation technique for continuous piecewise affine splines.
result Batch normalization adapts the geometry of a deep network's partition to match the data, improving learning and generalization.
Exact LAD line fitting via PALB with linear scaling and speed.
problem Robust line fitting for data with outliers.
method Piecewise Affine Lower-Bounding (PALB) method using supporting lines and subdivision scheme.
result Empirical log-linear scaling and significantly faster than LP and IRLS methods.
New NN design for nonlinear systems control with guarantees.
problem Designing NN architectures for nonlinear system control with guarantees.
method Exploits system model to design NN architecture, uses TLL NN for approximation.
result Guaranteed NN architecture sufficient for implementing a controller.
This paper shows how to hedge financial risks with integer investments.
problem Evaluating the minimal super-hedging price with integer-valued strategies for arbitrary payoffs.
method Formulated a dynamic programming principle to evaluate the minimal super-hedging price with integer-valued strategies for continuous piecewise affine terminal claims.
result It is possible to evaluate the minimal super-hedging price with integer-valued strategies for discrete-time, arbitrary Ω.
We consider the quasiconformal dilatation of projective transformations of the real projective plane. For non-affine transformations, the contour lines of dilatation form a hyperbolic pencil of circles, and these are the only circles that are mapped to circles. We apply this result to analyze the dilatation of the circ…
Tropical Geometry and Mathematical Morphology share the same max-plus and min-plus semiring arithmetic and matrix algebra. In this chapter we summarize some of their main ideas and common (geometric and algebraic) structure, generalize and extend both of them using weighted lattices and a max-⋆ algebra with an ar…
We present new families of continuous piecewise linear (CPWL) functions in Rn having a number of affine pieces growing exponentially in n. We show that these functions can be seen as the high-dimensional generalization of the triangle wave function used by Telgarsky in 2016. We prove that they can be computed by ReLU…
I study Gromov-Hausdorff limits of complex curves endowed with singular flat metrics of constant diameter. I formulate a criterion that the limit is collapsed in terms of a certain piecewise affine weight function on the dual intersection complex of a semi-stable model of the degeneration introduced by Kontsevich and S…
The paper provides results regarding the computational complexity of hybrid system identification. More precisely, we focus on the estimation of piecewise affine (PWA) maps from input-output data and analyze the complexity of computing a global minimizer of the error. Previous work showed that a global solution could b…
New EM algorithm improves deep generative network training.
problem Training deep generative networks with complex posterior and likelihood distributions.
method Derive analytical posterior and marginal distributions using CPA property, derive analytical EM algorithm.
result EM training yields higher likelihood than Variational Autoencoders (VAEs).
A method for identifying NPWARX models with arbitrary domains using probabilistic mixture models.
problem Identifying hybrid system models with discontinuous maps.
method Probabilistic mixture model with a neural network for nonlinear partitioning and Expectation Maximization for parameter estimation.
result Demonstrated on a nonlinear piece-wise problem with discontinuous maps.
A celebrated theorem of Hadwiger states that the Euler-Poincaré characteristic is the the unique invariant and continuous valuation on the distributive lattice of compact polyhedra in R^n that assigns value one to each convex non-empty such polyhedron. This paper provides an analogue of Hadwiger's result for finitely p…
A positive space is a space with a positive atlas, i.e. a collection of rational coordinate systems with subtraction free transition functions. The set of positive real points of a positive space is well defined. We define a tropical compactification of the latter. We show that it generalizes the Thurston compactificat…
The paper explores how ReLU DNNs can represent MPC policies and vice versa.
problem Representing MPC policies as ReLU DNNs and vice versa.
method Developed an approximate method for identifying input-space in ReLU nets resulting in PWA functions over polyhedral regions. Studied inverse multiparametric linear or quadratic programs for reconstruction of constraints and cost functions given a PWA function.
result Identification and representation of MPC policies as ReLU DNNs and vice versa.
This work generalizes bounds on the number of linear regions in CPWL NNs.
problem Determining the number of linear regions in CPWL neural networks is challenging.
method Generalized bounds on the maximal number of linear regions for arbitrary CPWL activation functions.
result Depth significantly increases the number of linear regions, but not exponentially.
A generalized semitoric system F:=(J,H): M --> R^2 on a symplectic 4-manifold is an integrable system whose essential properties are that F is a proper map, its set of regular values is connected, J generates an S^1-action and is not necessarily proper. These systems can exhibit focus-focus singularities, which corresp…
Discretizes Helfrich-type energies on surfaces using triangular complexes.
problem Discretizing curvature energies on surfaces of specific type.
method Asymptotic lower bound combined with recovery sequence of triangulations and edge director fields.
result Valid discrete versions of integral curvature energies on surfaces.
New GP model estimates piecewise continuous functions.
problem Piecewise continuous regression functions in scientific and engineering applications.
method Local Gaussian process model with partitioned local data and joint estimation of boundaries.
result Superior performance over conventional GP models in estimating piecewise regression functions.
Neural network models improve survival analysis with reduced computation time.
problem Limited expressiveness of standard survival models.
method Piecewise neural network models of hazard and density functions.
result Models outperform state-of-the-art models with less computation time.
Proposes adaptive ridge regression for functional linear models with piecewise shapes.
problem Functional linear regression with unknown coefficient function.
method Adaptive piecewise function template with L2 penalization. result Improves predictive power and interpretability compared to standard methods.
Deep neural networks with piecewise-polynomial activations can approximate smooth functions and their derivatives.
problem Approximating smooth functions and their derivatives with neural networks.
method Derives the depth, width, and sparsity required for approximation in Hölder norms.
result Deep neural networks with bounded weights can approximate Hölder smooth functions and their derivatives.
The paper extends a variance gamma model to quadratic functions, reducing arbitrage and computational costs.
problem Creating an arbitrage-free interpolation for option pricing models.
method Generalizing the local variance gamma model to a piecewise quadratic local variance function.
result The quadratic model results in an arbitrage-free interpolation of class C3, reducing knots and computational cost.
We propose a novel, theoretically-grounded, acquisition function for Batch Bayesian optimization informed by insights from distributionally ambiguous optimization. Our acquisition function is a lower bound on the well-known Expected Improvement function, which requires evaluation of a Gaussian Expectation over a multiv…
Global approximation for piecewise linear paths via signatures.
problem Global approximation theorems for piecewise linear paths.
method Using signatures of piecewise linear paths and their density in Lp-norms. result Linear functionals of signatures are dense in Lp-norms under an integrability condition. We study the geometry of deep (neural) networks (DNs) with piecewise affine and convex nonlinearities. The layers of such DNs have been shown to be {\em max-affine spline operators} (MASOs) that partition their input space and apply a region-dependent affine mapping to their input to produce their output. We demonstrat…
This technical note extends recent results on the computational complexity of globally minimizing the error of piecewise-affine models to the related problem of minimizing the error of switching linear regression models. In particular, we show that, on the one hand the problem is NP-hard, but on the other hand, it admi…
GraN-GAN normalizes gradients for better GAN performance.
problem Improving image generation in GANs with piecewise linear discriminators.
method Piecewise Gradient Normalization (GraN) for input-dependent normalization.
result Significant performance gains in image generation across various datasets.