Neural networks can represent complex piecewise functions efficiently.
problem Representing continuous piecewise affine functions with neural networks.
method Two hidden layers with ReLU activation, O(p) neurons for p pieces. result CPA functions can be represented by a neural network with linear size.
We use partial actions, as formalized by Exel, to construct various commensurating actions. We use this in the context of groups piecewise preserving a geometric structure, and we interpret the transfixing property of these commensurating actions as the existence of a model for which the group acts preserving the geome…
BN refines local partition geometry in piecewise-affine networks during training.
problem Understanding the effect of BN on the function realized during training in piecewise-affine networks.
method Analyzing the geometry of switching hyperplanes and affine-region partition conditioned on a mini-batch.
result BN increases expected local partition refinement in ReLU and piecewise-affine networks.
The paper designs neural networks with assurance for controlling nonlinear systems.
problem Designing neural networks with assurance for nonlinear system control.
method Bounding the number of affine functions needed for a CPWA function, connecting it to a TLL NN architecture.
result The TLL NN architecture is parameterized by the number of affine functions in the CPWA function it realizes.
PAR provides a flexible framework for quantization in optimization problems.
problem Challenges in optimization problems over discrete or quantized variables.
method Piecewise-affine regularization (PAR) for modeling and computational optimization.
result PAR-regularized loss functions exhibit high quantization at critical points in the overparameterized regime.
New EM algorithm improves deep generative network training.
problem Training deep generative networks with complex posterior and likelihood distributions.
method Derive analytical posterior and marginal distributions using CPA property, derive analytical EM algorithm.
result EM training yields higher likelihood than Variational Autoencoders (VAEs).
In exchange for large quantities of data and processing power, deep neural networks have yielded models that provide state of the art predication capabilities in many fields. However, a lack of strong guarantees on their behaviour have raised concerns over their use in safety-critical applications. A first step to unde…
Paper presents ABGD for efficient piecewise linear regression in high dimensions.
problem Efficiently solving piecewise linear regression in high-dimensional spaces.
method Parametrizes piecewise linear functions as difference of max-affine functions, using ABGD algorithm.
result ABGD converges linearly to an ε-accurate estimate with optimal sample complexity.
Method identifies latent variables from high-dimensional data with piecewise affine mixing.
problem Identifying latent variables from high-dimensional observations with dependencies and piecewise affine transformations.
method Proposes a two-stage method with sparsity and Gaussianity regularization.
result Effectively recovers ground-truth latent variables from synthetic and image data.
Quantum Monte Carlo speeds up option pricing for complex payoff functions.
problem Efficiently pricing options with complex payoff functions using quantum computing.
method Developed a quantum Monte Carlo algorithm for multidimensional Black-Scholes PDEs.
result Proved polynomial computational complexity and speed-up over classical methods.
The paper provides results regarding the computational complexity of hybrid system identification. More precisely, we focus on the estimation of piecewise affine (PWA) maps from input-output data and analyze the complexity of computing a global minimizer of the error. Previous work showed that a global solution could b…
Paper develops algorithms for PWA systems with polynomial regret.
problem Learning in piecewise affine systems due to discontinuities.
method Smoothed online learning framework applied to PWA systems.
result First algorithms with polynomial regret in PWA systems.
New NN design for nonlinear systems control with guarantees.
problem Designing NN architectures for nonlinear system control with guarantees.
method Exploits system model to design NN architecture, uses TLL NN for approximation.
result Guaranteed NN architecture sufficient for implementing a controller.
New insights into Deep Autoencoders for better data approximation and generalization.
problem Understanding and improving generalization of deep learning models with more parameters than data.
method Interpreting Deep Autoencoders' structure and using Lie group theory for regularization.
result Regularizations enable Deep Autoencoders to better approximate data manifolds and generalize.
We study the coarse geometry of the moduli space of dilation tori with two singularities and the dynamical properties of the action of the Teichmuller flow on this moduli space. This leads to a proof that the vertical foliation of a dilation torus is almost always Morse-Smale. As a corollary, we get that the generic pi…
This paper shows how to hedge financial risks with integer investments.
problem Evaluating the minimal super-hedging price with integer-valued strategies for arbitrary payoffs.
method Formulated a dynamic programming principle to evaluate the minimal super-hedging price with integer-valued strategies for continuous piecewise affine terminal claims.
result It is possible to evaluate the minimal super-hedging price with integer-valued strategies for discrete-time, arbitrary Ω.
Max-affine regression method converges linearly using GD and SGD.
problem Regression of max-affine models in signal processing and statistics.
method Gradient descent and mini-batch stochastic gradient descent analysis.
result GD and SGD converge linearly to a neighborhood of the ground truth under sub-Gaussian assumptions.
Study growth patterns in random networks using i.i.d. perturbations.
problem Understanding the growth of affine regions in random piecewise-linear networks.
method Analyzes a random compositional model with i.i.d. perturbations of the tent map, proving submultiplicative pressure and using finite-state defect process for upper-tail lower bounds.
result Proves the existence of a submultiplicative pressure for \(N_n\) and gives exponential upper bounds for \(n^{-1}\log N_n\).
New GP model estimates piecewise continuous functions.
problem Piecewise continuous regression functions in scientific and engineering applications.
method Local Gaussian process model with partitioned local data and joint estimation of boundaries.
result Superior performance over conventional GP models in estimating piecewise regression functions.
Batch normalization improves deep networks by aligning their decision boundaries with data.
problem Improving the performance and generalization of deep networks.
method Theoretical analysis of batch normalization as a function approximation technique for continuous piecewise affine splines.
result Batch normalization adapts the geometry of a deep network's partition to match the data, improving learning and generalization.
Paper proposes a new method for SP with covariates using PADR and ERM.
problem Stochastic programming with covariate information.
method Empirical risk minimization (ERM) with nonconvex piecewise affine decision rules (PADR).
result The method provides theoretical consistency and computational tractability for nonconvex SP problems.
This paper considers affine analogues of the isoperimetric inequality in the sense of piecewise linear topology. Given a closed polygon P embedded in R^d having n edges, we give upper and lower bounds for the minimal number of triangles needed to forma triangulated embedded orientable surface in R^d having P as its geo…
Two-dimensional affine A-nets in 3-space are quadrilateral meshes that discretize surfaces parametrized along asymptotic lines. The characterizing property of A-nets is planarity of vertex stars, so for generic A-nets the elementary quadrilaterals are skew. We classify the simply connected affine A-nets that can be ext…
A celebrated theorem of Hadwiger states that the Euler-Poincaré characteristic is the the unique invariant and continuous valuation on the distributive lattice of compact polyhedra in R^n that assigns value one to each convex non-empty such polyhedron. This paper provides an analogue of Hadwiger's result for finitely p…
Theorem proves integrability for piecewise-smooth distributions.
problem Integrability of piecewise-smooth distributions.
method Generalizations of Frobenius integrability theorem.
result Sufficient criteria for complete integrability with bi-Lipschitz coordinates.
For a bounded domain equipped with a piecewise Lipschitz continuous Riemannian metric g, we consider harmonic map from (Ω,g) to a compact Riemannian manifold (N,h)⊂Rk without boundary. We generalize the notion of stationary harmonic map and prove the partial regularity. We also discuss the global Li…
Exact LAD line fitting via PALB with linear scaling and speed.
problem Robust line fitting for data with outliers.
method Piecewise Affine Lower-Bounding (PALB) method using supporting lines and subdivision scheme.
result Empirical log-linear scaling and significantly faster than LP and IRLS methods.
A generalized semitoric system F:=(J,H): M --> R^2 on a symplectic 4-manifold is an integrable system whose essential properties are that F is a proper map, its set of regular values is connected, J generates an S^1-action and is not necessarily proper. These systems can exhibit focus-focus singularities, which corresp…
This paper tackles discontinuous neural networks for better approximation of piecewise continuous functions.
problem Limitation of neural networks in approximating piecewise continuous functions due to discontinuities.
method Proposes a decoupled two-step procedure to train a discontinuous deep neural network model.
result Provides approximation guarantees for the proposed model in piecewise continuous function spaces.
PARC uses piecewise linear predictors for regression and classification.
problem Multivariate regression and classification problems.
method Alternates between ridge and softmax regression, and cluster assignment based on accuracy and separability.
result Converges to a local minimum in a finite number of steps.
We present new families of continuous piecewise linear (CPWL) functions in Rn having a number of affine pieces growing exponentially in n. We show that these functions can be seen as the high-dimensional generalization of the triangle wave function used by Telgarsky in 2016. We prove that they can be computed by ReLU…
New method samples from piecewise smooth distributions using Hamiltonian Monte Carlo.
problem Sampling from distributions with discontinuous gradients.
method Generalized Randomized Hamiltonian Monte Carlo (GRHMC) for piecewise smooth targets.
result GRHMC processes sample from piecewise smooth target distributions with the desired distribution as the invariant distribution.
New method uses DC functions for piecewise linear regression.
problem Regression with piecewise linear constraints.
method Estimates piecewise linear convex functions using a difference of convex functions.
result Method achieves close to minimax statistical risk and comparable performance to existing methods.
Nonlinearity is crucial to the performance of a deep (neural) network (DN). To date there has been little progress understanding the menagerie of available nonlinearities, but recently progress has been made on understanding the rôle played by piecewise affine and convex nonlinearities like the ReLU and absolute value …
New algorithm reduces regret in online learning for piecewise continuous functions.
problem Exponential loss in efficiency when moving from classical to adversarial learning.
method Introduces generalized bracketing numbers and Follow-the-Perturbed-Leader algorithm.
result Optimal scaling of optimization oracle calls with average regret.
We consider the quasiconformal dilatation of projective transformations of the real projective plane. For non-affine transformations, the contour lines of dilatation form a hyperbolic pencil of circles, and these are the only circles that are mapped to circles. We apply this result to analyze the dilatation of the circ…
Deep Jump Gaussian Processes model high-dimensional piecewise functions.
problem Modeling high-dimensional piecewise continuous functions with limited accuracy.
method Integrates region-specific locally linear projections with Jump Gaussian Processes (JGP) to capture local low-dimensional subspace structures.
result DJGP achieves superior predictive accuracy and more reliable uncertainty quantification compared to existing methods.
This work generalizes bounds on the number of linear regions in CPWL NNs.
problem Determining the number of linear regions in CPWL neural networks is challenging.
method Generalized bounds on the maximal number of linear regions for arbitrary CPWL activation functions.
result Depth significantly increases the number of linear regions, but not exponentially.
In non-linear incompatible elasticity, the configurations are maps from a non-Euclidean body manifold into the ambient Euclidean space, Rk. We prove the Γ-convergence of elastic energies for configurations of a converging sequence, Mn→M, of body manifolds. This convergence result …
New method detects symmetries beyond affine transformations.
problem Current methods limit symmetry detection to affine transformations.
method Framework for discovering continuous symmetry beyond affine transformations.
result Method is competitive for large sample sizes and superior for small sample sizes.
Efficiently finds sparse solutions to max-plus equations for convex regression.
problem Finding sparse solutions to max-plus equations for convex multivariate regression.
method Polynomial-time algorithm for sparse approximate solutions.
result Optimal piecewise-linear fitting with minimum number of regions.
This research studies affine invariance in continuous-domain convolutional neural networks.
problem Recognizing patterns and features under affine transformations in continuous domains.
method Introduces a new criterion for assessing affine invariance, embeds images into the affine Lie group, and analyzes convolution over this group.
result Extends the scope of geometrical transformations that deep-learning pipelines can handle.
Normalizing flows attempt to model an arbitrary probability distribution through a set of invertible mappings. These transformations are required to achieve a tractable Jacobian determinant that can be used in high-dimensional scenarios. The first normalizing flow designs used coupling layer mappings built upon affine …
A triangulated piecewise-linear minimal surface in Euclidean 3-space defined using a variational characterization is critical for area amongst all continuous piecewise-linear variations with compact support that preserve the simplicial structure. We explicitly construct examples of such surfaces that are embedded and a…
In this paper we study time-inhomogeneous affine processes beyond the common assumption of stochastic continuity. In this setting times of jumps can be both inaccessible and predictable. To this end we develop a general theory of finite dimensional affine semimartingales under very weak assumptions. We show that the co…
Constructs finite element spaces for (p,q)-forms, excluding one subspace.
problem Constructing finite element spaces for (p,q)-forms. method Piecewise polynomial finite element spaces for all natural subspaces of (p,q)-forms, excluding one subspace. result Recovers known finite element spaces and introduces new ones.
Recently there have been exciting developments in Monte Carlo methods, with the development of new MCMC and sequential Monte Carlo (SMC) algorithms which are based on continuous-time, rather than discrete-time, Markov processes. This has led to some fundamentally new Monte Carlo algorithms which can be used to sample f…
Discretizes Helfrich-type energies on surfaces using triangular complexes.
problem Discretizing curvature energies on surfaces of specific type.
method Asymptotic lower bound combined with recovery sequence of triangulations and edge director fields.
result Valid discrete versions of integral curvature energies on surfaces.