Piecewise-linear approximation improves feature selection in logit models.
problem Improving feature subset selection in sequential logit models.
method Applied piecewise-linear approximation to logistic loss function to frame feature selection as a mixed integer linear optimization problem.
result Piecewise-linear approximation found a better subset of features than quadratic approximation.
Global approximation for piecewise linear paths via signatures.
problem Global approximation theorems for piecewise linear paths.
method Using signatures of piecewise linear paths and their density in Lp-norms. result Linear functionals of signatures are dense in Lp-norms under an integrability condition. The paper approximates smooth isotropic surfaces with piecewise linear ones.
problem Approximating smooth isotropic surfaces with piecewise linear ones.
method Using analogies with infinite dimensional moment map geometry, the authors prove the approximation of smooth isotropic immersions by piecewise linear ones.
result Smooth isotropic immersions can be approximated by piecewise linear isotropic maps.
Investigates neural network approximation power with bounds.
problem Understanding neural network capacity for approximation.
method Established lower and upper bounds on network size and difference.
result Improved bounds for certain function classes.
Efficiently finds sparse solutions to max-plus equations for convex regression.
problem Finding sparse solutions to max-plus equations for convex multivariate regression.
method Polynomial-time algorithm for sparse approximate solutions.
result Optimal piecewise-linear fitting with minimum number of regions.
A hybrid model combines piecewise linear and neural components for interpretable predictions.
problem Post-hoc interpretable methods lead to contradictory explanations and lower prediction accuracy.
method Hybrid model with piecewise linear and neural components.
result The model achieves good interpretability and state-of-the-art accuracy.
Smooth symplectic manifolds can be approximated by PL symplectic manifolds.
problem Understanding the relationship between smooth and piecewise linear symplectic structures.
method Defining PL symplectic manifolds and proving approximations.
result Smooth symplectic manifolds can be C0-approximated by PL symplectic manifolds. Tropical geometry and weighted lattices improve curve and surface fitting.
problem Fitting max-⋆ tropical curves and surfaces to data. method Max-⋆ algebra, weighted lattices, morphological adjunctions. result Optimal piecewise-linear regression for max-⋆ curves and surfaces. Proposes new models to solve portfolio selection with cardinality constraints using factor models.
problem Solving portfolio selection with cardinality constraints using factor models.
method Developed 0-1 linear models and a minimum edge-weighted clique problem to solve the cardinality constrained portfolio problem.
result Piecewise linear approximation reduces computation time for solving the quadratic problem.
Deep networks with ReLU outperform piecewise linear spline methods in function approximation.
problem Comparing expressive power of deep neural networks with ReLU activation to piecewise linear spline methods.
method Comparison of function approximation capabilities between deep neural networks with ReLU activation and piecewise linear spline methods.
result Deep neural networks with ReLU activation can approximate functions better or only slightly worse than piecewise linear spline methods.
Given a piecewise linear (PL) function p defined on an open subset of Rn, one may construct by elementary means a unique polyhedron with multiplicities $\D(p)$ in the cotangent bundle Rn×Rn∗ representing the graph of the differential of p. Restricting to dimension 2, we show that any smooth functi…
The paper develops a method to approximate arbitrary Bregman divergences from supervision.
problem Approximating an arbitrary Bregman divergence from supervision.
method Develops a formulation and algorithm for learning arbitrary Bregman divergences by approximating their convex generating function via a piecewise linear function.
result The method achieves a generalization error of Op(m−1/2) for metric learning, matching known bounds. The paper tackles prescribing discrete Gaussian curvature on polyhedral surfaces.
problem Prescribing discrete Gaussian curvature on polyhedral surfaces.
method Discrete conformal theory and variational principles with constraints.
result Proves Kazdan-Warner type theorems for polyhedral surfaces.
Improved Gaussian Process model for predicting trajectories without independence assumption errors.
problem Incorrect independence assumption in previous work on Gaussian Process uncertainty propagation.
method Proposed a novel piecewise linear approximation to correct the independence assumption in continuous models.
result Corrected the independence assumption in Gaussian Process models for predicting trajectories.
Explicit formula derived for Slepian process boundary non-crossing probabilities.
problem Calculating boundary non-crossing probabilities for Slepian processes.
method Derived explicit formula and approximation formula for general continuous boundaries.
result Easy to implement formulas for boundary non-crossing probabilities.
Two new criteria help understand the advantage of deep neural networks.
problem Understanding the advantage of deepening neural networks.
method Proposed two new criteria to evaluate the expressivity of functions computable by deep neural networks.
result Increasing layers is more effective than increasing units in improving the expressivity of deep neural networks.
New method uses DC functions for piecewise linear regression.
problem Regression with piecewise linear constraints.
method Estimates piecewise linear convex functions using a difference of convex functions.
result Method achieves close to minimax statistical risk and comparable performance to existing methods.
We obtain a criterion for approximability by embeddings of piecewise linear maps of a circle to the plane, analogous to the one proved by Minc for maps of a segment to the plane. Theorem. Let S be a triangulation of a circle with s vertices. Let f be a simplicial map of the graph S to the plane. The map f is approximab…
Paper proposes variational inference for piecewise-linear systems.
problem Intractability of switching dynamical systems.
method Variational approximation and expectation-maximization algorithms.
result Parameters can be estimated off-line, including the number of linear modes.
Paper proposes LANN to measure model complexity of neural networks with curve activation functions.
problem Measuring model complexity of neural networks with curve activation functions.
method Proposes LANN, a piecewise linear framework to approximate curve activation functions, and derives complexity measure based on the number of linear regions.
result Demonstrates positive correlation between overfitting and model complexity during training.
New applications of trace embedding lemma show exotic 4-manifolds properties.
problem Detecting exotic phenomena in 4-dimensional smooth structures.
method New applications of trace embedding lemma to study piecewise-linear surfaces.
result Found infinitely many pairs of homeomorphic 4-manifolds with distinct properties.
DNNs can approximate fractal functions with exponential linear regions.
problem Understanding neural network approximations of complex functions.
method Using Iterated Function Systems (IFS) and neural networks to generate fractal functions.
result DNNs can generate fractal functions with a number of linear regions exponential in the number of parameters.
Extends Tanimoto kernel to real-valued functions.
problem Measuring similarity between real-valued functions.
method Unified representation of real-valued functions via sets, derived general form of the kernel, explicit feature representation, and smooth approximation.
result General Tanimoto kernel for real-valued functions.
New CPWL functions for deep neural networks with practical applications in coding.
problem Expressivity of deep ReLU neural networks in high dimensions.
method Developed new families of CPWL functions and showed they can be computed by ReLU networks.
result Proved approximation error of CPWL functions by shallower networks and separation result.
Paper introduces ℓ-DER for regression tasks using morphological operators and convex-concave procedure.
problem Developing a universal approximator for regression tasks.
method Introduces ℓ-DER model, trains it using a convex-concave procedure (CCP) to minimize least-squares. result Outperforms other hybrid morphological models and state-of-the-art approaches.
The Immersed Boundary (IB) method is a widely-used numerical methodology for the simulation of fluid-structure interaction problems. The IB method utilizes an Eulerian discretization for the fluid equations of motion while maintaining a Lagrangian representation of structural objects. Operators are defined for transmit…
The paper extends Riemannian concepts to piecewise linear spaces, defining Einstein metrics and flows.
problem Extending Riemannian concepts to piecewise linear spaces.
method Defining an Einstein vector field and Ricci flows on piecewise linear spaces.
result Criteria for flows to approach Einstein metrics are formulated.
Label assignment problems with large state spaces are important tasks especially in computer vision. Often the pairwise interaction (or smoothness prior) between labels assigned at adjacent nodes (or pixels) can be described as a function of the label difference. Exact inference in such labeling tasks is still difficul…
SyMPLER improves time series forecasting in nonstationary environments with explainable models.
problem Nonstationary time series forecasting with limited interpretability.
method Dynamic piecewise-linear approximations based on Statistical Learning Theory generalization bounds.
result SyMPLER achieves comparable performance to black-box and explainable models while maintaining interpretability.
The center of a quotient group of piecewise linear homeomorphisms is trivial.
problem Understanding the structure of a specific group of homeomorphisms.
method Analyzing a quotient of piecewise linear homeomorphisms of the real line.
result The center of the quotient group is trivial.
Smooth structures on manifolds with group actions.
problem Smooth structures on manifolds with group actions.
method Piecewise linear manifolds with group actions, equivariant smoothing.
result Smooth structures compatible with group actions exist.
Properties of a parametric curve in R^3 are often determined by analysis of its piecewise linear (PL) approximation. For Bezier curves, there are standard algorithms, known as subdivision, that recursively create PL curves that converge to the curve in distance . The exterior angles of PL curves under subdivision are s…
This paper studies PL cobordism categories and their homotopy types.
problem Understanding the homotopy types of PL cobordism categories.
method Introducing a bordism category and showing weak homotopy equivalence to an infinite loop space.
result The classifying space BCdPL is weak homotopy equivalent to an infinite loop space. The preservation of ambient isotopic equivalence under piecewise linear (PL) approximation for smooth knots are prominent in molecular modeling and simulation. Sufficient conditions are given regarding: (1) Hausdorff distance, and (2) a sum of total curvature and derivative. High degree Bezier curves are often used as …
Estimates SDEs from sparse data using Gaussian processes.
problem Estimating drift and diffusion functions in SDEs from sparse observations.
method Gaussian processes for flexible modeling, approximate EM algorithm, sparse Gaussian process approximation.
result Accurate estimation of SDE parameters from sparse data.
Signature uniquely identifies piecewise linear surfaces up to thin homotopy.
problem Characterizing piecewise linear surfaces up to equivalence.
method Crossed module of piecewise linear surfaces and signature homomorphism.
result Signature uniquely characterizes surfaces up to translation and thin homotopy.
PARC uses piecewise linear predictors for regression and classification.
problem Multivariate regression and classification problems.
method Alternates between ridge and softmax regression, and cluster assignment based on accuracy and separability.
result Converges to a local minimum in a finite number of steps.
Piecewise linear activations create many spurious local minima in neural networks.
problem Understanding the loss surface of neural networks with piecewise linear activations.
method Proved the existence of infinite spurious local minima and partitioned the loss surface into smooth cells.
result Piecewise linear activations create many spurious local minima that are invariant under a continuous path.
Convex regression is a promising area for bridging statistical estimation and deterministic convex optimization. New piecewise linear convex regression methods are fast and scalable, but can have instability when used to approximate constraints or objective functions for optimization. Ensemble methods, like bagging, sm…
The regularization path of the Lasso can be shown to be piecewise linear, making it possible to "follow" and explicitly compute the entire path. We analyze in this paper this popular strategy, and prove that its worst case complexity is exponential in the number of variables. We then oppose this pessimistic result to a…
Paper finds how many neurons are needed to approximate histogram distributions.
problem How many neurons are needed to approximate a target probability distribution?
method Examined for uniform input distribution and histogram target distributions, using efficient neural net construction.
result Obtained a new upper bound on the number of required neurons, strictly better than previous bounds.
Piecewise-linear regression trees improve tree-based regression with theoretical and practical benefits.
problem Improving tree-based regression models with theoretical guarantees and practical tractability.
method Regularized piecewise-linear node-splitting criterion, LASSO-type and ℓ2 regularization, variable selection procedure. result New high-probability generalization error bounds for piecewise-linear regression trees.
Study approximates nonlinear functionals using deep ReLU networks.
problem Approximating nonlinear continuous functionals with neural networks.
method Constructs continuous piecewise linear interpolation under simple triangulation, analyzes rates of approximation.
result Established rates of approximation for functional deep ReLU networks.
New approach treats neural networks with piecewise linear activations using tropical geometry.
problem Upper bounds on linear regions of neural networks with ReLU or leaky ReLU activations.
method Treat neural network layers with piecewise linear activations as tropical polynomials, refining upper bounds using tropical geometry.
result Upper bounds on linear regions improved to $\min\left\{ 2^m, \sum_{j=0}^n \binom{m}{j}
ight\}$, where n,m are the number of inputs and outputs, respectively. Embeddings in 5D can be isotoped to closed braids.
problem Embeddings of manifolds in high dimensions.
method Piecewise linear co-dimension two embeddings, isotopy to closed braids.
result Embeddings in 5D can be isotoped to closed braids.
Proposes a method to make deep networks' derivatives more stable.
problem Making deep networks' derivatives more stable over larger regions.
method A learning problem to encourage stable derivatives, with an inference step and optimization step.
result Proposes a novel relaxation to scale the algorithm to realistic models.
Paper presents ABGD for efficient piecewise linear regression in high dimensions.
problem Efficiently solving piecewise linear regression in high-dimensional spaces.
method Parametrizes piecewise linear functions as difference of max-affine functions, using ABGD algorithm.
result ABGD converges linearly to an ε-accurate estimate with optimal sample complexity.
New cones in 4D space found with minimal mass.
problem Finding mass-minimizing piecewise linear cones in 4D space.
method Mass minimization via Lipschitz maps, classification of candidates.
result No additional mass-minimizing cones found outside five known cases.