Deep neural networks with piecewise-polynomial activations can approximate smooth functions and their derivatives.
problem Approximating smooth functions and their derivatives with neural networks.
method Derives the depth, width, and sparsity required for approximation in Hölder norms.
result Deep neural networks with bounded weights can approximate Hölder smooth functions and their derivatives.
We study algebraic varieties of ReLU networks to understand their representable functions.
problem Understanding the functions that ReLU neural networks can represent.
method We introduce algebraic varieties associated with ReLU networks and derive polynomial equations to characterize representable functions.
result Conditions under which ReLU networks attain their expected dimension, providing insight into their structural properties.
Proposed by Donoho (1997), Dyadic CART is a nonparametric regression method which computes a globally optimal dyadic decision tree and fits piecewise constant functions in two dimensions. In this article we define and study Dyadic CART and a closely related estimator, namely Optimal Regression Tree (ORT), in the contex…
New knots share same Upsilon invariant despite different Alexander polynomials.
problem Identifying concordant knots via Upsilon invariant.
method Examined hyperbolic L-space knots and their Upsilon invariants.
result Infinitely many pairs of hyperbolic L-space knots with distinct Alexander polynomials share the same Upsilon invariant.
New algorithm predicts piecewise regular functions online.
problem Online prediction of piecewise regular functions.
method Modified sleeping experts aggregation algorithm.
result Oracle risk bounds for all local regions.
Piecewise polynomial interpolation-based gradient descent reduces oracle complexity for smooth loss functions.
problem Optimizing empirical risk minimization loss functions
method Piecewise polynomial interpolation-based gradient descent
result Oracle complexity is reduced for smooth loss functions
This paper introduces a novel mixture model-based approach for simultaneous clustering and optimal segmentation of functional data which are curves presenting regime changes. The proposed model consists in a finite mixture of piecewise polynomial regression models. Each piecewise polynomial regression model is associat…
Efficiently finds sparse solutions to max-plus equations for convex regression.
problem Finding sparse solutions to max-plus equations for convex multivariate regression.
method Polynomial-time algorithm for sparse approximate solutions.
result Optimal piecewise-linear fitting with minimum number of regions.
This paper uses linear rational splines for invertible modeling, offering a simpler inverse and similar costs.
problem Creating expressive invertible models with tractable Jacobian determinants.
method Replacing affine transformations with linear rational splines in coupling layers.
result Linear rational splines offer a simpler inverse and similar costs for inference and generation.
Paper proposes algorithms to accurately identify breakpoints in piecewise regression.
problem Identifying accurate breakpoints in piecewise regression for better data fitting.
method Proposes novel greedy algorithms to minimize error and determine optimal breakpoints.
result The proposed algorithms outperform existing methods in accuracy and efficiency.
SURF simplifies distribution estimation with simple, robust, and fast algorithms.
problem Efficient and accurate distribution estimation in statistics and machine learning.
method Piecewise polynomial approximation using empirical probability interpolation and divide-and-conquer merging.
result Surpassing state-of-the-art algorithms in efficiency and accuracy, SURF estimates distributions robustly and quickly.
ParamBoost uses gradient boosting to create interpretable non-linear models with constraints.
problem Creating interpretable non-linear models with expert knowledge constraints.
method Gradient Boosting of cubic polynomials with specified constraints.
result ParamBoost outperforms state-of-the-art GAMs in real-world datasets.
We give a highly efficient "semi-agnostic" algorithm for learning univariate probability distributions that are well approximated by piecewise polynomial density functions. Let p be an arbitrary distribution over an interval I which is τ-close (in total variation distance) to an unknown probability distribution $…
Wide networks with polynomial activations have proven asymptotic behavior.
problem Understanding the behavior of neural networks in the large width limit.
method Proving a conjecture for deep networks with polynomial activation functions.
result Tight bounds on the behavior of wide networks during stochastic gradient descent and derivation of their finite-width dynamics.
We consider an appoximation of a catenoid constructed from "odd" truncated cones that maintains minimality in a certain sense. Thorough this procedure, we obtain a discrete curve approximating a catenary by exploiting the fact that it is the function that generates a catenoid. In this investigation, the theory of the G…
New algorithm reduces dynamic regret for noisy gradient feedback with piecewise polynomial comparators.
problem Online estimation of piecewise polynomial trends with noisy feedback.
method Introduces variational constraint for piecewise polynomial comparators, designs adaptive algorithm.
result Achieves nearly optimal dynamic regret of $ ilde{O}(n^{ rac{1}{2k+3}}C_n^{ rac{2}{2k+3}})$.
Quantum Monte Carlo speeds up option pricing for complex payoff functions.
problem Efficiently pricing options with complex payoff functions using quantum computing.
method Developed a quantum Monte Carlo algorithm for multidimensional Black-Scholes PDEs.
result Proved polynomial computational complexity and speed-up over classical methods.
The paper approximates Levi-Civita connection and curvature on 2D manifolds using finite elements.
problem Approximating Levi-Civita connection and curvature on 2D manifolds with finite elements.
method Using Regge finite elements, piecewise polynomial symmetric (0,2)-tensor fields, and distributional sense for non-regular tensors.
result Distributional quantities converge to their smooth counterparts under refinement of triangulation.
Paper develops algorithms for PWA systems with polynomial regret.
problem Learning in piecewise affine systems due to discontinuities.
method Smoothed online learning framework applied to PWA systems.
result First algorithms with polynomial regret in PWA systems.
This note is an addendum to our earlier work \cite{humi}. In \cite{humi}, we studied a Hamiltonian action for a generalized Calabi-Yau manifold and showed that the Duistermaat-Heckman theorem holds. The purpose of this note is to show that the density function of the Duistermaa-Heckman measure is a piecewise polynomial…
The study describes a cell structure for multisets in a rectangle.
problem Understanding the space of multisets in a rectangle.
method Developed a piecewise Euclidean bi-simplicial cell structure.
result Connected to spaces of complex polynomials and permutahedra.
Neural networks can represent complex piecewise functions efficiently.
problem Representing continuous piecewise affine functions with neural networks.
method Two hidden layers with ReLU activation, O(p) neurons for p pieces. result CPA functions can be represented by a neural network with linear size.
We present a new, unifying approach following some recent developments on the complexity of neural networks with piecewise linear activations. We treat neural network layers with piecewise linear activations as tropical polynomials, which generalize polynomials in the so-called (max,+) or tropical algebra, with pos…
We study additive models built with trend filtering, i.e., additive models whose components are each regularized by the (discrete) total variation of their kth (discrete) derivative, for a chosen integer k≥0. This results in kth degree piecewise polynomial components, (e.g., k=0 gives piecewise constant co…
PolyLUT uses polynomials to reduce FPGA latency.
problem Reducing latency in FPGA-based neural network inference.
method Training neural networks using multivariate polynomials as basic building blocks.
result Achieved significant latency and area improvements.
Constructs finite element spaces for (p,q)-forms, excluding one subspace.
problem Constructing finite element spaces for (p,q)-forms. method Piecewise polynomial finite element spaces for all natural subspaces of (p,q)-forms, excluding one subspace. result Recovers known finite element spaces and introduces new ones.
Paper studies efficient function approximation in high-dimensional spaces with low-dimensional structures.
problem Regression of functions varying along a central subspace in high-dimensional spaces.
method Generalized Contour Regression (GCR) algorithm for estimating the central subspace using piecewise polynomials.
result GCR leads to a mean squared estimation error of O(n−1) for the central subspace, improving the mean squared regression error of f to $O(n^{- rac{2s}{2s+d}})$. Hard problem of learning simple generative models from i.i.d. samples.
problem Learning simple neural network distributions from samples.
method Statistical query model, ODE-based construction of piecewise-linear functions.
result No polynomial-time algorithm can solve this problem even with one-hidden-layer ReLU networks.
Let G be a connected compact Lie group acting on a manifold M and let D be a transversally elliptic operator on M. The multiplicity of the index of D is a function on the set of irreducible representations of G. Let T be a maximal torus of G with Lie algebra Lie(T). We construct a finite number of piecewise polynomial …
Functional data analysis involves data described by regular functions rather than by a finite number of real valued variables. While some robust data analysis methods can be applied directly to the very high dimensional vectors obtained from a fine grid sampling of functional data, all methods benefit from a prior simp…
In this paper, we investigate adaptive nonlinear regression and introduce tree based piecewise linear regression algorithms that are highly efficient and provide significantly improved performance with guaranteed upper bounds in an individual sequence manner. We use a tree notion in order to partition the space of regr…
We recast basic topological concepts underlying differential geometry using the language and tools of noncommutative geometry. This way we characterize principal (free and proper) actions by a density condition in (multiplier) C*-algebras. We introduce the concept of piecewise triviality to adapt the standard notion of…
New GP model estimates piecewise continuous functions.
problem Piecewise continuous regression functions in scientific and engineering applications.
method Local Gaussian process model with partitioned local data and joint estimation of boundaries.
result Superior performance over conventional GP models in estimating piecewise regression functions.
Neural network models improve survival analysis with reduced computation time.
problem Limited expressiveness of standard survival models.
method Piecewise neural network models of hazard and density functions.
result Models outperform state-of-the-art models with less computation time.
New method uses DC functions for piecewise linear regression.
problem Regression with piecewise linear constraints.
method Estimates piecewise linear convex functions using a difference of convex functions.
result Method achieves close to minimax statistical risk and comparable performance to existing methods.
Proposes adaptive ridge regression for functional linear models with piecewise shapes.
problem Functional linear regression with unknown coefficient function.
method Adaptive piecewise function template with L2 penalization. result Improves predictive power and interpretability compared to standard methods.
The paper extends a variance gamma model to quadratic functions, reducing arbitrage and computational costs.
problem Creating an arbitrage-free interpolation for option pricing models.
method Generalizing the local variance gamma model to a piecewise quadratic local variance function.
result The quadratic model results in an arbitrage-free interpolation of class C3, reducing knots and computational cost.
Global approximation for piecewise linear paths via signatures.
problem Global approximation theorems for piecewise linear paths.
method Using signatures of piecewise linear paths and their density in Lp-norms. result Linear functionals of signatures are dense in Lp-norms under an integrability condition. GraN-GAN normalizes gradients for better GAN performance.
problem Improving image generation in GANs with piecewise linear discriminators.
method Piecewise Gradient Normalization (GraN) for input-dependent normalization.
result Significant performance gains in image generation across various datasets.
A derivation of the Cesàro-Fedorov relation from the Selberg trace formula on an orbifolded 2-sphere is elaborated and extended to higher dimensions using the known heat-kernel coefficients for manifolds with piecewise-linear boundaries. Several results are obtained that relate the coefficients, bi, in the Shephard-…
New method efficiently interpolates nonparametric density estimators.
problem Efficient evaluation of nonparametric density estimators.
method Piecewise multivariate polynomial interpolation scheme.
result New estimator with low space requirements and efficient querying.
New method for robust learning from batches, even adversarial ones.
problem Learning from batches that may be corrupt or adversarial.
method General framework for robust learning, derived from optimal robust algorithms.
result First robust agnostic learning algorithms for various distributions.
The paper studies geometric structures of polynomial spaces.
problem Understanding the geometric and combinatorial structures of polynomial spaces.
method Introducing and analyzing finite piecewise Euclidean cell complexes.
result The branched rectangle and annulus complexes are homeomorphic to specific polynomial spaces.
New proof shows efficient ReLU networks for piecewise linear functions.
problem Existence of efficient ReLU neural networks for piecewise linear functions.
method Degree 1 triangulations of the relative homology class bounded by polyhedra.
result Existence of efficient ReLU neural networks for functions with compact support.
Empirical risk minimization frequently employs convex surrogates to underlying discrete loss functions in order to achieve computational tractability during optimization. However, classical convex surrogates can only tightly bound modular loss functions, sub-modular functions or supermodular functions separately while …
Sampling logconcave functions arising in statistics and machine learning has been a subject of intensive study. Recent developments include analyses for Langevin dynamics and Hamiltonian Monte Carlo (HMC). While both approaches have dimension-independent bounds for the underlying continuous processes under s…
The paper analyzes a simple neural network model with algebraic methods.
problem Finding minima of a ridge-regularized mean squared error for ReLU perceptrons.
method Developed a Divide-Enumerate-Merge strategy using computational algebra.
result Identifies both isolated and connected minima of the RR-MSE.
A piecewise flat manifold is a triangulated manifold given a geometry by specifying edge lengths (lengths of 1-simplices) and specifying that all simplices are Euclidean. We consider the variation of angles of piecewise flat manifolds as the geometry varies in a particular way, which we call a conformal variation. This…