Regularizers change the geometric properties of loss functions in neural networks.
problem Understanding how different regularizers affect the geometric properties of loss functions in neural networks.
method Examined several regularizers, including weight decay, to determine if the regularized loss function becomes Morse.
result For certain regularizers, the regularized loss function becomes Morse, indicating a change in geometric properties.
Study slice-regular polynomial functions via twistor space group actions.
problem Characterize slice-regular functions and their polynomial subclasses.
method Employ the twistor construction and group actions of PGL(2,H). result Characterize slice-regular functions with planar twistor lifts and normal classes of polynomials.
Entropy-regularized NPG converges linearly with linear function approximation.
problem Analyzing convergence of entropy-regularized NPG with function approximation.
method Established finite-time convergence analyses with entropy regularization and linear function approximation.
result Entropy-regularized NPG achieves linear convergence up to a function approximation error.
Research examines octonionic slice regular functions and their automorphisms and invariants.
problem Analyzing slice regular functions in the octonionic algebra.
method Investigates automorphisms and invariants of octonionic slice regular functions.
result Characterizes the automorphisms and invariants of octonionic slice regular functions.
Study uses property elicitation to understand how fairness regularizers affect optimal decisions.
problem Understanding how fairness regularizers change the optimal decision in predictive algorithms.
method Property elicitation to analyze the relationship between loss, regularization, and optimal decision.
result Necessary and sufficient condition for when a property changes with the addition of a regularizer.
New method encodes function preferences into neural nets for better generalization.
problem Challenges in encoding explicit function preferences in neural network training.
method Function-space empirical Bayes (FSEB) regularization.
result FSEB leads to near-perfect semantic shift detection and improved generalization.
Deep networks adapt to function regularity and data distribution.
problem Understanding deep learning's adaptability to function regularity and data distribution.
method Developed nonparametric approximation and estimation theories for a broad class of functions using deep ReLU networks.
result Deep neural networks are adaptive to different regularity of functions and nonuniform data distributions.
This work explores Target Networks and Functional Regularization in deep Reinforcement Learning.
problem Stability and performance issues in deep Reinforcement Learning due to target value instability.
method Proposes and studies an explicit Functional Regularization approach as a replacement for Target Networks.
result Functional Regularization improves performance and stability compared to Target Networks.
Proves regularity of extremal function on compact Kähler manifolds.
problem Regularity of extremal function on compact Kähler manifolds.
method Local property analysis and equivalence of continuity and Hölder continuity.
result Equivalence of classical notions of local L-regularity and locally Hölder continuous property. Extends Campanato theory to multi-valued functions for geometric variational problems.
problem Regularity of multi-valued functions in geometric variational problems.
method Adapting Campanato's ideas to multi-valued functions, proving regularity theorems.
result Established regularity for multi-valued harmonic functions and stationary integral varifolds.
Paper describes invariants of slice regular functions' automorphism group.
problem Understanding invariants of slice regular functions' automorphism group.
method Analyzes automorphism group of slice regular functions over Clifford algebras.
result Describes invariants of the automorphism group of slice regular functions.
Study connects Gaussian processes and regularization for sequence-function mappings.
problem Understanding and interpreting sequence-function maps in biology.
method Relates Gaussian process priors, regularization, and gauge fixing in overparameterized weight space.
result Established the relationship between regularized regression and Gaussian processes in function space.
Study introduces a new method for multiple parameter regularization in polynomial functional regression.
problem Handling varying regularization parameters in polynomial functional regression.
method Developed a theoretically grounded algorithm for multiple parameter regularization and model aggregation.
result Promising results from evaluations on synthetic and real-world data.
Proposes an L1-regularized functional SVM for binary classification with functional covariates.
problem Binary classification with multivariate functional covariates.
method L1-regularized functional support vector machine (SVM) with an accompanying algorithm.
result The proposed classifier performs well in prediction and feature selection.
Given a slice regular function f:Ω⊂H→H, with Ω∩R=∅, it is possible to lift it to a surface in the twistor space CP3 of S4≃H∪{∞} (see~\cite{gensalsto}). In this paper we show that the same result is true if one rem…
Establish C^{1,2} regularity of American value functions in Heston model
problem Regularity of American put options in Heston model
method PDE techniques
result C^{1,2} regularity in exercise domain and smooth-fit principle
We establish continuous maximal regularity results for parabolic differential operators acting on sections of tensor bundles on Riemannian manifolds. As an application, we show that solutions to the Yamabe flow instantaneously regularize and become real analytic in space and time. The regularity result is obtained by i…
New definition of regular points for PL functions on manifolds.
problem Defining regular points for PL functions on combinatorial manifolds.
method Definition based on link of the point, stratification of Jacobi set, Stein factorization of Reeb space.
result Our definition of regularity is distinct from existing definitions.
Develops local elliptic regularity for geometrically-natural operators with low regularity coefficients.
problem Local elliptic regularity for operators with low regularity coefficients in Sobolev-type spaces.
method Rescaling estimates and multiplication results for function spaces.
result Unified set of interior estimates and regularity inference for operators with Sobolev-type coefficients.
Optimally shows the distance between perturbed convex functions and their Γ-regularizations.
problem Understanding the difference between perturbed convex functions and their Γ-regularizations.
method Analyzing the compactly supported perturbation and the Γ-regularization of a strictly convex function.
result The optimal estimate of the distance between perturbed convex functions and their Γ-regularizations is shown to be o(ε). The purpose of this article is to present a new regularization technique of quasi-plurisubharmoinc functions on a compact Kaehler manifold. The idea is to regularize the function on local coordinate balls first, and then glue each piece together. Therefore, all the higher order terms in the complex Hessian of this regu…
A quaternionic version of Picard's theorem limits how many values a slice regular function can avoid.
problem How many values can a non-constant slice regular function of a quaternionic variable avoid?
method Investigates slice regular functions of quaternionic variables, extending the classical Picard theorem.
result A non-constant slice regular function of a quaternionic variable can avoid at most one value, similar to the classical Picard theorem.
Proves optimal regularity for sphere minimizers in 3-sphere.
problem Finding optimal regularity for sphere minimizers.
method Proves C1,1 regularity for minimizers of prescribed mean curvature over isotopy classes. result Proves optimal C1,1 regularity for minimizers. New iterative regularization method tackles non-smooth, non-strongly convex functionals.
problem Tackles non-smooth, non-strongly convex functionals in regularization problems.
method Primal-dual algorithm with convergence and stability analysis.
result First iterative regularization procedure for non-smooth, non-strongly convex functionals.
Paper improves learning rates for GSC loss functions using iterated Tikhonov regularization.
problem Improving learning rates for GSC loss functions.
method Iterated Tikhonov regularization using proximal point method.
result Achieves fast and optimal rates for GSC loss functions.
Defines weak geodesics on specific subsets of manifolds.
problem Characterizing geodesics on prox-regular subsets of Riemannian manifolds.
method Defining weak geodesics as continuous curves with weak regularities, and characterizing them as viscosity critical points of the energy functional.
result Characterizes weak geodesics on prox-regular subsets of Riemannian manifolds.
Study improves estimation of functions from noisy data using convex penalties.
problem Estimating functions from noisy point evaluations of linear operators.
method Tikhonov regularization with convex and p-homogeneous penalty functionals. result Derives concentration rates for regularized solutions in symmetric Bregman distance.
We provide adaptive inference methods, based on ℓ1 regularization, for regular (semi-parametric) and non-regular (nonparametric) linear functionals of the conditional expectation function. Examples of regular functionals include average treatment effects, policy effects, and derivatives. Examples of non-regular f…
This study explores star-shaped regularizers learned from critic-based losses.
problem Understanding the structure of regularizers learned from critic-based losses.
method Optimizing critic-based loss functions over star-shaped regularizers.
result Derives exact expressions for optimal regularizers in certain cases.
Unified framework for constructing nonconvex sparse recovery methods.
problem Constructing valid nonconvex regularization functions remains open.
method Unified framework based on probability density function, using Weibull distribution.
result New nonconvex sparse recovery method based on Weibull distribution.
Unified theory for representation learning using learnable functions.
problem Insufficient theoretical understanding of unsupervised and self-supervised learning.
method Discriminative theoretical framework for analyzing sample complexity.
result Learnable regularization functions can reduce the amount of labeled data needed.
Study on isoperimetric inequalities and regularity of A-harmonic functions on surfaces.
problem Investigating isoperimetric inequalities and regularity of A-harmonic functions on smooth surfaces. method Logarithmic and power-type convexity of the length of level curves, higher Sobolev regularity properties, and estimates for derivatives.
result Higher Sobolev regularity properties of solutions, including W2,2 regularity. We consider the problem of supervised learning with convex loss functions and propose a new form of iterative regularization based on the subgradient method. Unlike other regularization approaches, in iterative regularization no constraint or penalization is considered, and generalization is achieved by (early) stoppin…
New bounds on ReLU networks for low-regular functions.
problem Bounding approximation error for ReLU networks on low-regular functions.
method Complexity analysis of Fourier features residual networks to ReLU networks.
result Approximation error bound proportional to target function norm and inversely proportional to network width and depth.
Paper proposes DC functions for better regularization of inverse problems with theoretical guarantees.
problem Improving regularization for ill-posed inverse problems.
method Introduces difference-of-convex (DC) functions and uses them with optimization algorithms like DCA and PSM.
result DC functions yield improved performance and theoretical guarantees compared to weakly convex functions.
Develops a functional mix-of-experts model for multiclass classification.
problem Multiclass classification with univariate functional predictors.
method Functional mix-of-experts model with regularization and sparsity constraints.
result Regularized maximum likelihood estimation yields interpretable coefficient functions.
Unified definition of mass aspect function for weakly regular hyperbolic manifolds.
problem Ambiguity in mass definition for asymptotically hyperbolic manifolds.
method Introduced an ADM-style mass aspect function for broad asymptotics and low regularity.
result Unified mass aspect function exhibits favorable covariance properties.
New regularizer improves neural network robustness and generalization.
problem Ineffective weight decay for networks with homogeneous activation functions.
method Proposes an invariant regularizer to penalize intrinsic weight norms.
result Improves generalization and adversarial robustness on various datasets.
Koopman Regularization learns governing equations from sparse data.
problem Learning governing equations from sparse and corrupted data.
method Constrained optimization using Koopman Eigenfunctions.
result Restores dynamics precisely with minimal assumptions.
In this paper, we revisit implicit regularization from the ground up using notions from dynamical systems and invariant subspaces of Morse functions. The key contributions are a new criterion for implicit regularization---a leading contender to explain the generalization power of deep models such as neural networks---a…
Rapidly growing product lines and services require a finer-granularity forecast that considers geographic locales. However the open question remains, how to assess the quality of a spatio-temporal forecast? In this manuscript we introduce a metric to evaluate spatio-temporal forecasts. This metric is based on an Opti- …
Double descent risk in L2-regularized models explained and mitigated.
problem Risk of overparameterized models in machine learning.
method Analysis of L2-regularized models, two-layer neural networks, and CNNs.
result Double descent risk in L2-regularized models can be explained and mitigated by adjusting regularization strengths.
Study evaluates various regularization methods for electricity price forecasting.
problem Improving accuracy of electricity price predictions.
method Applied ten different penalty functions to two model structures in two electricity markets.
result LQ and elastic net consistently produce more accurate forecasts than other regularization types.
Deep neural networks (DNNs) have become increasingly important due to their excellent empirical performance on a wide range of problems. However, regularization is generally achieved by indirect means, largely due to the complex set of functions defined by a network and the difficulty in measuring function complexity. …
We establish an optimal regularity result for parametrized two-dimensional stationary varifolds. Namely, we show that the parametrization map is a smooth minimal branched immersion and that the multiplicity function is constant. We provide some applications of this regularity result, especially in the calculus of varia…
Evolutionary methods improve neural network loss functions, reducing overfitting.
problem Improving neural network performance and preventing overfitting.
method Evolutionary computation to optimize loss functions, balancing error pull and overfitting push.
result Evolved loss functions effectively reduce overfitting, leading to better performance and robustness.
Introduces top-k regularization for better feature selection in machine learning.
problem Limited ability of existing feature selection methods to reconcile feature representativeness and inter-correlations.
method Top-k regularization, which induces a sub-architecture on the model's architecture to select informative features and model complex relationships. result Uniform approximation error bound for top-k regularization approximating high-dimensional sparse functions. Dealing with high variance is a significant challenge in model-free reinforcement learning (RL). Existing methods are unreliable, exhibiting high variance in performance from run to run using different initializations/seeds. Focusing on problems arising in continuous control, we propose a functional regularization appr…