arXiv research

We establish continuous maximal regularity results for parabolic differential operators acting on sections of tensor bundles on Riemannian manifolds. As an application, we show that solutions to the Yamabe flow instantaneously regularize and become real analytic in space and time. The regularity result is obtained by i…

A regular F-manifold is an F-manifold (with Euler field) (M, \circ, e, E), such that the endomorphism {\mathcal U}(X) := E \circ X of TM is regular at any p\in M. We prove that the germ ((M,p), \circ, e, E) is uniquely determined (up to isomorphism) by the conjugacy class of {\mathcal U}_{p} : T_{p}M \rightarrow T_{p}M…

Manifold regularization, such as laplacian regularized least squares (LapRLS) and laplacian support vector machine (LapSVM), has been widely used in semi-supervised learning, and its performance greatly depends on the choice of some hyper-parameters. Cross-validation (CV) is the most popular approach for selecting the …

We consider the problem of constructing a reduced-rank regression model whose coefficient parameter is represented as a singular value decomposition with sparse singular vectors. The traditional estimation procedure for the coefficient parameter often fails when the true rank of the parameter is high. To overcome this …

Manifold regularization is a commonly used technique in semi-supervised learning. It enforces the classification rule to be smooth with respect to the data-manifold. Here, we derive sample complexity bounds based on pseudo-dimension for models that add a convex data dependent regularization term to a supervised learnin…

Develops a new asymptotic efficiency theory for non-Euclidean parameter spaces.

Study shows consistency of shallow GCNNs on sampled point clouds under manifold assumption.

Due to the growing ubiquity of unlabeled data, learning with unlabeled data is attracting increasing attention in machine learning. In this paper, we propose a novel semi-supervised kernel learning method which can seamlessly combine manifold structure of unlabeled data and Regularized Least-Squares (RLS) to learn a ne…

The paper tackles safe reinforcement learning with convex regularization.

The paper studies how regularization parameters affect sparsity in deep neural networks.

Paper uses 'manifold attack' to improve model accuracy and robustness.

Study introduces a new method for multiple parameter regularization in polynomial functional regression.

We establish a version of the complex Frobenius theorem in the context of a complex subbundle S of the complexified tangent bundle of a manifold, having minimal regularity. If the subbundle S defines the structure of a Levi-flat CR-manifold, it suffices that S be Lipschitz for our results to apply. A principal tool in …

The study finds generic regularity of minimal hypersurfaces in Riemannian manifolds.

This research smooths out fluid equations to avoid sudden shocks.

We prove that every projective special Kähler manifold with \emph{regular boundary behaviour} is complete and defines a family of complete quaternionic Kähler manifolds depending on a parameter $c\ge 0$. We also show that, irrespective of its boundary behaviour, every complete projective special Kähler manifold with \e…

Let M be a six dimensional manifold, endowed with a cohomogeneity one action of G= SU_2 x SU_2, and M_reg its subset of regular points. We show that M_reg admits a smooth, 2-parameter family of G-invariant, non-isometric strict nearly Kaehler structures and that a 1-parameter subfamily of such structures smoothly exten…

New estimators outperform maximum likelihood without hyper-parameter estimation.

New method encodes function preferences into neural nets for better generalization.

Proposes a new method for selecting regularization parameters in sparse precision matrix estimation.

This paper studies least-square regression penalized with partly smooth convex regularizers. This class of functions is very large and versatile allowing to promote solutions conforming to some notion of low-complexity. Indeed, they force solutions of variational problems to belong to a low-dimensional manifold (the so…

Regularization methods, specifically those which directly alter weights like $L_1$ and $L_2$, are an integral part of many learning algorithms. Both the regularizers mentioned above are formulated by assuming certain priors in the parameter space and these assumptions, in some cases, induce sparsity in the parameter sp…

Deep neural networks (DNNs) have been successfully applied to a wide variety of acoustic modeling tasks in recent years. These include the applications of DNNs either in a discriminative feature extraction or in a hybrid acoustic modeling scenario. Despite the rapid progress in this area, a number of challenges remain …

Careful tuning of a regularization parameter is indispensable in many machine learning tasks because it has a significant impact on generalization performances. Nevertheless, current practice of regularization parameter tuning is more of an art than a science, e.g., it is hard to tell how many grid-points would be need…

New insights into Deep Autoencoders for better data approximation and generalization.

Gradient descent on MMD GAN parameter space converges globally to target distribution.

We provide adaptive inference methods, based on $\ell_1$ regularization, for regular (semi-parametric) and non-regular (nonparametric) linear functionals of the conditional expectation function. Examples of regular functionals include average treatment effects, policy effects, and derivatives. Examples of non-regular f…

Few-shot learning algorithms aim to learn model parameters capable of adapting to unseen classes with the help of only a few labeled examples. A recent regularization technique - Manifold Mixup focuses on learning a general-purpose representation, robust to small changes in the data distribution. Since the goal of few-…

In sparse regression modeling via regularization such as the lasso, it is important to select appropriate values of tuning parameters including regularization parameters. The choice of tuning parameters can be viewed as a model selection and evaluation problem. Mallows' $C_p$ type criteria may be used as a tuning param…

Large scale, streaming datasets are ubiquitous in modern machine learning. Streaming algorithms must be scalable, amenable to incremental training and robust to the presence of non-stationarity. In this work consider the problem of learning $\ell_1$ regularized linear models in the context of streaming data. In particu…

Proposes R2LDA for improved LDA classifier performance.

Many applied settings in empirical economics involve simultaneous estimation of a large number of parameters. In particular, applied economists are often interested in estimating the effects of many-valued treatments (like teacher effects or location effects), treatment effects for many groups, and prediction models wi…

This study proposes sparse estimation methods for the generalized linear models, which run one of least angle regression (LARS) and least absolute shrinkage and selection operator (LASSO) in the tangent space of the manifold of the statistical model. This study approximates the statistical model and subsequently uses e…

This paper presents a bias-variance tradeoff of graph Laplacian regularizer, which is widely used in graph signal processing and semi-supervised learning tasks. The scaling law of the optimal regularization parameter is specified in terms of the spectral graph properties and a novel signal-to-noise ratio parameter, whi…

Method estimates parameters of complex nonlinear systems.

Regularization is a popular technique in machine learning for model estimation and avoiding overfitting. Prior studies have found that modern ordered regularization can be more effective in handling highly correlated, high-dimensional data than traditional regularization. The reason stems from the fact that the ordered…

Piecewise constant denoising can be solved either by deterministic optimization approaches, based on the Potts model, or by stochastic Bayesian procedures. The former lead to low computational time but require the selection of a regularization parameter, whose value significantly impacts the achieved solution, and whos…

It is the purpose of this article to establish a technical tool to study regularity of solutions to parabolic equations on manifolds. As applications of this technique, we prove that solutions to the Ricci-DeTurck flow, the surface diffusion flow and the mean curvature flow enjoy joint analyticity in time and space, an…

This paper identifies a problem with the usual procedure for L2-regularization parameter estimation in a domain adaptation setting. In such a setting, there are differences between the distributions generating the training data (source domain) and the test data (target domain). The usual cross-validation procedure requ…

NDM incorporates geometric structure into neural networks for better optimization and interpretability.

We consider adaptive system identification problems with convex constraints and propose a family of regularized Least-Mean-Square (LMS) algorithms. We show that with a properly selected regularization parameter the regularized LMS provably dominates its conventional counterpart in terms of mean square deviations. We es…

A new method for learning function parameters in operators using data-adaptive RKHS.

Optimal ridge regularization computed iteratively from generative parameters.

New bounds explain neural network generalization by considering local Lipschitz properties.

Parameter pruning is a promising approach for CNN compression and acceleration by eliminating redundant model parameters with tolerable performance degrade. Despite its effectiveness, existing regularization-based parameter pruning methods usually drive weights towards zero with large and constant regularization factor…

Regularization can improve both privacy and performance in machine learning models.

We introduce and study new spectral invariant of two elliptic partial differential operators of Laplace and Dirac type on compact smooth manifolds without boundary that depends on both the eigenvalues and the eigensections of the operators, which is a equal to the regularized number of created particles from the vacuum…

Cross-regularization adapts model complexity during training.