arXiv research

We consider a class of operator-induced norms, acting as finite-dimensional surrogates to the L2 norm, and study their approximation properties over Hilbert subspaces of L2 . The class includes, as a special case, the usual empirical norm encountered, for example, in the context of nonparametric regression in reproduci…

Improved stochastic Halpern iteration for fixed-point approximation in normed spaces.

Study approximates operator learning for PDEs using Fourier multipliers.

Extends importance sampling to nonlinear models using adjoint operators.

Study efficient neural operator learning using variation spaces.

SAM improves generalization by operating near the edge of stability.

The OSCAR (octagonal selection and clustering algorithm for regression) regularizer consists of a L_1 norm plus a pair-wise L_inf norm (responsible for its grouping behavior) and was proposed to encourage group sparsity in scenarios where the groups are a priori unknown. The OSCAR regularizer has a non-trivial proximit…

The paper studies the metric and algebraic structures on section rings of projective manifolds.

This research develops approximation theory for OOMs of infinite-dimensional processes.

Improved singular value approximation for convolutional layers.

This work provides closed-form solutions and minimum achievable errors for a large class of low-rank approximation problems in Hilbert spaces. The proposed theorem generalizes to the case of bounded linear operators the previous results obtained in the finite dimensional case for the Frobenius norm. The theorem provide…

New method improves stability of soft FQI for offline RL.

New method stabilizes FQE by reweighting Bellman targets.

Researchers develop neural networks for approximating functions in Banach spaces.

Researchers approximate conditional expectation operators using kernel methods.

Study variance-reduced method for estimating fixed points in Banach spaces.

Recovering a large matrix from limited measurements is a challenging task arising in many real applications, such as image inpainting, compressive sensing and medical imaging, and this kind of problems are mostly formulated as low-rank matrix approximation problems. Due to the rank operator being non-convex and discont…

Unified analysis of stochastic iterative algorithms using Lyapunov functions.

Study convergence and approximations of entropic regularized Wasserstein distances for Gaussian and RKHS measures.

This article concerns upper bounds for $L^\infty$-norms of random approximate eigenfunctions of the Laplace operator on a compact aperiodic Riemannian manifold $(M,g).$ We study $f_λ$ chosen uniformly at random from the space of $L^2$-normalized linear combinations of Laplace eigenfunctions with eigenvalues in the inte…

Online learning of linear operators between infinite-dimensional spaces is possible but with limitations.

In reinforcement learning, temporal difference (TD) is the most direct algorithm to learn the value function of a policy. For large or infinite state spaces, exact representations of the value function are usually not available, and it must be approximated by a function in some parametric family. However, with \emph{no…

Unique inhomogeneous ruled hypersurface found in complex hyperbolic space.

We show how the discovery of robust scalable numerical solvers for arbitrary bounded linear operators can be automated as a Game Theory problem by reformulating the process of computing with partial information and limited resources as that of playing underlying hierarchies of adversarial information games. When the so…

Deep networks with path norm regularization can approximate analytic functions.

Motivated by the study of $Q$-learning algorithms in reinforcement learning, we study a class of stochastic approximation procedures based on operators that satisfy monotonicity and quasi-contractivity conditions with respect to an underlying cone. We prove a general sandwich relation on the iterate error at each time,…

The Laplace-Beltrami operator in the curved Möbius strip is investigated in the limit when the width of the strip tends to zero. By establishing a norm-resolvent convergence, it is shown that spectral properties of the operator are approximated well by an unconventional flat model whose spectrum can be computed explici…

We establish a theoretical link between adversarial training and operator norm regularization for deep neural networks. Specifically, we prove that $\ell_p$-norm constrained projected gradient ascent based adversarial training with an $\ell_q$-norm loss on the logits of clean and perturbed inputs is equivalent to data-…

Kernel methods are powerful learning methodologies that allow to perform non-linear data analysis. Despite their popularity, they suffer from poor scalability in big data scenarios. Various approximation methods, including random feature approximation, have been proposed to alleviate the problem. However, the statistic…

Paper shows deep neural networks can approximate Korobov functions nearly optimally.

Applying standard techniques from Toeplitz operator theory, we analyze the asymptotics of the Hilbert-Smith norms of the TQFT operators coming from isotopy classes of one dimensional oriented submanifolds on a closed oriented surface. We thereby obtain a Toeplitz operator interpretation and generalization of the asympt…

Optimal scaling found to depend on operator norm across large models and datasets.

Improved spectral convergence bounds for diffusion maps on tori.

Kernel methods represent one of the most powerful tools in machine learning to tackle problems expressed in terms of function values and derivatives due to their capability to represent and model complex relations. While these methods show good versatility, they are computationally intensive and have poor scalability t…

A new distance metric compares probability distributions using kernel covariance operators.

The use of certain critical-exponent Sobolev norms is an important feature of methods employed by Taubes to solve the anti-self-dual and similar non-linear elliptic partial differential equations. Indeed, the estimates one can obtain using these critical-exponent norms appear to be the best possible when one needs to b…

Low-rank modeling has a lot of important applications in machine learning, computer vision and social network analysis. While the matrix rank is often approximated by the convex nuclear norm, the use of nonconvex low-rank regularizers has demonstrated better recovery performance. However, the resultant optimization pro…

Study bounds Rademacher complexity of Fourier neural operators.

Low-rank modeling has many important applications in computer vision and machine learning. While the matrix rank is often approximated by the convex nuclear norm, the use of nonconvex low-rank regularizers has demonstrated better empirical performance. However, the resulting optimization problem is much more challengin…

Sparse coding consists in representing signals as sparse linear combinations of atoms selected from a dictionary. We consider an extension of this framework where the atoms are further assumed to be embedded in a tree. This is achieved using a recently introduced tree-structured sparse regularization norm, which has pr…

Low-rank matrix is desired in many machine learning and computer vision problems. Most of the recent studies use the nuclear norm as a convex surrogate of the rank operator. However, all singular values are simply added together by the nuclear norm, and thus the rank may not be well approximated in practical problems. …

New pivoting strategy improves trace norm contraction in low-rank approximation.

Proximal operators are of particular interest in optimization problems dealing with non-smooth objectives because in many practical cases they lead to optimization algorithms whose updates can be computed in closed form or very efficiently. A well-known example is the proximal operator of the vector $\ell_1$ norm, whic…

We model how Lipschitz continuity changes during neural network training.

The paper bounds neural networks' approximation error and applies it to regression and GANs.

Any Sasakian structure can be closely mimicked by embeddings into weighted spheres.

Study of unitary and groupoid orbits of normal operators, focusing on manifold structures and spectral conditions.

A new method for learning function parameters in operators using data-adaptive RKHS.