Stochastic Gradient Descent improved for various Hilbert scales and misspecified models.
problem Understanding and optimizing SGD in Hilbert scales for machine learning.
method Extending SGD analysis to Hilbert scales, including Sobolev and Diffusion spaces, and showing the effects of smoothness and preconditioning.
result Violation of smoothness assumption affects learning rate; preconditioning in Hilbert scales reduces the number of iterations for misspecified models.
The paper analyzes Tikhonov regularization in Hilbert scales for statistical inverse problems.
problem Statistical inverse problems in Hilbert scales with general noise.
method Tikhonov regularization scheme with conditional stability estimates and high probability error bounds.
result Explicit rates of convergence for oversmoothing and regular cases over defined regularity classes.
Study optimizes KSD estimation from samples, revealing Hilbert-Schmidt vs trace scales.
problem Optimizing estimation of Kernel Stein Discrepancy from samples.
method Identifying and comparing minimax scales for U-statistic and V-statistic.
result Hilbert-Schmidt norm of Stein covariance operator gives optimal scale.
New learning rates derived for Tikhonov-regularized problems without kernel assumptions.
problem Learning rates for Tikhonov-regularized learning problems.
method Minimax adaptive rates derived using Fourier isocapacitary condition and interpolation theory.
result Derivation of minimax adaptive rates without requiring kernel assumptions.
We study the linear ill-posed inverse problem with noisy data in the statistical learning setting. Approximate reconstructions from random noisy data are sought with general regularization schemes in Hilbert scale. We discuss the rates of convergence for the regularized solution under the prior assumptions and a certai…
The paper identifies a 'small' set of functions containing Gaussian process samples.
problem Identifying a small set of functions containing Gaussian process samples.
method Using scaled RKHSs and Karhunen-Loève theorem, the paper defines the sample support set.
result The sample support set consists of functions with bounded squared basis coefficients.
Model shows feature learning can improve neural scaling laws for hard tasks.
problem Understanding and improving neural network scaling laws for various task difficulties.
method Developed a solvable model of neural scaling laws, identified three scaling regimes, and demonstrated feature learning's impact on scaling exponents.
result Feature learning can improve scaling with training time and compute for hard tasks, nearly doubling the exponent.
Introduces Floer functions and Floerfolds for intrinsic properties.
problem Complex transformation of Hessian under chart transition.
method Introduces Floer functions and Floerfolds to address intrinsic properties.
result Floer functions and Floerfolds provide intrinsic conditions for Hessian.
Researchers develop Orlov-Schulman symmetries for self-dual conformal structures.
problem Developing symmetries for self-dual conformal structures.
method Explicit proof of compatibility with Lax-Sato flows, dressing scheme based on Riemann-Hilbert problem.
result Construction and proof of compatibility of Orlov-Schulman symmetries.
Efficient RL in large POMDPs with latent determinism and embeddings.
problem Efficient reinforcement learning in large-scale POMDPs with latent states and observations.
method Conditional Hilbert space embeddings, linear optimal Q-function, deterministic latent transitions, gap assumption. result Computationally and statistically efficient algorithm for exact optimal policy.
Kernel interpolation is inconsistent for norms with smoothness above a constant.
problem Inconsistency of kernel interpolation in reproducing kernel Hilbert spaces.
method Lower bounds for generalization error in Sobolev norms.
result Kernel interpolation is always inconsistent for norms with smoothness above a constant.
In this paper we derive a scaling limit for an infinite dimensional limit order book model driven by Hawkes random measures. The dynamics of the incoming order flow is allowed to depend on the current market price as well as on a volume indicator. With our choice of scaling the dynamics converges to a coupled SDE-ODE s…
MOCK learns complex systems from trajectories efficiently.
problem Learning nonparametric differential equations from high-dimensional data.
method MOCK uses multivariate occupation kernel functions to learn vector fields linearly.
result MOCK outperforms other methods on various datasets.
Sparse model for noisy datasets using hierarchical regularization.
problem Learning from large noisy datasets with sparse representations.
method Hierarchical learning strategy with projection-based penalty operators.
result Efficient sparse model reconstruction and generalizability on real datasets.
This paper improves Koopman operator approximations by pruning subspaces in RKHS.
problem Improving predictive accuracy of Koopman operator approximations.
method Computes principal angles and vectors in RKHS to prune subspaces.
result Validated approach enhances Koopman operator approximations for large datasets.
Poor approximators found in neural networks and random feature models.
problem Understanding why certain neural networks and models perform poorly in approximating functions.
method Established a scale separation of Kolmogorov width type and applied it to neural networks and random feature models.
result Reproducing kernel Hilbert spaces and two-layer neural networks are poor L2-approximators for certain functions. In supervised learning using kernel methods, we often encounter a large-scale finite-sum minimization over a reproducing kernel Hilbert space (RKHS). Large-scale finite-sum problems can be solved using efficient variants of Newton method, where the Hessian is approximated via sub-samples of data. In RKHS, however, the …
Extends metrics for SPD matrices to infinite dimensions.
problem Lack of generalized forms for Riemannian metrics.
method Unitized Hilbert-Schmidt operators and extended Mahalanobis norm.
result Improved performance in high-dimensional comparisons.
Kernel-based methods improve policy evaluation in MRP models.
problem Estimating value functions in infinite-horizon discounted MRP models.
method Kernel-based temporal difference methods using reproducing kernel Hilbert spaces.
result Optimal error bounds derived for the kernel-based LSTD estimate.
In the paper "Direct Images, Fields of Hilbert Spaces, and Geometric Quantization", Lempert and Szőke proved that any flat analytic Hilbert field will induce a hermitian Hilbert bundle and gave an example of a flat Hilbert field that does not induce any Hilbert bundle. In this paper, we will provide an example of an an…
New method speeds up HSIC for multiple variables.
problem Quadratic computational complexity of HSIC for multiple variables.
method Nyström approximation to HSIC for M≥2. result Consistent Nyström HSIC estimator for M≥2. Study on kernel methods in large-scale machine learning problems.
problem Large-scale machine learning with many interacting variables.
method Mean field limit analysis of kernels and their Hilbert spaces.
result Mean field convergence of empirical and infinite-sample solutions.
We show that a space with a finite asymptotic dimension is embeddable in a non-positively curved manifold. Then we prove that if a uniformly contractible manifold X is uniformly embeddable in Rn or non-positively curved n-dimensional simply connected manifold then X×Rn is integrally hyperspherical. If a un…
New algorithm uses GNNs to optimize rewards in graph-structured data.
problem Optimizing rewards in molecule design with graph-structured data.
method Embedding permutation invariance into GNNs and using GNTK for regret bounds.
result First GNN confidence bound and phased-elimination algorithm with sublinear regret.
We prove an abstract criterion stating resolvent convergence in the case of operators acting in different Hilbert spaces. This result is then applied to the case of Laplacians on a family $X_\eps$ of branched quantum waveguides. Combining it with an exterior complex scaling we show, in particular, that the resonances o…
The Hilbert map's image is discussed, showing when it's surjective.
problem Understanding when the Hilbert map is surjective.
method Analyzing the Hilbert map's properties to determine surjectivity.
result Necessary and sufficient conditions for the Hilbert map to be surjective.
New Hilbert bundles with ends defined from indexed bases.
problem Defining new structures in Hilbert bundles.
method Indexed bases and unitary operators of finite propagation.
result Characteristic classes of Hilbert bundles with ends.
Study examines Hilbert area of inscribed polygons in projective geometry.
problem Understanding Hilbert area of inscribed polygons in projective geometry.
method Examined correspondence between Fock-Goncharov and Cartesian coordinates, analyzed degeneration and Hilbert area of inscribed quadrilaterals, developed microlocal condition.
result Sequence of strictly convex domains with bounded Hilbert area and divergent Goldman parameters.
Study finds Hilbert square of real surfaces can be maximal even when the surface has disconnected real locus.
problem Exploring conditions for maximality of Hilbert square of real surfaces.
method Analyzing Hilbert square of maximal real surfaces and examining specific examples.
result Hilbert square can be maximal even for surfaces with disconnected real locus.
Proves Thom's conjecture for parabolic flows on Hilbert spaces.
problem Gradient flows on infinite-dimensional spaces and geometric flows with symmetry.
method Analytic functions, Hilbert spaces, Yang-Mills Flow, Ricci flow, critical points, Lojasiewicz inequality.
result Gradient conjecture holds for parabolic flows on Hilbert spaces, including flows with gauge symmetry.
Theory developed for Hilbert geometry over valued fields, linking real and non-Archimedean geometries.
problem Understanding Hilbert geometry over general valued fields and their limits.
method Developed a theory of Hilbert geometry over general ordered valued fields, proving ultralimit results.
result Ultralimit of rescaled real Hilbert geometries is isometric to a non-Archimedean Hilbert metric space.
Paper introduces robust distribution regression using kernel methods.
problem Distribution regression from probability measures to real-valued responses.
method Introduces a robust loss function lσ and a windowing function V for two-stage sampling problems. result Shows improved learning rates and robustness with the robust distribution regression (RDR) scheme.
Representations of probability measures in reproducing kernel Hilbert spaces provide a flexible framework for fully nonparametric hypothesis tests of independence, which can capture any type of departure from independence, including nonlinear associations and multivariate interactions. However, these approaches come wi…
We introduce an efficient algorithmic framework for model selection in online learning, also known as parameter-free online learning. Departing from previous work, which has focused on highly structured function classes such as nested balls in Hilbert space, we propose a generic meta-algorithm framework that achieves o…
Kernel dependence measures yield accurate estimates of nonlinear relations between random variables, and they are also endorsed with solid theoretical properties and convergence rates. Besides, the empirical estimates are easy to compute in closed form just involving linear algebra operations. However, they are hampere…
Paper studies a robust online learning algorithm for regression.
problem Develops a robust online learning algorithm for regression problems.
method Introduces an online learning algorithm with a robust loss function over RKHS.
result The algorithm achieves optimal convergence rates in mean square and RKHS.
We implement an all-optical setup demonstrating kernel-based quantum machine learning for two-dimensional classification problems. In this hybrid approach, kernel evaluations are outsourced to projective measurements on suitably designed quantum states encoding the training data, while the model training is processed o…
A commuting n-tuple (T1,…,Tn) of bounded linear operators on a Hilbert space $\clh$ associate a Hilbert module H over C[z1,…,zn] in the following sense: \[\mathbb{C}[z_1, \ldots, z_n] \times \mathcal{H} \rightarrow \mathcal{H}, \quad \quad (p, h) \mapsto p(T_1, \ldots, T_n)h…
Improves kernel ridge regression by optimizing scale and feature parameters.
problem Kernel ridge regression with fixed kernel.
method Introduces a matrix parameter U to optimize scale and feature parameters.
result Solves a nonlinear variational problem to optimize U.
The paper tackles data-driven optimal control of unknown nonlinear systems using RKHS.
problem Unknown nonlinear dynamics and stage cost functions.
method Embed state densities into RKHS, learn Markov operators, solve Hamilton-Jacobi-Bellman recursions.
result Solves a wide range of nonlinear control problems, including depth regulation.
Develop a variational framework for statistical inference on cyclic interactions.
problem Estimating and comparing large-scale recurrent organization in directed interactions.
method Represent directed interactions as edge flows on a simplicial complex and evolve under an energy-minimizing dynamical system.
result Separate transient interaction components from persistent harmonic flows, yielding a low-dimensional cycle space.
Formulates Hilbert reciprocity law on 3-manifolds.
problem Developing arithmetic topology on 3-manifolds.
method Formulated an analogue of Hilbert reciprocity law using intersection forms and Kummer extensions.
result Cyclic covers of links are analogues of Kummer extensions.
New GP-based method improves uncertainty quantification for causal functions.
problem Challenges in quantifying uncertainty for causal effects, especially for entire functions.
method GP-based approach using inner-product of observational functions in RKHS, with tractable posterior moments and calibration.
result Improves uncertainty quantification while maintaining causal effect estimation performance.
We give first examples of finitely generated groups having an intermediate, with values in (0,1), Hilbert space compression (which is a numerical parameter measuring the distortion required to embed a metric space into Hilbert space). These groups include certain diagram groups. In particular, we show that the Hilbert …
Motivated by a question of Hirzebruch on the possible topological types of cusp cross-sections of Hilbert modular varieties, we give a necessary and sufficient condition for a manifold M to be diffeomorphic to a cusp cross-section of a Hilbert modular variety. Specialized to Hilbert modular surfaces, this proves that e…
Proves existence and uniqueness of solutions for a nonlinear equation on Hilbert manifold.
problem Proving existence and uniqueness of solutions for a nonlinear equation on Hilbert manifold.
method Analyzes the equation on Hilbert manifold, proving existence and uniqueness of solutions.
result Demonstrates that solutions are in the Hilbert manifold and are gradient flows.
Generatability in metric spaces studied with novel novelty parameters.
problem Understanding generatability in metric spaces with asymmetric novelty parameters.
method Introducing (ε,ε′)-closure dimension to characterize uniform and non-uniform generatability. result Generatability is stable across novelty scales in doubling spaces but can be highly scale-sensitive in general metric spaces.
Let {T1,…,Tn} be a set of n commuting bounded linear operators on a Hilbert space H. Then the n-tuple (T1,…,Tn) turns H into a module over C[z1,…,zn] in the following sense: \[\mathbb{C}[z_1, \ldots, z_n] \times \mathcal{H} \raro \clh, \quad \quad …