Transformers struggle to approximate smooth functions, relying on piecewise constant approximations.
problem Understanding the expressivity of Transformers for function approximation.
method Theoretical analysis and experimental validation of Transformer's ability to approximate smooth functions.
result Transformers cannot reliably approximate smooth functions, relying on piecewise constant approximations.
New method certifies images against transformations like rotations and translations.
problem Certifying robustness of images against transformations like rotations and translations.
method Randomized smoothing with three different kinds of defenses.
result Individual certificates can be obtained via statistical error bounds or efficient online inverse computation.
Vision transformers benefit from non-smooth components in adaptation.
problem Understanding the role of non-smoothness in vision transformer adaptation.
method Theoretical analysis and extensive experiments on large-scale vision transformers.
result High plasticity of attention modules and feedforward layers leads to better finetuning performance.
Permutability of surface transforms yields discrete analogs.
problem Discretization of smooth surfaces with specific properties.
method Permutability of transforms of smooth surfaces.
result Discrete surfaces with discrete analogs of original properties.
Unified proof of smooth fibration theorems for collapsed manifolds.
problem Smooth fibration theorems for collapsed manifolds with Ricci curvature bounded below.
method Generalized Reifenberg condition and transformation technique for almost splitting maps.
result Unified proof of smooth fibration theorems in many previous works.
Transformers handle infinite dimensional inputs effectively by feature extraction and dynamic feature selection.
problem Understanding the approximation and estimation ability of Transformers with infinite dimensional inputs.
method Anisotropic smoothness analysis and feature extraction properties of Transformers.
result Transformers avoid the curse of dimensionality and dynamically select important features.
Injectivity of X-ray transform proven for non-smooth metrics.
problem Injectivity of X-ray transform on non-smooth metrics.
method Microlocal analysis of the normal operator, establishing ellipticity and smoothing properties.
result Injectivity of X-ray transform on L2 for metrics with finitely differentiable tensor. Regularity results for geodesic X-ray transform on nonsmooth manifolds
problem Geodesic X-ray transform on nonsmooth simple manifolds
method Symbol smoothing arguments and pseudodifferential operators with low regularity symbols
result Improved injectivity results for Lp functions New actions found on exotic spheres using group theory.
problem Understanding smooth transformations on exotic spheres.
method Recent progress in stable homotopy groups of spheres and group theory.
result Smooth circle and cyclic group actions on exotic spheres produced.
Transforms solutions of Davey-Stewartson II equation geometrically.
problem Solving the Davey-Stewartson II equation.
method Moutard transform and spinor representation of surfaces.
result Constructs examples of solutions with smooth initial data losing regularity.
Geodesic X-ray transform proves injective for smooth one-forms on gas giant manifolds.
problem Injectivity of geodesic X-ray transform for one-forms on specific manifolds.
method Pestov identity and asymptotic analysis of short geodesics.
result Geodesic X-ray transform is solenoidally injective for smooth one-forms on gas giant manifolds.
New STKR estimators use unlabeled data for smoother function learning.
problem Leveraging unlabeled data for smoother function learning.
method Spectrally transformed kernel regression (STKR) with scalable implementations.
result STKR can learn any sufficiently smooth function.
We discuss natural transformations in the context of Lie groupoids, and their infinitesimal counterpart. Our main result is an integration procedure that provides smooth natural transformations between Lie groupoid morphisms.
Smooth manifold structure on Möbius transformations of quaternionic ball identified.
problem Identifying the manifold structure of Möbius transformations of quaternionic unit ball.
method Realizing M(B) as a quotient of Sp(1,1) and using Lie group properties. result The manifold M(B) is diffeomorphic to R4imesS3. New findings suggest Barron space doesn't defy curse of dimensionality for certain types of smoothness.
problem Understanding the curse of dimensionality in neural networks with different smoothness notions.
method Defined ADZ spaces via Mellin transform to encapsulate nonclassical smoothness, compared to classical smoothness.
result Evidence provided that Barron space doesn't defy curse of dimensionality for certain smoothness types.
We discuss results for the Ribaucour transformation of curves or of higher dimensional smooth and discrete submanifolds. In particular, a result for the reduction of the ambient dimension of a submanifold is proved and the notion of Ribaucour coordinates is derived using a Bianchi permutability result. Further, we disc…
We develop some theory of double fibration transforms where the cycle space is a smooth manifold and apply it to complex projective space.
FineMorphs models smooth transformations for multivariate regression.
problem Efficiently modeling complex transformations for multivariate regression.
method Optimal control of affine and diffeomorphic transformations using smooth vector fields.
result FineMorphs can reduce dimensionality and adapt to large datasets.
Unified framework TSS certifies robustness against semantic transformations.
problem Certifying robustness of ML models against semantic transformations.
method Unified framework TSS categorizes transformations into resolvable and differentially resolvable, proposing randomized smoothing and stratified sampling strategies.
result Significantly outperforms state of the art on over ten types of semantic transformations.
We define parametrized cobordism categories and study their formal properties as bivariant theories. Bivariant transformations to a strongly excisive bivariant theory give rise to characteristic classes of smooth bundles with strong additivity properties. In the case of cobordisms between manifolds with boundary, we pr…
New geometric transformations link discrete and continuous curve motions.
problem Establishing a connection between discrete and continuous curve motions.
method Infinitesimal Darboux transformations of smooth curves.
result Alternate geometric interpretation for semi-discrete mKdV equation.
New algebraic-geometry method for Ribaucour transformations.
problem Classical differential geometry problems.
method Algebraic-geometry approach to constructing orthogonal nets.
result Obtains smooth orthogonal nets as Ribaucour transformations.
ABHT boosts regression by filtering regions with different smoothness.
problem Improving regression performance through local adaptivity.
method Gradient boosting with adaptive histogram transform.
result ABHT converges faster than PEHT in Hölder continuous spaces.
Researchers relax the CVF's smoothness requirement to create more flexible flow models.
problem Challenges in constructing flexible density models due to the CVF's smoothness requirement.
method Introduce L-diffeomorphisms as generalized transformations that may violate smoothness on zero Lebesgue-measure sets. result The relaxation allows for the use of non-smooth activation functions like ReLU in residual flows.
Multisections generalize trisections for 4-manifolds, allowing complex operations and explicit diagrams.
problem Decomposing arbitrary smooth 4-manifolds into simpler pieces.
method Introducing multisections as a sequence of cut systems on a surface, enabling smooth operations and explicit diagrams.
result Elliptic fibrations admit genus 3 multisections, with explicit diagrams.
This paper analyzes convergence of large-scale Transformers with weight decay.
problem Understanding optimization guarantees in large-scale Transformer training.
method Construct mean-field limit, show gradient flow convergence to PDE, demonstrate global minimum consistency.
result Gradient flow reaches global minimum in large-scale Transformers with small weight decay.
The p-Laplacian Transformer improves transformer models by assigning higher attention weights to tokens in close proximity.
problem The self-attention mechanism in transformers does not effectively distinguish attention weights between tokens in close and non-close proximity.
method Proposes a novel class of transformers, p-Laplacian Transformers, that use p-Laplacian regularization to assign higher attention weights to tokens in close proximity. result Empirically demonstrates that p-Laplacian Transformers outperform baseline transformers on various benchmark datasets.
We propose a construction which transforms a self-similar zipper in Rn to a self-affine zipper Rn+1 whose attractor is a smooth curve.
Weingarten transformations which, by definition, preserve the asymptotic lines on smooth surfaces have been studied extensively in classical differential geometry and also play an important role in connection with the modern geometric theory of integrable systems. Their natural discrete analogues have been investigated…
CutMix training technique improves spatial locality in Vision Transformers.
problem Improving spatial locality in Vision Transformers trained from scratch.
method Comparison of Baseline and Modern training protocols on CIFAR-10, CIFAR-100, and Tiny-ImageNet.
result CutMix training component significantly reduces Mean Attention Distance (MAD) in early layers of Vision Transformers.
Paper proves Fourier transform for valuations, simplifying previous work.
problem Existence of isomorphism for translation-invariant smooth valuations.
method Directly describes Alesker's isomorphism in terms of Fourier transform on functions.
result Simple proofs of Alesker's Fourier transform properties, including a previously conjectured result.
In this paper we consider the geodesic X-ray transform with attenuation coefficient as a combination of smooth complex function and 1-form. We show that attenuated X-ray transform applied to the pair of tensors is injective modulo the natural obstruction.
Using symplectic topology and the Radon transform, we prove that smooth 4-dimensional projective planes are diffeomorphic to CP2. We define the notion of a plane curve in a smooth projective plane, show that plane curves in high dimensional regular planes are lines, prove that homeomorphisms preserving plan…
Algorithm learns polynomial transformations of Gaussian distributions.
problem Learning high-dimensional polynomial transformations of Gaussian distributions.
method Polynomial-time algorithms for smoothed settings, tensor ring decomposition.
result First end-to-end guarantees for learning pushforwards under neural networks.
We study the weighted integral transform on a compact manifold with boundary over a smooth family of curves Γ. We prove generic injectivity and a stability estimate under the condition that the conormal bundle of Γ covers T∗M.
Unique Poincaré type cscK metric with singularity at smooth divisor is unique up to holomorphic transformations.
problem Proving uniqueness of Poincaré type cscK metric with singularity at a smooth divisor.
method Holomorphic transformations, asymptotic behavior analysis, fixed point problem.
result Unique Poincaré type cscK metric with singularity at a smooth divisor is unique up to holomorphic transformations.
We show that the group of isometries (i.e., distance-preserving homeomorphisms) of an equiregular subRiemannian manifold is a finite-dimensional Lie group of smooth transformations. The proof is based on a new PDE argument, in the spirit of harmonic coordinates, establishing that in an arbitrary subRiemannian manifold …
This paper analyzes kNN convergence over feature transformations.
problem The curse of dimensionality affects kNN performance in transformed feature spaces.
method Developed a novel analysis on kNN convergence rates over transformed features, linking properties of the transformed space to raw feature space.
result Theoretical analysis explains why some feature transformations are better for kNN.
In this note we study logarithmic transformations in the sense of differential topology on two fibers of the Hopf surface. It is known that such transformations are susceptible to yield exotic smooth structures on four-manifolds. We will show here that this is not the case for the Hopf surface, all integer homology Hop…
Improved signal classification using multiple wavelets and their smooth coefficients.
problem Signal classification accuracy declines with reduced attributes.
method Transform data with multiple wavelets, combine outputs, apply ensemble classifiers.
result Proposed technique outperforms raw data and single wavelet approaches.
We construct explicit solutions to the discrete motion of discrete plane curves that has been introduced by one of the authors recently. Explicit formulas in terms the τ function are presented. Transformation theory of the motions of both smooth and discrete curves is developed simultaneously.
Transformer models can approximate smooth functions with prompts, enhancing LLMs' dynamic capabilities.
problem Lack of theoretical framework for prompt engineering in transformer models.
method Formal framework demonstrating transformer models can approximate β-times differentiable functions with prompts. result Transformer models can approximate β-times differentiable functions with arbitrary precision using appropriately structured prompts. Some methods based on simple regularizing geometric element transformations have heuristically been shown to give runtime efficient and quality effective smoothing algorithms for meshes. We describe the mathematical framework and a systematic approach to global optimization-based versions of such methods for mixed volu…
Smooth flows for physical systems with smooth energies and forces.
problem Smooth energies for physical simulations and force computation.
method Smooth mixture transformations on compact intervals and hypertori, using root-finding and the inverse function theorem.
result Smooth flows allow training by force matching and use as molecular dynamics potentials.
We introduce and study a new Radon-like transform that averages projected differential p-forms in R^n over affine (n-k)-planes. We then prove an explicit inversion formula for our transform on the space of rapidly-decaying smooth p-forms. Our transform differs from the one in Gelfand-Graev-Shapiro. Moreover, if it can …
We show that the Radon transform related to closed geodesics is injective on a Lie group if and only if the connected components are not homeomorphic to S1 nor to S3. This is true for both smooth functions and distributions. The key ingredients of the proof are finding totally geodesic tori and realizing the Rado…
Anti-self-dual (ASD) connections for a compact smooth four manifold arise as critical values for the Yang-Mills action functional. Nahm transform is a nice correspondence between a vector bundle with ASD connections and a vector bundle with ASD connections over Picard torus associated to X. In this talk we propose a no…
Let (M,g) be an analytic, compact, Riemannian manifold with boundary, of dimension n >= 2. We study a class of generalized Radon transforms, integrating over a family of hypersurfaces embedded in M, satisfying the Bolker condition [23]. Using analytic microlocal analysis, we prove a microlocal regularity theorem for ge…