Transformers struggle to approximate smooth functions, relying on piecewise constant approximations.
problem Understanding the expressivity of Transformers for function approximation.
method Theoretical analysis and experimental validation of Transformer's ability to approximate smooth functions.
result Transformers cannot reliably approximate smooth functions, relying on piecewise constant approximations.
Novel power transform unifies various mathematical functions.
problem Normalizing and standardizing datasets.
method Presented a novel power transform.
result Unified various mathematical functions.
Develops a framework for consistent loss functions with variable transformations.
problem Lack of theoretical understanding of variable transformations in consistent loss functions.
method Formal characterizations of consistency for transformed loss functions in two cases: realization and prediction variables.
result Establishes new identifiable and elicitable functionals for complex predictive tasks.
Paper proposes a new e-exponentiated transformation to make convex loss functions more robust to outliers.
problem Making convex loss functions robust to outliers in the presence of label noise.
method Introduces a novel e-exponentiated transformation for loss functions and proves its effectiveness through theoretical and empirical analysis. result The transformed loss function achieves tighter generalization error bounds and higher accuracy in noisy datasets.
DGPFM uses deep Gaussian processes to map functions accurately and quantify uncertainty.
problem Learning mappings between functional spaces, especially when data are noisy, sparse, or irregularly sampled.
method Constructs a sequence of GP-based linear and nonlinear transformations directly in function space, leveraging kernel integral transforms, GP conditional means, and nonlinear activations sampled from Gaussian processes.
result Empirical results show DGPFM outperforms existing methods in predictive accuracy and uncertainty calibration.
Functions with constant geodesic X-ray transform are restricted to manifolds with specific geometrical properties.
problem Existence of functions with constant geodesic X-ray transform on manifolds.
method Analyzing the geometrical properties of manifolds based on the existence of such functions.
result Functions with constant geodesic X-ray transform impose specific geometrical restrictions on the manifold.
Injectivity of geodesic ray transform for piecewise constants on compact manifolds.
problem Injectivity of geodesic ray transform for piecewise constant functions.
method Injectivity of geodesic ray transform on piecewise constant functions weighted by a continuous matrix weight.
result Injectivity of the geodesic X-ray transform on piecewise constant functions.
We propose a new class of transforms that we call {\it Lehmer Transform} which is motivated by the {\it Lehmer mean function}. The proposed {\it Lehmer transform} decomposes a function of a sample into their constituting statistical moments. Theoretical properties of the proposed transform are presented. This transform…
Transformers learn low-dimensional target functions efficiently in-context.
problem Efficiently learning nonlinear target functions in-context using transformers.
method Nonlinear MLP layer in transformers optimized by gradient descent, focusing on single-index target functions.
result Transformers can learn target functions with low-dimensional structures efficiently in-context.
A new method for discrete data normalizing flows using latent transformations.
problem Challenges in parameterizing bijective transformations for discrete data.
method Predict a distribution over latent transformations to make the marginal likelihood differentiable.
result Discrete-data normalizing flows can be trained using gradient-based learning with unbiased score function estimation.
New analysis shows RPE-based Transformers can't approximate all functions.
problem Understanding the limitations of RPE-based Transformers in approximating continuous functions.
method Mathematical analysis and development of a novel attention module (URPE) to overcome limitations.
result RPE-based Transformers can't approximate all continuous sequence-to-sequence functions, even with depth and width.
Sharp mapping properties and regularization for X-ray transform on disks of constant curvature.
problem Sharp mapping properties and regularization of X-ray transform.
method Derive functional relations and mapping properties using elliptic differential operators.
result Theoretical possibility of regularized inversions for X-ray transform.
We solve for functions from their truncated Hilbert transforms using Chebyshev series.
problem Finding functions from their truncated Hilbert transforms.
method Express functions in Chebyshev series and numerically estimate coefficients.
result Numerical methods work well for extrapolating functions from truncated Hilbert transforms.
Standard Transformers approximate Hölder functions and achieve optimal nonparametric regression rate.
problem Approximating Hölder functions and achieving optimal nonparametric regression rate with Transformers.
method Using the size tuple and dimension vector metrics, the paper characterizes Transformer structures and derives upper bounds for their Lipschitz constant and memorization capacity.
result Standard Transformers achieve the minimax optimal rate in nonparametric regression for Hölder target functions.
The complex wave representation (CWR) converts unsigned 2D distance transforms into their corresponding wave functions. Here, the distance transform S(X) appears as the phase of the wave function φ(X)---specifically, φ(X)=exp(iS(X)/τwhere τis a free parameter. In this work, we prove a novel result using the higher-orde…
Transformer models outperform LSTM in financial forecasting with MADL loss.
problem Optimizing loss functions for Transformer models in financial forecasting.
method Empirical experiments with MADL loss function on equity and cryptocurrency assets.
result Transformer models significantly outperform LSTM models in financial forecasting.
The paper develops methods to generate invariant quantities in Metric-Affine Geometry.
problem Developing methods to generate invariant quantities in Metric-Affine Geometry.
method The paper introduces a theorem to generate invariant quantities under transformations of the affine connection, proving invariance conditions.
result Theorem establishing conditions for invariance of functionals under transformations of the affine connection.
Transformers can approximate any sequence-to-sequence function, surprising given their complexity.
problem Understanding the expressive power of Transformer models for sequence-to-sequence functions.
method Established that Transformers are universal approximators of continuous permutation equivariant sequence-to-sequence functions with compact support, and extended this to arbitrary functions using positional encodings.
result Transformers are universal approximators of arbitrary continuous sequence-to-sequence functions on a compact domain.
New Hermite approximations accelerate convergence with adaptive coordinate transformations.
problem Accelerating convergence of spectral approximations for Hermite expansions.
method Using normalizing flows for adaptive coordinate transformations and deriving error estimates.
result Error estimates for Hermite expansions under adaptive coordinate transformations.
New bounds for classifier robustness to adversarial attacks.
problem Characterizing robustness of classifiers to adversarial perturbations.
method Introducing function transformations to relate adversarial risk to standard learning-theoretic risk bounds.
result Error rates on the same order as generalization error of original function classes.
Researchers calculate the Laplace transform of a geometric Brownian motion integral.
problem Calculating the Laplace transform of a specific integral functional of geometric Brownian motion.
method Analytical calculation of the Laplace transform of the cumulative distribution and probability density functions.
result The Laplace transform of the integral functional of geometric Brownian motion is derived.
Unified framework for learning function representations using INRs and Transformers.
problem Scalability and efficiency limitations in existing generative models.
method Integrates INRs and Transformer-based hypernetworks into latent variable models.
result Improved scalability, expressiveness, and generalization over existing models.
Transformers handle infinite dimensional inputs effectively by feature extraction and dynamic feature selection.
problem Understanding the approximation and estimation ability of Transformers with infinite dimensional inputs.
method Anisotropic smoothness analysis and feature extraction properties of Transformers.
result Transformers avoid the curse of dimensionality and dynamically select important features.
Lower bounds set for infinite-precision transformers.
problem Understanding limitations of infinite-precision transformers.
method Used VC dimension technique to prove lower bounds.
result First lower bounds for two tasks: function composition and SUM2. Study light ray transform on Lorentzian manifolds without conjugate points.
problem Recovering spacelike singularities from weighted light ray transforms.
method Fourier Integral Operator analysis and filtered back-projection.
result Recovery of spacelike singularities from weighted light ray transforms without conjugate points.
We consider the generalized Segal-Bargmann transform, defined in terms of the heat operator, for a noncompact symmetric space of the complex type. For radial functions, we show that the Segal-Bargmann transform is a unitary map onto a certain L^2 space of meromorphic functions. For general functions, we give an inversi…
The paper explores the injectivity of the light ray transform on Lorentzian manifolds.
problem Injectivity of the light ray transform on functions and tensors.
method Analyzes injectivity conditions for scalar and tensor fields on stationary and static Lorentzian manifolds.
result Injectivity of the light ray transform on functions and tensors is proven under specific conditions.
Study spherical Fourier transform on hypergeometric type harmonic manifolds.
problem Spherical Fourier transform on harmonic Hadamard manifolds.
method Representation of spherical functions by Gauss hypergeometric functions.
result Inversion formula, convolution rule, and Plancherel theorem are derived.
Study X-ray transform on manifolds, desingularize, and improve mapping properties.
problem Characterize mapping properties of X-ray transform and its adjoint on manifolds with strictly convex boundary.
method Use b-fibrations, desingularize, and apply Melrose's Pushforward Theorem to analyze polyhomogeneous functions.
result Improved mapping properties of X-ray transform and its adjoint, recovering sharp results.
Paper classifies minimal graph transformations into new families of surfaces.
problem Classifying minimal graph transformations into new families of surfaces.
method Formulated and solved a coupled system of partial differential equations, reduced to solving an ordinary differential equation.
result Established rigorous equivalence to a modified problem for a harmonic function, yielding new families of minimal surfaces.
Enhances Transformers for better risk assessment in finance.
problem Transformer models lack sensitivity to extreme financial losses.
method Integrates Loss-at-Risk function with Value at Risk (VaR) and Conditional Value at Risk (CVaR).
result Improves risk prediction and management in financial datasets.
Transformers struggle to learn Markovian dynamics, showing NP-hard optimization challenges.
problem Understanding transformers' limitations in learning Markovian dynamical functions.
method Investigated through a structured ICL setup, analyzing loss landscapes and parameter optimization.
result Recovering optimal transformer parameters for Markovian functions is NP-hard.
Transformer architecture struggles with complex tasks due to limitations in function composition.
problem Transformer architecture's limitations in handling complex tasks.
method Used Communication Complexity to prove limitations in composing functions.
result Transformer layer is incapable of handling large domain functions, even when domains are small.
Kernel estimator optimally recovers function from noisy exponential Radon transform.
problem Inverting noisy exponential Radon transform of a function.
method Proposed a kernel estimator to estimate the true function.
result The estimator converges to the true function at minimax optimal rate.
Efficient algorithms find optimal monotone transforms for calibration under strictly convex losses.
problem Calibrating estimations to improve performance with monotone transforms.
method Proposed linear-time and space algorithm for finding optimal monotone transforms for specific loss functions. Also proposed an anytime algorithm with linear space and pseudo-linearithmic time complexity.
result Optimal monotone transforms are unique and can be found efficiently for various strictly convex loss functions.
The visibility transformation embeds data position into signature features for efficient pattern recognition.
problem Embedding absolute position into signature features for efficient pattern recognition.
method The visibility transformation is put on a theoretical footing and used to embed absolute position into signature features efficiently.
result The generated feature set simplifies pattern recognition by accommodating nonlinear functions of absolute and relative values.
Transformers tend to learn more symmetric functions in sequence data.
problem Understanding inductive bias in Transformers with infinitely over-parameterized models.
method Analyzing Transformers in the Gaussian process limit, using representation theory of the symmetric group.
result Transformers are biased towards more permutation symmetric functions, and this can be quantitatively predicted.
Transformation models are a very important tool for applied statisticians and econometricians. In many applications, the dependent variable is transformed so that homogeneity or normal distribution of the error holds. In this paper, we analyze transformation models in a high-dimensional setting, where the set of potent…
Holonomy-preserving transformations help recover Alexander polynomials from graph zeta functions.
problem Recovering Alexander polynomials from graph zeta functions.
method Introducing holonomy to preserve zeta functions of matrix-weighted graphs and extending to group elements and quandles.
result Holonomy-preserving transformations correspond to transformations of group presentations and preserve the twisted Alexander polynomial.
Transformers learn sparse Boolean functions through RL and SFT, revealing distinct learning behaviors.
problem Learning sparse Boolean functions with Transformers.
method Reinforcement Learning (RL) with process rewards and Supervised Fine-Tuning (SFT).
result RL learns the whole CoT chain simultaneously, while SFT learns step by step.
Transformer networks approximate Hölder and Sobolev functions with fixed-depth networks.
problem Nonparametric regression with dependent observations.
method Established novel upper bounds for Transformer networks approximating Hölder and Sobolev functions under various β-mixing data assumptions. result Explicit convergence rates for nonparametric regression problems under β-mixing data assumptions. Bayesian model for discrete data with conditional transformations.
problem Handling discrete ordinal and count data with excess zeros.
method Bayesian framework with conditional transformation functions and modular MCMC algorithm.
result Flexible modeling of linear and nonlinear covariate effects for ordinal and count data.
We reduce boundary determination of an unknown function and its normal derivatives from the (possibly weighted and attenuated) broken ray data to the injectivity of certain geodesic ray transforms on the boundary. For determination of the values of the function itself we obtain the usual geodesic ray transform, but for…
Paper shows invertibility of tensor X-ray transform on certain manifolds.
problem Invertibility of tensor X-ray transform on asymptotically conic manifolds.
method Used 1-cusp pseudodifferential operator algebra and modified solenoidal gauge condition.
result Invertibility of tensor X-ray transform up to natural obstruction.
Study of line congruences for Appell's rank-4 hypergeometric functions.
problem Understanding line congruences for Appell's rank-4 hypergeometric functions.
method Derived original formulae for Laplace transform of rank-4 system, applied to geometry of surfaces defined by these functions.
result Natural line congruences for Laplace transforms of Appell's rank-4 functions form a W-congruence.
Paper computes link determinants using Fourier-Hadamard transforms.
problem Computing determinants of complex link structures.
method Fourier-Hadamard transforms of Boolean functions.
result Determinant of centrally symmetric links with even components equals zero.
We interpret the setting for a Radon transform as a submanifold of the space of generalized functions, and compute its extrinsic curvature: it is the Hessian composed with the Radon transform.
The Funk--Minkowski transform F associates a function f on the sphere S2 with its mean values (integrals) along all great circles of the sphere. Thepresented analytical inversion formula reconstruct the unknown function f completely if two Funk--Minkowski transforms, Ff and ${…