Transformers struggle to approximate smooth functions, relying on piecewise constant approximations.
problem Understanding the expressivity of Transformers for function approximation.
method Theoretical analysis and experimental validation of Transformer's ability to approximate smooth functions.
result Transformers cannot reliably approximate smooth functions, relying on piecewise constant approximations.
Novel power transform unifies various mathematical functions.
problem Normalizing and standardizing datasets.
method Presented a novel power transform.
result Unified various mathematical functions.
Develops a framework for consistent loss functions with variable transformations.
problem Lack of theoretical understanding of variable transformations in consistent loss functions.
method Formal characterizations of consistency for transformed loss functions in two cases: realization and prediction variables.
result Establishes new identifiable and elicitable functionals for complex predictive tasks.
Paper proposes a new e-exponentiated transformation to make convex loss functions more robust to outliers.
problem Making convex loss functions robust to outliers in the presence of label noise.
method Introduces a novel e-exponentiated transformation for loss functions and proves its effectiveness through theoretical and empirical analysis. result The transformed loss function achieves tighter generalization error bounds and higher accuracy in noisy datasets.
DGPFM uses deep Gaussian processes to map functions accurately and quantify uncertainty.
problem Learning mappings between functional spaces, especially when data are noisy, sparse, or irregularly sampled.
method Constructs a sequence of GP-based linear and nonlinear transformations directly in function space, leveraging kernel integral transforms, GP conditional means, and nonlinear activations sampled from Gaussian processes.
result Empirical results show DGPFM outperforms existing methods in predictive accuracy and uncertainty calibration.
Functions with constant geodesic X-ray transform are restricted to manifolds with specific geometrical properties.
problem Existence of functions with constant geodesic X-ray transform on manifolds.
method Analyzing the geometrical properties of manifolds based on the existence of such functions.
result Functions with constant geodesic X-ray transform impose specific geometrical restrictions on the manifold.
Injectivity of geodesic ray transform for piecewise constants on compact manifolds.
problem Injectivity of geodesic ray transform for piecewise constant functions.
method Injectivity of geodesic ray transform on piecewise constant functions weighted by a continuous matrix weight.
result Injectivity of the geodesic X-ray transform on piecewise constant functions.
New Lehmer Transform for analyzing non-stationary signals.
problem Analyzing non-stationary signals like brain waves.
method Proposes a new Lehmer Transform to decompose statistical moments.
result Theoretical properties of the Lehmer Transform are presented.
Transformers learn low-dimensional target functions efficiently in-context.
problem Efficiently learning nonlinear target functions in-context using transformers.
method Nonlinear MLP layer in transformers optimized by gradient descent, focusing on single-index target functions.
result Transformers can learn target functions with low-dimensional structures efficiently in-context.
A new method for discrete data normalizing flows using latent transformations.
problem Challenges in parameterizing bijective transformations for discrete data.
method Predict a distribution over latent transformations to make the marginal likelihood differentiable.
result Discrete-data normalizing flows can be trained using gradient-based learning with unbiased score function estimation.
New analysis shows RPE-based Transformers can't approximate all functions.
problem Understanding the limitations of RPE-based Transformers in approximating continuous functions.
method Mathematical analysis and development of a novel attention module (URPE) to overcome limitations.
result RPE-based Transformers can't approximate all continuous sequence-to-sequence functions, even with depth and width.
Support theorems for light ray transforms on analytic manifolds.
problem Understanding functions on Lorentzian manifolds via lightlike geodesics.
method Analyticity of manifold and weight, support theorems for ray transforms.
result Support theorems for integrating functions on analytic Lorentzian manifolds.
Study characterizes geodesic ray transform on surfaces, isolating and separating sub-ranges.
problem Characterizing the range of the attenuated geodesic ray transform on surfaces.
method Isolating and separating sub-ranges of sums of functions and one-forms, deriving new inversion formulas.
result Range characterizations and new inversion formulas for geodesic ray transform.
Sharp mapping properties and regularization for X-ray transform on disks of constant curvature.
problem Sharp mapping properties and regularization of X-ray transform.
method Derive functional relations and mapping properties using elliptic differential operators.
result Theoretical possibility of regularized inversions for X-ray transform.
We solve for functions from their truncated Hilbert transforms using Chebyshev series.
problem Finding functions from their truncated Hilbert transforms.
method Express functions in Chebyshev series and numerically estimate coefficients.
result Numerical methods work well for extrapolating functions from truncated Hilbert transforms.
Standard Transformers approximate Hölder functions and achieve optimal nonparametric regression rate.
problem Approximating Hölder functions and achieving optimal nonparametric regression rate with Transformers.
method Using the size tuple and dimension vector metrics, the paper characterizes Transformer structures and derives upper bounds for their Lipschitz constant and memorization capacity.
result Standard Transformers achieve the minimax optimal rate in nonparametric regression for Hölder target functions.
The complex wave representation (CWR) converts unsigned 2D distance transforms into their corresponding wave functions. Here, the distance transform S(X) appears as the phase of the wave function φ(X)---specifically, φ(X)=exp(iS(X)/τwhere τis a free parameter. In this work, we prove a novel result using the higher-orde…
Transformer models outperform LSTM in financial forecasting with MADL loss.
problem Optimizing loss functions for Transformer models in financial forecasting.
method Empirical experiments with MADL loss function on equity and cryptocurrency assets.
result Transformer models significantly outperform LSTM models in financial forecasting.
The Funk-Minkowski transform and spherical convolution reconstruct functions and vector fields on the sphere.
problem Reconstructing functions and vector fields on the sphere using Funk-Minkowski transform and Hilbert type spherical convolution.
method Inversion formula for Funk-Minkowski transform and Helmholtz-Hodge decomposition solution using spherical convolution.
result Complete reconstruction of functions and vector fields on the sphere.
The paper develops methods to generate invariant quantities in Metric-Affine Geometry.
problem Developing methods to generate invariant quantities in Metric-Affine Geometry.
method The paper introduces a theorem to generate invariant quantities under transformations of the affine connection, proving invariance conditions.
result Theorem establishing conditions for invariance of functionals under transformations of the affine connection.
Transformers can approximate any sequence-to-sequence function, surprising given their complexity.
problem Understanding the expressive power of Transformer models for sequence-to-sequence functions.
method Established that Transformers are universal approximators of continuous permutation equivariant sequence-to-sequence functions with compact support, and extended this to arbitrary functions using positional encodings.
result Transformers are universal approximators of arbitrary continuous sequence-to-sequence functions on a compact domain.
New Hermite approximations accelerate convergence with adaptive coordinate transformations.
problem Accelerating convergence of spectral approximations for Hermite expansions.
method Using normalizing flows for adaptive coordinate transformations and deriving error estimates.
result Error estimates for Hermite expansions under adaptive coordinate transformations.
New bounds for classifier robustness to adversarial attacks.
problem Characterizing robustness of classifiers to adversarial perturbations.
method Introducing function transformations to relate adversarial risk to standard learning-theoretic risk bounds.
result Error rates on the same order as generalization error of original function classes.
Researchers calculate the Laplace transform of a geometric Brownian motion integral.
problem Calculating the Laplace transform of a specific integral functional of geometric Brownian motion.
method Analytical calculation of the Laplace transform of the cumulative distribution and probability density functions.
result The Laplace transform of the integral functional of geometric Brownian motion is derived.
Unified framework for learning function representations using INRs and Transformers.
problem Scalability and efficiency limitations in existing generative models.
method Integrates INRs and Transformer-based hypernetworks into latent variable models.
result Improved scalability, expressiveness, and generalization over existing models.
Study geodesic X-ray transform on curved spaces, proving injectivity for decaying functions.
problem Injectivity of geodesic X-ray transform on curved spaces.
method Proving injectivity for decaying functions and tensor fields of any order.
result Injectivity of the geodesic X-ray transform for functions and tensor fields of any order on Cartan-Hadamard manifolds.
Legendre transform connects Upsilon function of L-space knots to large surgeries d-invariants.
problem Understanding the Upsilon function of L-space knots and its relation to knot concordance.
method Using Legendre transform to relate Upsilon function to d-invariants of surgeries.
result The Upsilon function's unknotting obstruction is contained in d-invariants of surgeries.
Transformers handle infinite dimensional inputs effectively by feature extraction and dynamic feature selection.
problem Understanding the approximation and estimation ability of Transformers with infinite dimensional inputs.
method Anisotropic smoothness analysis and feature extraction properties of Transformers.
result Transformers avoid the curse of dimensionality and dynamically select important features.
Study light ray transform on Lorentzian manifolds without conjugate points.
problem Recovering spacelike singularities from weighted light ray transforms.
method Fourier Integral Operator analysis and filtered back-projection.
result Recovery of spacelike singularities from weighted light ray transforms without conjugate points.
Lower bounds set for infinite-precision transformers.
problem Understanding limitations of infinite-precision transformers.
method Used VC dimension technique to prove lower bounds.
result First lower bounds for two tasks: function composition and SUM2. The paper explores the injectivity of the light ray transform on Lorentzian manifolds.
problem Injectivity of the light ray transform on functions and tensors.
method Analyzes injectivity conditions for scalar and tensor fields on stationary and static Lorentzian manifolds.
result Injectivity of the light ray transform on functions and tensors is proven under specific conditions.
We consider the generalized Segal-Bargmann transform, defined in terms of the heat operator, for a noncompact symmetric space of the complex type. For radial functions, we show that the Segal-Bargmann transform is a unitary map onto a certain L^2 space of meromorphic functions. For general functions, we give an inversi…
Study spherical Fourier transform on hypergeometric type harmonic manifolds.
problem Spherical Fourier transform on harmonic Hadamard manifolds.
method Representation of spherical functions by Gauss hypergeometric functions.
result Inversion formula, convolution rule, and Plancherel theorem are derived.
The paper analyzes transformation models in high-dimensional settings.
problem Analyzing transformation models in high-dimensional data.
method Proposed an estimator for transformation parameter and showed asymptotic normality.
result The proposed estimator works well in small samples and tests the log-wage transformation.
Gaussian processes with linear constraints ensure function adherence.
problem Ensuring function adherence to known constraints in Gaussian processes.
method Modification of covariance function and explicit incorporation of linear constraints through a transformation operator.
result Guaranteed fulfillment of constraints in predictions and samples.
Study X-ray transform on manifolds, desingularize, and improve mapping properties.
problem Characterize mapping properties of X-ray transform and its adjoint on manifolds with strictly convex boundary.
method Use b-fibrations, desingularize, and apply Melrose's Pushforward Theorem to analyze polyhomogeneous functions.
result Improved mapping properties of X-ray transform and its adjoint, recovering sharp results.
Paper classifies minimal graph transformations into new families of surfaces.
problem Classifying minimal graph transformations into new families of surfaces.
method Formulated and solved a coupled system of partial differential equations, reduced to solving an ordinary differential equation.
result Established rigorous equivalence to a modified problem for a harmonic function, yielding new families of minimal surfaces.
Spectral Convolution Networks speed up computation by applying convolution and activation in the frequency domain.
problem Performance increase in convolution networks comes with repeated transform computations.
method Implement convolution and activation in the frequency domain using Fourier or Laplace transformations.
result Reduced number of transforms and overall complexity by computing both convolution and activation in the frequency domain.
Enhances Transformers for better risk assessment in finance.
problem Transformer models lack sensitivity to extreme financial losses.
method Integrates Loss-at-Risk function with Value at Risk (VaR) and Conditional Value at Risk (CVaR).
result Improves risk prediction and management in financial datasets.
Transformers struggle to learn Markovian dynamics, showing NP-hard optimization challenges.
problem Understanding transformers' limitations in learning Markovian dynamical functions.
method Investigated through a structured ICL setup, analyzing loss landscapes and parameter optimization.
result Recovering optimal transformer parameters for Markovian functions is NP-hard.
Transformer architecture struggles with complex tasks due to limitations in function composition.
problem Transformer architecture's limitations in handling complex tasks.
method Used Communication Complexity to prove limitations in composing functions.
result Transformer layer is incapable of handling large domain functions, even when domains are small.
Kernel estimator optimally recovers function from noisy exponential Radon transform.
problem Inverting noisy exponential Radon transform of a function.
method Proposed a kernel estimator to estimate the true function.
result The estimator converges to the true function at minimax optimal rate.
Efficient algorithms find optimal monotone transforms for calibration under strictly convex losses.
problem Calibrating estimations to improve performance with monotone transforms.
method Proposed linear-time and space algorithm for finding optimal monotone transforms for specific loss functions. Also proposed an anytime algorithm with linear space and pseudo-linearithmic time complexity.
result Optimal monotone transforms are unique and can be found efficiently for various strictly convex loss functions.
Defines Reidemeister transformations for potential function and solution.
problem Analyzing changes in complex volume formula under Reidemeister moves.
method Defines Reidemeister transformations for potential function and solution, showing explicit formulas.
result Explicit formulas for Reidemeister transformations enable changes in complex volume formula.
New non-commutative geometric Nahm transform for ASD connections.
problem Generalizing Nahm transform to non-commutative geometry.
method Formulated using Dixmier trace, generalizes Connes-Yang-Mills action.
result Proposes a new non-commutative geometric Nahm transform.
The visibility transformation embeds data position into signature features for efficient pattern recognition.
problem Embedding absolute position into signature features for efficient pattern recognition.
method The visibility transformation is put on a theoretical footing and used to embed absolute position into signature features efficiently.
result The generated feature set simplifies pattern recognition by accommodating nonlinear functions of absolute and relative values.
Transformers tend to learn more symmetric functions in sequence data.
problem Understanding inductive bias in Transformers with infinitely over-parameterized models.
method Analyzing Transformers in the Gaussian process limit, using representation theory of the symmetric group.
result Transformers are biased towards more permutation symmetric functions, and this can be quantitatively predicted.
Transformers learn sparse Boolean functions through RL and SFT, revealing distinct learning behaviors.
problem Learning sparse Boolean functions with Transformers.
method Reinforcement Learning (RL) with process rewards and Supervised Fine-Tuning (SFT).
result RL learns the whole CoT chain simultaneously, while SFT learns step by step.