Multivariate splines linked to infinitely-wide neural networks with improved numerical performance.
problem Understanding the relationship between multivariate splines and neural networks.
method Showed multivariate splines can be represented as random features in infinitely-wide neural networks with a homogeneous activation function.
result The function space of multivariate splines is a Sobolev space on a Euclidean ball with explicit norm bounds on derivatives.
Sparse random features improve accuracy in data-scarce settings.
problem Limited accuracy of random feature methods in data-scarce applications.
method Sparse random feature expansion using compressive sensing.
result Improved generalization bounds for sparse random features.
TaylorPODA uses Taylor expansions to improve feature attributions for opaque models.
problem Lack of systematic framework for quantifying feature contributions in opaque models.
method Taylor expansion framework with postulates (precision, federation, zero-discrepancy, adaptation).
result TaylorPODA achieves competitive results and provides principled explanations.
Paper develops a new kernel expansion method using entropic optimal features for sparse and efficient kernel approximation.
problem Efficient kernel approximation with reduced computational cost and feature dissimilarity.
method Develops a novel optimal design maximizing entropy among kernel features, resulting in a sparse kernel expansion.
result Achieves optimal statistical accuracy with only $O(N^{ rac{1}{4}})$ features, significantly reducing time and space costs.
Ensembles dynamic models using random feature approximations.
problem Online scalable Bayesian learning with dynamic models and ensembling.
method Random feature approximations and dynamic models using random walks.
result Better performance with alternative basis expansions like Hilbert space Gaussian processes.
New method improves nonlinear filtering accuracy with reduced computation.
problem Complex nonlinear filtering with small system noise.
method Asymptotic expansion with ordinary differential equations and Edgeworth-type correction.
result Significantly lower computational cost with improved accuracy.
HARFE approximates sparse additive functions using random features and ridge regression.
problem Approximating high-dimensional sparse additive functions.
method Hard-ridge random feature expansion with sparse ridge regression and hard-thresholding pursuit.
result HARFE method converges with a given error bound and achieves lower error than other algorithms.
FFCP improves FCP's speed without sacrificing accuracy.
problem Inefficient feature transformation in FCP.
method Introduces FFCP using Taylor expansion for faster computation.
result FFCP achieves a 50x speedup with comparable accuracy.
Zero-shot learning (ZSL) aims at recognizing unseen classes with knowledge transferred from seen classes. This is typically achieved by exploiting a semantic feature space (FS) shared by both seen and unseen classes, i.e., attributes or word vectors, as the bridge. However, due to the mutually disjoint of training (see…
SLEIPNIR improves Gaussian process regression with derivatives, scaling up efficiently and accurately.
problem Scaling Gaussian process regression with derivatives for large datasets.
method Quadrature Fourier features for feature expansion, proving error bounds.
result Deterministic, non-asymptotic, exponentially fast decaying error bounds for approximated kernel and posterior.
We derive a higher-order expansion for rough volatility models.
problem Characterizing and estimating rough volatility models.
method Higher-order asymptotic expansion of characteristic functions.
result Distinct roles of rough and jump dynamics in volatility.
This work presents a new classifier that is specifically designed to be fully interpretable. This technique determines the probability of a class outcome, based directly on probability assignments measured from the training data. The accuracy of the predicted probability can be improved by measuring more probability es…
The composition of multiple Gaussian Processes as a Deep Gaussian Process (DGP) enables a deep probabilistic nonparametric approach to flexibly tackle complex machine learning problems with sound quantification of uncertainty. Existing inference approaches for DGP models have limited scalability and are notoriously cum…
A new Lagrangian formulation of the Raychaudhuri equation in non-Riemannian geometry.
problem Formulating the Raychaudhuri equation in non-Riemannian geometries.
method Established a formal connection between the expansion scalar and the cross-sectional volume of the congruence. Derived a Lagrangian and Hamiltonian formulation.
result The expansion scalar equals the fractional rate of change of volume, weighted by a scalar factor.
In this work, we propose a new policy iteration algorithm for pricing Bermudan options when the payoff process cannot be written as a function of a lifted Markov process. Our approach is based on a modification of the well-known Longstaff Schwartz algorithm, in which we basically replace the standard least square regre…
ESE-FN improves elderly activity recognition accuracy.
problem Recognizing individual actions and human-object interactions in elderly activities.
method Exploits multi-modal features from RGB videos and skeleton sequences using ESE attentions and a new Multi-modal Loss.
result ESE-FN achieves best accuracy on ETRI-Activity3D dataset.
Develops a new feature theory for robust machine learning.
problem Creating robust machine learning features from training data.
method Stochastic tensor space feature theory with Karhunen-Loeve expansion and hierarchical subspaces.
result Dramatic increases in accuracy for predicting Alzheimer's disease stages.
We study finite-dimensional integrals in a way that elucidates the mathematical meaning behind the formal manipulations of path integrals occurring in quantum field theory. This involves a proper understanding of how Wick's theorem allows one to evaluate integrals perturbatively, i.e., as a series expansion in a formal…
We propose a novel framework for combining datasets via alignment of their intrinsic geometry. This alignment can be used to fuse data originating from disparate modalities, or to correct batch effects while preserving intrinsic data structure. Importantly, we do not assume any pointwise correspondence between datasets…
Enhances random forest performance with exogenous randomness.
problem Improving random forest performance through exogenous randomness.
method Developed non-asymptotic MSE expansions for individual trees and forests, identified two types of randomness, and conducted simulations.
result Exogenous randomness, particularly feature subsampling, reduces both bias and variance of random forests.
Study on future stability of FLRW spacetime solutions with decelerated expansion.
problem Stability of solutions to Einstein equations coupled with a nonlinear scalar field.
method Decomposition of metric and scalar field perturbations into spatial averages and oscillatory remainders.
result Future-stability of FLRW spacetime solutions for 1/3<p<1. Matching pursuit (MP) methods are a promising class of feature construction algorithms for value function approximation. Yet existing MP methods require creating a pool of potential features, mandating expert knowledge or enumeration of a large feature pool, both of which hinder scalability. This paper introduces batch…
We consider a class of assets whose risk-neutral pricing dynamics are described by an exponential Lévy-type process subject to default. The class of processes we consider features locally-dependent drift, diffusion and default-intensity as well as a locally-dependent Lévy measure. Using techniques from regular perturba…
We show that kernel-based quadrature rules for computing integrals can be seen as a special case of random feature expansions for positive definite kernels, for a particular decomposition that always exists for such kernels. We provide a theoretical analysis of the number of required samples for a given approximation e…
Expands sparse disparity cues from LiDAR to improve stereo matching performance.
problem Improving stereo estimation performance with limited dense data.
method Proposes a sparsity expansion technique to enhance local features from sparse disparity cues.
result Significantly boosts stereo algorithms with sparse cues, outperforming previous methods.
New method uses sparse random features for crashworthiness analysis.
problem Efficient surrogate modelling for uncertainty quantification.
method Sparse Random Features combined with self-supervised dimensionality reduction.
result Superiority over state-of-the-art techniques in crashworthiness analysis.
General Relativity in 4 dimensions can be equivalently described as a dynamical theory of SO(3)-connections rather than metrics. We introduce the notion of asymptotically hyperbolic connections, and work out an analog of the Fefferman-Graham expansion in the language of connections. As in the metric setup, one can solv…
The short-time heat kernel expansion of elliptic operators provides a link between local and global features of classical geometries. For many geometric structures related to (non-)involutive distributions, the natural differential operators tend to be Rockland, hence hypoelliptic. In this paper we establish a universa…
Study quantifies how LLMs capture higher-order statistical structure using cumulant expansion.
problem Understanding how LLMs internalize statistical structure during next-token prediction.
method Cumulant-expansion framework treating softmax entropy as perturbation around center distribution.
result Cumulants reveal distinct signatures for mathematical vs. general text prompts, quantifying feature-learning dynamics.
Hyperfitting improves LLM generation quality by enhancing diversity, contrary to simple temperature scaling.
problem Improving open-ended generation quality of LLMs with minimal fine-tuning effort.
method Demonstrates that hyperfitting, a phenomenon where LLMs are fine-tuned to near-zero training loss, enhances generation quality and mitigates repetition.
result Hyperfitting is distinct from temperature scaling and involves a dynamic, context-dependent rank reordering mechanism in the final transformer block.
RE enhances DL by learning model behavior, enabling iterative self-improvement.
problem Static data representations limit DL's potential for evolving models.
method RE uses multiple mappings of data through identical deep architectures, analyzing internal representations and performance signals.
result Models can gain insight from predecessors, leading to iterative self-improvement.
Basis adaptation in Homogeneous Chaos spaces rely on a suitable rotation of the underlying Gaussian germ. Several rotations have been proposed in the literature resulting in adaptations with different convergence properties. In this paper we present a new adaptation mechanism that builds on compressive sensing algorith…
We introduce a statistical physics inspired supervised machine learning algorithm for classification and regression problems. The method is based on the invariances or stability of predicted results when known data is represented as expansions in terms of various stochastic functions. The algorithm predicts the classif…
New spatiotemporal Besov process improves CT image reconstruction and other inverse problems.
problem Handling abrupt changes and sharp contrasts in spatiotemporal data.
method Generalized Besov process (STBP) with Q-exponential process for temporal correlation.
result STBP outperforms traditional methods in dynamic reconstruction and inverse problems.
Rescaling expansiveness proven for k*-expansive vector fields.
problem Proving rescaling expansiveness for k*-expansive vector fields.
method Introducing and exploring singular-expansive flows.
result Rescaling expansiveness established for k*-expansive vector fields.
Can we effectively learn a nonlinear representation in time comparable to linear learning? We describe a new algorithm that explicitly and adaptively expands higher-order interaction features over base linear representations. The algorithm is designed for extreme computational efficiency, and an extensive experimental …
New method uses deep neural networks to interpolate spatiotemporal data.
problem Scalable interpolation of spatiotemporal data from growing earth observation systems.
method Bayesian deep learning with random feature expansions.
result Competitive or superior results compared to existing methods.
A new method integrates Fourier basis expansion and mapping for improved time series forecasting.
problem Inconsistent starting cycles and series length issues in Fourier-based methods.
method Fourier Basis Mapping (FBM) method that integrates time-frequency features through Fourier basis expansion and mapping.
result FBM addresses inconsistencies and preserves temporal characteristics, achieving SOTA performance.
The implied volatility skew has received relatively little attention in the literature on short-term asymptotics for financial models with jumps, despite its importance in model selection and calibration. We rectify this by providing high-order asymptotic expansions for the at-the-money implied volatility skew, under a…
Paper extracts features from time series to improve forecasting accuracy.
problem Forecasting time series generated by Itô-type processes with unknown coefficients.
method Statistical adjustment of mixture-type models to extract features from time series data.
result Additional statistical features enhance time series prediction accuracy.
Many contemporary statistical learning methods assume a Euclidean feature space. This paper presents a method for defining similarity based on hyperspherical geometry and shows that it often improves the performance of support vector machine compared to other competing similarity measures. Specifically, the idea of usi…
The paper derives expansions for Green's operators and resolvents using Hadamard methods.
problem Analyzing normally hyperbolic operators and their Green's functions.
method Hadamard expansions for powers of Green's operators and resolvents.
result Derives expansions involving Hadamard coefficients for advanced/retarded Green's operators.
The study assesses low-rank approximations in Gaussian Process regression.
problem Improving Gaussian Process regression efficiency with low-rank approximations.
method Analyzes two low-rank approximations: random Fourier features and Mercer expansion truncation.
result Bounds on the divergence and error between exact and approximate GP models.
Gaussian equivalence fails for simple polynomial embeddings in quadratic scaling RF models.
problem Failure of Gaussian equivalence in polynomial feature embeddings under quadratic scaling.
method Introduced Conditional Gaussian Equivalent (CGE) model to capture non-Gaussian behavior.
result Correct asymptotics derived for training and test errors in CGE model.
The study assesses low-rank approximations in Gaussian Process regression.
problem Improving the efficiency of Gaussian Process regression while maintaining accuracy.
method Analyzes two low-rank approximations: random Fourier features and Mercer expansion truncation, and bounds the divergence and error between exact and approximate models.
result Theoretical bounds on the divergence and error between exact and approximate Gaussian Process models are provided.
This paper offers a characterization of fundamental limits on the classification and reconstruction of high-dimensional signals from low-dimensional features, in the presence of side information. We consider a scenario where a decoder has access both to linear features of the signal of interest and to linear features o…
Develops a martingale expansion for stochastic volatility models.
problem Approximating marginal distributions of stochastic volatility models.
method Martingale expansion framework for continuous stochastic volatility models.
result First-order perturbation expansions for small volatility-of-volatility and fast mean-reversion models.
Proposes efficient model for continual learning that grows model over task-specific parameters.
problem Limited transfer learning ability and forgetting of earlier knowledge in existing methods.
method Filter and channel expansion method that grows model over previous task parameters.
result Better knowledge transfer and improved performance in task incremental learning.