arXiv research

A novel model uses ODE-based random features to model nonlinear dynamical systems.

The paper provides convergence guarantees for ODE-based generative models using transformers.

This work establishes near-minimax optimal guarantees for ODE-based samplers under mild assumptions.

Realizations of stochastic process are often observed temporal data or functional data. There are growing interests in classification of dynamic or functional data. The basic feature of functional data is that the functional data have infinite dimensions and are highly correlated. An essential issue for classifying dyn…

Enhances DGPs with adaptive RKHS Fourier features for better non-stationary pattern modeling.

It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introduction of so-called Neural ODEs, which allow more general discretization schemes with adaptive time stepping. Here, we propose ANODEV2, which is…

Paper analyzes convergence of ODE samplers in Wasserstein distances.

The paper examines the sampling dynamics of diffusion models using ODEs.

Rough Transformers improve efficiency for medical time-series data.

Novel methods improve Bayesian analysis of chaotic dynamical systems.

Rough Transformers improve time series modeling with lower costs and better performance.

Orthogonal random features approximate a Bessel kernel, offering sharper bounds than random Fourier features.

Poisson Midpoint Method improves Langevin Dynamics for diffusion models.

We present a particle flow realization of Bayes' rule, where an ODE-based neural operator is used to transport particles from a prior to its posterior after a new observation. We prove that such an ODE operator exists. Its neural parameterization can be trained in a meta-learning framework, allowing this operator to re…

Sparse random features improve accuracy in data-scarce settings.

A two-phase algorithm identifies the best arm in sparse linear bandits with fixed budget.

Learning rate needs to decrease with higher data moments for effective ICA in high dimensions.

Random feature maps improve forecasting with cheaper computation.

We study the approximation properties of random ReLU features through their reproducing kernel Hilbert space (RKHS). We first prove a universality theorem for the RKHS induced by random features whose feature maps are of the form of nodes in neural networks. The universality result implies that the random ReLU features…

Time series with non-uniform intervals occur in many applications, and are difficult to model using standard recurrent neural networks (RNNs). We generalize RNNs to have continuous-time hidden dynamics defined by ordinary differential equations (ODEs), a model we call ODE-RNNs. Furthermore, we use ODE-RNNs to replace t…

Kernel ridgeless regression with random features shows good generalization without explicit regularization.

RFRBoost uses random features to boost deep residual neural networks, improving performance and computational efficiency.

We propose semi-random features for nonlinear function approximation. The flexibility of semi-random feature lies between the fully adjustable units in deep learning and the random features used in kernel methods. For one hidden layer models with semi-random features, we prove with no unrealistic assumptions that the m…

The randomized-feature approach has been successfully employed in large-scale kernel approximation and supervised learning. The distribution from which the random features are drawn impacts the number of features required to efficiently perform a learning task. Recently, it has been shown that employing data-dependent …

A faster graph kernel using optical random features.

Gradient descent amplifies random features in neural networks to useful ones.

New method improves feature importance assessment in random forests.

A new method for approximating softmax and Gaussian kernels with reduced error.

Kernel method has been developed as one of the standard approaches for nonlinear learning, which however, does not scale to large data set due to its quadratic complexity in the number of samples. A number of kernel approximation methods have thus been proposed in the recent years, among which the random features metho…

Hermite polynomials improve private data generation by reducing feature count.

Random feature matrices' singular values concentrate near their full expectation in high dimensions.

HARFE approximates sparse additive functions using random features and ridge regression.

Enhances random forest performance with exogenous randomness.

Random Fourier features classification achieves fast learning rates with fewer features.

New random feature maps for Laplacian and related kernels.

Random feature maps are ubiquitous in modern statistical machine learning, where they generalize random projections by means of powerful, yet often difficult to analyze nonlinear operators. In this paper, we leverage the "concentration" phenomenon induced by random matrix theory to perform a spectral analysis on the Gr…

Improves efficiency of random feature approximations for dot product kernels.

This paper uses ODE to improve RNN models for time series data.

One-bit quantization improves inference speed for Random Features models.

We study the generalization properties of ridge regression with random features in the statistical learning framework. We show for the first time that $O(1/\sqrt{n})$ learning bounds can be achieved with only $O(\sqrt{n}\log n)$ random features rather than $O({n})$ as suggested by previous results. Further, we prove fa…

TRF uses ternary random features to improve ML performance without extra computation.

Survey on random features for kernel approximation, focusing on algorithms, theory, and practical applications.

Random feature models approximate functions in Banach spaces efficiently.

Improved kernel ridge regression for large datasets using weighted random binning.

A stability metric compares feature selection algorithms in machine learning.

This study examines a single attention layer's capabilities using random features.

We analyze the condition number of random feature matrices and prove their well-conditioned nature.

A new model combines diffusion and random features for better interpretability and comparable performance.