New framework models complex spatial data with basis functions and graphical vectors.
problem Modeling highly-multivariate spatial processes with varying resolutions.
method Extends graphical lasso to multivariate Gaussian processes with independent graphical vectors at different resolutions, using an orthogonal basis and fusion penalty.
result Linear complexity and parsimonious conditional independence structure in multilevel graphical model.
We study a novel spline-like basis, which we name the "falling factorial basis", bearing many similarities to the classic truncated power basis. The advantage of the falling factorial basis is that it enables rapid, linear-time computations in basis matrix multiplication and basis matrix inversion. The falling factoria…
Machine learning model predicts DFT total energy to complete basis set limit.
problem Finding a model to extrapolate DFT calculations to complete basis set limit.
method Quantile-random-forest model trained on binary solids data.
result Random-forest model achieves <25% symmetric MAPE for both DFT codes.
Introduces tunable basis functions for Gaussian processes.
problem Reduces computational complexity in Gaussian process approximations.
method Introduces tunable, local, and bounded basis functions for kernel approximation.
result Demonstrates superior performance compared to state-of-the-art methods, especially with poorly chosen kernel functions.
Ordinal Regression (OR) aims to model the ordering information between different data categories, which is a crucial topic in multi-label learning. An important class of approaches to OR models the problem as a linear combination of basis functions that map features to a high dimensional non-linear space. However, most…
A number of fundamental quantities in statistical signal processing and information theory can be expressed as integral functions of two probability density functions. Such quantities are called density functionals as they map density functions onto the real line. For example, information divergence functions measure t…
Derives representations invariant under crystallographic groups for functions.
problem Representing and learning functions invariant under crystallographic groups.
method Derives linear and nonlinear representations of functions invariant under crystallographic groups.
result Derives orthonormal crystallographically invariant basis functions and embedding maps.
We propose a new method for learning deep neural network models that is based on a greedy learning approach: we add one basis function at a time, and a new basis function is generated as a non-linear activation function applied to a linear combination of the previous basis functions. Such a method (growing deep neural …
Optimizes basis functions for learning dynamical systems from data.
problem Learning suitable basis functions for dynamical systems from data.
method Gradient-based optimization framework for learning basis functions.
result Efficacy demonstrated on various benchmark problems.
Several multiscale methods account for sub-grid scale features using coarse scale basis functions. For example, in the Multiscale Finite Volume method the coarse scale basis functions are obtained by solving a set of local problems over dual-grid cells. We introduce a data-driven approach for the estimation of these co…
RI-DeepONet learns neural operators from arbitrary sensor data.
problem Discretization of input functions limits practical applications of DeepONet.
method Introduces RI-DeepONet and two dictionary learning algorithms for INRs.
result RINO handles arbitrary sensor data robustly and applies to various problems.
New ODE-Block handles stateful layers with continuous-in-depth functions using basis functions.
problem Handling stateful layers in ODE-Nets.
method Formulate ODE-Block using continuous-in-depth functions with basis function expansions.
result Enables state-of-the-art performance and reduces memory footprint.
Adaptive neural networks learn functional data bases for improved performance.
problem Applying deep learning to functional data is challenging due to high dimensionality.
method Proposes adaptive neural networks with Basis Layers that learn relevant basis functions.
result Empirically outperforms other neural network approaches across various tasks.
Radial-basis-function networks are traditionally defined for sets of vector-based observations. In this short paper, we reformulate such networks so that they can be applied to adjacency-matrix representations of weighted, directed graphs that represent the relationships between object pairs. We re-state the sum-of-squ…
Paper projects GP basis functions using tensor networks to reduce complexity.
problem Efficiently approximating Gaussian process regression with a large number of basis functions.
method Develops a method using tensor networks to approximate GP regression with an exponential number of basis functions without exponential computational complexity.
result Shows efficient GP regression on an 18-dimensional benchmark data set.
In many applications (in particular information systems, such as pattern recognition, machine learning, cheminformatics, bioinformatics to name but a few) the assessment of uncertainty is essential - i.e., the estimation of the underlying probability distribution function. More often than not, the form of this function…
New sparse Gaussian process method tackles unconstrained regression problems.
problem Dealing with physical systems that satisfy inequality constraints.
method Extends constrained Gaussian process by redefining hat basis functions.
result Reduces computational complexity from O(n3) to O(nm2). Deep neural network predicts molecular wave functions in minimal basis.
problem Improving accuracy and efficiency in quantum chemistry calculations.
method Adapted SchNet for Orbitals (SchNOrb) model in quasi-atomic minimal basis.
result Model accurately predicts molecular orbital energies and wavefunctions for large molecules.
The study explores various localized bases and their duals for scattered data approximation.
problem Scattered data approximation using radial basis functions.
method Examines different localized bases including Lagrange, Newton, and multiresolution versions, and their duals.
result Localized orthogonal bases, such as the Newton basis, offer symmetric preconditioners and are feasible for scattered data approximation.
s-RBFN integrates multiple hypotheses for efficient and diverse prediction.
problem Integrating multiple hypotheses into learning models for regression.
method Structured Radial Basis Function Network (s-RBFN) using Voronoi tessellations and least-squares training.
result s-RBFN achieves superior generalization and efficiency compared to other models.
New neural network models for functional data.
problem Handling non-linear functional data.
method Functional Direct Neural Network (FDNN) and Functional Basis Neural Network (FBNN) with gradient-based optimization.
result Demonstrated effectiveness in complex functional models.
A core operation in reinforcement learning (RL) is finding an action that is optimal with respect to a learned value function. This operation is often challenging when the learned value function takes continuous actions as input. We introduce deep radial-basis value functions (RBVFs): value functions learned using a de…
BASIS improves LLM reasoning by sharing batchwise rollout info, reducing MSE by 69%.
problem Improving large language model reasoning with limited rollouts and batch information.
method BASIS samples only one rollout per prompt but uses batch information to improve value function estimation.
result BASIS reduces MSE in value function estimation by 69% compared to REINFORCE++.
Archetype and archetypoid analysis can be extended to functional data. Each function is represented as a mixture of actual observations (functional archetypoids) or functional archetypes, which are a mixture of observations in the data set. Well-known Canadian temperature data are used to illustrate the analysis develo…
Efficiently analyzes multidimensional functional data using separable basis functions.
problem Curse of dimensionality in traditional functional data analysis.
method Marginal product basis systems for multidimensional data, tensor decomposition, differential operator-based penalties.
result Efficient estimation of multidimensional functional data representations.
Gradient-based training and pruning for radial basis function networks in materials physics.
problem Interpretable and robust machine learning for materials physics problems.
method Gradient-based training and pruning of radial basis function networks with closed-form optimization criteria.
result Pruned models provide compact and interpretable versions of larger models, offering insights into atom-level migration processes.
We propose two localized Radial Basis Function (RBF) methods, the Radial Basis Function Partition of Unity method (RBF-PUM) and the Radial Basis Function generated Finite Differences method (RBF-FD), for solving financial derivative pricing problems arising from market models with multiple stochastic factors. We demons…
Neural networks can approximate functionals on RKHS with error bounds.
problem Approximating functionals on RKHS using neural networks.
method Interpolating orthogonal projections in RKHS using point evaluations.
result Explicit error bounds for various kernels (inverse multiquadric, Gaussian, Sobolev).
New RBF networks can approximate any continuous function.
problem Approximating any continuous function on a compact subset.
method Replacing smoothing factors with shifts in RBF networks and proving approximation under certain conditions.
result RBF networks can approximate any continuous function on any compact subset.
New method improves speed of estimating bivariate functional data.
problem Estimating bivariate functional data at faster rates.
method Adapting to directional regularity of bivariate processes.
result Faster rates of convergence achieved through change-of-basis.
In this paper, we propose a generic framework for devising an adaptive approximation scheme for value function approximation in reinforcement learning, which introduces multiscale approximation. The two basic ingredients are multiresolution analysis as well as tree approximation. Starting from simple refinable function…
We consider a nonlinear state-space model with the state transition and observation functions expressed as basis function expansions. The coefficients in the basis function expansions are learned from data. Using a connection to Gaussian processes we also develop priors on the coefficients, for tuning the model flexibi…
We study the problem of estimating multiple predictive functions from a dictionary of basis functions in the nonparametric regression setting. Our estimation scheme assumes that each predictive function can be estimated in the form of a linear combination of the basis functions. By assuming that the coefficient matrix …
Enhances FM models for numerical features using function basis encoding.
problem Challenges in incorporating numerical features into FM variants.
method Encoding numerical features into a vector of function values for learning segmentized functions.
result Improves model accuracy by learning segmentized functions of numerical features.
DDD reformulated for sparse matrices, integrating trajectory and snapshot time series data.
problem Efficiently integrate trajectory and snapshot time series data.
method Reformulate DDD to use compact basis functions, reducing parameter scaling.
result Inference of sparse matrices reduces the number of parameters in DDD.
Optimizes basis for density-based atomic representations to enhance compactness and accuracy.
problem Improving the efficiency and accuracy of machine learning models for atomic properties.
method An unsupervised approach to determine the optimal basis set for atom density representations using splines.
result Optimal basis sets that encode structural information more compactly and accurately.
Study shows Calabi-Yau metrics converge to a specific form under certain conditions.
problem Degeneration of Calabi-Yau metrics and their limits.
method Optimal transport problem and minimisation of Kontorovich functional.
result Limit data of Calabi-Yau metrics can be encoded into a unique minimiser.
Global optimization problems whose objective function is expensive to evaluate can be solved effectively by recursively fitting a surrogate function to function samples and minimizing an acquisition function to generate new samples. The acquisition step trades off between seeking for a new optimization vector where the…
Neural Chaos uses neural networks instead of polynomials for stochastic modeling.
problem Challenges in constructing surrogate models with uncertainty quantification for complex or high-dimensional stochastic processes.
method Adopting spectral expansion formalism with neural network basis functions, identifying them data-drivenly without prior assumptions.
result Demonstrates effectiveness of the proposed scheme through numerical examples of varying complexity.
OKSVM optimizes RBF kernel hyperparameter for SVMs, improving classification performance.
problem Intrinsic dependence of RBF kernel hyperparameter on SVM performance.
method Gradient descent method for automatic hyperparameter learning and SVM weights adjustment.
result OKSVM outperforms classical SVM regardless of initial RBF hyperparameter values.
We study the effectiveness of various approaches that defend against adversarial attacks on deep networks via manipulations based on basis function representations of images. Specifically, we experiment with low-pass filtering, PCA, JPEG compression, low resolution wavelet approximation, and soft-thresholding. We evalu…
Many problems in financial engineering involve the estimation of unknown conditional expectations across a time interval. Often Least Squares Monte Carlo techniques are used for the estimation. One method that can be combined with Least Squares Monte Carlo is the "Regress-Later" method. Unlike conventional methods wher…
Many applications that use empirically estimated functions face a curse of dimensionality, because the integrals over most function classes must be approximated by sampling. This paper introduces a novel regression-algorithm that learns linear factored functions (LFF). This class of functions has structural properties …
We consider the problem of designing a sparse Gaussian process classifier (SGPC) that generalizes well. Viewing SGPC design as constructing an additive model like in boosting, we present an efficient and effective SGPC design method to perform a stage-wise optimization of a predictive loss function. We introduce new me…
Adaptive RBF-KAN improves KANs by dynamically adjusting kernel parameters.
problem Efficiently approximating multivariate functions using learnable univariate edge functions.
method Integrates LOOCV-based kernel scale estimation with adaptive kernel learning.
result Adaptive RBF-KAN outperforms fixed kernel KANs on various benchmark functions.
A new method integrates Fourier basis expansion and mapping for improved time series forecasting.
problem Inconsistent starting cycles and series length issues in Fourier-based methods.
method Fourier Basis Mapping (FBM) method that integrates time-frequency features through Fourier basis expansion and mapping.
result FBM addresses inconsistencies and preserves temporal characteristics, achieving SOTA performance.
A machine learning method selects optimal orthonormal bases for functional data analysis.
problem Lack of formal criteria for choosing initial orthonormal bases in functional data methods.
method Proposes a machine learning algorithm to learn and place knots for efficient orthogonal spline bases (splinets).
result Demonstrates efficiency, especially for sparse functional data and complex physical systems.
While Bayesian neural networks have many appealing characteristics, current priors do not easily allow users to specify basic properties such as expected lengthscale or amplitude variance. In this work, we introduce Poisson Process Radial Basis Function Networks, a novel prior that is able to encode amplitude stationar…