Ordinal Regression (OR) aims to model the ordering information between different data categories, which is a crucial topic in multi-label learning. An important class of approaches to OR models the problem as a linear combination of basis functions that map features to a high dimensional non-linear space. However, most…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Bayesian linear basis functions”
Introduces a neural network-based method for efficient state and parameter estimation in complex systems.
problem Efficiently estimating state paths and parameters from noisy measurements in high-dimensional nonlinear systems.
method Bayesian Information Field Theory with neural network parameterization and optimization algorithms.
result Proposes a method to simplify and enrich state path parameterizations using neural networks, improving inference accuracy.
The paper presents a method to infer unknown forcing functions in differential equations using Gaussian processes and adjoints.
problem Inferring unknown forcing functions in differential equations from noisy observations.
method Using adjoint methods to efficiently infer Gaussian process (GP) driven differential equations, with truncated basis expansions of the GP kernel.
result Efficient Bayesian inference of forcing functions modeled as GPs using adjoints, with lower computation than MCMC methods.
Paper projects GP basis functions using tensor networks to reduce complexity.
problem Efficiently approximating Gaussian process regression with a large number of basis functions.
method Develops a method using tensor networks to approximate GP regression with an exponential number of basis functions without exponential computational complexity.
result Shows efficient GP regression on an 18-dimensional benchmark data set.
LLA shows strong performance in Bayesian optimization but has unbounded search space issues.
problem Applying LLA in unbounded search spaces for Bayesian optimization.
method Linearized-Laplace approximation applied to Bayesian optimization problems.
result LLA demonstrates strong performance but also presents unbounded search space challenges.
Active learning method reduces labeling cost for regression models with aggregated data.
problem Reducing labeling cost for training regression models with aggregated data.
method Sequentially selects sets to be labeled using mutual information quantifying model parameter uncertainty.
result Achieves better predictive performance with fewer labeled sets.
New sparse Gaussian process method tackles unconstrained regression problems.
problem Dealing with physical systems that satisfy inequality constraints.
method Extends constrained Gaussian process by redefining hat basis functions.
result Reduces computational complexity from to .
Bayesian-TPNN improves ANOVA-TPNN for detecting higher-order components.
problem Difficulty in incorporating higher-order components in ANOVA-TPNN due to computational and memory constraints.
method Bayesian inference procedure for functional ANOVA model with TPNN basis functions.
result Bayesian-TPNN detects higher-order components with reduced computational cost.
While Bayesian neural networks have many appealing characteristics, current priors do not easily allow users to specify basic properties such as expected lengthscale or amplitude variance. In this work, we introduce Poisson Process Radial Basis Function Networks, a novel prior that is able to encode amplitude stationar…
Nonlocal Bayesian modeling for continuous spatio-temporal dynamics
problem Handling irregular time points, sparse observations, and nonlocal interactions in spatio-temporal forecasting
method Hierarchical Bayesian framework with coordinate-based spatial basis expansion and continuous-time ODE
result Strong forecasting and uncertainty calibration
We propose a new method for learning deep neural network models that is based on a greedy learning approach: we add one basis function at a time, and a new basis function is generated as a non-linear activation function applied to a linear combination of the previous basis functions. Such a method (growing deep neural …
DBKs enable scalable GPs with tractable inference for large datasets.
problem Scaling Gaussian processes to large and complex datasets while maintaining tractable inference.
method DBKs constructed from neural-network-parameterized basis functions with explicit low-rank structure, enabling linear-complexity inference.
result DBKs provide a unified perspective and improve predictive accuracy, uncertainty quantification, and computational efficiency.
Bayesian method learns PDEs from noisy data.
problem Discovering PDEs from noisy data.
method Combining variational Bayes and sparse linear regression.
result Proposes a new method to discover PDEs accurately.
Modeling massive highly-multivariate nonstationary spatial data with the basis graphical lassostat.ME
New framework models complex spatial data with basis functions and graphical vectors.
problem Modeling highly-multivariate spatial processes with varying resolutions.
method Extends graphical lasso to multivariate Gaussian processes with independent graphical vectors at different resolutions, using an orthogonal basis and fusion penalty.
result Linear complexity and parsimonious conditional independence structure in multilevel graphical model.
New neural network models for functional data.
problem Handling non-linear functional data.
method Functional Direct Neural Network (FDNN) and Functional Basis Neural Network (FBNN) with gradient-based optimization.
result Demonstrated effectiveness in complex functional models.
Robust Bayesian models are appealing alternatives to standard models, providing protection from data that contains outliers or other departures from the model assumptions. Historically, robust models were mostly developed on a case-by-case basis; examples include robust linear regression, robust mixture models, and bur…
To solve a machine learning problem, one typically needs to perform data preprocessing, modeling, and hyperparameter tuning, which is known as model selection and hyperparameter optimization.The goal of automated machine learning (AutoML) is to design methods that can automatically perform model selection and hyperpara…
Bayesian sparse reconstruction: a brute-force approach to astronomical imaging and machine learningastro-ph.IM
We present a principled Bayesian framework for signal reconstruction, in which the signal is modelled by basis functions whose number (and form, if required) is determined by the data themselves. This approach is based on a Bayesian interpretation of conventional sparse reconstruction and regularisation techniques, in …
Derives representations invariant under crystallographic groups for functions.
problem Representing and learning functions invariant under crystallographic groups.
method Derives linear and nonlinear representations of functions invariant under crystallographic groups.
result Derives orthonormal crystallographically invariant basis functions and embedding maps.
Bayesian approach models match and non-match score distributions over continuous covariates.
problem Complex evaluation of model performance over continuous covariates in biometric verification.
method Generative model of score distributions, mixture models, local basis functions, Bayesian inference.
result Accurate and effective method for studying model performance over continuous covariates.
We study a novel spline-like basis, which we name the "falling factorial basis", bearing many similarities to the classic truncated power basis. The advantage of the falling factorial basis is that it enables rapid, linear-time computations in basis matrix multiplication and basis matrix inversion. The falling factoria…
AEN-RBF kernel improves robustness in Bayesian optimization for complex systems.
problem Bayesian optimization struggles with outliers in RBF kernel, leading to poor performance.
method Proposes AEN-RBF kernel function, demonstrating improved robustness and convergence.
result The AEN-RBF kernel function reduces mean squared prediction error and improves convergence.
We reframe linear dimensionality reduction as a problem of Bayesian inference on matrix manifolds. This natural paradigm extends the Bayesian framework to dimensionality reduction tasks in higher dimensions with simpler models at greater speeds. Here an orthogonal basis is treated as a single point on a manifold and is…
Nonparametric Bayesian label prediction on a large graph using truncated Laplacian regularizationstat.CO
This article describes an implementation of a nonparametric Bayesian approach to solving binary classification problems on graphs. We consider a hierarchical Bayesian approach with a prior that is constructed by truncating a series expansion of the soft label function using the graph Laplacian eigenfunctions as basis f…
Bayesian approach learns linear operators from noisy data.
problem Learning linear operators from noisy data in infinite-dimensional spaces.
method Bayesian approach with Gaussian priors.
result Establishes posterior contraction rates and generalization error guarantees.
Self-guided ALPs improve MDP policies without domain knowledge.
problem Improving MDP policies with minimal domain knowledge.
method Self-guided sequence of ALPs with random basis functions and state-relevance distribution.
result High probability error bounds and improved policy performance.
Dual Bayesian Affine Estimators for Wiener-type state-space models
problem Estimating parameters in Wiener-type state-space models
method Fixed-point architecture combining two affine estimators
result Dual basis-parameter estimator achieves comparable parameter MSE to purely affine estimator
A linear model approximates Gaussian processes for efficient control.
problem Efficiently modeling and controlling Gaussian processes with many parameters.
method Developed a linear model using basis functions to approximate Gaussian processes.
result The linear model improves computational efficiency and feasibility of control strategies.
A major problem for the learning of Bayesian networks (BNs) is the exponential number of parameters needed for conditional probability tables. Recent research reduces this complexity by modeling local structure in the probability tables. We examine the use of log-linear local models. While log-linear models in this con…
A method for optimal Bayesian filtering using progressive particle flow and optimal transport maps.
problem Optimizing Bayesian filtering with deterministic particles to avoid degeneration.
method Progressive flow of particles through a sequence of sub-steps, each using an optimal transport map to replace non-equally weighted particles with equally weighted ones.
result The method avoids particle degeneration and simplifies the filtering process by not requiring inversions or monotonicity constraints.
Debiased Machine Learning of Conditional Average Treatment Effects and Other Causal Functionsstat.ME
This paper provides estimation and inference methods for the best linear predictor (approximation) of a structural function, such as conditional average structural and treatment effects, and structural derivatives, based on modern machine learning (ML) tools. We represent this structural function as a conditional expec…
Bayesian method identifies dynamical models with uncertainty quantification.
problem Uncertainty in selecting governing equations for dynamical systems.
method Bayesian sparse identification with model averaging.
result Accurately recovers sparse interaction structures with uncertainty quantification.
This work introduces a fixed-point optimization for variational inference.
problem Improving quantified uncertainty in predictions by optimizing a simplified distribution over parameters.
method Projective integral updates for high-dimensional variational inference.
result Efficient quasirandom quadrature sequence for mean-field distributions, leading to quasi-Newton variational Bayes (QNVB).
Bayesian model estimates treatment effects near cutoffs in regression discontinuity designs.
problem Estimating conditional average treatment effects in regression discontinuity designs.
method Develops a Bayesian additive regression tree (BART) model with linear leaf-level regressions.
result Adapts to different slopes on the running variable near the cutoff, providing interpretable inference.
Many applications that use empirically estimated functions face a curse of dimensionality, because the integrals over most function classes must be approximated by sampling. This paper introduces a novel regression-algorithm that learns linear factored functions (LFF). This class of functions has structural properties …
ADASAP accelerates GP inference for large datasets.
problem Scaling Gaussian process inference to large datasets.
method Approximate sketch-and-project algorithm for solving linear systems.
result Sketch-and-project rapidly converges to true posterior mean.
We propose a sequential learning policy for noisy discrete global optimization and ranking and selection (R\&S) problems with high dimensional sparse belief functions, where there are hundreds or even thousands of features, but only a small portion of these features contain explanatory power. We aim to identify the spa…
Proposes a new Bayesian learning method for optimal treatment regimes.
problem Sub-optimal policies in offline data due to lack of exploration.
method Integrates pessimism principle with Thompson sampling and Bayesian machine learning.
result Derives a credible set that uniformly lower bounds the optimal Q-function.
New ODE-Block handles stateful layers with continuous-in-depth functions using basis functions.
problem Handling stateful layers in ODE-Nets.
method Formulate ODE-Block using continuous-in-depth functions with basis function expansions.
result Enables state-of-the-art performance and reduces memory footprint.
Approximate Bayesian inference on the basis of summary statistics is well-suited to complex problems for which the likelihood is either mathematically or computationally intractable. However the methods that use rejection suffer from the curse of dimensionality when the number of summary statistics is increased. Here w…
Global optimization problems whose objective function is expensive to evaluate can be solved effectively by recursively fitting a surrogate function to function samples and minimizing an acquisition function to generate new samples. The acquisition step trades off between seeking for a new optimization vector where the…
We show theoretical similarities between the Least Squares Support Vector Regression (LS-SVR) model with a Radial Basis Functions (RBF) kernel and maximum a posteriori (MAP) inference on Bayesian RBF networks with a specific Gaussian prior on the regression weights. Although previous works have pointed out similar expr…
Gaussian processes retain the linear model either as a special case, or in the limit. We show how this relationship can be exploited when the data are at least partially linear. However from the perspective of the Bayesian posterior, the Gaussian processes which encode the linear model either have probability of nearly…
Sharp results link DLN gradient flow to basis pursuit optimization and GHA phase transitions.
problem Understanding implicit regularization in Diagonal Linear Networks.
method Sharp convergence bounds and characterization of minimizers.
result Gradient flow of DLNs with tiny initialization approximates minimizers of basis pursuit optimization problem.
We propose a novel reversible jump Markov chain Monte Carlo (MCMC) simulated annealing algorithm to optimize radial basis function (RBF) networks. This algorithm enables us to maximize the joint posterior distribution of the network parameters and the number of basis functions. It performs a global search in the joint …
Optimizes expensive functions using adaptive RBF surrogate model.
problem Global optimization of expensive, possibly non-differentiable functions.
method Adaptive Radial Basis Function (RBF) surrogate model with uncertainty quantification.
result The proposed method identifies optimal points efficiently, especially for non-smooth surfaces.
Bayesian methods estimate regression functions on submanifolds using graph Laplacian eigenbasis.
problem Estimating regression functions on unknown smooth submanifolds.
method Random geometric graph structure, Bayesian priors based on random basis expansion in graph Laplacian eigenbasis.
result Posterior contraction rates are minimax optimal for any positive smoothness index.
Real world experiments are expensive, and thus it is important to reach a target in minimum number of experiments. Experimental processes often involve control variables that changes over time. Such problems can be formulated as a functional optimisation problem. We develop a novel Bayesian optimisation framework for s…