arXiv research

We introduce a Gaussian process model of functions which are additive. An additive function is one which decomposes into a sum of low-dimensional functions, each depending on only a subset of the input variables. Additive GPs generalize both Generalized Additive Models, and the standard GP models which use squared-expo…

Proposes a new model for high-dimensional data analysis with unknown link function.

Improved Gaussian process models for interpretable predictions.

Bayesian Optimisation (BO) is a technique used in optimising a $D$-dimensional function which is typically expensive to evaluate. While there have been many successes for BO in low dimensions, scaling it to high dimensions has been notoriously difficult. Existing literature on the topic are under very restrictive setti…

Gradient-free optimization for additive models achieves optimal error.

The main purpose of this note is to provide a topological approach to defining additive functions on Riemannian co-compact normal coverings.

Existing approaches to combine both additive and multiplicative neural units either use a fixed assignment of operations or require discrete optimization to determine what function a neuron should perform. This leads either to an inefficient distribution of computational resources or an extensive increase in the comput…

Sparse additive modeling is a class of effective methods for performing high-dimensional nonparametric regression. In this work we show how shape constraints such as convexity/concavity and their extensions, can be integrated into additive models. The proposed sparse difference of convex additive models (SDCAM) can est…

Existing approaches to combine both additive and multiplicative neural units either use a fixed assignment of operations or require discrete optimization to determine what function a neuron should perform. However, this leads to an extensive increase in the computational complexity of the training procedure. We present…

Establishes relationships between prudence and stability properties of risk functionals.

Many biological learning systems such as the mushroom body, hippocampus, and cerebellum are built from sparsely connected networks of neurons. For a new understanding of such networks, we study the function spaces induced by sparse random features and characterize what functions may and may not be learned. A network wi…

Novel covariance function improves Bayesian optimization efficiency.

TSL learns separable models to avoid signal cancellation and off-support extrapolation.

A new class of risk measures called cash sub-additive risk measures is introduced to assess the risk of future financial, nonfinancial and insurance positions. The debated cash additive axiom is relaxed into the cash sub additive axiom to preserve the original difference between the numeraire of the current reserve amo…

Neural model improves option pricing by calibrating additive process term structure.

High dimensional nonparametric regression is an inherently difficult problem with known lower bounds depending exponentially in dimension. A popular strategy to alleviate this curse of dimensionality has been to use additive models of \emph{first order}, which model the regression function as a sum of independent funct…

The paper tackles high-dimensional Bayesian optimization using tree-structured additive models.

Improved Bayesian optimization for conditional parameter spaces.

HARFE approximates sparse additive functions using random features and ridge regression.

We consider the Willmore functional on graphs, with an additional penalization of the area where the curvature is non-zero. Interpreting the penalization parameter as a Lagrange multiplier, this corresponds to the Willmore functional with a constraint on the area where the graph is flat. Sending the penalization parame…

Additive models play an important role in semiparametric statistics. This paper gives learning rates for regularized kernel based methods for additive models. These learning rates compare favourably in particular in high dimensions to recent results on optimal learning rates for purely nonparametric regularized kernel …

Improves adversarial robustness by constraining logits with a bounded function.

Projection pursuit model improves Gaussian process regression for high-dimensional data.

New model estimates higher-order interactions in stochastic processes using lower-dimensional projections.

We provide a unified view of additive explanations for dependent inputs.

Study on signal detection in sparse additive models with nonasymptotic minimax rates.

Probit Monotone BART estimates binary outcomes using monotonic functions.

We study additive models built with trend filtering, i.e., additive models whose components are each regularized by the (discrete) total variation of their $k$th (discrete) derivative, for a chosen integer $k \geq 0$. This results in $k$th degree piecewise polynomial components, (e.g., $k=0$ gives piecewise constant co…

BARK optimizes black-box functions using Bayesian Additive Regression Trees.

Support vector machines (SVMs) are special kernel based methods and belong to the most successful learning methods since more than a decade. SVMs can informally be described as a kind of regularized M-estimators for functions and have demonstrated their usefulness in many complicated real-life problems. During the last…

Sparse matrices simplify computation of GP variances and likelihoods.

Bayesian optimization tackles non-smooth tuning problems.

Many methods to explain black-box models, whether local or global, are additive. In this paper, we study global additive explanations for non-additive models, focusing on four explanation methods: partial dependence, Shapley explanations adapted to a global setting, distilled additive explanations, and gradient-based e…

This article arose from a series of three lectures given at the Banach Center, Warsaw, during period of 24 March to 13 April, 2003. Morse functions are useful tool in revealing the geometric formation of its domain manifolds $M$. They define the handle decompositions of $M$ from which the additive homologies $H_{\ast}(…

Improved convergence speed of principal component analysis through modified learning rules.

No-regret optimization for time-varying functions using uncertainty injection.

First introduced by Fernholz in stochastic portfolio theory, functionally generated portfolio allows its investment performance to be attributed to directly observable and easily interpretable market quantities. In previous works we showed that Fernholz's multiplicatively generated portfolio has deep connections with o…

SurvFD and SurvSHAP-IQ provide interpretable survival models by analyzing feature interactions.

Much recent work has concerned sparse approximations to speed up the Gaussian process regression from the unfavorable O(n3) scaling in computational time to O(nm2). Thus far, work has concentrated on models with one covariance function. However, in many practical situations additive models with multiple covariance func…

Bayesian Additive Distribution Regression (DistBART) predicts distributions from grouped data.

New method finds profitable investment opportunities by considering additional financial variables.

Inexact subgradient methods work well for semialgebraic functions with additive errors.

A new method for analyzing shapes and forms using additive models on manifolds.

FAST optimizes additive segmentation for faster, more interpretable models.

Additive models form a widely popular class of regression models which represent the relation between covariates and response variables as the sum of low-dimensional transfer functions. Besides flexibility and accuracy, a key benefit of these models is their interpretability: the transfer functions provide visual means…

Thompson Sampling with bilateral uncertainty improves performance in Bayesian Optimization.

RAMs improve GAMs' accuracy by fitting components to subregions of feature space.

The main point of this paper is to prove the following useful result: If the almost everywhere 2-jet of a locally quasi-convex function u satisfies a degenerate elliptic constraint F, then u is F-subharmonic, i.e., u is a viscosity F-subsolution. This AE Theorem makes otherwise difficult results transparent. Some insta…