arXiv research

Forest-guided smoothing uses random forest outputs for interpretable local smoothers.

Convolutional Neural Networks (CNN) and the locally connected layer are limited in capturing the importance and relations of different local receptive fields, which are often crucial for tasks such as face verification, visual question answering, and word sequence prediction. To tackle the issue, we propose a novel loc…

Combining deep learning and ensemble smoothers for better history matching.

We present a Kalman smoothing framework based on modeling errors using the heavy tailed Student's t distribution, along with algorithms, convergence theory, open-source general implementation, and several important applications. The computational effort per iteration grows linearly with the length of the time series, a…

New nonlinear smoothers improve state estimation in chaotic systems.

EnKBS smoothes complex systems with future observations for causal inference.

New method calibrates LV surfaces for exotic derivatives with smoother, more stable Greeks.

This paper presents a general iterative bias correction procedure for regression smoothers. This bias reduction schema is shown to correspond operationally to the $L_2$ Boosting algorithm and provides a new statistical interpretation for $L_2$ Boosting. We analyze the behavior of the Boosting algorithm applied to commo…

Exponential smoothers are a simple and memory efficient way to compute running averages of time series. Here we define and describe practical properties of exponential smoothers for signals observed at constant and variable intervals.

dSMC improves parallel processing of state-space models.

Paper uses UKS to improve BLE RSSI for proximity inference in mobile phone apps.

Two ML approaches learn local volatility surfaces from option prices, with GP being arbitrage-free.

We consider a self-exciting counting process, the parameters of which depend on a hidden finite-state Markov chain. We derive the optimal filter and smoother for the hidden chain based on observation of the jump process. This filter is in closed form and is finite dimensional. We demonstrate the performance of this fil…

Improved Thompson Sampling for smoother functions with noise.

We introduce a family of adaptive estimators on graphs, based on penalizing the $\ell_1$ norm of discrete graph differences. This generalizes the idea of trend filtering [Kim et al. (2009), Tibshirani (2014)], used for univariate nonparametric regression, to graphs. Analogous to the univariate case, graph trend filteri…

We present a general probabilistic perspective on Gaussian filtering and smoothing. This allows us to show that common approaches to Gaussian filtering/smoothing can be distinguished solely by their methods of computing/approximating the means and covariances of joint probabilities. This implies that novel filters and …

Develops a novel ML smoothing method for incomplete data in state-space models.

ResNets promote smoother interpolations than MLPs, enhancing generalization.

Bayesian convolutional deep sets improve ambiguity in stationary process modeling.

Method approximates Lipschitz domains with smoother shapes.

Auto-regressive models improve smoothing efficiency with exponentially tapered windows.

This paper proposes a new optimization algorithm called Entropy-SGD for training deep neural networks that is motivated by the local geometry of the energy landscape. Local extrema with low generalization error have a large proportion of almost-zero eigenvalues in the Hessian with very few positive or negative eigenval…

Paper proposes a new Taylor moment expansion for non-linear Gaussian filtering and smoothing.

This paper presents a tensor multiplication based smoothing algorithm that follows a two step denoising method. Unlike other traditional averaging approaches, our approach uses an element based normal voting tensor to compute smooth surfaces. By introducing a binary optimization on the proposed tensor together with a l…

We introduce a new algorithm, called adaptive sparse backfitting algorithm, for solving high dimensional Sparse Additive Model (SpAM) utilizing symmetric, non-negative definite smoothers. Unlike the previous sparse backfitting algorithm, our method is essentially a block coordinate descent algorithm that guarantees to …

New method for robust fixed-point smoothing without state augmentation.

Estimating the state of a dynamical system from a series of noise-corrupted observations is fundamental in many areas of science and engineering. The most well-known method, the Kalman smoother (and the related Kalman filter), relies on assumptions of linearity and Gaussianity that are rarely met in practice. In this p…

Unified framework for ensemble transport-based smoothing of non-Gaussian time series.

The simplicity of gradient descent (GD) made it the default method for training ever-deeper and complex neural networks. Both loss functions and architectures are often explicitly tuned to be amenable to this basic local optimization. In the context of weakly-supervised CNN segmentation, we demonstrate a well-motivated…

Paper explains how tree ensembles improve predictions by smoothing and regulating smoothness.

We show that $C^0$-fine approximation of convex functions by smooth (or real analytic) convex functions on $\R^d$ is possible in general if and only if $d=1$. Nevertheless, for $d\geq 2$ we give a characterization of the class of convex functions on $\R^d$ which can be approximated by real analytic (or just smoother) c…

Paper uses SLT to improve model selection for SHM.

In the context of clustering, we assume a generative model where each cluster is the result of sampling points in the neighborhood of an embedded smooth surface; the sample may be contaminated with outliers, which are modeled as points sampled in space away from the clusters. We consider a prototype for a higher-order …

Entropy regularization is commonly used to improve policy optimization in reinforcement learning. It is believed to help with \emph{exploration} by encouraging the selection of more stochastic policies. In this work, we analyze this claim using new visualizations of the optimization landscape based on randomly perturbi…

Proposes LRR and LRLR for improving stock prediction accuracy.

Unified framework for efficient Gaussian process inference.

An important feature of successful supervised machine learning applications is to be able to explain the predictions given by the regression or classification model being used. However, most state-of-the-art models that have good predictive power lead to predictions that are hard to interpret. Thus, several model-agnos…

Study generalizes Möbius energy to non-smooth sets in arbitrary dimensions.

Generative models learn smoother densities to sample from unknown distributions.

Measures neural network decision boundary volume to predict model performance.

Bayesian approach improves neural network recurrence.

We study reproducing kernel Hilbert spaces (RKHS) on a Riemannian manifold. In particular, we discuss under which condition Sobolev spaces are RKHS and characterize their reproducing kernels. Further, we introduce and discuss a class of smoother RKHS that we call diffusion spaces. We illustrate the general results with…

Two of the authors have defined the class $WDC(M)$ as the class of all subsets of a smooth manifold $M$ that may be expressed in local coordinates as certain sublevel sets of DC (differences of convex) functions. If $M$ is Riemanian and $G$ is a group of isometries acting transitively on the sphere bundle $SM$, we def…

We present a framework to train a structured prediction model by performing smoothing on the inference algorithm it builds upon. Smoothing overcomes the non-smoothness inherent to the maximum margin structured prediction objective, and paves the way for the use of fast primal gradient-based optimization algorithms. We …

Study optimal stopping problems with finite-time horizon and proves continuity and strict monotonicity of the boundary.

We develop theory for nonlinear dimensionality reduction (NLDR). A number of NLDR methods have been developed, but there is limited understanding of how these methods work and the relationships between them. There is limited basis for using existing NLDR theory for deriving new algorithms. We provide a novel framework …

Entropy regularization improves power k-means for high-dimensional data.

Unified approach to multiclass classification using Gabriel graphs.