arXiv research

Forest-guided smoothing uses random forest outputs for interpretable local smoothers.

Convolutional Neural Networks (CNN) and the locally connected layer are limited in capturing the importance and relations of different local receptive fields, which are often crucial for tasks such as face verification, visual question answering, and word sequence prediction. To tackle the issue, we propose a novel loc…

Combining deep learning and ensemble smoothers for better history matching.

We present a Kalman smoothing framework based on modeling errors using the heavy tailed Student's t distribution, along with algorithms, convergence theory, open-source general implementation, and several important applications. The computational effort per iteration grows linearly with the length of the time series, a…

New nonlinear smoothers improve state estimation in chaotic systems.

EnKBS smoothes complex systems with future observations for causal inference.

New method calibrates LV surfaces for exotic derivatives with smoother, more stable Greeks.

This paper presents a general iterative bias correction procedure for regression smoothers. This bias reduction schema is shown to correspond operationally to the $L_2$ Boosting algorithm and provides a new statistical interpretation for $L_2$ Boosting. We analyze the behavior of the Boosting algorithm applied to commo…

Exponential smoothers are a simple and memory efficient way to compute running averages of time series. Here we define and describe practical properties of exponential smoothers for signals observed at constant and variable intervals.

dSMC improves parallel processing of state-space models.

Paper uses UKS to improve BLE RSSI for proximity inference in mobile phone apps.

Two ML approaches learn local volatility surfaces from option prices, with GP being arbitrage-free.

We consider a self-exciting counting process, the parameters of which depend on a hidden finite-state Markov chain. We derive the optimal filter and smoother for the hidden chain based on observation of the jump process. This filter is in closed form and is finite dimensional. We demonstrate the performance of this fil…

We introduce a family of adaptive estimators on graphs, based on penalizing the $\ell_1$ norm of discrete graph differences. This generalizes the idea of trend filtering [Kim et al. (2009), Tibshirani (2014)], used for univariate nonparametric regression, to graphs. Analogous to the univariate case, graph trend filteri…

The paper is concerned with non-linear Gaussian filtering and smoothing in continuous-discrete state-space models, where the dynamic model is formulated as an Itô stochastic differential equation (SDE), and the measurements are obtained at discrete time instants. We propose novel Taylor moment expansion (TME) Gaussian …

We present a general probabilistic perspective on Gaussian filtering and smoothing. This allows us to show that common approaches to Gaussian filtering/smoothing can be distinguished solely by their methods of computing/approximating the means and covariances of joint probabilities. This implies that novel filters and …

Develops a novel ML smoothing method for incomplete data in state-space models.

ResNets promote smoother interpolations than MLPs, enhancing generalization.

Bayesian convolutional deep sets improve ambiguity in stationary process modeling.

Method approximates Lipschitz domains with smoother shapes.

Auto-regressive models improve smoothing efficiency with exponentially tapered windows.

This paper proposes a new optimization algorithm called Entropy-SGD for training deep neural networks that is motivated by the local geometry of the energy landscape. Local extrema with low generalization error have a large proportion of almost-zero eigenvalues in the Hessian with very few positive or negative eigenval…

This paper presents a tensor multiplication based smoothing algorithm that follows a two step denoising method. Unlike other traditional averaging approaches, our approach uses an element based normal voting tensor to compute smooth surfaces. By introducing a binary optimization on the proposed tensor together with a l…

We introduce a new algorithm, called adaptive sparse backfitting algorithm, for solving high dimensional Sparse Additive Model (SpAM) utilizing symmetric, non-negative definite smoothers. Unlike the previous sparse backfitting algorithm, our method is essentially a block coordinate descent algorithm that guarantees to …

New method for robust fixed-point smoothing without state augmentation.

Estimating the state of a dynamical system from a series of noise-corrupted observations is fundamental in many areas of science and engineering. The most well-known method, the Kalman smoother (and the related Kalman filter), relies on assumptions of linearity and Gaussianity that are rarely met in practice. In this p…

Unified framework for ensemble transport-based smoothing of non-Gaussian time series.

The simplicity of gradient descent (GD) made it the default method for training ever-deeper and complex neural networks. Both loss functions and architectures are often explicitly tuned to be amenable to this basic local optimization. In the context of weakly-supervised CNN segmentation, we demonstrate a well-motivated…

Paper explains how tree ensembles improve predictions by smoothing and regulating smoothness.

We show that $C^0$-fine approximation of convex functions by smooth (or real analytic) convex functions on $\R^d$ is possible in general if and only if $d=1$. Nevertheless, for $d\geq 2$ we give a characterization of the class of convex functions on $\R^d$ which can be approximated by real analytic (or just smoother) c…

Paper uses SLT to improve model selection for SHM.

In the context of clustering, we assume a generative model where each cluster is the result of sampling points in the neighborhood of an embedded smooth surface; the sample may be contaminated with outliers, which are modeled as points sampled in space away from the clusters. We consider a prototype for a higher-order …

Entropy regularization is commonly used to improve policy optimization in reinforcement learning. It is believed to help with \emph{exploration} by encouraging the selection of more stochastic policies. In this work, we analyze this claim using new visualizations of the optimization landscape based on randomly perturbi…

Unified framework for efficient Gaussian process inference.

Proposes LRR and LRLR for improving stock prediction accuracy.

An important feature of successful supervised machine learning applications is to be able to explain the predictions given by the regression or classification model being used. However, most state-of-the-art models that have good predictive power lead to predictions that are hard to interpret. Thus, several model-agnos…

Study generalizes Möbius energy to non-smooth sets in arbitrary dimensions.

Despite its well-known shortcomings, $k$-means remains one of the most widely used approaches to data clustering. Current research continues to tackle its flaws while attempting to preserve its simplicity. Recently, the \textit{power $k$-means} algorithm was proposed to avoid trapping in local minima by annealing throu…

Generative models learn smoother densities to sample from unknown distributions.

Measures neural network decision boundary volume to predict model performance.

We study reproducing kernel Hilbert spaces (RKHS) on a Riemannian manifold. In particular, we discuss under which condition Sobolev spaces are RKHS and characterize their reproducing kernels. Further, we introduce and discuss a class of smoother RKHS that we call diffusion spaces. We illustrate the general results with…

Two of the authors have defined the class $WDC(M)$ as the class of all subsets of a smooth manifold $M$ that may be expressed in local coordinates as certain sublevel sets of DC (differences of convex) functions. If $M$ is Riemanian and $G$ is a group of isometries acting transitively on the sphere bundle $SM$, we def…

We present a framework to train a structured prediction model by performing smoothing on the inference algorithm it builds upon. Smoothing overcomes the non-smoothness inherent to the maximum margin structured prediction objective, and paves the way for the use of fast primal gradient-based optimization algorithms. We …

Study optimal stopping problems with finite-time horizon and proves continuity and strict monotonicity of the boundary.

We develop theory for nonlinear dimensionality reduction (NLDR). A number of NLDR methods have been developed, but there is limited understanding of how these methods work and the relationships between them. There is limited basis for using existing NLDR theory for deriving new algorithms. We provide a novel framework …

Unified approach to multiclass classification using Gabriel graphs.

We describe notions of tautness that arise in the study of $C^0$ foliations, $C^{1,0}$ or smoother foliations, and in geometry. We give examples to show that these notions are different, and discuss how these differences impact some classical foliation results. We construct examples of smoothly taut $C^{\infty,0}$ foli…

Sparse Markovian Gaussian processes improve probabilistic model inference for large datasets.