arXiv research

Forest-guided smoothing uses random forest outputs for interpretable local smoothers.

We introduce a new algorithm, called adaptive sparse backfitting algorithm, for solving high dimensional Sparse Additive Model (SpAM) utilizing symmetric, non-negative definite smoothers. Unlike the previous sparse backfitting algorithm, our method is essentially a block coordinate descent algorithm that guarantees to …

Paper explains how tree ensembles improve predictions by smoothing and regulating smoothness.

We present a Kalman smoothing framework based on modeling errors using the heavy tailed Student's t distribution, along with algorithms, convergence theory, open-source general implementation, and several important applications. The computational effort per iteration grows linearly with the length of the time series, a…

New nonlinear smoothers improve state estimation in chaotic systems.

We introduce a family of adaptive estimators on graphs, based on penalizing the $\ell_1$ norm of discrete graph differences. This generalizes the idea of trend filtering [Kim et al. (2009), Tibshirani (2014)], used for univariate nonparametric regression, to graphs. Analogous to the univariate case, graph trend filteri…

This paper presents a general iterative bias correction procedure for regression smoothers. This bias reduction schema is shown to correspond operationally to the $L_2$ Boosting algorithm and provides a new statistical interpretation for $L_2$ Boosting. We analyze the behavior of the Boosting algorithm applied to commo…

Exponential smoothers are a simple and memory efficient way to compute running averages of time series. Here we define and describe practical properties of exponential smoothers for signals observed at constant and variable intervals.

dSMC improves parallel processing of state-space models.

Combining deep learning and ensemble smoothers for better history matching.

AutoClip automatically adjusts gradient clipping for better audio separation.

We consider a self-exciting counting process, the parameters of which depend on a hidden finite-state Markov chain. We derive the optimal filter and smoother for the hidden chain based on observation of the jump process. This filter is in closed form and is finite dimensional. We demonstrate the performance of this fil…

EnKBS smoothes complex systems with future observations for causal inference.

The paper is concerned with non-linear Gaussian filtering and smoothing in continuous-discrete state-space models, where the dynamic model is formulated as an Itô stochastic differential equation (SDE), and the measurements are obtained at discrete time instants. We propose novel Taylor moment expansion (TME) Gaussian …

We present a general probabilistic perspective on Gaussian filtering and smoothing. This allows us to show that common approaches to Gaussian filtering/smoothing can be distinguished solely by their methods of computing/approximating the means and covariances of joint probabilities. This implies that novel filters and …

Convolutional Neural Networks (CNN) and the locally connected layer are limited in capturing the importance and relations of different local receptive fields, which are often crucial for tasks such as face verification, visual question answering, and word sequence prediction. To tackle the issue, we propose a novel loc…

Develops a novel ML smoothing method for incomplete data in state-space models.

ResNets promote smoother interpolations than MLPs, enhancing generalization.

Bayesian convolutional deep sets improve ambiguity in stationary process modeling.

Auto-regressive models improve smoothing efficiency with exponentially tapered windows.

New method for robust fixed-point smoothing without state augmentation.

Estimating the state of a dynamical system from a series of noise-corrupted observations is fundamental in many areas of science and engineering. The most well-known method, the Kalman smoother (and the related Kalman filter), relies on assumptions of linearity and Gaussianity that are rarely met in practice. In this p…

The study examines the dynamic behavior of RMSprop and Adam algorithms.

Unified framework for ensemble transport-based smoothing of non-Gaussian time series.

An unsupervised learning algorithm to cluster hyperspectral image (HSI) data is proposed that exploits spatially-regularized random walks. Markov diffusions are defined on the space of HSI spectra with transitions constrained to near spatial neighbors. The explicit incorporation of spatial regularity into the diffusion…

The paper explores how structured representations influence learning dynamics in neural networks.

We propose a novel algorithm for large-scale regression problems named histogram transform ensembles (HTE), composed of random rotations, stretchings, and translations. First of all, we investigate the theoretical properties of HTE when the regression function lies in the Hölder space $C^{k,α}$, $k \in \mathbb{N}_0$, $…

Paper uses SLT to improve model selection for SHM.

Unified framework for efficient Gaussian process inference.

Proposes a new framework for investing that adapts to market regimes.

Paper uses UKS to improve BLE RSSI for proximity inference in mobile phone apps.

Generative models use kernel smoothing for conditioning on small example sets.

Generative models learn smoother densities to sample from unknown distributions.

We present a framework to train a structured prediction model by performing smoothing on the inference algorithm it builds upon. Smoothing overcomes the non-smoothness inherent to the maximum margin structured prediction objective, and paves the way for the use of fast primal gradient-based optimization algorithms. We …

AngularGrad optimizes CNNs by considering gradient direction, improving convergence.

We study reproducing kernel Hilbert spaces (RKHS) on a Riemannian manifold. In particular, we discuss under which condition Sobolev spaces are RKHS and characterize their reproducing kernels. Further, we introduce and discuss a class of smoother RKHS that we call diffusion spaces. We illustrate the general results with…

Robo-advisors use MPC to create dynamic investment strategies.

New method estimates and optimizes policy differences using orthogonal learning.

We investigate an algorithm named histogram transform ensembles (HTE) density estimator whose effectiveness is supported by both solid theoretical analysis and significant experimental performance. On the theoretical side, by decomposing the error term into approximation error and estimation error, we are able to condu…

Layer normalization (LayerNorm) is a technique to normalize the distributions of intermediate layers. It enables smoother gradients, faster training, and better generalization accuracy. However, it is still unclear where the effectiveness stems from. In this paper, our main contribution is to take a step further in und…

Unified approach to multiclass classification using Gabriel graphs.

Reliable 4D aircraft trajectory prediction, whether in a real-time setting or for analysis of counterfactuals, is important to the efficiency of the aviation system. Toward this end, we first propose a highly generalizable efficient tree-based matching algorithm to construct image-like feature maps from high-fidelity m…

We describe notions of tautness that arise in the study of $C^0$ foliations, $C^{1,0}$ or smoother foliations, and in geometry. We give examples to show that these notions are different, and discuss how these differences impact some classical foliation results. We construct examples of smoothly taut $C^{\infty,0}$ foli…

New method calibrates LV surfaces for exotic derivatives with smoother, more stable Greeks.

The ropelength of a knot is the quotient of its length and its thickness, the radius of the largest embedded normal tube around the knot. We prove existence and regularity for ropelength minimizers in any knot or link type; these are $C^{1,1}$ curves, but need not be smoother. We improve the lower bound for the ropelen…

Study applies HRP to Latin American markets, showing smoother risk-return profile.

Deep model predicts shapes of curves with multiple covariates.

GDM models time series with smoother transitions and interpretable states.