Forest-guided smoothing uses random forest outputs for interpretable local smoothers.
problem Creating interpretable local smoothers from complex random forest outputs.
method Uses random forest outputs to define spatially adaptive bandwidth matrices for a linear smoother.
result Improves interpretability and applicability of random forest outputs for various analyses.
We introduce a new algorithm, called adaptive sparse backfitting algorithm, for solving high dimensional Sparse Additive Model (SpAM) utilizing symmetric, non-negative definite smoothers. Unlike the previous sparse backfitting algorithm, our method is essentially a block coordinate descent algorithm that guarantees to …
Paper explains how tree ensembles improve predictions by smoothing and regulating smoothness.
problem Understanding why tree ensembles perform well despite their complexity.
method Interpreting tree ensembles as adaptive and self-regularizing smoothers.
result Ensemble trees make more smooth predictions than individual trees and adjust smoothness based on input dissimilarity.
We present a Kalman smoothing framework based on modeling errors using the heavy tailed Student's t distribution, along with algorithms, convergence theory, open-source general implementation, and several important applications. The computational effort per iteration grows linearly with the length of the time series, a…
New nonlinear smoothers improve state estimation in chaotic systems.
problem Improving state estimation in chaotic dynamical systems with non-Gaussian behavior.
method Developed nonlinear backward ensemble transport smoothers with parameterization and regularization of transport maps.
result Nonlinear smoothers yield lower estimation error than conventional methods for comparable model evaluations.
We introduce a family of adaptive estimators on graphs, based on penalizing the ℓ1 norm of discrete graph differences. This generalizes the idea of trend filtering [Kim et al. (2009), Tibshirani (2014)], used for univariate nonparametric regression, to graphs. Analogous to the univariate case, graph trend filteri…
This paper presents a general iterative bias correction procedure for regression smoothers. This bias reduction schema is shown to correspond operationally to the L2 Boosting algorithm and provides a new statistical interpretation for L2 Boosting. We analyze the behavior of the Boosting algorithm applied to commo…
Exponential smoothers are a simple and memory efficient way to compute running averages of time series. Here we define and describe practical properties of exponential smoothers for signals observed at constant and variable intervals.
dSMC improves parallel processing of state-space models.
problem Processing multiple observations efficiently in state-space models.
method A parallel-in-time particle smoother that reduces complexity to log(T).
result dSMC achieves O(log(T)) time complexity on parallel architectures.
Combining deep learning and ensemble smoothers for better history matching.
problem Dealing with complex facies distributions in history matching.
method Using autoencoders and generative adversarial networks to parameterize facies models, applying distance-based localization.
result Improved history matching performance with deep learning parameterizations.
AutoClip automatically adjusts gradient clipping for better audio separation.
problem Improving generalization in audio source separation networks.
method Adaptive gradient clipping based on historical gradient norms.
result Improves generalization performance in audio source separation networks.
We consider a self-exciting counting process, the parameters of which depend on a hidden finite-state Markov chain. We derive the optimal filter and smoother for the hidden chain based on observation of the jump process. This filter is in closed form and is finite dimensional. We demonstrate the performance of this fil…
EnKBS smoothes complex systems with future observations for causal inference.
problem Improving state estimation in complex systems with rapid dynamics.
method Continuous-time ensemble Kalman-Bucy smoother for nonlinear dynamical systems.
result EnKBS provides derivative-free framework with high skill in various scientific problems.
Improved Thompson Sampling for smoother functions with noise.
problem Applying Thompson Sampling to continuum armed bandits with weak conditions.
method Analysis of eluder dimension for function classes with smooth derivatives.
result New bounds on eluder dimension for classes of functions with Lipschitz derivatives.
We present a general probabilistic perspective on Gaussian filtering and smoothing. This allows us to show that common approaches to Gaussian filtering/smoothing can be distinguished solely by their methods of computing/approximating the means and covariances of joint probabilities. This implies that novel filters and …
Convolutional Neural Networks (CNN) and the locally connected layer are limited in capturing the importance and relations of different local receptive fields, which are often crucial for tasks such as face verification, visual question answering, and word sequence prediction. To tackle the issue, we propose a novel loc…
Develops a novel ML smoothing method for incomplete data in state-space models.
problem Estimating states in stochastic systems with incomplete information.
method Introduces score function and conditional observed information matrices for incomplete data, and uses them to derive the ML smoother.
result The ML smoother provides more accurate state estimates with lower standard errors compared to the standard ML state estimator.
ResNets promote smoother interpolations than MLPs, enhancing generalization.
problem Understanding the difference in smoothness between ResNets and MLPs.
method Neural Tangent Kernel (NTK) analysis during gradient descent training.
result ResNet's NTK results in smoother interpolations than MLPs.
Bayesian convolutional deep sets improve ambiguity in stationary process modeling.
problem Ambiguity in translation equivariant functional representations due to insufficient data points.
method Introduce Bayesian convolutional deep sets with task-dependent stationary prior.
result Improves representation quality compared to kernel smoother and non-parametric models.
Auto-regressive models improve smoothing efficiency with exponentially tapered windows.
problem Improving time-series smoothing efficiency.
method An auto-regressive formulation for time-series smoothing.
result Auto-regressive models result in moving means with exponentially tapered windows.
Paper proposes a new Taylor moment expansion for non-linear Gaussian filtering and smoothing.
problem Non-linear Gaussian filtering and smoothing in continuous-discrete state-space models.
method Taylor moment expansion (TME) for moment functions directly and in time variable.
result Significantly outperforms state-of-the-art methods in terms of estimation accuracy and numerical stability.
New method for robust fixed-point smoothing without state augmentation.
problem Estimating initial states in Gaussian smoothing algorithms.
method Cholesky-based formulation without state augmentation.
result Matches runtime and robustness of existing methods.
Estimating the state of a dynamical system from a series of noise-corrupted observations is fundamental in many areas of science and engineering. The most well-known method, the Kalman smoother (and the related Kalman filter), relies on assumptions of linearity and Gaussianity that are rarely met in practice. In this p…
The study examines the dynamic behavior of RMSprop and Adam algorithms.
problem Understanding the training loss curve patterns of adaptive gradient algorithms.
method Careful numerical experiments and theoretical explanations using the signGD flow.
result Adam converges smoother and faster when momentum factors are close to each other.
Unified framework for ensemble transport-based smoothing of non-Gaussian time series.
problem Bayesian time series re-analysis with non-Gaussian distributions.
method Measure transport approach to derive consistent prior-to-posterior transformations.
result General ensemble framework for transport-based smoothing of state-space models.
An unsupervised learning algorithm to cluster hyperspectral image (HSI) data is proposed that exploits spatially-regularized random walks. Markov diffusions are defined on the space of HSI spectra with transitions constrained to near spatial neighbors. The explicit incorporation of spatial regularity into the diffusion…
The paper explores how structured representations influence learning dynamics in neural networks.
problem Understanding the training dynamics of deep neural networks.
method Investigates a family of enriched transformation layers with constrained pathways and adaptive corrections.
result Improved robustness, smoother optimization, and scalable depth behavior are achieved through structured representations.
Paper uses SLT to improve model selection for SHM.
problem Model selection for SHM using data-based systems.
method Utilizes Statistical Learning Theory to rigorously estimate generalisation.
result Incorporating domain knowledge improves model generalisation.
Proposes a new framework for investing that adapts to market regimes.
problem Adapting to dynamic market regimes for better investment performance.
method Wasserstein Hidden Markov Model (HMM) with transaction-cost-aware optimization.
result Significantly higher risk-adjusted performance compared to benchmarks.
Unified framework for efficient Gaussian process inference.
problem Efficient inference in non-conjugate Gaussian process models.
method Combines expectation propagation with linearization for improved efficiency.
result Unified view of various inference schemes, including classical smoothers and EP.
Paper uses UKS to improve BLE RSSI for proximity inference in mobile phone apps.
problem Improper BLE RSSI for accurate proximity inference during pandemics.
method Single-dimensional Unscented Kalman Smoother (UKS) with Gaussian process observation transforms.
result UKS outperforms traditional methods in predicting infection risk from BLE RSSI.
Generative models use kernel smoothing for conditioning on small example sets.
problem Improving generative models' performance with limited conditioning examples.
method Showed that cross-attention conditioning is equivalent to kernel smoothing, specifically a Nadaraya--Watson kernel smoother.
result The approach predicts and confirms three failure regimes for kernel-based conditioning.
Generative models learn smoother densities to sample from unknown distributions.
problem Sampling from unknown distributions in high-dimensional spaces.
method Formalizes sampling problem, introduces multimeasurement noise model, derives Bayes estimator, and uses underdamped Langevin MCMC.
result Formulation leads to efficient sampling methods and theoretical connections with denoising autoencoders.
We present a framework to train a structured prediction model by performing smoothing on the inference algorithm it builds upon. Smoothing overcomes the non-smoothness inherent to the maximum margin structured prediction objective, and paves the way for the use of fast primal gradient-based optimization algorithms. We …
A new algorithm HTE for large-scale regression improves accuracy compared to single estimators.
problem Improving accuracy in large-scale regression problems.
method Histogram transform ensembles (HTE) with random transformations and kernel histogram transforms (KHT).
result Ensemble HTE outperforms single estimators in accuracy for various Hölder spaces.
AngularGrad optimizes CNNs by considering gradient direction, improving convergence.
problem Dying gradient problem and inefficiency in exploiting gradient curvature.
method AngularGrad considers gradient direction/angle, generating a score for step size control.
result AngularGrad outperforms state-of-the-art optimizers in benchmark tests.
Bayesian approach improves neural network recurrence.
problem Improving neural network recurrence mechanisms.
method Introducing Bayesian recurrence relations and gates.
result Bayesian approach can perform as well as or better than conventional recurrent networks.
We study reproducing kernel Hilbert spaces (RKHS) on a Riemannian manifold. In particular, we discuss under which condition Sobolev spaces are RKHS and characterize their reproducing kernels. Further, we introduce and discuss a class of smoother RKHS that we call diffusion spaces. We illustrate the general results with…
Robo-advisors use MPC to create dynamic investment strategies.
problem Static allocation methods limit robo-advisors' effectiveness.
method Combines MPC with Hidden Markov Model and Black-Litterman for dynamic asset allocation.
result MPC-based strategies outperform static approaches in dynamic and risk-budgeting criteria.
New method estimates and optimizes policy differences using orthogonal learning.
problem Offline reinforcement learning with safety concerns and cost limitations.
method Dynamic R-learner for estimating and optimizing Qπ(s,1)−Qπ(s,0), leveraging orthogonal estimation. result Consistent policy optimization with improved convergence rates.
Unified approach to multiclass classification using Gabriel graphs.
problem Improving multiclass classification accuracy and efficiency.
method Integrates Gabriel graphs for binary and multiclass classification, proposing new activation functions and support edge neurons.
result Experimental results show superior performance compared to previous GG-based classifiers.
Reliable 4D aircraft trajectory prediction, whether in a real-time setting or for analysis of counterfactuals, is important to the efficiency of the aviation system. Toward this end, we first propose a highly generalizable efficient tree-based matching algorithm to construct image-like feature maps from high-fidelity m…
We describe notions of tautness that arise in the study of C0 foliations, C1,0 or smoother foliations, and in geometry. We give examples to show that these notions are different, and discuss how these differences impact some classical foliation results. We construct examples of smoothly taut C∞,0 foli…
Histogram transform ensembles improve density estimation accuracy.
problem Improving density estimation accuracy for various distributions.
method Histogram transform ensembles (HTE) with theoretical analysis and experimental validation.
result HTE outperforms single histogram transforms and offers almost optimal convergence rates in Hölder space C0,α. New method calibrates LV surfaces for exotic derivatives with smoother, more stable Greeks.
problem Challenges in LV calibration leading to spiky surfaces and unstable Greeks.
method Automatic local regression to pre-process market observables and smooth LV surfaces.
result Significantly smoother LV surfaces and greatly improved Greek stability with negligible additional cost.
The ropelength of a knot is the quotient of its length and its thickness, the radius of the largest embedded normal tube around the knot. We prove existence and regularity for ropelength minimizers in any knot or link type; these are C1,1 curves, but need not be smoother. We improve the lower bound for the ropelen…
Study applies HRP to Latin American markets, showing smoother risk-return profile.
problem Lack of empirical analyses of HRP in Latin American markets.
method Hierarchical Risk Parity (HRP) with hierarchical clustering and recursive bisection.
result HRP portfolio outperforms Max Sharpe portfolio in NUAM markets, with smoother risk-return profile.
Deep model predicts shapes of curves with multiple covariates.
problem Predicting shapes of planar curves with various covariates.
method Deep learning model using complex-valued functions, conditional covariance smoother with modality-specific encoders.
result Model accurately predicts shapes of curves with multimodal covariates.