New nonlinear smoothers improve state estimation in chaotic systems.
problem Improving state estimation in chaotic dynamical systems with non-Gaussian behavior.
method Developed nonlinear backward ensemble transport smoothers with parameterization and regularization of transport maps.
result Nonlinear smoothers yield lower estimation error than conventional methods for comparable model evaluations.
Combining deep learning and ensemble smoothers for better history matching.
problem Dealing with complex facies distributions in history matching.
method Using autoencoders and generative adversarial networks to parameterize facies models, applying distance-based localization.
result Improved history matching performance with deep learning parameterizations.
EnKBS smoothes complex systems with future observations for causal inference.
problem Improving state estimation in complex systems with rapid dynamics.
method Continuous-time ensemble Kalman-Bucy smoother for nonlinear dynamical systems.
result EnKBS provides derivative-free framework with high skill in various scientific problems.
Unified framework for ensemble transport-based smoothing of non-Gaussian time series.
problem Bayesian time series re-analysis with non-Gaussian distributions.
method Measure transport approach to derive consistent prior-to-posterior transformations.
result General ensemble framework for transport-based smoothing of state-space models.
Paper explains how tree ensembles improve predictions by smoothing and regulating smoothness.
problem Understanding why tree ensembles perform well despite their complexity.
method Interpreting tree ensembles as adaptive and self-regularizing smoothers.
result Ensemble trees make more smooth predictions than individual trees and adjust smoothness based on input dissimilarity.
Workflow uses deep learning to improve geosteering accuracy in Goliat Field.
problem Uncertainty in geological models and forward simulations affects real-time estimations.
method Offline DNN training, online FlexIES with probabilistic estimation.
result Median probabilistic estimation matches proprietary inversion.
Forest-guided smoothing uses random forest outputs for interpretable local smoothers.
problem Creating interpretable local smoothers from complex random forest outputs.
method Uses random forest outputs to define spatially adaptive bandwidth matrices for a linear smoother.
result Improves interpretability and applicability of random forest outputs for various analyses.
Study uses DNNs for real-time EM inversion, highlighting model errors and proposing solutions.
problem Model errors in DNNs affect real-time geosteering decisions in EM measurements.
method Bayesian ensemble smoothing with DNNs for thousands of model evaluations, identifying multimodality.
result Model errors can lead to biased estimates, necessitating error reduction techniques.
A new algorithm HTE for large-scale regression improves accuracy compared to single estimators.
problem Improving accuracy in large-scale regression problems.
method Histogram transform ensembles (HTE) with random transformations and kernel histogram transforms (KHT).
result Ensemble HTE outperforms single estimators in accuracy for various Hölder spaces.
Data assimilation for subsurface flow using latent diffusion models shows that ensemble Kalman methods may overestimate posterior uncertainty, while Monte Carlo sampling is more reliable.
problem Data assimilation for subsurface flow
method Ensemble Kalman smoother and Markov chain Monte Carlo sampling
result Monte Carlo sampling is more reliable than ensemble Kalman methods
The literature about history matching is vast and despite the impressive number of methods proposed and the significant progresses reported in the last decade, conditioning reservoir models to dynamic data is still a challenging task. Ensemble-based methods are among the most successful and efficient techniques current…
Flexible tree ensemble learning framework supports arbitrary loss functions and multi-task learning.
problem Limited modeling capabilities of existing tree ensemble learning toolkits.
method Differentiable tree ensembles with tensor-based formulation for efficient training.
result Our framework leads to 100x more compact and 23% more expressive tree ensembles.
Estimating the state of a dynamical system from a series of noise-corrupted observations is fundamental in many areas of science and engineering. The most well-known method, the Kalman smoother (and the related Kalman filter), relies on assumptions of linearity and Gaussianity that are rarely met in practice. In this p…
New methods improve tree ensemble models by compressing them while maintaining accuracy.
problem Theoretical understanding and practical compression of tree ensembles like random forests and gradient boosting machines.
method Spectral perspective on tree ensembles, deriving minimax rates and developing compression schemes.
result Leading eigenfunctions/singular vectors capture dominant predictive directions, leading to smaller, competitive models.
This paper studies recursive ensembles driven by Fibonacci updates, improving learning dynamics.
problem Improving learning dynamics in recursive ensemble learning.
method Develops second-order recursive architectures with Fibonacci-type update flows.
result Establishes global convergence conditions and generalization bounds for recursive ensembles.
Histogram transform ensembles improve density estimation accuracy.
problem Improving density estimation accuracy for various distributions.
method Histogram transform ensembles (HTE) with theoretical analysis and experimental validation.
result HTE outperforms single histogram transforms and offers almost optimal convergence rates in Hölder space C0,α. In this article, we propose a novel ensemble technique with a multi-scheme weighting based on a technique called coopetitive soft gating. This technique combines both, ensemble member competition and cooperation, in order to maximize the overall forecasting accuracy of the ensemble. The proposed algorithm combines the …
We present a Kalman smoothing framework based on modeling errors using the heavy tailed Student's t distribution, along with algorithms, convergence theory, open-source general implementation, and several important applications. The computational effort per iteration grows linearly with the length of the time series, a…
Spatial smoothing improves BNNs' accuracy, uncertainty, and robustness without increasing computational cost.
problem Large ensembles in BNNs increase computational cost and reduce performance.
method Spatial smoothing adds blur layers to convolutional neural networks to ensemble neighboring feature map points.
result Spatial smoothing improves BNNs' performance with fewer ensembles and enhances robustness.
Recent studies have shown that the aggregated dynamic flexibility of an ensemble of thermostatic loads can be modeled in the form of a virtual battery. The existing methods for computing the virtual battery parameters require the knowledge of the first-principle models and parameter values of the loads in the ensemble.…
AdaNet is a lightweight TensorFlow-based (Abadi et al., 2015) framework for automatically learning high-quality ensembles with minimal expert intervention. Our framework is inspired by the AdaNet algorithm (Cortes et al., 2017) which learns the structure of a neural network as an ensemble of subnetworks. We designed it…
BI-EqNO improves Bayesian inference with flexible neural operators.
problem Inaccurate estimation of marginal likelihoods in approximate Bayesian methods.
method Equivariant neural operator framework for generalized approximate Bayesian inference.
result BI-EqNO enhances both deterministic and stochastic approaches to Bayesian inference.
Combines deep generative models with ensemble methods for subsurface property estimation.
problem Estimating spatially distributed subsurface properties from sparse measurements.
method Wasserstein Generative Adversarial Network (WGAN-GP) and Ensemble Smoother with Multiple Data Assimilation (ES-MDA).
result The proposed method outperforms variational inversion methods, especially for channelized and fractured fields.
This paper presents a general iterative bias correction procedure for regression smoothers. This bias reduction schema is shown to correspond operationally to the L2 Boosting algorithm and provides a new statistical interpretation for L2 Boosting. We analyze the behavior of the Boosting algorithm applied to commo…
Method approximates Lipschitz domains with smoother shapes.
problem Approximating bounded Lipschitz domains.
method Sequence of smooth, bounded domains with weak curvatures.
result Uniform isocapacitary estimates for approximating sets.
Exponential smoothers are a simple and memory efficient way to compute running averages of time series. Here we define and describe practical properties of exponential smoothers for signals observed at constant and variable intervals.
ControlBurn selects few features from tree ensembles for better model interpretability.
problem Improving model interpretability in machine learning models.
method Sparse tree ensembles with lasso optimization.
result ControlBurn selects feature-sparse subsets for better model interpretability.
CDST improves ensemble prediction by adjusting model weights based on covariates.
problem Improving ensemble prediction accuracy in complex scenarios.
method Covariate-dependent stacking (CDST) with flexible model weights estimated via cross-validation.
result CDST consistently outperforms conventional model averaging methods in complex datasets.
dSMC improves parallel processing of state-space models.
problem Processing multiple observations efficiently in state-space models.
method A parallel-in-time particle smoother that reduces complexity to log(T).
result dSMC achieves O(log(T)) time complexity on parallel architectures.
Generative adversarial network improves geosteering in fluvial reservoirs.
problem Improving geosteering in complex reservoirs with high uncertainties.
method Generative adversarial deep neural network (GAN) trained to model fluvial successions.
result Reduces uncertainty and correctly predicts geological features up to 500 meters ahead of drill-bit.
Paper improves training physics-informed neural networks with model ensembles.
problem Training physics-informed neural networks (PINNs) is difficult due to convergence to wrong solutions.
method Proposes training an ensemble of PINNs, using ensemble agreement to expand the solution interval.
result Algorithm stabilizes PINN training and yields competitive performance.
The recent success of Deep Neural Networks (DNNs) has drastically improved the state of the art for many application domains. While achieving high accuracy performance, deploying state-of-the-art DNNs is a challenge since they typically require billions of expensive arithmetic computations. In addition, DNNs are typica…
Ensemble quantile classifier improves performance on high-dimensional data.
problem Discriminating high-dimensional data with heavy-tailed or skewed inputs.
method Regularized quantile classifier that assigns variable weights.
result Consistently estimates minimal population loss and is Bayes optimal.
Study models weather index insurance pricing by insurers and farmers, finding flexible pricing kernels boost profits.
problem Monopoly pricing of weather index insurance with risk and flexibility considerations.
method Bowley-type sequential game with insurer and farmer, using neural networks for farmer's payoff.
result Flexible pricing kernels increase insurer profits closer to indemnity insurance levels.
MAC combines models without locking them, improving ensemble performance.
problem Improving ensemble learning performance with flexibility.
method Model agnostic combination technique that dynamically combines models.
result MAC outperforms classical methods and competitive to boosting.
New ensemble models classify mouse movement trajectories to assess survey question difficulty.
problem Assessing survey question difficulty based on respondents' interaction data.
method Ensemble models combining semi-metric-based weak learners to classify multivariate functional data.
result Improved survey data quality through better identification of respondent difficulty.
We present power low rank ensembles (PLRE), a flexible framework for n-gram language modeling where ensembles of low rank matrices and tensors are used to obtain smoothed probability estimates of words in context. Our method can be understood as a generalization of n-gram modeling to non-integer n, and includes standar…
Ensembles of decision trees perform well on many problems, but are not interpretable. In contrast to existing approaches in interpretability that focus on explaining relationships between features and predictions, we propose an alternative approach to interpret tree ensemble classifiers by surfacing representative poin…
ATHENA builds flexible defenses against adversarial attacks.
problem Extensive research on adversarial attacks is domain-specific and cannot be easily extended.
method Designing an extensible framework based on diverse weak defenses.
result Comprehensive empirical study demonstrates the effectiveness of ATHENA.
Neural architecture search (NAS) is gaining more and more attention in recent years due to its flexibility and remarkable capability to reduce the burden of neural network design. To achieve better performance, however, the searching process usually costs massive computations that might not be affordable for researcher…
When randomized ensemble methods such as bagging and random forests are implemented, a basic question arises: Is the ensemble large enough? In particular, the practitioner desires a rigorous guarantee that a given ensemble will perform nearly as well as an ideal infinite ensemble (trained on the same data). The purpose…
Parallel-in-time solver reduces ODE simulation time from linear to logarithmic.
problem Efficiently solving ordinary differential equations (ODEs) with reduced computational cost.
method Formulated a parallel-in-time probabilistic numerical ODE solver using time-parallel formulation of iterated extended Kalman smoothers.
result Reduces span cost from linear to logarithmic in the number of time steps.
Vote-boosting is a sequential ensemble learning method in which the individual classifiers are built on different weighted versions of the training data. To build a new classifier, the weight of each training instance is determined in terms of the degree of disagreement among the current ensemble predictions for that i…
Decentralized Gaussian processes for multi-agent systems.
problem Scalable and flexible learning solutions for multi-agent systems.
method Asymptotically exact decentralized solution to Gaussian processes, with online Bayesian model averaging for hyperparameter selection.
result Asymptotically exact decentralized Gaussian process approximation and online Bayesian model averaging.
New algorithm reduces overfitting in neural networks.
problem Overfitting in neural networks.
method Integrates SMC with SGHMC for mini-batch sampling.
result SMCSGHMC outperforms SGD and deep ensembles.
Proposes a new method for ensembling neural subnetworks.
problem Computational expense and limited flexibility of traditional deep ensembles.
method Sequential Bayesian neural subnetwork ensembling.
result Outperforms traditional ensembles in various metrics.
Ensembling smaller models can outperform larger models in terms of accuracy and efficiency.
problem The inefficiency of training larger models for boosting performance.
method Training ensembles of smaller models and comparing their performance to single larger models.
result Ensembles of smaller models outperform single larger models in accuracy and efficiency, especially as models grow large.
We consider a self-exciting counting process, the parameters of which depend on a hidden finite-state Markov chain. We derive the optimal filter and smoother for the hidden chain based on observation of the jump process. This filter is in closed form and is finite dimensional. We demonstrate the performance of this fil…