Paper uses tensor completion to estimate HVAC fan power baselines.
problem Estimating HVAC fan power without demand response.
method Tensor completion for multi-dimensional data analysis.
result Tensor completion outperforms existing baselining methods.
Previous work has shown the unreliability of existing algorithms in the batch Reinforcement Learning setting, and proposed the theoretically-grounded Safe Policy Improvement with Baseline Bootstrapping (SPIBB) fix: reproduce the baseline policy in the uncertain state-action pairs, in order to control the variance on th…
Paper tackles online learning for DR management with incentives.
problem Estimating baseline consumption in DR programs with consumer incentives.
method Online learning scheme using least-squares with perturbed reward prices.
result Achieves low regret of $\mathcal{O}\left((\log{T})^2
ight)$ compared to optimal.
Policy gradient methods are a widely used class of model-free reinforcement learning algorithms where a state-dependent baseline is used to reduce gradient estimator variance. Several recent papers extend the baseline to depend on both the state and action and suggest that this significantly reduces variance and improv…
BASIS improves LLM reasoning by sharing batchwise rollout info, reducing MSE by 69%.
problem Improving large language model reasoning with limited rollouts and batch information.
method BASIS samples only one rollout per prompt but uses batch information to improve value function estimation.
result BASIS reduces MSE in value function estimation by 69% compared to REINFORCE++.
Two-stage TMLE reduces bias and improves efficiency in CRTs.
problem Differential outcome measurement and imbalance in baseline predictors in CRTs.
method Two-stage targeted minimum loss-based estimator (TMLE) to adjust for baseline covariates.
result Our approach nearly eliminates bias due to differential outcome measurement.
C-SURE improves complex-valued deep learning models by shrinking estimates, outperforming MLE and SurReal.
problem Improving accuracy and robustness of complex-valued deep learning models.
method Proposes a Stein's unbiased risk estimate (SURE) for complex-valued data and integrates it into a prototype CNN classifier.
result C-SURE outperforms SurReal and MLE in accuracy and robustness on complex-valued datasets.
New linear algorithms improve wSVMs for multiclass probability estimation.
problem Estimating conditional probabilities for multiclass problems.
method Proposed baseline learning and OVA learning schemes to improve wSVMs.
result Linear algorithms achieve optimal computational efficiency and good estimation accuracy.
Study shows targeting students with intermediate predicted outcomes is most effective for financial aid renewal.
problem Determining which students to target for financial aid renewal to maximize effectiveness.
method Used causal forest to estimate heterogeneous treatment effects and targeted students accordingly; compared targeting low vs high predicted probability outcomes.
result Targeting students with intermediate predicted outcomes yields the highest effectiveness in financial aid renewal.
We show a connection between the Fourier spectrum of Boolean functions and the REINFORCE gradient estimator for binary latent variable models. We show that REINFORCE estimates (up to a factor) the degree-1 Fourier coefficients of a Boolean function. Using this connection we offer a new perspective on variance reduction…
There exist a number of reinforcement learning algorithms which learnby climbing the gradient of expected reward. Their long-runconvergence has been proved, even in partially observableenvironments with non-deterministic actions, and without the need fora system model. However, the variance of the gradient estimator ha…
Bayesian Scattering offers a simple baseline for image data uncertainty.
problem Lack of interpretable, mathematically grounded uncertainty quantification methods for image data.
method Coupling wavelet scattering transform with a simple probabilistic head.
result Bayesian Scattering provides sensible uncertainty estimates under distribution shifts.
BAMS uses Bayesian sampling to discover AV failures more efficiently and accurately.
problem Discovering potential failure cases in autonomous vehicles efficiently and accurately.
method Bayesian adaptive multifidelity sampling (BAMS) prioritizes exploration of low performance regions.
result BAMS discovers 10 times more issues than traditional methods with narrower rate estimates.
Policy gradient methods have enjoyed great success in deep reinforcement learning but suffer from high variance of gradient estimates. The high variance problem is particularly exasperated in problems with long horizons or high-dimensional action spaces. To mitigate this issue, we derive a bias-free action-dependent ba…
Tabular foundation models outperform other methods in conditional density estimation across various datasets.
problem Estimating the full conditional distribution of a response given tabular covariates, especially in settings with heteroscedasticity, multimodality, or asymmetric uncertainty.
method Benchmarked three tabular foundation model variants (TabPFN and TabICL) against six CDE baselines on 39 real-world datasets.
result Tabular foundation models achieve the best CDE loss, log-likelihood, and CRPS across all sample sizes, outperforming other methods.
Extracting and detecting spike activities from the fluorescence observations is an important step in understanding how neuron systems work. The main challenge lies in that the combination of the ambient noise with dynamic baseline fluctuation, often contaminates the observations, thereby deteriorating the reliability o…
In classical Hawkes process, the baseline intensity and triggering kernel are assumed to be a constant and parametric function respectively, which limits the model flexibility. To generalize it, we present a fully Bayesian nonparametric model, namely Gaussian process modulated Hawkes process and propose an EM-variation…
Improved A/B testing by leveraging system similarities.
problem Traditional A/B testing ignores potential system similarities.
method Off-policy estimation to exploit system propensities.
result Improved A/B testing estimators achieve better accuracy.
WaveGrad generates high-fidelity audio using gradient estimation.
problem Generating high-fidelity audio efficiently.
method Conditional model using score matching and diffusion models, iteratively refining a Gaussian white noise signal.
result WaveGrad can generate high-fidelity audio samples using as few as six iterations.
A new model predicts discrete events with flexible, nonparametric baseline and excitation.
problem Limited flexibility in discrete Hawkes models for event prediction.
method Gaussian Process Discrete Hawkes Process (GP-DHP) with collapsed latent representation.
result Improves predictive log-likelihood for diverse event patterns.
Unified framework for stable RL learning with theoretical guarantees.
problem Lack of systematic theoretical principles guiding RL post-training methods.
method Unified theoretical framework for policy-gradient estimators and optimization algorithms.
result Establishes unbiasedness, variance expressions, and convergence guarantees.
STAR framework reduces OPE variance by distilling complex problems into discrete ARPs.
problem High variance and bias in off-policy evaluation methods.
method STAR framework that includes various OPE estimators and leverages state abstraction.
result Predictions from ARPs estimated from off-policy data are asymptotically correct.
Paper identifies problematic baselines in Shapley value explanations and proposes a reweighting mechanism.
problem Identifying and addressing the suboptimality of baselines in Shapley value feature importance analysis.
method Analyzed suboptimality of baselines, identified problematic baseline, generalized uninformativeness, and designed a reweighting mechanism.
result Proposed uncertainty-based reweighting mechanism effectively accelerates computation and improves explanation quality.
In traditional reinforcement learning, an agent maximizes the reward collected during its interaction with the environment by approximating the optimal policy through the estimation of value functions. Typically, given a state s and action a, the corresponding value is the expected discounted sum of rewards. The optima…
Modeling high-frequency order book data with Hawkes-Markovian process.
problem Capturing the dynamics of high-frequency order book events.
method Hawkes process with Markovian baseline intensities, LASSO regularization, and Akaike Information Criteria.
result Effective modeling of order book dynamics with reduced parameter redundancy.
UCB algorithms estimate uplifts in multi-variable reward systems.
problem Marketing campaigns and recommender systems with multiple outcome variables.
method UCB-style algorithms for estimating uplifts over a baseline.
result Sublinear regret bounds for multiple problem variants.
The paper reveals that baselines significantly impact RL algorithms' convergence.
problem Understanding the true impact of baselines on policy optimization.
method Theoretical analysis of bandit and RL problems, focusing on natural policy gradient and EXP3.
result Baselines can determine algorithm convergence, contradicting traditional optimization theory.
We introduce a Cox-type model for relative intensities of orders flows in a limit order book. The model assumes that all intensities share a common baseline intensity, which may for example represent the global market activity. Parameters can be estimated by quasi likelihood maximization, without any interference from …
We provide new approximation guarantees for greedy low rank matrix estimation under standard assumptions of restricted strong convexity and smoothness. Our novel analysis also uncovers previously unknown connections between the low rank estimation and combinatorial optimization, so much so that our bounds are reminisce…
Due to the high variance of policy gradients, on-policy optimization algorithms are plagued with low sample efficiency. In this work, we propose Augment-Reinforce-Merge (ARM) policy gradient estimator as an unbiased low-variance alternative to previous baseline estimators on tasks with binary action space, inspired by …
Node-perturbation learning is a type of statistical gradient descent algorithm that can be applied to problems where the objective function is not explicitly formulated, including reinforcement learning. It estimates the gradient of an objective function by using the change in the object function in response to the per…
DSI improves tail-risk estimation in generative models by averaging checkpoints.
problem Generative models' instability in rare adverse scenarios.
method Diachronic Sample Integration (DSI) ensembles generated samples across checkpoints.
result DSI reduces tail-estimation error compared to single-checkpoint baselines.
M3E2 neural network estimates multiple treatment effects.
problem Estimating effects of multiple treatments simultaneously.
method Multi-task learning neural network model for multiple treatments, continuous and binary.
result M3E2 outperforms baselines in synthetic datasets.
New method estimates hazard ratios without bias in observational studies.
problem Uninterpretable hazard ratios due to unspecified baseline hazard.
method Kernel-based machine learning to model risk set changes.
result Debiased maximum-likelihood estimators identify true hazard ratios.
ActiveCQ improves causal quantity estimation with active learning and Gaussian Processes.
problem Estimating causal quantities requires large datasets, which are costly.
method Unified framework using Gaussian Processes and conditional mean embeddings for distribution estimation. Derived principled acquisition strategies based on information gain and total variance reduction.
result Framework significantly outperforms baselines in sample efficiency across various causal quantities.
Survival forests estimate treatment effects with censored data.
problem Estimating treatment effects in survival analysis with censored data.
method Causal survival forests using orthogonal estimating equations.
result Survival forests perform well relative to baselines in treatment effect estimation.
VSCOUT detects anomalies in high-dimensional data using a hybrid VAE approach.
problem Challenges in classical SPC for high-dimensional, non-Gaussian data.
method Hybrid VAE architecture with ARD prior, ensemble filtering, and changepoint detection.
result VSCOUT achieves superior sensitivity to special-cause structure and controlled false alarms.
We consider network sparsification as an L0-norm regularized binary optimization problem, where each unit of a neural network (e.g., weight, neuron, or channel, etc.) is attached with a stochastic binary gate, whose parameters are jointly optimized with original network parameters. The Augment-Reinforce-Merge (ARM),…
Fundamental frequency (f0) estimation from polyphonic music includes the tasks of multiple-f0, melody, vocal, and bass line estimation. Historically these problems have been approached separately, and only recently, using learning-based approaches. We present a multitask deep learning architecture that jointly estimate…
Improved IV estimates by weighting on compliance reduces noise in treatment effect estimation.
problem Noisy IV estimates in settings with non-random treatment receipt.
method Weighting observations by estimated compliance, leveraging machine learning for compliance estimation.
result Compliance weighting reduces IV variance, improving precision of treatment effect estimates.
This work establishes safe reinforcement learning for LQR with nonlinear baselines.
problem Safe reinforcement learning in LQR with unknown dynamics and safety constraints.
method General framework for nonlinear baselines, focusing on 1D spaces.
result Achieves optimal regret bounds for constrained reinforcement learning.
This paper tackles the reduction of redundant repeating generation that is often observed in RNN-based encoder-decoder models. Our basic idea is to jointly estimate the upper-bound frequency of each target vocabulary in the encoder and control the output words based on the estimation in the decoder. Our method shows si…
Curiosity-Critic improves world model training by focusing on cumulative prediction error.
problem Training world models with intrinsic rewards that consider cumulative prediction error.
method Curiosity-Critic uses a surrogate reward based on the difference between current and asymptotic prediction errors, estimated online by a co-trained critic.
result Curiosity-Critic outperforms other methods in training speed and final world model accuracy.
Proposes HetSNGP method for joint model and data uncertainty modeling.
problem Uncertainty estimation in deep learning for safety-critical applications.
method Jointly models model and data uncertainty with HetSNGP method.
result Outperforms baseline methods on challenging out-of-distribution datasets.
ALIEN improves uncertainty estimation of language models by refining entropy-based methods.
problem Overconfidence in uncertainty estimation for language models, especially for difficult inputs.
method ALIEN refines entropy-based uncertainty by aligning it with prediction reliability, using a lightweight uncertainty head.
result ALIEN consistently outperforms strong baselines in detecting incorrect predictions and achieving the lowest calibration error.
Second-order estimator improves continuous-time policy evaluation.
problem Estimating value surfaces from discrete data with time-inhomogeneous dynamics.
method Moment-matching coefficients for high-order generator regression.
result Second-order estimator consistently outperforms Bellman baseline.
BODE enhances deep neural network predictions and uncertainty quantification in safety modeling.
problem Uncertainty in deep neural network predictions for safety-critical applications.
method Bayesian optimization combined with deep ensembles (BODE).
result BODE reduces total uncertainty by over 30% compared to a manually tuned baseline ensemble.
We design, conduct and present the results of a highly personalized baseline emotion recognition experiment, which aims to set reliable ground-truth estimates for the subject's emotional state for real-life prediction under similar conditions using a small number of physiological sensors. We also propose an adaptive st…