New method combines experimental and observational data for causal inference.
problem Combining internal validity of experiments and larger sample sizes of observations.
method Empirical risk minimization (ERM) framework with cross-validation.
result Efficacy and reliability demonstrated on real and synthetic data.
Estimates personalized policies robust to shifts in target populations.
problem Estimating policies that perform well in diverse target populations.
method Develops methods for estimating robust policies considering shifts in outcomes and characteristics.
result Welfare-maximizing policies are robust to certain shifts in potential outcomes.
LOOCV is often useful for analyzing small, structured experimental designs.
problem The effectiveness of cross-validation in analyzing designed experiments.
method Empirical study comparing LOOCV and other model selection methods.
result LOOCV is often useful in the analysis of small, structured experimental designs.
Theoretical limits show experimental data can falsify but not validate causal estimates from observational studies.
problem Fundamental limits on validating causal estimates using experimental data in observational studies.
method Impossible inference framework, Gaussian Process based approach.
result Experimental data can falsify but not validate causal estimates from observational studies.
New framework validates counterfactual estimations in network interference settings.
problem Challenges in causal effect estimation and validation in network interference settings.
method Introduces a distribution-preserving network bootstrap and counterfactual cross-validation procedure.
result Validates counterfactual estimations in diverse network interference settings.
Bayesian DOE accelerates experimental design with improved efficiency.
problem Enhancing experimental design efficiency and reliability.
method Bayesian framework, conditional density estimation, informative data selection.
result Significantly improved computational efficiency of experimental design.
Design of experiments improves validation of biomolecular networks.
problem Efficiently validate non-machine learning designed biomolecular networks.
method Use Gaussian processes and Bayesian optimization to select experimental points.
result Developed a stopping criterion based on discrepancy metric and uncertainty.
The paper proposes an efficient method for estimating ATEs using adaptive experiments.
problem Estimating average treatment effects (ATEs) with minimal sample size and high accuracy.
method The paper defines and uses the efficient treatment-assignment probability to sequentially assign treatments, estimating ATEs using an Adaptive Augmented Inverse Probability Weighting (A2IPW) estimator.
result The proposed experimental design and A2IPW estimator achieve the minimized semiparametric efficiency bound and provide anytime valid confidence intervals for early stopping.
GAAVI offers anytime-valid tests for CMF global null and contrasts.
problem Inference on the conditional mean function for high confidence decisions.
method Asymptotic anytime-valid tests for CMF global null and contrasts.
result Achieves asymptotic type-I error guarantees, power one, and optimal sample complexity.
Novel approach to Bayesian experimental design for non-exchangeable data.
problem Optimal experimental design for non-exchangeable data.
method Inside-Out SMC2 algorithm embedded in particle Markov chain Monte Carlo framework. result Efficacy demonstrated on a set of dynamical systems.
Paper proposes a method to use in silico experiments with foundation models to reduce sample size.
problem Costly and uncertain randomized experiments.
method Integrates predictions from multiple foundation models with experimental data.
result Estimator offers substantial precision gains, equivalent to a 20% reduction in sample size.
Valid inference from data and predictions.
problem Valid statistical inference with machine learning predictions.
method Framework for valid inference using machine learning predictions.
result Valid confidence intervals without assumptions on predictions.
With the increasing size of today's data sets, finding the right parameter configuration in model selection via cross-validation can be an extremely time-consuming task. In this paper we propose an improved cross-validation procedure which uses nonparametric testing coupled with sequential analysis to determine the bes…
New IF method improves accuracy in deep neural networks with noisy data.
problem Inaccurate influence estimates in deep neural networks, especially with noisy data.
method Established a connection between influence estimation error, validation set risk, and sharpness, introducing a novel estimation form for flat validation minima.
result Our novel Influence Function approach provides more accurate influence estimates, validated across various tasks.
We study signal recovery on graphs based on two sampling strategies: random sampling and experimentally designed sampling. We propose a new class of smooth graph signals, called approximately bandlimited, which generalizes the bandlimited class and is similar to the globally smooth class. We then propose two recovery s…
A new cross-validation method reduces redundancy and improves model performance.
problem Redundancy in traditional k-fold cross-validation leads to biased results.
method Irredundant k-fold cross-validation, where each instance is used exactly once for training and testing.
result Consistent performance estimates with reduced variance and lower computational cost.
Study re-evaluates MIMIC-III codes, finding many are under-coded.
problem Validity of MIMIC-III clinical codes is questionable.
method Open-source, reproducible methodology for assessing codes.
result Most frequently assigned codes are under-coded up to 35%
Optimizes experimental designs for intractable models using mutual information bounds.
problem Finding optimal experimental designs for models with intractable data-generating distributions.
method Maximizes mutual information lower bounds parametrized by neural networks, updating network parameters and designs simultaneously.
result Framework enables experimental design for various tasks including parameter estimation and model discrimination.
GeneDisco benchmarks experimental design for drug discovery.
problem Vast experimental design space in drug discovery.
method Machine learning for optimal experimental design.
result Standardised benchmark suite for active learning.
This paper proposes a new AED framework for multi-metric experiments with fixed budget.
problem Statistical power challenges in testing multiple metrics simultaneously.
method Two-phase structure: adaptive exploration followed by validation. SHRVar algorithm with relative-variance-based sampling.
result Achieves provable error probability that decreases exponentially.
The paper benchmarks OS with RCTs, accounting for right-censoring.
problem Benchmarking observational studies with experimental data under censoring.
method Two cases: independent and dependent censoring. Censoring-doubly-robust signal for CATE.
result Effectiveness of censoring-aware tests verified via experiments and real data.
The formalism of anchor words has enabled the development of fast topic modeling algorithms with provable guarantees. In this paper, we introduce a protocol that allows users to interact with anchor words to build customized and interpretable topic models. Experimental evidence validating the usefulness of our approach…
A new method for experimental design focuses on predicting downstream quantities of interest.
problem Designs that maximize parameter learning may not maximize downstream quantity prediction.
method Likelihood-free goal-oriented optimal experimental design (LF-GO-OED) using ABC density ratio estimation.
result LF-GO-OED maximizes the expected information gain for downstream quantities.
Deep generative models have achieved remarkable success in various data domains, including images, time series, and natural languages. There remain, however, substantial challenges for combinatorial structures, including graphs. One of the key challenges lies in the difficulty of ensuring semantic validity in context. …
PGAE uses predictions to guide active experimentation.
problem Efficiently guiding experimental sampling based on predictions.
method PGAE framework that combines predictions and actual outcomes.
result PGAE achieves asymptotic optimality and efficiency.
The paper optimizes spatial experimental designs to improve causal effect estimation.
problem Optimizing spatial experimental designs to enhance causal effect estimation accuracy.
method Proposes a surrogate function for MSE and uses graph cut algorithms to learn optimal designs.
result The method accommodates spatial interference and covariance, is computationally efficient, and validated by theoretical and numerical experiments.
We explore non-acyclic GFlowNets in discrete settings.
problem Training and understanding non-acyclic GFlowNets in discrete environments.
method Relaxing acyclicity assumption, simpler theoretical framework, novel theoretical insights, experimental validation.
result Theoretical and experimental validation of non-acyclic GFlowNets in discrete environments.
Like all sub-fields of machine learning Bayesian Deep Learning is driven by empirical validation of its theoretical proposals. Given the many aspects of an experiment it is always possible that minor or even major experimental flaws can slip by both authors and reviewers. One of the most popular experiments used to eva…
To define oscillatory movements of securities market, we put in the non-local extension of Ito- equation for wavelet-images of random processes. It is proposed an algorithm of creation of evolutionary equation and a model of prediction of the most probable price movement path. It is carried out experimental validation …
In this work we propose a method for reducing the dimensionality of tensor objects in a binary classification framework. The proposed Common Mode Patterns method takes into consideration the labels' information, and ensures that tensor objects that belong to different classes do not share common features after the redu…
Validates composite systems using discrepancy propagation.
problem Validation of industrial systems with costly real-world tests.
method Propagates bounds on distributional discrepancy measures through a composite system.
result Derives upper bound on real system failure probability from simulations.
Paper uses ML to improve A/B testing for complex treatment effects.
problem Detecting treatment effects in A/B experiments with complex variables.
method Combines ML models with randomization tests for better detection of treatment effects.
result ML-assisted tests improve detection of complex treatment effects.
Motivated by the problem of tuning hyperparameters in machine learning, we present a new approach for gradually and adaptively optimizing an unknown function using estimated gradients. We validate the empirical performance of the proposed idea on both low and high dimensional problems. The experimental results demonstr…
CPTD improves prediction intervals in time series regression with cross-sectional data.
problem Constructing valid prediction intervals in time series regression with a cross-section.
method Conformal Prediction with Temporal Dependence (CPTD) for post-hoc, light-weight approach.
result CPTD maintains cross-sectional validity while improving longitudinal coverage.
GEAR uses auxiliary data to estimate optimal decisions in studies with limited primary outcomes.
problem Estimating optimal decisions when primary outcomes are not available in experimental samples.
method GEAR uses augmented inverse propensity weighting to estimate optimal decisions based on auxiliary data.
result GEAR estimators and value estimators have established asymptotic properties and are validated in simulations and a real application.
In this paper, the use of the Generalized Beta Mixture (GBM) and Horseshoe distributions as priors in the Bayesian Compressive Sensing framework is proposed. The distributions are considered in a two-layer hierarchical model, making the corresponding inference problem amenable to Expectation Maximization (EM). We prese…
LALR adapts learning rate for faster convergence in regression and neural nets.
problem Finding optimal learning rates for faster convergence in regression and neural networks.
method Lipschitz continuity theory applied to Mean Absolute Error and Quantile loss functions.
result Adaptive learning rate policy enables up to 20x faster convergence.
In this paper, an unsupervised machine learning method for geometric constellation shaping is investigated. By embedding a differentiable fiber channel model within two neural networks, the learning algorithm is optimizing for a geometric constellation shape. The learned constellations yield improved performance to sta…
Research in natural language processing proceeds, in part, by demonstrating that new models achieve superior performance (e.g., accuracy) on held-out test data, compared to previous results. In this paper, we demonstrate that test-set performance scores alone are insufficient for drawing accurate conclusions about whic…
Simulation workflow is a top-level model for the design and control of simulation process. It connects multiple simulation components with time and interaction restrictions to form a complete simulation system. Before the construction and evaluation of the component models, the validation of upper-layer simulation work…
Recent work has introduced a simple numerical method for solving partial differential equations (PDEs) with deep neural networks (DNNs). This paper reviews and extends the method while applying it to analyze one of the most fundamental features in numerical PDEs and nonlinear analysis: irregular solutions. First, the S…
As high-throughput biological sequencing becomes faster and cheaper, the need to extract useful information from sequencing becomes ever more paramount, often limited by low-throughput experimental characterizations. For proteins, accurate prediction of their functions directly from their primary amino-acid sequences h…
New method narrows prediction intervals for individual treatment effects.
problem Insufficiently conservative prediction intervals for individual treatment effects.
method Conformal inference using conditional density estimates.
result Narrower prediction intervals compared to existing methods.
Method removes misleading data to improve ML model accuracy.
problem Unhealthy fear of missing out on data leads to model instability and poor performance.
method Bayesian sequential selection method that identifies and selects critical information.
result Improves sample-wise error convergence and eliminates model instabilities.
Optimal sensor placement minimizes information loss from simulations.
problem Designing efficient sensor networks for spatiotemporal processes.
method Model-based sensor placement criterion with sparse variational inference and Gauss-Markov priors.
result Our method identifies sensor networks that minimize information loss from simulated data.
This work proposes validation diagnostics for SBI algorithms using Normalizing Flows.
problem Lack of appropriate validation methods for SBI algorithms with complex, high-dimensional data.
method Develops validation diagnostics based on Normalizing Flows with theoretical guarantees.
result Offers theoretical guarantees on consistency of NF-based estimators.
Dan Reznik discovered conserved quantities for ellipses using billiard maps.
problem Conservation laws in periodic billiard trajectories.
method Non-standard generating function for the billiard ball map.
result Proved identities valid for all smooth convex billiard tables.
In this paper, we advocate for representation learning as the key to mitigating unfair prediction outcomes downstream. Motivated by a scenario where learned representations are used by third parties with unknown objectives, we propose and explore adversarial representation learning as a natural method of ensuring those…