Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

156312467623 · Jun 202019922001200920172026
48 results for Empirical Experimentation

New method combines experimental and observational data for causal inference.

problem Combining internal validity of experiments and larger sample sizes of observations.
method Empirical risk minimization (ERM) framework with cross-validation.
result Efficacy and reliability demonstrated on real and synthetic data.

Develops a flexible batched experimentation framework for limited adaptivity.

problem Challenges of continual reallocation in bandit algorithms with delayed feedback.
method Computational framework leveraging Gaussian sequential experiment and dynamic programming.
result Improves statistical power over standard methods, even compared to Bayesian bandit algorithms.

Bayesian optimal experimental design (BOED) is a principled framework for making efficient use of limited experimental resources. Unfortunately, its applicability is hampered by the difficulty of obtaining accurate estimates of the expected information gain (EIG) of an experiment. To address this, we introduce several …

2019-03-13abs ↗pdf ↗

Step-DAD improves BED by periodically updating a design policy during experiments.

problem Improving flexibility and robustness in Bayesian experimental design.
method Semi-amortized, policy-based approach that updates a design policy during data collection.
result Consistently superior decision-making and robustness compared to current BED methods.

The abstract warns against flawed empirical research in machine learning.

problem Flawed empirical research in machine learning leading to unreliable results.
method Call for more awareness of experimental knowledge plurality and epistemic limitations.
result Current empirical machine learning research should be exploratory, not confirmatory.

Optimizes experimental designs for intractable models using mutual information bounds.

problem Finding optimal experimental designs for models with intractable data-generating distributions.
method Maximizes mutual information lower bounds parametrized by neural networks, updating network parameters and designs simultaneously.
result Framework enables experimental design for various tasks including parameter estimation and model discrimination.

Certified training improves robustness against adversarial attacks.

problem Certified training's gap with empirical robustness limits its practical utility.
method Combining adversarial attacks with network over-approximations.
result Certified training can prevent catastrophic overfitting and bridge the gap to multi-step baselines.

New method optimizes experiments under constraints.

problem Adapting BED to dynamic constraints in real-world tasks.
method Offline pre-training of an amortized policy and posterior network with online multi-step lookahead planning.
result Significantly more informative design sequences than existing methods.

Unified framework for robust A/B testing under model misspecification.

problem Improving sample efficiency in A/B testing with model misspecification.
method Unified framework for contextual bandit and dynamic settings, proving worst-case mean squared error bounds.
result Empirically validated approach using synthetic and real-world datasets.

Enhances robustness in experimental design through Generalised Bayesian inference.

problem Poor inference and estimates of information gain when statistical model is incorrectly specified.
method Generalised Bayesian (Gibbs) inference framework applied to experimental design.
result GBOED enhances robustness to outliers and incorrect assumptions about noise distribution.

Proposes rounding method for precise treatment effect estimation under budget constraints.

problem Resource-constrained experimental design for precise treatment effect estimation.
method Dependent randomized rounding procedure to convert assignment probabilities into binary treatment decisions.
result Improved estimator precision through variance reduction and efficient inference.

Study evaluates reinforcement learning algorithms for sequential experimental design.

problem Lack of generalization in reinforcement learning for experimental design.
method Investigated several reinforcement learning algorithms for sequential experimental design.
result Certain algorithms, using dropout or ensemble approaches, show attractive generalization properties.

We use a large census of hyperbolic 3-manifolds to experimentally investigate a conjecture of Neumann regarding the Bloch Group. We present an augmented census including, for feasible invariant trace fields, explicit manifolds (associated to that field) that appear to generate the Bloch group of that field. We also mak…

2016-09-28abs ↗pdf ↗

Like all sub-fields of machine learning Bayesian Deep Learning is driven by empirical validation of its theoretical proposals. Given the many aspects of an experiment it is always possible that minor or even major experimental flaws can slip by both authors and reviewers. One of the most popular experiments used to eva…

2018-11-23abs ↗pdf ↗

GoBOED optimizes experiments for specific decision-making objectives, improving downstream outcomes.

problem Reducing parameter uncertainty does not always improve decision-making in critical settings.
method Combines variational posterior surrogate and differentiable convex decision layer for gradient-based design optimization.
result GoBOED identifies designs that better align with specific decision objectives and reveals wider optimal design windows.

GEAR uses auxiliary data to estimate optimal decisions in studies with limited primary outcomes.

problem Estimating optimal decisions when primary outcomes are not available in experimental samples.
method GEAR uses augmented inverse propensity weighting to estimate optimal decisions based on auxiliary data.
result GEAR estimators and value estimators have established asymptotic properties and are validated in simulations and a real application.

The paper develops methods to estimate optimal treatment sequences under policy constraints.

problem Estimating the best sequence of treatments over multiple stages for individuals.
method Empirical welfare maximization approach, solving treatment assignment sequentially or simultaneously.
result Established convergence rates and upper bounds for estimation methods.

Novel neural architecture improves Bayesian experimental design efficiency.

problem Intractable evaluation of expected information gain (EIG) in Bayesian optimal experimental design.
method Develops a neural architecture that optimizes a single variational model for estimating EIG across many designs, using a lower bound for computational efficiency.
result Significantly improves accuracy in Bayesian experimental design with better sample efficiency.

We propose a new randomized ensemble technique with a provable security guarantee against black-box transfer attacks. Our proof constructs a new security problem for random binary classifiers which is easier to empirically verify and a reduction from the security of this new model to the security of the ensemble classi…

2019-06-07abs ↗pdf ↗

We describe a method to determine the eigenvalue density of empirical covariance matrix in the presence of correlations between samples. This is a straightforward generalization of the method developed earlier by the authors for uncorrelated samples. The method allows for exact determination of the experimental spectru…

2005-08-19abs ↗pdf ↗

We give improved constants for data dependent and variance sensitive confidence bounds, called empirical Bernstein bounds, and extend these inequalities to hold uniformly over classes of functionswhose growth function is polynomial in the sample size n. The bounds lead us to consider sample variance penalization, a nov…

2009-07-21abs ↗pdf ↗

Motivated by the problem of tuning hyperparameters in machine learning, we present a new approach for gradually and adaptively optimizing an unknown function using estimated gradients. We validate the empirical performance of the proposed idea on both low and high dimensional problems. The experimental results demonstr…

2019-06-04abs ↗pdf ↗

PINNACLE optimizes point selection for PINNs, improving accuracy.

problem Challenges in selecting points for training Physics-Informed Neural Networks (PINNs).
method Introduces PINNACLE, an algorithm that jointly optimizes collocation and experimental points selection, adjusting point proportions dynamically.
result PINNACLE outperforms existing methods in forward, inverse, and transfer learning problems.

We study the risk performance of distributed learning for the regularization empirical risk minimization with fast convergence rate, substantially improving the error analysis of the existing divide-and-conquer based distributed learning. An interesting theoretical finding is that the larger the diversity of each local…

2018-12-19abs ↗pdf ↗

In high dimensional settings, density estimation algorithms rely crucially on their inductive bias. Despite recent empirical success, the inductive bias of deep generative models is not well understood. In this paper we propose a framework to systematically investigate bias and generalization in deep generative models …

2018-11-08abs ↗pdf ↗

We study methods for simultaneous analysis of many noisy experiments in the presence of rich covariate information. The goal of the analyst is to optimally estimate the true effect underlying each experiment. Both the noisy experimental results and the auxiliary covariates are useful for this purpose, but neither data …

2019-06-04abs ↗pdf ↗

Many applications require that we learn the parameters of a model from data. EM is a method used to learn the parameters of probabilistic models for which the data for some of the variables in the models is either missing or hidden. There are instances in which this method is slow to converge. Therefore, several accele…

2013-01-23abs ↗pdf ↗

Paper proposes a method to use in silico experiments with foundation models to reduce sample size.

problem Costly and uncertain randomized experiments.
method Integrates predictions from multiple foundation models with experimental data.
result Estimator offers substantial precision gains, equivalent to a 20% reduction in sample size.

In this paper, we advocate for representation learning as the key to mitigating unfair prediction outcomes downstream. Motivated by a scenario where learned representations are used by third parties with unknown objectives, we propose and explore adversarial representation learning as a natural method of ensuring those…

2018-02-17abs ↗pdf ↗

As researchers and practitioners of applied machine learning, we are given a set of requirements on the problem to be solved, the plausibly obtainable data, and the computational resources available. We aim to find (within those bounds) reliably useful combinations of problem, data, and algorithm. An emphasis on algori…

2018-12-04abs ↗pdf ↗

Bayesian adaptive designs can be biased by active learning, especially with misspecified models.

problem Active learning bias in Bayesian adaptive experimental designs.
method Analysis of linear and preference learning models, empirical testing.
result Model misspecification and noise influence active learning bias in Bayesian designs.

A framework for designing and evaluating new GCN variants.

problem Designing and evaluating new graph convolutional network (GCN) variants.
method Propose a framework to compose networks using building blocks of GCN.
result Several newly composed variants are useful alternatives and competitive with original GCNs.

Multi-domain learning (MDL) aims at obtaining a model with minimal average risk across multiple domains. Our empirical motivation is automated microscopy data, where cultured cells are imaged after being exposed to known and unknown chemical perturbations, and each dataset displays significant experimental bias. This p…

2019-03-21abs ↗pdf ↗

The paper examines the tilted empirical risk's generalization and robustness under negative tilt.

problem The generalization error of machine learning algorithms under negative tilt.
method Uniform and information-theoretic bounds on the tilted generalization error under negative tilt.
result The tilted empirical risk's generalization error has a convergence rate of \(O(n^{-ε/(1+ε)})\).

Customer scoring models are the core of scalable direct marketing. Uplift models provide an estimate of the incremental benefit from a treatment that is used for operational decision-making. Training and monitoring of uplift models require experimental data. However, the collection of data under randomized treatment as…

2019-10-01abs ↗pdf ↗

We propose a generic model for multiple choice situations in the presence of herding and compare it with recent empirical results from a Web-based music market experiment. The model predicts a phase transition between a weak imitation phase and a strong imitation, `fashion' phase, where choices are driven by peer press…

2006-06-26abs ↗pdf ↗

Meta-algorithm for efficient reinforcement learning from human preferences.

problem Learning from human preference comparisons in Markov decision processes.
method Randomized exploration and experimental design for batch comparison queries.
result Meta-algorithm achieves both regret and last-iterate guarantees with minimal preference queries.

New algorithm improves active learning in agnostic pool-based classification.

problem Efficient active learning in the agnostic setting with minimized sample complexity.
method Solves an experimental design problem to determine a distribution over examples for label requests.
result Achieves sample complexity bounds never worse than best disagreement coefficient-based bounds, sometimes significantly smaller.

This work formalizes and extends parameter sharing in multi-agent reinforcement learning.

problem Parameter sharing limits multi-agent learning to a single policy, preventing different tasks or action spaces.
method Introduces agent indication and extends parameter sharing to heterogeneous observation and action spaces.
result Proves convergence to optimal policies for parameter sharing in heterogeneous environments.

In the absence of explicit regularization, Kernel "Ridgeless" Regression with nonlinear kernels has the potential to fit the training data perfectly. It has been observed empirically, however, that such interpolated solutions can still generalize well on test data. We isolate a phenomenon of implicit regularization for…

2018-08-01abs ↗pdf ↗