WALL-E accelerates reinforcement learning by parallelizing experience collection.
problem Slow experience collection time in reinforcement learning.
method Parallel rollout samplers to speed up experience generation.
result Achieved higher average rewards with parallel processes.
Scientific and business practices are increasingly resulting in large collections of randomized experiments. Analyzed together, these collections can tell us things that individual experiments in the collection cannot. We study how to learn causal relationships between variables from the kinds of collections faced by m…
From scientific experiments to online A/B testing, the previously observed data often affects how future experiments are performed, which in turn affects which data will be collected. Such adaptivity introduces complex correlations between the data and the collection procedure. In this paper, we prove that when the dat…
Study shows small groups can influence machine learning algorithms.
problem How small groups can influence machine learning algorithms deployed on digital platforms.
method Proposed a theoretical model and conducted experiments on a large-scale language model.
result Small groups can exert significant control over machine learning algorithms.
Designs a single policy for collecting data to train near-optimal policies.
problem Engineering overhead in deploying minimax procedures for stochastic linear contextual bandits.
method Designs a single stochastic policy to collect data from which a near-optimal policy can be extracted.
result The designed policy can collect data from which a near-optimal policy can be extracted.
Study uses contextual bandits to optimize charity exposure in donation solicitation.
problem Optimizing charity exposure in donation solicitation using survey responses.
method Adaptive experiment design to balance cumulative regret minimization and simple regret minimization.
result Adaptive experimentation yields better policy learning outcomes than uniform randomization.
Adapting policy learning for data collected from evolving systems.
problem Challenges in learning optimal policies from adaptively collected data.
method Proposes an algorithm based on generalized augmented inverse propensity weighted (AIPW) estimators to control worst-case estimation variance.
result Achieves minimax rate optimal regret guarantees even with diminishing exploration.
Supervised randomization makes randomized experiments more cost-effective for uplift modeling.
problem Costly randomized experiments for uplift modeling.
method Integrates existing scoring models into randomized trials to target relevant customers while correcting for selection bias.
result Cost-efficient data collection under supervised randomization with competitive uplift model performance.
New algorithm for collective Gaussian hidden Markov models inference.
problem Inference of collective Gaussian hidden Markov models from aggregate data.
method Collective Gaussian forward-backward algorithm, extending Sinkhorn belief propagation.
result Convergence guarantee and applicability to single individual Kalman filter.
A universal collection of 4 invariants improves neural network accuracy for molecular dynamics.
problem Improving accuracy of neural networks in molecular dynamics.
method Developed a universal collection of 4 smooth scalar invariants on M(3) x M(3) and evaluated their effectiveness in a PONITA neural network architecture.
result Using a universal collection of invariants significantly improves neural network accuracy.
Study experiment planning with function approximation in contextual bandit problems.
problem Designing effective data collection strategies in settings with limited rewards.
method Proposes two experiment planning strategies compatible with function approximation.
result Eluder planning and sampling procedure achieves optimality guarantees.
Generative model predicts NFT collection transactions based on early history.
problem Predict future transactions of newly minted NFT collections.
method Unsupervised learning to extract contexts, then generate future transactions.
result Projected market value of new NFT collections.
Improved disability insurance model with collective health claims.
problem Enhance disability insurance model with collective health claims.
method Expand classic semi-Markov model with collective health claims, solve many-body problem using mean-field approach.
result Mean-field approach simplifies complex model into a transparent pricing method.
We address the collective matrix completion problem of jointly recovering a collection of matrices with shared structure from partial (and potentially noisy) observations. To ensure well--posedness of the problem, we impose a joint low rank structure, wherein each component matrix is low rank and the latent space of th…
We describe parallel Markov chain Monte Carlo methods that propagate a collective ensemble of paths, with local covariance information calculated from neighboring replicas. The use of collective dynamics eliminates multiplicative noise and stabilizes the dynamics thus providing a practical approach to difficult anisotr…
A new method for analyzing adaptive experiments using kernel treatment effects.
problem Efficiently analyzing adaptive experiments that adjust treatment assignments based on outcomes.
method Kernel Treatment Effects (KTE) framework combining RKHS scores and witness functions.
result Effective for both mean shifts and higher-moment differences, outperforming adaptive baselines.
Advances neural tri-factorization for clustering and discordance analysis of multi-typed data.
problem Challenges in analyzing heterogeneous, multimodal relational data.
method Deep collective matrix tri-factorization for spectral clustering and cluster association learning.
result Demonstrates efficacy over previous non-neural approaches in clustering and discordance analysis.
Algorithm improves reinforcement learning policies using offline data.
problem Improving reinforcement learning policies with limited online data.
method Designs a single non-reactive policy using offline data with provable guarantees.
result Algorithm achieves better policy quality with less online data.
A scalable topic model for large document collections using MapReduce.
problem Scalability issues in topic modeling for large document collections.
method Correlated Topic Model with variational Expectation-Maximization in MapReduce framework.
result Comparable topic coherences with LDA in MapReduce framework.
Dual behavior policy improves reinforcement learning across various environments.
problem Challenges in reinforcement learning with inconsistent or unknown environments.
method Dual, advantage-based behavior policy using counterfactual regret minimization.
result Demonstrated improved performance over baseline models in diverse environments.
DiFF-RF detects point-wise and collective anomalies using random partitioning trees.
problem Detecting anomalies in data, especially collective anomalies.
method Random partitioning binary trees with distance-based leaves and semi-supervised learning.
result DiFF-RF significantly outperforms isolation forest and one-class SVM.
CDP improves RL performance and sample-efficiency by prioritizing rare goal states.
problem Learning from imbalanced data in RL.
method Curiosity-Driven Prioritization (CDP) framework.
result CDP improves both performance and sample-efficiency of RL agents.
DisCor corrects reinforcement learning issues by re-weighting collected data.
problem Reinforcement learning algorithms struggle with instability and sensitivity to hyperparameters.
method DisCor reweights collected data to mitigate issues caused by the distribution of experience.
result DisCor improves reinforcement learning in challenging settings like multi-task learning and noisy reward signals.
This paper introduces a new method to compare collections of distributions on manifolds and graphs.
problem Comparing collections of probability distributions over diverse domains.
method Intrinsic slicing construction for Wasserstein distances, Hilbert embedding, resampling, p-value combination.
result Powerful and well-calibrated p-values for comparing distributions on manifolds and graphs.
Learning by integrating multiple heterogeneous data sources is a common requirement in many tasks. Collective Matrix Factorization (CMF) is a technique to learn shared latent representations from arbitrary collections of matrices. It can be used to simultaneously complete one or more matrices, for predicting the unknow…
Probabilistic active meta-learning improves data efficiency in robotics.
problem Data-efficient learning in robotics where data collection is expensive.
method Conceptualizing meta-learning with a probabilistic latent variable model for sequential task selection.
result Improves data efficiency compared to baselines on simulated robotic experiments.
Framework for systemic risk modeling using jointly exchangeable arrays.
problem Systemic risk in insurance portfolios with interactions.
method Jointly exchangeable arrays, central limit theorems, simulation-based validation.
result Asymptotic approximations for total portfolio losses in large portfolios over long time horizons.
Unified method for balancing simulation and data collection.
problem Finding optimal solutions with uncertain inputs.
method Bayesian Information Collection and Optimisation (BICO) procedure.
result Algorithm automatically determines optimal balance between optimisation and data collection.
New algorithm for aggregate inference in HMMs with continuous observations.
problem Inference in large populations with indistinguishable individuals and continuous measurements.
method Continuous observation collective forward-backward algorithm extending existing discrete case algorithm.
result Efficacy demonstrated through numerical experiments.
Novel framework uses few data for Bayesian inference in imaging.
problem Uncertainty estimation in machine learning for imaging requires large data volumes.
method Variational inference framework combining few data, domain expertise, and existing datasets.
result Bayesian models achieve state-of-the-art reconstructions with minimal data collection.
This paper improves self-play learning in games by manipulating experience distributions.
problem Improving self-play learning in games through better experience sampling.
method Three approaches: weighted sampling, Prioritized Experience Replay, and diversifying trajectories.
result Major improvements in early training performance in some games, minor improvements overall.
TUDataset provides benchmark datasets for graph learning.
problem Lack of meaningful benchmark datasets and standardized evaluation procedures for graph learning.
method Collection and standardization of 120 graph datasets from various applications.
result Standardized evaluation procedures and baseline experiments provided.
Active learning with Gaussian processes improves crop phenotype data collection.
problem Scalability issue in high throughput phenotyping for crop improvement.
method Active learning algorithm with Gaussian Process model.
result Superior performance compared to current practices on sorghum data.
One of the open challenges in designing robots that operate successfully in the unpredictable human environment is how to make them able to predict what actions they can perform on objects, and what their effects will be, i.e., the ability to perceive object affordances. Since modeling all the possible world interactio…
Improves likelihood-free inference by using a new sampling approach to avoid biased data collection.
problem Efficient Bayesian inference without likelihood evaluation for real-world datasets.
method Introduces Neural Proposal (NP) to sample simulation inputs i.i.d. for unbiased posterior inference.
result Demonstrates improved performance, especially for multi-modal posteriors, through experiments.
Paper proposes a new loss function for PU learning without negative examples.
problem Traditional machine learning struggles with negative examples, leading to biased predictions.
method Developed a collective loss function (cPU) for positive and unlabeled data.
result The cPU consistently outperforms existing methods in PU learning benchmarks and real-world datasets.
Twitter, a popular social network, presents great opportunities for on-line machine learning research. However, previous research has focused almost entirely on learning from passively collected data. We study the problem of learning to acquire followers through normative user behavior, as opposed to the mass following…
We analyze the sample complexity of learning from multiple experiments where the experimenter has a total budget for obtaining samples. In this problem, the learner should choose a hypothesis that performs well with respect to multiple experiments, and their related data distributions. Each collected sample is associat…
SynthER uses generative models to augment limited RL experience.
problem Limited data for reinforcement learning agents.
method SynthER leverages diffusion models to generate synthetic experience data.
result SynthER significantly improves sample efficiency and training of RL agents.
Compared to other biometrics, gait is difficult to conceal and has the advantage of being unobtrusive. Inertial sensors, such as accelerometers and gyroscopes, are often used to capture gait dynamics. These inertial sensors are commonly integrated into smartphones and are widely used by the average person, which makes …
Collective motion is an intriguing phenomenon, especially considering that it arises from a set of simple rules governing local interactions between individuals. In theoretical models, these rules are normally \emph{assumed} to take a particular form, possibly constrained by heuristic arguments. We propose a new class …
Develops a method to efficiently use offline data for RL policy optimization.
problem Lack of online data for offline RL in mobile health applications.
method Advantage learning framework using optimal Q-estimators.
result New policy converges faster than existing methods.
Matrix completion aims to reconstruct a data matrix based on observations of a small number of its entries. Usually in matrix completion a single matrix is considered, which can be, for example, a rating matrix in recommendation system. However, in practical situations, data is often obtained from multiple sources whic…
Study uses Android smartphones to measure road roughness, finds ML better than RMS.
problem Measuring road roughness using smartphones.
method Three smartphones, Android OS, RMS and ML methods, comparison with IRI.
result ML method outperforms RMS in detecting road roughness.
Meta-learning approach for spatial-temporal prediction across cities.
problem Spatial-temporal prediction for cities with limited data.
method Meta-learning paradigm with spatial-temporal network and pattern-based memory.
result Meta-learning model improves prediction accuracy over multiple tasks.
ISOKANN learns collective variables and effective dynamics for metastable transitions.
problem Understanding metastable transitions in complex molecular systems.
method Integrates Koopman operators with neural networks to extract CVs and effective dynamics.
result Reconstructs coarse-grained kinetics and reproduces transition times across barriers.
Meta-learning improves model performance by optimizing data acquisition.
problem Lack of operationally realistic data limits model performance.
method Gaussian process surrogate fit to metadata-driven training data variations.
result Meta-learning enhances model performance compared to random data acquisition.
Modern machine learning-based recognition approaches require large-scale datasets with large number of labelled training images. However, such datasets are inherently difficult and costly to collect and annotate. Hence there is a great and growing interest in automatic dataset collection methods that can leverage the w…