Study finds real-world datasets contain natural experiments that can improve model performance.
problem Detecting natural experiments in real-world datasets for causal inference.
method Synthetic graph simulation and feature selection based on causal links.
result Real-world datasets contain natural experiments that can be exploited for improved model performance.
Natural experiment dataset reveals inconsistent treatment effect estimators.
problem Inconsistent results from over 20 estimators on a new dataset.
method Created a benchmark to evaluate estimator accuracy, derived variance formula, introduced new estimator.
result Doubly robust estimators outperform others by orders of magnitude.
Square-root natural-gradient improves variational inference convergence.
problem Challenges in establishing theoretical convergence guarantees for natural-gradient descent.
method Square-root parameterization for Gaussian covariance.
result Establishes novel convergence guarantees for natural-gradient Gaussian inference.
A new test improves statistical inference in bandit algorithms without sacrificing adaptiveness.
problem Challenges in statistical inference for adaptive randomised experiments in bandits.
method An allocation probability test for Thompson Sampling without trading-off regret or requiring large sample sizes.
result Improves statistical inference in small samples, showing advantages in mental health experiments.
iPrompt uses LLMs to generate natural-language explanations of data patterns.
problem Finding and explaining patterns in data using natural language.
method Interpretable autoprompting (iPrompt) that generates natural-language explanations based on LLMs.
result iPrompt can accurately find and explain data patterns, improving upon human-written prompts.
MAC Net is a compositional attention network designed for Visual Question Answering. We propose a modified MAC net architecture for Natural Language Question Answering. Question Answering typically requires Language Understanding and multi-step Reasoning. MAC net's unique architecture - the separation between memory an…
Probabilistic active meta-learning improves data efficiency in robotics.
problem Data-efficient learning in robotics where data collection is expensive.
method Conceptualizing meta-learning with a probabilistic latent variable model for sequential task selection.
result Improves data efficiency compared to baselines on simulated robotic experiments.
RL optimizes resource allocation in MG by balancing experience and exploration.
problem Optimal resource allocation in competitive scenarios.
method Introduced RL to MG, allowing dynamic strategy adjustment based on experience and expected rewards.
result Achieves optimal resource coordination by balancing exploitation and exploration.
Paper discusses new stochastic algorithms for sparse signal recovery.
problem Sparse signal recovery in medical imaging and remote sensing.
method Proposes and analyzes stochastic natural thresholding algorithms.
result Demonstrates improved performance of StoNT algorithms.
PixL2R maps natural language to pixel-based rewards for RL, improving sample efficiency.
problem Sparse reward settings in RL limit applicability to complex problems.
method Directly maps natural language descriptions to pixel-based rewards for guiding RL.
result Language-based rewards significantly improve sample efficiency in policy learning.
BERT improves fine-grained sentiment classification.
problem Fine-grained sentiment classification of text.
method Used BERT for fine-grained sentiment classification.
result BERT outperforms other models for fine-grained sentiment classification.
New method minimizes experiment cost while maintaining accuracy.
problem Minimizing cost in experiments with interference or other concerns.
method Synthetically Controlled Thompson Sampling (SCTS).
result Minimizes regret and maintains inferential ability.
Recent reinforcement learning (RL) approaches have shown strong performance in complex domains such as Atari games, but are often highly sample inefficient. A common approach to reduce interaction time with the environment is to use reward shaping, which involves carefully designing reward functions that provide the ag…
Automated PDE discovery from multiple noisy experiments.
problem Inherent variability in experiments makes single experiment inference unreliable.
method Randomised adaptive group Lasso sparsity estimator in deep learning framework.
result More generalizable PDEs found from multiple datasets.
Motivated by the widespread adoption of large-scale A/B testing in industry, we propose a new experimentation framework for the setting where potential experiments are abundant (i.e., many hypotheses are available to test), and observations are costly; we refer to this as the experiment-rich regime. Such scenarios requ…
DeFi doesn't fully remove trust, showing run risk and personal character's importance.
problem The need for trust in DeFi despite its code-based approach.
method Natural experiment revealing identities of DeFi participants, including a criminal.
result DeFi doesn't fully remove trust, showing run risk and personal character's relevance.
New insights into experience replay in RL algorithms.
problem Understanding the impact of replay capacity and replay ratio in Q-learning.
method Systematic and extensive analysis of experience replay in Q-learning methods, focusing on replay capacity and replay ratio.
result Greater replay capacity significantly improves performance for certain algorithms, while other techniques offer limited benefit.
We introduce a novel definition of curvature for hypergraphs, a natural generalization of graphs, by introducing a multi-marginal optimal transport problem for a naturally defined random walk on the hypergraph. This curvature, termed \emph{coarse scalar curvature}, generalizes a recent definition of Ricci curvature for…
This work investigates uncertainty quantification for black-box large language models in natural language generation.
problem Lack of trustworthiness in responses generated by black-box large language models.
method Differentiated uncertainty vs confidence, proposed and compared several confidence/uncertainty measures, applied to selective NLG.
result A simple measure for semantic dispersion can predict the quality of LLM responses.
Develops an efficient method for large-scale deep learning problems.
problem Efficiently addressing the computational and memory challenges of natural gradient methods in deep learning.
method Randomized techniques for sketching the empirical Fisher information matrix in neural network layers.
result Global convergence to stationary points and fast linear convergence under the NTK case.
The paper proposes a method to train time-varying generative models using natural gradients.
problem Training time-varying generative models efficiently and accurately.
method Projecting generative model parameters onto an exponential family manifold and optimizing using natural gradient descent.
result The proposed method efficiently approximates the natural gradient and can be applied to various exponential family models.
Improved natural gradient boosting with leaf number clipping for faster and better performance.
problem Slower training speed and poor performance on large datasets for natural gradient boosting.
method Leaf number clipping regularization to optimize hyperparameters and improve performance.
result Significant improvement in performance and up to 4.85x speed up on various datasets.
DDPM encoder matches optimal transport for natural images.
problem Understanding theoretical properties of DDPM latent space.
method Showed DDPM encoder matches optimal transport for common distributions.
result DDPM encoder map coincides with optimal transport map for natural images.
Structured prediction requires searching over a combinatorial number of structures. To tackle it, we introduce SparseMAP: a new method for sparse structured inference, and its natural loss function. SparseMAP automatically selects only a few global structures: it is situated between MAP inference, which picks a single …
Algorithm selects optimal experiments in Markov chains to learn unknown quantities.
problem Designing efficient experiments in Markov chains to learn about unknown quantities.
method Proposes extsc{markov-design} algorithm for sequential policy selection.
result Algorithm provably converges to optimal measurement allocation.
MuLan links music audio to natural language tags.
problem Traditional music tagging systems use rigid attributes; MuLan aims to link audio directly to natural language.
method Joint audio-text embedding model trained on 44 million music recordings and text annotations.
result MuLan's embeddings enable zero-shot functionalities and transfer learning.
Embedding graph nodes into a vector space can allow the use of machine learning to e.g. predict node classes, but the study of node embedding algorithms is immature compared to the natural language processing field because of a diverse nature of graphs. We examine the performance of node embedding algorithms with respe…
Since deep learning became a key player in natural language processing (NLP), many deep learning models have been showing remarkable performances in a variety of NLP tasks, and in some cases, they are even outperforming humans. Such high performance can be explained by efficient knowledge representation of deep learnin…
A-ICP selects experiments to learn causal effects efficiently.
problem Learning causal effects from observational data is difficult.
method Active learning framework based on Invariant Causal Prediction.
result Proposes intervention selection policies to reveal direct causes.
It is well known that in stochastic multi-armed bandits (MAB), the sample mean of an arm is typically not an unbiased estimator of its true mean. In this paper, we decouple three different sources of this selection bias: adaptive \emph{sampling} of arms, adaptive \emph{stopping} of the experiment, and adaptively \emph{…
Functional Magnetic Resonance Imaging (fMRI) provides dynamical access into the complex functioning of the human brain, detailing the hemodynamic activity of thousands of voxels during hundreds of sequential time points. One approach towards illuminating the connection between fMRI and cognitive function is through dec…
A method for disentangling text representations without supervision.
problem Challenges in learning disentangled representations of natural language.
method Information-theoretic guidance to induce independent style and content embeddings.
result High quality disentangled representations in terms of content and style preservation.
BICauseTree improves causal effect estimation by identifying clusters and balancing treatment allocation.
problem Improving interpretability and transparency in causal effect models from observational data.
method Hierarchical bias-driven stratification using decision trees with a customized objective function.
result BICauseTree provides interpretable causal effect estimation and is comparable to existing methods.
The techniques of deep learning have become the state of the art methodology for executing complicated tasks from various domains of computer vision, natural language processing, and several other areas. Due to its rapid development and promising benchmarks in those fields, researchers started experimenting with this t…
Optimizes staggered treatment rollouts to minimize cost and error.
problem Efficiently scheduling treatment initiation times for staggered rollouts.
method Non-adaptive and adaptive experimental designs, including a near-optimal solution for non-adaptive cases and a new algorithm for adaptive cases.
result Reduces experiment cost by over 50% compared to static design benchmarks.
NRWS improves training of SBNs and HMs using natural gradient.
problem Training Sigmoid Belief Networks and Helmholtz Machines efficiently.
method Exploits block-diagonal structure of Fisher Information Matrices to use natural gradient.
result NRWS and NBiHM achieve better log-likelihood and faster convergence.
Deep learning has improved performance on many natural language processing (NLP) tasks individually. However, general NLP models cannot emerge within a paradigm that focuses on the particularities of a single metric, dataset, and task. We introduce the Natural Language Decathlon (decaNLP), a challenge that spans ten ta…
SSMBA generates synthetic data to improve robustness in natural language tasks.
problem Improving out-of-domain generalization of models trained on natural language data.
method SSMBA uses corruption and reconstruction functions to generate synthetic data points on the manifold assumption.
result SSMBA consistently outperforms existing methods on robustness benchmarks across multiple tasks and datasets.
Conformal prediction fails under severe feature turnover in COVID-19 supply chain tasks.
problem Dealing with distribution shift in conformal prediction models.
method Using COVID-19 as a natural experiment across 8 supply chain tasks, analyzing SHAP explanations.
result Coverage drops vary widely (0% to 86.7%) and correlate with single-feature dependence.
Active learning improves neutron spectroscopy experiments by automating measurement selection.
problem Efficiently searching for signals in neutron scattering experiments with limited beam time.
method Probabilistic active learning using log-Gaussian processes.
result Automated selection of informative measurements improves experimental efficiency.
Sparse reward is one of the most challenging problems in reinforcement learning (RL). Hindsight Experience Replay (HER) attempts to address this issue by converting a failed experience to a successful one by relabeling the goals. Despite its effectiveness, HER has limited applicability because it lacks a compact and un…
Proposes model-based robust deep learning to handle natural variation in data.
problem Deep learning's fragility to natural variation in data.
method Develops model-based robust training algorithms using deep generative models to learn natural variation.
result Deep neural networks trained with model-based algorithms outperform standard and norm-bounded robust algorithms.
Associating image regions with text queries has been recently explored as a new way to bridge visual and linguistic representations. A few pioneering approaches have been proposed based on recurrent neural language models trained generatively (e.g., generating captions), but achieving somewhat limited localization accu…
Robust tensor CP decomposition involves decomposing a tensor into low rank and sparse components. We propose a novel non-convex iterative algorithm with guaranteed recovery. It alternates between low-rank CP decomposition through gradient ascent (a variant of the tensor power method), and hard thresholding of the resid…
A new algorithm for training generative models using Sinkhorn divergence.
problem Training generative adversarial networks (GANs).
method Sinkhorn Natural Gradient (SiNG) algorithm for steepest descent on probability space.
result Explicit expression and efficient evaluation of the Sinkhorn information matrix (SIM).
3D models vulnerable to adversarial attacks, new method improves success rate and naturalness.
problem Vulnerability of 3D deep learning models to adversarial examples in the physical world.
method ε-isometric (ε-ISO) attack considering geometric properties and invariance to physical transformations. result Significantly improved attack success rate and naturalness of 3D adversarial examples.
LLMs are compared to Markov chains for natural language processing.
problem Theoretical analysis of LLMs' generalization capabilities.
method Equivalence between LLMs and Markov chains, studying multi-step inference.
result Derives generalization bounds for LLMs, capturing their behavior in practice.
Context: Conducting experiments is central to research machine learning research to benchmark, evaluate and compare learning algorithms. Consequently it is important we conduct reliable, trustworthy experiments. Objective: We investigate the incidence of errors in a sample of machine learning experiments in the domain …