GAN Q-learning uses GANs for distributional RL in tabular and gym environments.
problem Complex MDPs in nonlinear function approximation.
method Generative adversarial networks (GANs) for distributional reinforcement learning.
result Empirically shows GAN Q-learning is a viable alternative to traditional methods.
New framework for PMD convergence in non-tabular environments.
problem Applying PMD to general policy classes with weak closure conditions.
method Develops a theoretical framework with a novel smoothness notion.
result Obtains upper bounds on convergence rate for non-tabular environments.
Improved RL algorithm for robustness against parameter mismatches.
problem Learning robust control policies against parameter mismatches between training and testing environments.
method Formulated as DR-RL problem, proposed RPVL algorithm for tabular episodic learning with four divergences.
result Achieved i l d e O ( ∣ S ∣ ∣ A ∣ H 5 ) ilde{\mathcal{O}}(|\mathcal{S}||\mathcal{A}| H^{5}) i l d e O ( ∣ S ∣∣ A ∣ H 5 ) sample complexity uniformly better than existing results. MADE improves exploration in RL by maximizing deviation from explored regions.
problem Efficient exploration in high-dimensional RL tasks with sparse rewards.
method Proposes a new exploration approach via maximizing the deviation of the occupancy of the next policy from explored regions, adding it as an adaptive regularizer to the RL objective.
result Significantly improves sample efficiency in navigation and locomotion tasks.
We consider an agent's uncertainty about its environment and the problem of generalizing this uncertainty across observations. Specifically, we focus on the problem of exploration in non-tabular reinforcement learning. Drawing inspiration from the intrinsic motivation literature, we use density models to measure uncert…
Efficiently explores concurrent RL agents for practical scale problems.
problem Efficiently exploring multiple RL agents in a shared environment.
method Combines seed sampling and randomized value function learning.
result Approach is competitive and learns quickly with fewer agents.
Deep reinforcement learning methods attain super-human performance in a wide range of environments. Such methods are grossly inefficient, often taking orders of magnitudes more data than humans to achieve reasonable performance. We propose Neural Episodic Control: a deep reinforcement learning agent that is able to rap…
Optimistic algorithms and Thompson sampling use info-theory for better reinforcement learning.
problem Designing algorithms that balance exploration and exploitation in reinforcement learning.
method Integrating information-theoretic concepts into optimistic algorithms and Thompson sampling.
result Cumulative regret bound depends on uncertainty and quantifies prior information value.
A new method for prioritizing sequences in reinforcement learning improves learning efficiency.
problem Efficient learning in reinforcement learning algorithms.
method Prioritized Sequence Experience Replay (PSER) for prioritizing sequences of experience.
result PSER is guaranteed to converge faster than PER and substantially improves performance.
Self-consistent models improve reinforcement learning by aligning predictions with future values.
problem Improving reinforcement learning by aligning model predictions with future values.
method Proposes multiple self-consistency updates to encourage a learned model and value function to be consistent with each other.
result Self-consistency helps both policy evaluation and control in both tabular and function approximation settings.
New algorithm reduces sample size for robust reinforcement learning.
problem Creating robust policies for multi-agent reinforcement learning.
method Model-based algorithm RTZ-VI-LCB for tabular robust two-player zero-sum games.
result Establishes near-optimal sample complexity guarantees for offline robust reinforcement learning.
RandQL is a new model-free algorithm for MDPs with a novel learning rate randomization approach.
problem Minimizing regret in episodic MDPs with model-free methods.
method Randomized Q-learning with learning rate randomization.
result RandQL achieves optimal regret bounds in both tabular and metric state-action spaces.
A method for self-supervised representation learning in partially observable environments.
problem Sparse rewards and stochasticity in partially observable environments.
method World model in latent space for estimating missing information.
result Significant improvement in exploration compared to prior work.
This work bridges offline RL and DRL to address distributional shift.
problem Distributional shift in offline RL due to difference in state-action visitation distributions.
method Proposes offline RL algorithms using DRL framework, characterizes sample complexity under single policy concentrability.
result Demonstrates superior performance of proposed algorithms through simulations.
Paper improves Bayesian regret bounds for Thompson Sampling in reinforcement learning.
problem Improving Bayesian regret bounds for Thompson Sampling in reinforcement learning.
method Using a discrete set of surrogate environments and posterior consistency analysis, the authors derive an upper bound of order O ( H d l 1 T ) O(H\sqrt{d_{l_1}T}) O ( H d l 1 T ) . result The derived upper bound of O ( H d l 1 T ) O(H\sqrt{d_{l_1}T}) O ( H d l 1 T ) is a significant improvement over previous bounds. VaST maps complex environments to simpler models for faster learning.
problem Sample inefficiency in deep reinforcement learning.
method Variational State Tabulation (VaST) and prioritized sweeping planning.
result VaST enables rapid learning and adaptation in complex tasks.
XLVINs improve data efficiency in implicit planning by leveraging latent space.
problem Improving data efficiency in implicit planning algorithms.
method XLVINs use a high-dimensional latent space to perform planning computations, breaking the algorithmic bottleneck.
result XLVINs significantly improve data efficiency across various settings compared to value iteration-based implicit planners and model-free baselines.
A new method approximates Laplacian eigenvectors for RL efficiently.
problem Efficiently learning state representations in RL.
method General and scalable approach to approximating Laplacian eigenvectors.
result Empirically shows improved performance in RL tasks.
Neural Replicator Dynamics improves deep RL performance in nonstationary environments.
problem Nonstationarity and instability in multiagent reinforcement learning.
method Derive a new algorithm using replicator dynamics to bypass softmax in policy gradient methods.
result Neural Replicator Dynamics (NeuRD) outperforms policy gradient methods in nonstationary environments.
New research finds uncertainty estimation techniques fail to reliably detect abnormal medical cases.
problem Uncertainty estimation does not reliably detect out-of-distribution patients in medical tabular data.
method A series of tests on various uncertainty estimation techniques on real-world medical data.
result Almost all techniques fail to identify out-of-distribution patients, contradicting earlier findings.
This work expands state-action aggregation methods for non-Markovian environments.
problem Real-world problems with large state and action spaces are not tractable with existing methods.
method Expands Extreme State Aggregation (ESA) framework to non-Markovian homomorphisms and relaxes policy uniformity.
result Near-optimal performance is guaranteed even for non-Markovian homomorphisms.
Unified AI system for data quality control and governance in regulated environments.
problem Isolated data quality control steps in existing systems.
method AI-driven framework integrating rule-based, statistical, and AI methods.
result Empirical gains in anomaly detection, reduced manual remediation, improved auditability.
New RL method learns K-step lookahead Q-functions for fixed-horizon MDPs.
problem Challenges in online reinforcement learning for non-episodic, finite-horizon MDPs.
method Introduces a K-step lookahead Q-function with a time-varying threshold for selecting actions.
result Achieves minimax optimal constant regret for K=1 and O ( max ( ( K − 1 ) , C K − 1 ) S A T log ( T ) ) \mathcal{O}(\max((K-1),C_{K-1})\sqrt{SAT\log(T)}) O ( max (( K − 1 ) , C K − 1 ) S A T log ( T ) ) regret for K ≥ 2. Two new methods reduce random forest latency and improve accuracy.
problem High latency and memory demands in deep random forest models.
method DiNo and RanBu convert shallow random forests into efficient predictors.
result RanBu matches or exceeds full-depth random forest accuracy with up to 95% reduction in time.
PyTorch Frame simplifies multi-modal tabular learning with modular data and model handling.
problem Handling complex multi-modal tabular data in deep learning.
method A PyTorch-based framework that provides a data structure, model abstraction, and integration with external models.
result Demonstrated the effectiveness of PyTorch Frame in implementing and applying diverse tabular models to complex multi-modal tabular data.
FADE adapts machine learning models to evolving data efficiently.
problem Sequential covariate shift in dynamic environments.
method FADE uses Fisher information geometry for robust learning under SCS.
result FADE achieves up to 19% higher accuracy under severe shifts.
Paper examines actor-critic methods for RL in partially observable multiagent environments.
problem Optimizing policies in partially observable multiagent settings.
method Actor-critic algorithms, focusing on policy gradient and regret minimization.
result Shows convergence guarantees and improved performance in zero-sum games.
New mirror maps improve PMD performance in reinforcement learning.
problem Limited exploration of PMD's full potential due to focus on negative entropy.
method Evolutionary strategies to identify and learn more efficient mirror maps.
result Learned mirror maps outperform negative entropy in various environments.
MET learns tabular data representations without data augmentations.
problem Lack of effective self-supervised learning methods for tabular data.
method Reconstruction-based approach using masked encoding, with separate representations for each coordinate and adversarial reconstruction loss.
result MET achieves state-of-the-art performance on five diverse tabular datasets, improving up to 9% over current methods.
New algorithm for RL using mean embeddings of return distributions.
problem Improving reinforcement learning algorithms for dynamic programming.
method Mean embeddings of return distributions, novel algorithms for RL.
result Asymptotic convergence and improved performance in deep RL.
Algorithm tackles constrained reinforcement learning with concave-convex and knapsack constraints.
problem Constrained episodic reinforcement learning with concave rewards and convex constraints.
method Modular analysis with strong theoretical guarantees for concave-convex and knapsack settings.
result Significantly outperforms existing approaches in constrained episodic environments.
Deep tabular models outperform GBDT in medical diagnosis tasks.
problem Transfer learning for tabular data in medical diagnosis.
method Proposes a pseudo-feature method for transfer learning between different feature sets.
result Tabular neural networks outperform GBDT in medical diagnosis tasks.
GACTGAN synthesizes tabular data better with less computational overhead.
problem Synthesizing mixed tabular data while balancing risk and utility.
method Integrates Bayesian posterior approximation with Stochastic Weight Averaging-Gaussian (SWAG) in CTGAN.
result GACTGAN produces better synthetic data with reduced privacy risk.
Paper discusses imperceptible attacks on tabular data models.
problem Security of machine learning models on tabular data.
method Formalizes imperceptibility of adversarial attacks, proposes an approach to generate imperceptible adversarial examples.
result Demonstrates ability to generate imperceptible adversarial examples with high fooling rate.
AutoDiff combines auto-encoder and diffusion model for realistic tabular data synthesis.
problem Generating realistic synthetic tabular data with heterogeneous features.
method Employing auto-encoder architecture to handle tabular data's complexity.
result Synthetic tables from AutoDiff show good statistical fidelity and perform well in machine learning tasks.
This paper tackles RL issues with robust policies using historical data.
problem Limited data and mismatch between training and testing environments.
method Distributionally robust offline RL with linear function approximation.
result Achieved error bounds for sample complexity in RL.
Safe exploration in RF-RL doesn't increase sample complexity.
problem Achieving optimal policies with safety constraints in reward-free RL.
method Proposed SWEET framework for tabular and low-rank MDP settings, leveraging truncated value functions.
result Sample complexities match or outperform constraint-free counterparts, proving safety constraints have little impact.
Paper tackles test-time adaptation for tabular data.
problem Performance degradation due to distribution shifts in testing.
method Proposes FTAT for robustly adapting tabular models during testing.
result FTAT outperforms state-of-the-art methods on benchmark datasets.
Convolutional networks improve reinforcement learning for navigating all goals.
problem Expensive parallel updates limit reinforcement learning to small tabular cases.
method Use convolutional neural networks to generate Q-values and updates for all goals simultaneously.
result Demonstrated improved accuracy and generalization on various environments.
TabNet learns tabular data with attention, improving performance and interpretability.
problem Efficient learning and interpretability in tabular data.
method Sequential attention to feature selection, self-supervised learning.
result Outperforms other models on tabular datasets, interpretable feature attributions.
AdapTable adapts tabular models to shifts without source data, improving HELOC performance.
problem Distribution shifts in tabular data threaten model performance.
method Shift-aware uncertainty calibrator and label distribution handler.
result Up to 16% improvement on HELOC dataset.
GOTabPFN improves tabular model performance with compact tokenization for HDLSS data.
problem Making tabular models effective for high-dimensional, low-sample size data without retraining.
method Introducing Graph-guided Ordering with Local Refinement (GO-LR) and Neuro-Inspired Subunit Compression (NSC) to create compact meta-features.
result GOTabPFN improves stability and accuracy in tabular benchmarks with compact tokenization.
Tabular GAN generates synthetic tables from datasets.
problem Generating high-quality synthetic tabular data.
method Generative adversarial network (GAN) for tabular data.
result TGAN outperforms conventional models in generating correlated tabular data.
Improves tabular data augmentation for contrastive learning.
problem Ineffective augmentation techniques for tabular data.
method Class-conditioned and feature-correlation based augmentation.
result Consistently outperforms conventional corruption methods.
Orion-Bix combines biaxial attention and meta-learning for tabular few-shot learning.
problem Scaling and generalizing tabular models with mixed numeric and categorical fields, weak feature structure, and limited labeled data.
method Orion-Bix uses biaxial attention and meta-learned in-context reasoning to efficiently capture local and global dependencies.
result Orion-Bix outperforms gradient-boosting baselines and state-of-the-art tabular models on public benchmarks.
Proposes a proportional masking strategy for better tabular data imputation.
problem Heterogeneity of tabular data disrupts uniform random masking in MAEs.
method Computes missingness statistics, generates proportional masks, uses MLP token mixing.
result Proportional masking preserves missingness distribution, improves imputation performance.
NODE improves deep learning on tabular data, outperforming GBDT.
problem Limited performance of deep learning on tabular data compared to gradient boosting decision trees.
method Introducing Neural Oblivious Decision Ensembles (NODE), a deep learning architecture that generalizes ensembles of oblivious decision trees.
result NODE outperforms leading GBDT packages on most tabular tasks.
DNF-Net tackles tabular data challenges with neural architecture.
problem Handling tabular data efficiently using neural networks.
method DNF-Net uses a neural architecture with inductive bias corresponding to logical Boolean formulas in disjunctive normal form over affine soft-threshold decision terms.
result DNF-Net significantly outperforms fully connected networks on tabular data.