Paper introduces a new distributional successor measure for reinforcement learning.
problem Learning the distributional consequences of behavior in reinforcement learning.
method Formulates distributional successor measure as a distribution over distributions, proposes algorithm to learn it from data.
result Demonstrates zero-shot risk-sensitive policy evaluation.
Study on distributional TD learning with linear approximations for better return estimation.
problem Estimating the return distribution of a policy in reinforcement learning.
method Finite-sample analysis of distributional TD learning with linear function approximation, using the linear-categorical Bellman equation and exponential stability arguments for products of random matrices.
result Sample complexity of linear distributional TD learning matches that of classic linear TD learning, indicating similar difficulty in estimating return distribution versus its expectation.
Paper settles sample complexity for learning from multiple distributions.
problem Learning from multiple data distributions with a hypothesis class of bounded VC dimension.
method Introduced an algorithm with sample complexity of O((d+k)ε^-2)·(k/ε)^o(1).
result Algorithm matches lower bound up to sub-polynomial factor.
New algorithms improve distributional TD learning with linear approximations.
problem Estimating return distributions in reinforcement learning.
method Fine-grained analysis of linear-categorical Bellman equation, variance reduction techniques.
result Tight sample complexity bounds for distributional TD learning with linear approximations.
Paper develops a new algorithm for distribution regression with optimal learning rates.
problem Distribution regression with limited second-stage samples.
method Multi-penalty regularization in a reproducing kernel Hilbert space.
result Derives optimal learning rates for distribution regression.
Introduces NQ network for non-crossing quantile learning.
problem Quantile crossing issue in distributional learning.
method Non-negative activation functions ensure monotonic distributions.
result Effective for distributional reinforcement learning and causal effect estimation.
Unsupervised learning representations generalize better than supervised learning under distribution shifts.
problem Robustness of unsupervised representations to distribution shift.
method Extensive evaluation on synthetic and realistic datasets, including controllable domain generalization datasets.
result Unsupervised representations learned from SSL and AE generalize better than supervised learning under various distribution shifts.
Method learns statistics of return distributions via neural networks and maximum mean discrepancy.
problem Learning probability distributions in reinforcement learning.
method Maximum mean discrepancy (MMD) for learning unrestricted statistics of return distributions.
result Method outperforms standard distributional RL baselines on Atari games.
We provide new results for noise-tolerant and sample-efficient learning algorithms under s-concave distributions. The new class of s-concave distributions is a broad and natural generalization of log-concavity, and includes many important additional distributions, e.g., the Pareto distribution and t-distribution.…
The paper explores solutions to the distributional Bellman equation in reinforcement learning.
problem Distributional reinforcement learning considers complete return distributions, not just expected returns.
method Study existence and uniqueness of solutions to general distributional Bellman equations, linking them to multivariate affine equations.
result Any solution to a distributional Bellman equation can be derived from a multivariate affine distributional equation.
New approach for large-scale distributed learning systems that improve generalization performance.
problem Transitioning from centralized to distributed AI systems for complex learning tasks.
method Self-organizing hierarchical structuring mechanism based on agglomerative clustering, hierarchical generalization, and personalized learning.
result Demonstrates better generalization performance compared to conventional federated learning algorithms.
Paper proposes a new method to learn distribution kernels via entropy maximization.
problem Challenges in applying kernel methods to distribution regression tasks.
method Proposes a novel objective for unsupervised learning of data-dependent distribution kernels based on entropy maximization.
result Demonstrates the effectiveness of the learned kernel across different modalities.
Paper optimizes training data distribution for better model performance across various deployment conditions.
problem Improving model accuracy when deployed with parameters far from training data.
method Developed adaptive algorithms based on bilevel or alternating optimization in the space of probability measures.
result Optimized training distributions lead to models with improved sample complexity and robustness to distribution shift.
Equivariant flows learn symmetrical distributions on manifolds.
problem Learning symmetrical distributions on arbitrary manifolds.
method Equivariant manifold flows.
result Learned gauge invariant densities over SU(n) in quantum field theory.
Distributed machine learning algorithms enable learning of models from datasets that are distributed over a network without gathering the data at a centralized location. While efficient distributed algorithms have been developed under the assumption of faultless networks, failures that can render these algorithms nonfu…
Method enhances anomaly detection using contrastive learning and out-of-distribution data.
problem Improving anomaly detection in datasets with limited out-of-distribution data.
method Proposes a contrastive learning method that incorporates out-of-distribution data to enhance anomaly detection performance.
result The method significantly improves anomaly detection performance, even with limited out-of-distribution data.
New bounds on learning from multiple distributions for VC classes.
problem Understanding the sample complexity of learning from multiple data distributions.
method Analyzing the gap between known upper and lower bounds for PAC-learnable classes.
result Recent progress on sample complexity for VC dimension d classes on k distributions.
Proposes MRO to achieve uniformly low regret in distributionally robust learning.
problem Learning under unknown test distributions (distribution shift).
method Minimax Regret Optimization (MRO) for robust machine learning.
result MRO achieves uniformly low regret across all test distributions.
New perspective on distribution shift helps make learning easier.
problem Learning with different training and target distributions.
method Formalizing and exploring Positive Distribution Shift (PDS).
result Distribution shift can be positive, making learning easier.
Theory of learning with weight-distribution constraints.
problem Understanding how structure influences function in neural networks.
method Statistical mechanical theory and optimal transport.
result Reduction in capacity due to constrained weight-distribution is related to Wasserstein distance.
New RL algorithm minimizes distributional learning error.
problem Improving distributional reinforcement learning for better error minimization.
method Proposes a new model-based algorithm with theoretical minimax optimality.
result Proves minimax optimality for approximating return distributions.
In this paper we argue for the fundamental importance of the value distribution: the distribution of the random return received by a reinforcement learning agent. This is in contrast to the common approach to reinforcement learning which models the expectation of this return, or value. Although there is an established …
New findings on PAC learning and marginal distribution estimation.
problem Understanding how PAC learning relates to marginal distribution estimation under distributional constraints.
method Revisited the connection between PAC learning, uniform convergence, and density estimation, considering a known family of marginal distributions.
result PAC learning is sandwiched between two refined models of density estimation, differing only in whether the learner knows the set of well-estimated events in H.
New algorithms improve label complexity for active multi-distribution learning.
problem Active multi-distribution learning with improved label complexity.
method Developed new algorithms for active multi-distribution learning and established improved label complexity upper and lower bounds.
result Improved label complexity upper and lower bounds for active multi-distribution learning.
A new method for learning conditional distributions using ODEs and neural networks.
problem Learning conditional distributions efficiently and accurately.
method Conditional Föllmer Flow, discretized with Euler's method, using nonparametric velocity estimation.
result Effective approximation of target conditional distributions, with convergence results for Wasserstein-2 distance.
The theory of learning under the uniform distribution is rich and deep, with connections to cryptography, computational complexity, and the analysis of boolean functions to name a few areas. This theory however is very limited due to the fact that the uniform distribution and the corresponding Fourier basis are rarely …
SHIFT method optimally estimates heterogeneous discrete distributions with limited communication.
problem Collaborative learning of discrete distributions under heterogeneity and communication constraints.
method Two-stage method: First, users learn a central distribution; then, fine-tune this to estimate individual distributions.
result SHIFT is minimax optimal in the model of heterogeneity and under communication constraints.
Federated learning studies separate client data and distribution gaps.
problem Understanding performance differences in federated learning across different datasets.
method Proposed a framework to disentangle out-of-sample and participation gaps.
result Dataset synthesis strategy is crucial for realistic simulations of federated learning generalization.
Q-Distribution Guided Q-Learning corrects overestimation of uncertain OOD actions in offline RL.
problem Overestimation of Q-values for out-of-distribution actions in offline reinforcement learning.
method QDQ applies a pessimistic adjustment to Q-values in uncertain OOD regions based on a consistency model.
result QDQ improves performance on the D4RL benchmark and achieves significant improvements across many tasks.
Distributional approaches to value-based reinforcement learning model the entire distribution of returns, rather than just their expected values, and have recently been shown to yield state-of-the-art empirical performance. This was demonstrated by the recently proposed C51 algorithm, based on categorical distributiona…
New algorithms improve signal processing in federated learning.
problem Efficiently process distributed signal samples with privacy and communication constraints.
method Proposes overpredictive signal approximations using convex optimization.
result Quantifies tradeoffs between communication cost, sampling rate, and approximation error.
Paper tackles robust federated learning for affine distribution shifts.
problem Statistical heterogeneity and distribution shifts degrade model performance in federated learning.
method Develops a robust federated learning algorithm (FLRA) for affine distribution shifts.
result FLRA achieves significant performance gains against affine distribution shifts.
Many machine learning tasks, such as learning with invariance and policy evaluation in reinforcement learning, can be characterized as problems of learning from conditional distributions. In such problems, each sample x itself is associated with a conditional distribution p(z∣x) represented by samples $\{z_i\}_{i=1…
A novel Federated Learning scheme using Optimal Transport for personalized model training.
problem Training models with data from clients having non-identically distributed data.
method Personalized Federated Learning scheme based on Optimal Transport (FedOT).
result FedOT scheme effectively transfers data from multiple distributions to a common domain and optimizes the prediction model.
Study shows realizable learnability doesn't imply agnostic learnability for distributions.
problem Learnability and robustness of distribution classes.
method Analyzes the relationship between learnability and robustness for distribution learning.
result Realizable learnability does not imply agnostic learnability for distributions.
Transfer learning aims to learn robust classifiers for the target domain by leveraging knowledge from a source domain. Since the source and the target domains are usually from different distributions, existing methods mainly focus on adapting the cross-domain marginal or conditional distributions. However, in real appl…
New algorithm for RL using mean embeddings of return distributions.
problem Improving reinforcement learning algorithms for dynamic programming.
method Mean embeddings of return distributions, novel algorithms for RL.
result Asymptotic convergence and improved performance in deep RL.
New methods learn sampling distributions for particle filters without supervision.
problem Designing accurate sampling distributions for nonlinear dynamical systems.
method Proposed four unsupervised learning methods for multivariate Gaussian and nonparametric distributions.
result Learned sampling distributions outperform designed ones in accuracy.
This paper tackles sequential distribution shifts in representation learning.
problem Learning meaningful representations in a sequence of distribution shifts.
method Nonlinear Independent Component Analysis (ICA) framework for continual causal representation learning.
result The method achieves performance comparable to joint training on multiple offline distributions and shows no benefit from the incoming new distribution on all latent variables.
Enhanced reinforcement learning using ensemble methods.
problem Improving reinforcement learning performance.
method Distributional reinforcement learning with ensemble group-aided training.
result Ensemble methods lead to more robust and efficient learning.
Diverse projection ensembles improve distributional reinforcement learning.
problem Learning the distribution of returns in reinforcement learning.
method Combining multiple projection methods to improve model diversity and exploration.
result Diverse projection ensembles lead to significant performance improvements in exploration tasks.
This paper tackles learning functions on manifolds using parallel distributed learning.
problem Learning real-valued functions on manifolds from input-output data pairs.
method Filtered hyperinterpolation and parallel distributed learning.
result Optimal approximation order for non-distributed case, and quantitative relations for distributed case.
We analyze reinforcement learning algorithms using a distributional approach.
problem Theoretical analysis of reinforcement learning algorithms for constant step-sizes.
method Distributional approach to theoretical analyses of reinforcement learning algorithms.
result TD(λ) and Q-Learning have contractive update rules in the space of distributions of functions, leading to exponentially fast convergence. Method learns conditional distributions using neural entropic optimal transport.
problem Challenges in learning multiple conditional distributions.
method Neural entropic optimal transport method with two networks and regularization.
result Effective learning of conditional distributions with limited samples.
New algorithm for safer machine learning with different testing and training distributions.
problem Challenges in modern machine learning where training and testing distributions differ.
method First-order optimization algorithm for superquantile-based learning.
result Promising numerical results show the approach's effectiveness.
Proposes vMF distribution for skewed elliptical distributions.
problem Skewed distributions not adequately modeled by symmetric distributions.
method Introduces von-Mises-Fisher (vMF) distribution to represent skewed elliptical distributions.
result vMF distribution provides an explicit and simple probability representation of skewed elliptical distributions.
GANs can learn hierarchical distributions in real-world images efficiently.
problem Understanding and efficiently learning complex, real-world distributions with GANs.
method Formally studying how GANs can learn hierarchically generated distributions close to real-life image distributions using SGDA.
result Training GANs via SGDA can efficiently learn distributions with a 'forward super-resolution' structure, both in sample and time complexities.
New approach to machine learning optimization using distribution space.
problem Optimization challenges in machine learning with non-convex constraints.
method Relaxation to convex optimization in distribution space, numerical algorithm based on mixture distributions.
result Approximate optimization in distribution space is consistent and effective.