LoCo-RLHF models diverse human feedback with contextual information.
problem Heterogeneous human feedback from diverse contexts and preferences.
method Low-rank contextual preference model, PRS policy.
result LoCo-RLHF achieves tighter sub-optimality gap than existing methods.
USAC balances pessimism and optimism in actor-critic training for better exploration and performance.
problem Excessive pessimism limits exploration, while excessive optimism leads to high-risk behaviors.
method Utility Soft Actor-Critic (USAC) dynamically adapts exploration based on critic uncertainty.
result USAC consistently outperforms state-of-the-art algorithms in continuous control tasks.
Bayesian approach to portfolio selection reduces pessimism in frequent trading.
problem Tackling the challenge of estimating drift in Merton's portfolio selection model.
method Bayesian distributionally robust control with nonlinear Wasserstein projections.
result Reduced pessimism and improved performance in frequent rebalancing compared to existing methods.
Proposes a new Bayesian learning method for optimal treatment regimes.
problem Sub-optimal policies in offline data due to lack of exploration.
method Integrates pessimism principle with Thompson sampling and Bayesian machine learning.
result Derives a credible set that uniformly lower bounds the optimal Q-function.
The paper introduces Bellman-consistent pessimism to improve offline reinforcement learning without overly pessimistic bias.
problem Offline reinforcement learning's challenge of discovering good policies without exhaustive exploration.
method Introduces Bellman-consistent pessimism for function approximation, improving sample complexity and adaptability.
result Improves sample complexity by O(d) in the action space finite case, and automatically adapts to bias-variance tradeoff. New method reduces over-pessimism in Bayesian control under parameter uncertainty.
problem Over-pessimism in Bayesian control due to misspecified priors.
method Distributionally robust Bayesian control (DRBC) with strong duality and optimization.
result Validated algorithm on synthetic and real data, reducing over-pessimism.
Pessimistic Q-learning improves sample efficiency in offline reinforcement learning.
problem Insufficient coverage and sample scarcity in offline reinforcement learning datasets.
method Pessimistic Q-learning algorithm for offline reinforcement learning, focusing on variance reduction.
result Near-optimal sample complexity achieved with the proposed algorithm.
Unified framework for analyzing pessimism in off-policy learning with regularized importance sampling.
problem High variance in importance weighting for off-policy learning.
method Unified PAC-Bayesian study of pessimism with regularized importance sampling.
result Derivation of a tractable PAC-Bayesian generalization bound for common importance weight regularizations.
The paper improves Q-learning by incorporating pessimism for better sample efficiency.
problem Improving sample efficiency in asynchronous Q-learning with non-i.i.d. data.
method Developed an algorithmic framework that incorporates the principle of pessimism into asynchronous Q-learning, penalizing infrequently-visited state-action pairs based on suitable lower confidence bounds (LCBs).
result Achieved near-optimal sample complexity, providing theoretical support for the use of pessimism in non-i.i.d. data.
New method optimizes offline linear bandits using different confidence sets.
problem Optimizing offline learning for linear contextual bandits.
method Introduces a family of pessimistic learning rules based on ℓp confidence sets. result The π^∞ rule achieves minimax performance and strictly dominates other predictors. Bayesian methods reduce variance in subspace identification for small data sets.
problem High variance in traditional subspace identification methods for large models or small sample sizes.
method Investigation of Bayesian estimation solutions (regularized and shrinkage estimators) for subspace identification.
result Bayesian estimators reduce estimation risk by up to 40% compared to traditional methods.
Subspace clustering refers to the problem of clustering unlabeled high-dimensional data points into a union of low-dimensional linear subspaces, assumed unknown. In practice one may have access to dimensionality-reduced observations of the data only, resulting, e.g., from "undersampling" due to complexity and speed con…
Pessimistic RL algorithm improves offline RL performance.
problem Insufficient dataset coverage in offline RL.
method Proposes a pessimistic variant of value iteration (PEVI) with a penalty function.
result Establishes upper bound on suboptimality for general MDPs, matching lower bound.
A new algorithm improves offline reinforcement learning robustness.
problem Finding optimal policies in perturbed environments from offline data.
method Doubly Pessimistic Model-based Policy Optimization (P^2MPO) framework.
result Proves sample efficiency with robust partial coverage data.
This paper develops a model of reference-dependent assessment of subjective beliefs in which loss-averse people optimally choose the expectation as the reference point to balance the current felicity from the optimistic anticipation and the future disappointment from the realisation. The choice of over-optimism or over…
GPS model predicts subspace-valued functions efficiently.
problem Accurate and efficient prediction of subspace-valued functions.
method Gaussian Process Subspace regression (GPS) model, using multivariate Gaussian distributions on Euclidean space.
result GPS provides accurate, smooth predictions with uncertainty quantification.
Subspace clustering is the problem of partitioning unlabeled data points into a number of clusters so that data points within one cluster lie approximately on a low-dimensional linear subspace. In many practical scenarios, the dimensionality of data points to be clustered are compressed due to constraints of measuremen…
Subspace clustering refers to the problem of clustering unlabeled high-dimensional data points into a union of low-dimensional linear subspaces, whose number, orientations, and dimensions are all unknown. In practice one may have access to dimensionality-reduced observations of the data only, resulting, e.g., from unde…
PASTA optimizes assortment selection using pessimism principle.
problem Optimizing assortment selection with limited data coverage.
method Pessimistic Assortment Optimization (PASTA) based on the principle of pessimism.
result PASTA correctly identifies optimal assortment with minimal data coverage.
Motivated by the idea of turbomachinery active subspace performance maps, this paper studies dimension reduction in turbomachinery 3D CFD simulations. First, we show that these subspaces exist across different blades---under the same parametrization---largely independent of their Mach number or Reynolds number. This is…
New model explains price dynamics of Bitcoin with psychological factors.
problem Understanding price variations in cryptocurrency markets with psychological factors.
method Extended agent-based model with heterogeneous psychological parameters.
result Model shows diverse dynamics based on psychological correlation.
We investigate a Gaussian mixture model (GMM) with component means constrained in a pre-selected subspace. Applications to classification and clustering are explored. An EM-type estimation algorithm is derived. We prove that the subspace containing the component means of a GMM with a common covariance matrix also conta…
Agents collaborate to reduce regret in a multi-agent linear bandit problem with side information.
problem Reducing regret in a multi-agent stochastic linear bandit with side information.
method A decentralized algorithm where agents communicate subspace indices and each plays a projected LinUCB on the corresponding low-dimensional subspace.
result Per-agent finite-time regret is much smaller when agents communicate compared to non-communicating case.
Study examines how pandemic anxiety affects financial market trust.
problem Anxiety during pandemic and trust in financial markets.
method Used Google search volume and stock market data to create mood indicators.
result Different clusters of countries and markets in terms of pessimism and optimism emerged.
Paper improves ℓ0-SSC for noisy data by proving SDP and proposing Noisy-DR-ℓ0-SSC.
problem Noisy data and less restrictive subspace affinity in sparse subspace clustering.
method Proposes Noisy-DR-ℓ0-SSC, which projects data onto a lower dimensional space and then applies noisy ℓ0-SSC. result Theoretical guarantee on the correctness of noisy ℓ0-SSC in terms of SDP on noisy data. Union of Subspaces (UoS) is a popular model to describe the underlying low-dimensional structure of data. The fine details of UoS structure can be described in terms of canonical angles (also known as principal angles) between subspaces, which is a well-known characterization for relative subspace positions. In this pa…
Active sampling selects few points for accurate model reduction of high-fidelity systems.
problem Efficiently identify dominant subspaces for model reduction of large training sets.
method Proposes an active sampling strategy to select a few points from the training set to estimate dominant subspaces accurately.
result Active sampling can provide 17x speed-up without sacrificing accuracy.
New MCMC algorithm reduces subset selection passes to 2 for optimal k-dimensional subspace approximation.
problem Subset selection for k-dimensional subspace approximation with ε-approximation. method MCMC sampling algorithm reducing passes to 2 for p=2 case, poly(k/ε) size subset. result Subset selection of nearly optimal size in 2 passes, (1+ε) approximation. Computing optimal transport (OT) between measures in high dimensions is doomed by the curse of dimensionality. A popular approach to avoid this curse is to project input measures on lower-dimensional subspaces (1D lines in the case of sliced Wasserstein distances), solve the OT problem between these reduced measures, a…
When data is sampled from an unknown subspace, principal component analysis (PCA) provides an effective way to estimate the subspace and hence reduce the dimension of the data. At the heart of PCA is the Eckart-Young-Mirsky theorem, which characterizes the best rank k approximation of a matrix. In this paper, we prove …
LR-EDNN reduces PDE solver complexity by limiting network weights to low-rank subspace.
problem Efficiently solving time-dependent PDEs with deep neural networks.
method Low-rank constraint on network weights using SVD for efficient parameter updates.
result LR-EDNN achieves comparable accuracy to full EDNN with fewer parameters and lower cost.
RS-NSGD improves SGD convergence for heavy-tailed noise.
problem Nonconvex optimization with heavy-tailed noise.
method Integrates direction normalization into subspace updates.
result Achieves better oracle complexity than full-dimensional normalized SGD.
Stochastic Sparse Subspace Clustering improves subspace clustering by reducing over-segmentation through dropout.
problem Over-segmentation in subspace clustering.
method Introducing dropout regularization to enforce denser connections between points from the same subspace.
result Stochastic Sparse Subspace Clustering effectively handles large datasets and reduces over-segmentation.
New assumptions and algorithm solve offline two-player zero-sum Markov games.
problem Solving offline two-player zero-sum Markov games under insufficient assumptions.
method Proposed unilateral concentration assumption and pessimism-type algorithm.
result Algorithm efficiently learns Nash equilibrium under unilateral concentration.
Subspace clustering methods based on expressing each data point as a linear combination of all other points in a dataset are popular unsupervised learning techniques. However, existing methods incur high computational complexity on large-scale datasets as they require solving an expensive optimization problem and perfo…
Meta-learning bandits by reducing dimensionality with PCA.
problem Learning multiple bandit tasks with shared structure.
method Online Principal Component Analysis (PCA) for dimensionality reduction, combined with optimistic and Thompson sampling strategies.
result Significant reduction in expected regret compared to existing methods.
A low-rank transformation learning framework for subspace clustering and classification is here proposed. Many high-dimensional data, such as face images and motion sequences, approximately lie in a union of low-dimensional subspaces. The corresponding subspace clustering problem has been extensively studied in the lit…
Latent variable models improve RL by facilitating efficient learning and exploration.
problem Improving sample efficiency in reinforcement learning.
method Representation view of latent variable models for state-action value functions, incorporating kernel embeddings and UCB exploration.
result Established sample complexity of the proposed approach in online and offline settings, demonstrated superior performance in benchmarks.
Distributionally Robust Supervised Learning (DRSL) is necessary for building reliable machine learning systems. When machine learning is deployed in the real world, its performance can be significantly degraded because test data may follow a different distribution from training data. DRSL with f-divergences explicitly …
In applications ranging from communications to genetics, signals can be modeled as lying in a union of subspaces. Under this model, signal coefficients that lie in certain subspaces are active or inactive together. The potential subspaces are known in advance, but the particular set of subspaces that are active (i.e., …
In this work we propose a method for reducing the dimensionality of tensor objects in a binary classification framework. The proposed Common Mode Patterns method takes into consideration the labels' information, and ensures that tensor objects that belong to different classes do not share common features after the redu…
Algorithm selects public datasets for private machine learning.
problem Choosing the most suitable public dataset for private machine learning.
method Measures gradient subspace distance between public and private datasets.
result Excess risk scales with the subspace distance between gradients.
We present a framework for supervised subspace tracking, when there are two time series xt and yt, one being the high-dimensional predictors and the other being the response variables and the subspace tracking needs to take into consideration of both sequences. It extends the classic online subspace tracking work…
New method reduces high-dimensional data to key features.
problem Challenges of high-dimensional data analysis and interpretability.
method Randomized search to produce subspaces, ensemble of models for variable selection.
result Outperforms existing methods in prediction and variable selection.
The paper develops methods to reduce deployment risk under dynamic covariate shifts.
problem Reduction of deployment risk under dynamic covariate shifts.
method Time-domain Poincare inequality and Jacobian-velocity theorem to identify and control directional tangent energy.
result Drift-aligned tangent regularization (DTR) reduces risk volatility and directional gain in low-rank drift regimes.
The main contribution of the paper is a new approach to subspace clustering that is significantly more computationally efficient and scalable than existing state-of-the-art methods. The central idea is to modify the regression technique in sparse subspace clustering (SSC) by replacing the ℓ1 minimization with a g…
Active subspace is a model reduction method widely used in the uncertainty quantification community. In this paper, we propose analyzing the internal structure and vulnerability and deep neural networks using active subspace. Firstly, we employ the active subspace to measure the number of "active neurons" at each inter…
Proposes PredVAR model for reduced-dimensional dynamics from noisy data.
problem Extracting low-dimensional dynamics from high-dimensional noisy data.
method Probabilistic reduced-dimensional vector autoregressive model with oblique projection.
result Iterative algorithm yields dynamic latent variables with rank-ordered predictability.