Paper improves sample complexity for reward-free RL in low-rank MDPs.
problem Reward-free RL in low-rank MDPs with unknown representation and weights.
method Proposes a novel model-based algorithm RAFFLE with improved sample complexity.
result RAFFLE achieves ε-optimal policy and accurate system identification with significantly fewer samples. New reinforcement learning algorithm achieves instance-optimal sample complexity.
problem Achieving low regret and identifying optimal policies in reinforcement learning.
method A novel planning-based algorithm that explicitly accounts for state visitation distributions.
result The proposed algorithm attains nearly minimax optimal sample complexity, improving over worst-case bounds.
Study shows a tradeoff between sample complexity and computational efficiency for learning halfspaces with random noise.
problem PAC learning γ-margin halfspaces with Random Classification Noise.
method Established an information-computation tradeoff and provided a simple efficient algorithm with sample complexity O(1/(γ^2 ε^2)). Also, proved lower bounds for SQ algorithms and low-degree polynomial tests.
result Inherent gap between sample complexity and computational efficiency for learning halfspaces with random noise.
Paper tackles low sample and communication complexities in decentralized bilevel optimization.
problem Decentralized bilevel optimization problems with limited computation and communication capabilities.
method Proposes INTERACT and SVR-INTERACT algorithms to achieve low sample and communication complexities.
result Achieves both low sample and communication complexities for solving decentralized bilevel optimization problems.
Shallow diffusion models learn hidden low-dimensional structures effectively.
problem Learning from high-dimensional signals like images and video.
method Analysis of shallow diffusion models over the Barron space of single layer neural networks.
result Shallow diffusion models can adapt to simple low-dimensional structures, overcoming the curse of dimensionality.
New method reduces variance in stochastic optimization with high confidence.
problem Achieving high-probability guarantees in stochastic optimization with weaker noise assumptions.
method Stochastic proximal point method combining proximal subproblem solver and probability booster.
result Demonstrates convergence with low sample complexity under bounded variance assumptions.
LMC improves sampling from complex distributions using quasi-random sequences.
problem Sampling from complex high-dimensional distributions with high accuracy.
method Using completely uniformly distributed (CUD) sequences in Langevin Monte Carlo (LMC) to generate Gaussian perturbations.
result LMC with low-discrepancy CUD sequences achieves smaller estimation error than standard LMC.
Scaled gradient descent improves matrix recovery for ill-conditioned matrices with optimal sampling complexity.
problem Recovering low-rank matrices from limited measurements efficiently and accurately.
method Scaled gradient descent (ScaledGD) with optimal sample complexity and improved iteration complexity.
result ScaledGD achieves optimal sample complexity and improved iteration complexity for ill-conditioned matrices.
Wedge Sampling improves tensor completion with nearly-linear sample complexity.
problem Efficiently completing low-rank tensors from a subset of entries.
method Non-adaptive wedge sampling to promote structured connections in tensor completion.
result Polynomial-time algorithms achieve weak and exact recovery with nearly linear sample complexity.
A new algorithm reduces sample complexity for learning Q-functions in reinforcement learning.
problem Efficiently learning Q-functions in reinforcement learning with continuous state and action spaces.
method Developed a simple, iterative learning algorithm that estimates low-rank Q-functions.
result Achieved exponential improvement in sample complexity for low-rank Q-functions.
Paper finds a lower bound for estimating low-rank matrices in logistic regression.
problem Estimating low-rank coefficient matrices in logistic regression.
method Derives a minimax lower bound on the risk.
result The bound depends on matrix dimensions, rank, and sample size.
NPMD uses CNNs to optimize policies on low-dimensional manifolds, reducing sample complexity.
problem Explaining the effectiveness of deep policy gradient methods in high-dimensional RL.
method Neural policy mirror descent (NPMD) with CNNs, considering state spaces as low-dimensional manifolds.
result NPMD finds ε-optimal policies with O(ε^(-d/α-2)) samples, leveraging low-dimensional structure.
New BE dimension measure reveals rich RL problems with sample-efficient algorithms.
problem Finding sample-efficient algorithms for complex RL problems.
method Introducing Bellman Eluder (BE) dimension and designing GOLF and OLIVE algorithms.
result GOLF and OLIVE algorithms learn near-optimal policies for low BE dimension problems with polynomial samples.
We analyze low rank tensor completion (TC) using noisy measurements of a subset of the tensor. Assuming a rank-r, order-d, N×N×⋯×N tensor where r=O(1), the best sampling complexity that was achieved is O(N2d), which is obtained by solving a tensor nuclear-norm minimizatio…
Study shows low-complexity models can perform as well as state-of-the-art on small datasets.
problem Performance of deep learning models on small datasets.
method Wide variety of experiments with different deep learning architectures on small datasets.
result Low-complexity models can perform comparably well or better than state-of-the-art models on small datasets.
We develop a sequential low-complexity inference procedure for Dirichlet process mixtures of Gaussians for online clustering and parameter estimation when the number of clusters are unknown a-priori. We present an easily computable, closed form parametric expression for the conditional likelihood, in which hyperparamet…
New algorithm learns low-rank matrices with linear number of samples.
problem Learning low-rank matrices efficiently in latent-variable applications.
method Proposed algorithm that uses linear number of samples in high dimension.
result Learning kimesk, rank-r, matrices requires $Ω( rac{kr}{ε^2})$ samples. Safe exploration in RF-RL doesn't increase sample complexity.
problem Achieving optimal policies with safety constraints in reward-free RL.
method Proposed SWEET framework for tabular and low-rank MDP settings, leveraging truncated value functions.
result Sample complexities match or outperform constraint-free counterparts, proving safety constraints have little impact.
Hierarchical decoupling improves sample efficiency for complex robots.
problem Learning long-range behaviors on complex robots.
method Two-part policy: low-level imitation and high-level transfer, with KL regularization.
result Hierarchical transfer significantly improves zero-shot high-level transfer and stabilizes learning.
This paper explores and analyzes two randomized designs for robust Principal Component Analysis (PCA) employing low-dimensional data sketching. In one design, a data sketch is constructed using random column sampling followed by low dimensional embedding, while in the other, sketching is based on random column and row …
Derives a method to optimize high-dimensional functions on low-dimensional manifolds.
problem High-dimensional derivative-free optimization with high sample complexity.
method Online learning approach that learns the manifold while optimizing the function.
result Significantly reduces sample complexity compared to existing methods.
ETC learns minimal representations for reinforcement learning in POMDPs.
problem Sample complexity challenges in reinforcement learning for POMDPs.
method ETC learns low-dimensional features and embeddings at two levels, optimizing policy.
result ETC achieves polynomial sample complexity for POMDPs with low-rank transition kernels.
New tensor model reduces GLM estimation error and sample complexity.
problem Estimating GLM coefficients with reduced sample complexity.
method Developed LSR tensor model and block coordinate descent algorithm.
result Minimax lower bound on estimation error, suggesting lower sample complexity.
This paper improves diffusion models for low-dimensional data.
problem Theoretical foundations of diffusion models are lacking for low-dimensional data.
method Score approximation, estimation, and distribution recovery of diffusion models on low-dimensional data.
result Sample complexity bounds for distribution estimation using diffusion models are provided.
Two-layer NN with channel attention learns low-degree spherical polynomials efficiently.
problem Learning low-degree spherical polynomials with over-parameterized neural networks.
method Two-layer neural network with channel attention, vanilla gradient descent, learnable channel selection.
result Minimally improved sample complexity of $n \asymp Θ(d^{\ell_0}/\eps)$ for learning low-degree polynomials.
New method fits low-rank models for egocentrically sampled networks.
problem Statistical modeling of egocentrically sampled partial networks.
method Graph spectral properties-based approach for low-rank models.
result Consistent recovery of missing subnetworks for sparse networks.
Algorithm recovers multiple low-rank matrices from unlabeled data.
problem Learning mixtures of low-rank models from unlabelled data.
method Three-stage meta-algorithm that copes with non-convexity and noise.
result Near-optimal sample and computational complexities under Gaussian designs.
Improved sample complexity for Gaussian process approximations.
problem Efficiently approximating Gaussian processes with sparse spectrum.
method Improved sample complexity analysis and auto-encoding algorithm.
result Gaussian process predictions and model evidence can be well-approximated with low sample complexity.
Efficiently samples complex distributions using tensor train format.
problem Sampling from high-dimensional complex probability densities efficiently.
method Integrates tensor train format with backward stochastic differential equations (BSDEs) for fast, robust, and accurate sampling.
result Improved efficiency in sampling from challenging target distributions.
CADGMM detects anomalies by capturing complex correlations in data.
problem Detecting anomalies in complex, unstructured data.
method CADGMM uses a graph structure to encode correlations, then a dual-encoder to learn low-dimensional latent space, followed by a Gaussian Mixture Model for anomaly detection.
result CADGMM effectively detects anomalies in real-world datasets.
Study shows how diffusion models learn on low-dimensional manifolds.
problem Learning efficiency of diffusion models on manifolds.
method Analyzes denoising score matching with random feature neural networks.
result Sample complexity scales linearly with intrinsic dimension, not ambient dimension.
The paper analyzes how good initial guesses affect the amount of data needed for low-rank matrix recovery.
problem Theoretical guarantee of local optimization algorithms requires excessive data to prevent spurious local minima.
method Quantifies the relationship between initial guess quality and sample complexity using restricted isometry constant.
result A linear improvement in initial guess quality leads to a constant factor improvement in sample complexity.
Backward exploration reduces sample complexity in policy evaluation.
problem Empirical policy evaluation in reinforcement learning.
method Backward exploration algorithms from high-cost states.
result Reduced average-case sample complexity to O(logS). Latent DiTs improve data distribution recovery and inference efficiency under low-dimensional latent space.
problem Improving data distribution recovery and inference efficiency in latent DiTs.
method Investigates statistical and computational limits of latent DiTs under low-dimensional latent space assumption, deriving approximation error bounds, sample complexity, and efficient inference and training algorithms.
result Latent DiTs can bypass high dimensionality challenges and achieve almost-linear time inference and training.
We develop a method to summarize causal models with cycles in cubic time.
problem Cycles in high-dimensional causal models limit applicability of existing methods.
method We relax the acyclicity assumption in LiNG models and develop a low-dimensional DAG summary.
result Our method allows recovery of a low-dimensional DAG from high-dimensional data with cycles.
Projective DP-SGD reduces privacy error by identifying low-dimensional gradient subspaces.
problem Differentially private SGD's error rate scales with model's dimensionality, problematic for over-parameterized models.
method Projective DP-SGD, projecting noisy gradients to a low-dimensional subspace identified from a public dataset.
result The method reduces the dependence on model dimensionality, improving accuracy in high privacy regimes.
We study the problem of learning a distribution from samples, when the underlying distribution is a mixture of product distributions over discrete domains. This problem is motivated by several practical applications such as crowd-sourcing, recommendation systems, and learning Boolean functions. The existing solutions e…
Neural network learns low-dimensional polynomials with SGD near information-theoretic limit.
problem Learning a single-index target function with gradient descent.
method Two-layer neural network optimized by SGD on squared loss.
result Sample and runtime complexity of n≃T=Θ(d⋅polylogd) for polynomial single-index models, matching information theoretic limit up to polylogarithmic factors. This paper is concerned with the problem of low rank plus sparse matrix decomposition for big data. Conventional algorithms for matrix decomposition use the entire data to extract the low-rank and sparse components, and are based on optimization problems with complexity that scales with the dimension of the data, which…
SPREV simplifies visualization of complex labeled datasets.
problem Challenges of reducing dimensions and visualizing labeled datasets with small class size, high dimensionality, and low sample size.
method SPREV uses a novel dimensionality reduction technique integrating geometric principles.
result SPREV effectively visualizes hidden patterns in complex labeled datasets.
New methods evaluate data representations by complexity of low-loss predictor learning.
problem Evaluating quality of data representations for downstream tasks.
method Surplus Description Length (SDL) and ε Sample Complexity (εSC) methods.
result Methods measure the information needed to approximate optimal predictor up to specified tolerance.
Study efficient estimation of hidden subspaces in Gaussian Multi-index models.
problem Estimating hidden subspaces in Gaussian Multi-index models with low-dimensional projections.
method Introduced the generative leap exponent and developed an agnostic sequential estimation procedure using spectral U-statistics.
result Achieved optimal sample complexity of $n=Θ(d^{1 \vee \k/2})$ for efficient estimation.
New study shows Gaussian samplers struggle with heavy-tailed targets, while stable samplers excel.
problem The difficulty of sampling from heavy-tailed distributions using Gaussian versus stable oracles.
method Comparison of Gaussian and stable oracles for proximal samplers.
result Gaussian samplers have a fundamental barrier for high-accuracy guarantees in heavy-tailed sampling, while stable samplers excel.
Convex optimization method recovers low-rank matrices from rank-one projections efficiently.
problem Recovering low-rank matrices from limited rank-one projections.
method Unlifted convex optimization with subgradient method.
result The estimator succeeds with high probability if the number of measurements exceeds r2(d1+d2) up to logarithmic factors. We study the problem of estimating low-rank matrices from linear measurements (a.k.a., matrix sensing) through nonconvex optimization. We propose an efficient stochastic variance reduced gradient descent algorithm to solve a nonconvex optimization problem of matrix sensing. Our algorithm is applicable to both noisy and…
Gradient Descent with Projection learns low-degree polynomials efficiently.
problem Learning low-degree spherical polynomials with neural networks.
method Over-parameterized two-layer neural network with Gradient Descent with Projection.
result Achieves nearly minimax optimal sample complexity and risk bound.
Study shows sample complexity for logistic regression with normal covariates.
problem Estimating parameters of logistic regression with normal design.
method Analyzes sample complexity in terms of dimension and inverse temperature.
result Shows two change-points in sample complexity curve based on inverse temperature.
ConvResNets approximate Besov functions and classify on low-dimensional manifolds.
problem Lack of statistical theories for deep learning on high-dimensional data.
method Exploits low-dimensional geometric structures of real-world data sets using ConvResNets.
result ConvResNets can approximate Besov functions and learn classifiers with optimal excess risk.