BlitzWS is a working set algorithm for convex problems with theoretical guarantees.
problem Optimizing subproblem size and stopping criteria for working set algorithms.
method BlitzWS proposes a principled approach with theoretical guarantees, optimizing subproblem size and stopping criteria based on progress.
result BlitzWS achieves fast convergence times for convex problems, including L1-regularized models and support vector machines.
Improved RL algorithm with linear MDPs for offline learning with partial data coverage.
problem Efficient offline RL with linear MDPs under partial data coverage.
method Primal-dual algorithm with O ( ε − 2 ) O(ε^{-2}) O ( ε − 2 ) sample complexity. result First computationally efficient algorithm with O ( ε − 2 ) O(ε^{-2}) O ( ε − 2 ) sample complexity for offline RL with linear MDPs under partial data coverage. Paper proposes a working set algorithm for non-convex sparse regression with provable convergence.
problem Estimating sparse linear models from high-dimensional data using non-convex regularizers.
method FireWorks algorithm based on non-convex reformulation and leveraging residual geometry.
result Convergence to a stationary point of the full problem with provable guarantees.
Algorithm estimates Gaussian parameters under unknown truncation sets.
problem Estimating Gaussian parameters when samples are truncated to unknown sets.
method Efficient algorithm for arbitrary unknown truncation sets, using Gaussian surface area as complexity measure.
result Algorithm works for large families of sets including intersections of halfspaces and general convex sets.
Algorithm tackles constrained reinforcement learning with concave-convex and knapsack constraints.
problem Constrained episodic reinforcement learning with concave rewards and convex constraints.
method Modular analysis with strong theoretical guarantees for concave-convex and knapsack settings.
result Significantly outperforms existing approaches in constrained episodic environments.
New algorithms prove fast convergence in complex min-max problems.
problem Proving fast convergence in nonconvex min-max optimization.
method Hamiltonian Gradient Descent (HGD) and Consensus Optimization (CO) algorithms.
result HGD and CO achieve linear convergence in various settings.
New algorithm achieves small-loss bounds in online learning with improved rates.
problem Achieving strong stability in online learning algorithms.
method Introduces ρ ρ ρ -separation to enforce strong stability, unifying previous approaches. result Oracle-efficient algorithm achieves small-loss bounds with improved rates.
First robust bandit algorithm for contextual bandits with sub-linear regret.
problem Vulnerability of linear contextual bandit algorithms to adversarial attacks.
method Proposes a robust bandit algorithm for stochastic linear contextual bandits under fully adaptive and omniscient attacks.
result Sub-linear regret under various attacks without requiring attack information.
Improved regret bounds for structured linear contextual bandits with Gaussian noise.
problem Optimizing bandit learning algorithms for structured contexts with Gaussian perturbations.
method Proposed simple greedy algorithms for structured linear contextual bandits with Gaussian noise.
result Unified regret analysis for structured parameters with geometric quantities as bounds.
Paper proposes a new method for estimating mixture proportions without irreducibility assumption.
problem Estimating mixture proportions when component distributions are not irreducible.
method Developed a resampling-based meta-algorithm that adapts existing MPE algorithms to non-irreducible settings.
result Empirical results show improved estimation performance compared to baseline methods and regrouping-based algorithms.
Algorithm learns graph structure and weights from noisy epidemic cascade data.
problem Learning graph structure and weights from noisy infection times of multiple epidemics.
method Developed algorithms for two noisy settings: limited-noise and extreme-noise, with polynomial time complexity.
result Optimal sample complexity and efficient algorithms for various graph types.
Support vector machine (SVM) training is an active research area since the dawn of the method. In recent years there has been increasing interest in specialized solvers for the important case of linear models. The algorithm presented by Hsieh et al., probably best known under the name of the "liblinear" implementation,…
New robust algorithms improve learning with feature feedback.
problem Interactive learning with discriminative feature feedback.
method Developed new robust interactive learning algorithms with improved mistake bounds.
result Achieved significantly lower mistake bounds in adversarial and stochastic settings.
New algorithms learn sparse set functions in non-orthogonal Fourier bases.
problem Learning sparse set functions in non-orthogonal Fourier bases.
method Novel algorithms using non-orthogonal Fourier transforms.
result At most n k − k log 2 k + k nk - k \log_2 k + k nk − k log 2 k + k queries for k k k non-zero Fourier coefficients. New algorithms find half-optimal independent sets in sparse graphs.
problem Finding large independent sets in sparse random graphs.
method Low-degree polynomial algorithms.
result Low-degree polynomial algorithms can find independent sets of half-optimal size.
Labeled Latent Dirichlet Allocation (LLDA) is an extension of the standard unsupervised Latent Dirichlet Allocation (LDA) algorithm, to address multi-label learning tasks. Previous work has shown it to perform in par with other state-of-the-art multi-label methods. Nonetheless, with increasing label sets sizes LLDA enc…
Efficient algorithms for online convex optimization with limited switching decisions.
problem Online convex optimization with limited switching decisions.
method Presented computationally efficient algorithms for both general and strongly convex losses.
result Regret bounds of O ( T / S ) O(T/S) O ( T / S ) for general convex losses and O ~ ( T / S 2 ) \widetilde O(T/S^2) O ( T / S 2 ) for strongly convex losses. This work improves algorithm design for structured Pfaffian settings.
problem Designing algorithms for specific application domains with theoretical guarantees.
method Data-driven algorithm design using hyperparameter tuning and learning guarantees.
result Introduced the Pfaffian GJ framework for providing learning guarantees for Pfaffian function classes.
This work analyzes batch MARL with networked agents, providing finite-sample bounds.
problem Understanding the theoretical foundation of decentralized batch MARL with networked agents.
method Developed batch MARL algorithms for two settings: collaborative and competitive networks, without a central controller.
result Quantified finite-sample errors of estimated action-value functions for both settings.
Inverse reinforcement learning (IRL) infers a reward function from demonstrations, allowing for policy improvement and generalization. However, despite much recent interest in IRL, little work has been done to understand the minimum set of demonstrations needed to teach a specific sequential decision-making task. We fo…
Survey of automated algorithm selection for various problems.
problem Selecting the best algorithm for each problem instance.
method Exploiting performance complementarity among algorithms.
result Major improvements in solving discrete combinatorial problems.
Efficient algorithms for online learning with changing action sets, achieving no-approximate-regret guarantees.
problem Online learning with sleeping experts/bandits, where only a subset of actions are available each time.
method Developed computationally efficient algorithms providing no-approximate-regret guarantees for the general problem and better approximation ratios for special cases.
result Achieved no-approximate-regret guarantees for the general sleeping expert/bandit problems and better approximation ratios for specific cases.
Unified framework for combinatorial and rounding algorithms in experimental design.
problem Designing and analyzing combinatorial and rounding algorithms for experimental design problems.
method Local search framework for combinatorial algorithms and regret minimization framework for rounding algorithms.
result Unified approach to match and improve all known results in D/A/E-design and obtain new results in unknown settings.
New algorithm for sequential off-policy learning improves performance over batch methods.
problem Training policies from logged interaction data in a sequential setting.
method Combines Logarithmic Smoothing with online PAC-Bayesian tools.
result Improves performance and accelerates convergence in sequential off-policy learning.
This work introduces online meta-learning, merging paradigms to enhance continual learning.
problem Continuous learning of new tasks with fast adaptation.
method Follow the meta leader algorithm, extending MAML to online setting with theoretical guarantees.
result Significant performance improvement over traditional online learning approaches.
This work proves lower bounds on a greedy teaching set construction algorithm.
problem Characterize the best-case teaching dimension of a concept class.
method A greedy algorithm that iteratively adds points to the teaching set to restrict the concept class the most.
result Lower bounds on the performance of the greedy approach for small k, extending up to k ≤ c*d for small constant c.
Study online learning with feedback graphs and switching costs, providing algorithms and optimal regret bounds.
problem Online learning with partial feedback and switching costs.
method Analysis of feedback graphs, lower bound on expected regret, new algorithms (Threshold Based EXP3, EXP3. SC).
result Order optimal algorithms for specific cases and Threshold Based EXP3 outperforms in empirical evaluations.
New algorithm for truncated linear regression without knowing the survival set.
problem Estimating the unknown regressor in truncated linear regression with an unknown survival set.
method Sub-Gaussian feature vectors and novel subroutine for learning unions of intervals.
result First algorithm with poly(d/ε) runtime for truncated linear regression with unknown survival set.
The success of machine learning on a given task dependson, among other things, which learning algorithm is selected and its associated hyperparameters. Selecting an appropriate learning algorithm and setting its hyperparameters for a given data set can be a challenging task, especially for users who are not experts in …
Two algorithms achieve optimal regret with limited adaptivity in multinomial logistic bandits.
problem Achieving optimal regret with limited adaptivity in multinomial logistic bandits.
method Presented two algorithms, B-MNL-CB and RS-MNL, for batched and rarely-switching paradigms.
result Achieved i l d e O ( T ) ilde{O}(\sqrt{T}) i l d e O ( T ) regret with limited adaptivity. Boosting is a popular way to derive powerful learners from simpler hypothesis classes. Following previous work (Mason et al., 1999; Friedman, 2000) on general boosting frameworks, we analyze gradient-based descent algorithms for boosting with respect to any convex objective and introduce a new measure of weak learner p…
We study online optimization of smoothed piecewise constant functions over the domain [0, 1). This is motivated by the problem of adaptively picking parameters of learning algorithms as in the recently introduced framework by Gupta and Roughgarden (2016). Majority of the machine learning literature has focused on Lipsc…
New algorithms estimate parameters of Gaussian and non-Gaussian distributions from truncated samples.
problem Estimating distributional parameters from truncated samples.
method Polynomial time algorithms for exponential families and simple sets.
result Efficient algorithms for estimating parameters of various distributions from truncated samples.
New algorithm for online omniprediction with strong guarantees for continuous hypothesis classes.
problem Online adversarial learning with continuous hypothesis classes.
method Developed an oracle-efficient online multicalibration algorithm for infinite benchmark classes.
result First efficient online omnipredictor with strong guarantees for Lipschitz convex loss functions.
Learning sparse linear models with two-way interactions is desirable in many application domains such as genomics. l1-regularised linear models are popular to estimate sparse models, yet standard implementations fail to address specifically the quadratic explosion of candidate two-way interactions in high dimensions, a…
We extend a result regarding the Random Backward Iteration algorithm for drawing Julia sets (known to work for certain rational semigroups containing a non-Möbius element) to a class of Möbius semigroups which includes certain settings not yet been dealt with in the literature, namely, when the Julia set is not a thick…
New algorithms for hierarchical classification using conformal prediction.
problem Valid prediction sets in hierarchical classification tasks.
method Extended split conformal prediction framework with two inference algorithms.
result Empirical evaluations show effectiveness in achieving nominal coverage.
This work uncovers algorithm-dependent regularisation in diffusion models.
problem Understanding and improving generalisation in high-dimensional diffusion models.
method Algorithmic stability and score stability analysis.
result Identifies multiple sources of implicit regularisation unique to diffusion models.
Adaptive Bayesian Optimization for resource-constrained experiments with switching costs.
problem Sequential experimental design with varying costs for changing design variables.
method Adapted batch algorithms to sequential problem, proposing cost-aware and cost-ignorant methods.
result Cost-aware algorithm outperforms tuned process-constrained algorithms in all settings considered.
New algorithm optimizes linear system estimation from single trajectory.
problem Estimating LTI systems from a single trajectory.
method SGD with Reverse Experience Replay ( S G D − R E R \mathsf{SGD}-\mathsf{RER} SGD − RER ) result Optimal guarantees for parameter and prediction errors.
Scalable PnP-ADMM for large-scale imaging problems.
problem Heavy computational and memory requirements of current PnP algorithms.
method Incremental variant of PnP-ADMM with theoretical convergence guarantees.
result Fast convergence and scalability compared to existing PnP algorithms.
This paper is about randomized iterative algorithms for solving a linear system of equations X β = y X β= y X β = y in different settings. Recent interest in the topic was reignited when Strohmer and Vershynin (2009) proved the linear convergence rate of a Randomized Kaczmarz (RK) algorithm that works on the rows of X X X (data points…
Algorithm learns optimal dynamic mechanisms from data.
problem Designing optimal mechanisms for dynamic settings with unknown reward functions.
method Offline reinforcement learning with pessimism principle.
result Learned mechanisms are efficient, individually rational, and truthful.
New algorithms for fair data summarization in massive data models.
problem Fair k k k -center problem in massive data. method Streaming and distributed algorithms with provable guarantees.
result First distributed algorithm with constant approximation ratio.
Pipeline selects best RL policy from limited data.
problem Selecting best RL policy from small datasets.
method Task- and method-agnostic pipeline using multiple data splits.
result Pipeline produces higher-performing policies.
Novel framework for data sharing and coordinated exploration in concurrent RL with non-identical environments.
problem Learning more data-efficient and better policies in concurrent RL with non-identical environments.
method Proposes a novel algorithmic framework that leverages causal inference via ANM-MM to extract model parameters and a new data sharing scheme based on similarity measures.
result Demonstrates superior learning speeds on various tasks and effectiveness of diverse action selection.
New tighter bounds for learning algorithms from Steinke & Zakynthinou's supersample setting.
problem Improving generalization bounds for machine learning algorithms.
method Information-theoretic approach using projected loss and Rademacher sequence.
result The new bounds are tighter than previous information-theoretic bounds.
New algorithm reduces adaptive regret without projections.
problem Computational expense of projections in online convex optimization.
method Lazy gradient-based algorithm with set-membership computations.
result Near-optimal adaptive regret bounds for general convex functions.