New study shows Gaussian samplers struggle with heavy-tailed targets, while stable samplers excel.
problem The difficulty of sampling from heavy-tailed distributions using Gaussian versus stable oracles.
method Comparison of Gaussian and stable oracles for proximal samplers.
result Gaussian samplers have a fundamental barrier for high-accuracy guarantees in heavy-tailed sampling, while stable samplers excel.
New algorithm learns POMDPs without computational oracles.
problem Learning near-optimal policies in POMDPs with computationally hard oracles.
method Quasipolynomial-time algorithm using barycentric spanners for policy covers.
result First oracle-free learning algorithm for observable POMDPs.
New algorithm achieves small-loss bounds in online learning with improved rates.
problem Achieving strong stability in online learning algorithms.
method Introduces ρ ρ ρ -separation to enforce strong stability, unifying previous approaches. result Oracle-efficient algorithm achieves small-loss bounds with improved rates.
Adaptive algorithm reduces regret in causal bandits.
problem Minimize regret in causal bandits with unknown d-separators.
method Adaptive algorithm exploiting d-separators without prior knowledge.
result Significantly smaller regret than previous methods.
Audio source separation is the process of separating a mixture (e.g. a pop band recording) into isolated sounds from individual sources (e.g. just the lead vocals). Deep learning models are the state-of-the-art in source separation, given that the mixture to be separated is similar to the mixtures the deep model was tr…
New algorithms sample convex bodies using Markov chains and restricted Gaussian oracles.
problem Sampling uniformly from convex bodies efficiently.
method Markov chain Monte Carlo with proximal sampler and restricted Gaussian oracle.
result Efficient implementation of RGO for uniform sampling on convex bodies.
Gradient descent is efficient for solving feasibility problems with minimal memory and queries.
problem Finding a point in a given set using a memory-constrained algorithm with a separation oracle.
method Oracle complexity lower bounds for gradient descent and other algorithms.
result Gradient descent is Pareto-optimal in the oracle complexity/memory tradeoff for feasibility problems.
New PCstar algorithm discovers causal structure of max-linear Bayesian networks.
problem Discovering causal structure in max-linear Bayesian networks due to non-faithfulness.
method PC algorithm modified with C ∗ C^\ast C ∗ -separation assumptions. result PCstar algorithm can orient additional edges not possible with standard PC algorithm.
Paper tackles online learning on curved spaces without projections.
problem Online learning on Riemannian manifolds with computational constraints.
method Develops projection-free algorithms for geodesically convex optimization.
result Achieves sub-linear regret guarantees in online geodesically convex optimization.
New sampling methods for constrained and composite distributions.
problem Sampling from log-concave distributions with constraints and composite structures.
method Proximal sampler applied to lifted convex sets with separation and subgradient oracles.
result Practical and unbiased samplers for constrained and composite distributions.
New method recovers clusters in non-convex finite metric spaces with oracle queries.
problem Exact recovery of clusters in non-convex finite metric spaces.
method Introducing ( β , γ ) (β,γ) ( β , γ ) -convexity and a deterministic algorithm using oracle queries. result Clusters can be recovered using O ( k 2 log n + k 2 ( 6 / β γ ) d e n s ( X ) ) O(k^2 \log n + k^2 (6/βγ)^{dens(X)}) O ( k 2 log n + k 2 ( 6/ β γ ) d e n s ( X ) ) same-cluster queries. We give nearly matching upper and lower bounds on the oracle complexity of finding ε ε ε -stationary points ( ∥ ∇ F ( x ) ∥ ≤ ε \| \nabla F(x) \| \leqε ∥∇ F ( x ) ∥ ≤ ε ) in stochastic convex optimization. We jointly analyze the oracle complexity in both the local stochastic oracle model and the global oracle (or, statistical learning) model. This allows u…
Oracle-efficient algorithms for online learning with smoothed and hint-adversaries.
problem Online learning with beyond worst-case adversaries.
method Oracle-efficient algorithms for two settings: smoothed analysis and K K K -hint transductive learning. result Oracle-efficient regret bounds for learning real-valued and binary-valued functions.
In this paper, we propose a two-step training procedure for source separation via a deep neural network. In the first step we learn a transform (and it's inverse) to a latent space where masking-based separation performance using oracles is optimal. For the second step, we train a separation module that operates on the…
New algorithms solve convex optimization problems with limited memory.
problem Solving convex optimization problems with constrained memory.
method Recursive cutting-plane algorithms dividing variables into blocks.
result Achieves optimal memory usage and oracle complexity in certain regimes.
The study quantifies the information needed for causal queries at different levels of Pearl's hierarchy.
problem How much additional information is needed for interventional and counterfactual queries compared to observational queries?
method Formalized via query-class description length, using Kolmogorov complexity of answer oracles induced by SCMs.
result Binary acyclic SCMs show a quadratic gap between observational and interventional descriptions, and a logarithmic gap between interventional and counterfactual descriptions.
We consider the problem of estimating the phases of K mixed complex signals from a multichannel observation, when the mixing matrix and signal magnitudes are known. This problem can be cast as a non-convex quadratically constrained quadratic program which is known to be NP-hard in general. We propose three approaches t…
New algorithm reduces super-arm selection complexity exponentially.
problem Combinatorial multi-armed bandits with cardinality constraint.
method Combination of group-testing and quantized Thompson sampling.
result Achieves same regret order as state-of-the-art algorithms with perfect oracle, but with reduced complexity.
Develops new methods to estimate treatment effects in survival data with competing risks.
problem Estimating treatment effects in survival data with competing risks.
method Censoring Unbiased Transformations (CUTs) for survival outcomes with and without competing risks.
result Consistent estimates of heterogeneous cumulative incidence effects and total effects using HTE learners.
This paper improves universal sound separation using sound classification.
problem Separating acoustic sources from an open domain, regardless of their class.
method Utilizing semantic embeddings from a sound classifier to condition a separation network.
result Classifier embeddings provide nearly one dB of SNR gain, and iterative models achieve significant performance.
Study phase transitions in shuffled regression problems.
problem Phase transitions in shuffled regression problems.
method Transformed permutation recovery into probabilistic graphical model, used message passing (MP) algorithm and branching random walk process.
result Characterized impact of signal-to-noise-ratio ( $\snr$ ) on permutation recovery, proposed Gaussian approximation method.
Unified approach for quantum and classical learning from evaluation oracles.
problem Learning from evaluation oracles in quantum and classical settings.
method Inspired by Kearns' SQ and Valiant's weak evaluation oracle, a unified framework is established.
result Characterizes query complexity for learning linear function classes and extends learnability results for quantum circuits.
New algorithm for online omniprediction with strong guarantees for continuous hypothesis classes.
problem Online adversarial learning with continuous hypothesis classes.
method Developed an oracle-efficient online multicalibration algorithm for infinite benchmark classes.
result First efficient online omnipredictor with strong guarantees for Lipschitz convex loss functions.
This article establishes the performance of stochastic blockmodels in addressing the co-clustering problem of partitioning a binary array into subsets, assuming only that the data are generated by a nonparametric process satisfying the condition of separate exchangeability. We provide oracle inequalities with rate of c…
New algorithm solves complex optimization problems without needing projections.
problem Optimizing nested functions under convex constraints with noisy evaluations.
method Projection-free conditional gradient-type algorithm for smooth stochastic multi-level composition optimization.
result The algorithm achieves ε ε ε -stationary solutions with complexity bounds independent of ε ε ε and T T T . A method for finding most influential sets reduces a complex problem to a sequence of simpler top- k k k problems.
problem Identifying most influential subsets in complex models.
method Reduces the problem to a sequence of top- k k k problems using Dinkelbach's method. result The method returns a globally optimal set for the univariate ratio objective, including partial linear models.
Paper presents a novel waveform-to-waveform model for music source separation.
problem Isolating individual instruments from mixed music recordings.
method Adapted Conv-Tasnet to waveform domain and developed Demucs with U-Net and bidirectional LSTM.
result Demucs achieves superior performance on music source separation tasks, surpassing existing state-of-the-art.
New methods solve complex optimization problems without strong convexity assumptions.
problem Complex bilevel optimization problems with minimax lower-level structures.
method Penalty-based first-order methods for bilevel minimax optimization.
result Achieves ε ε ε -KKT point with improved oracle complexity. A desirable property of interpretable models is small size, so that they are easily understandable by humans. This leads to the following challenges: (a) small sizes typically imply diminished accuracy, and (b) bespoke levers provided by model families to restrict size, e.g., L1 regularization, might be insufficient to…
Oracle-efficient algorithms reduce combinatorial semi-bandit regret to logarithmic time.
problem Scalability issue in combinatorial semi-bandit problems due to high combinatorial optimization costs.
method Oracle-efficient frameworks that minimize oracle queries while maintaining tight regret guarantees.
result Achieved i l d e O ( T ) ilde{O}(\sqrt{T}) i l d e O ( T ) regret with O ( log log T ) O(\log\log T) O ( log log T ) oracle queries for worst-case linear rewards. New analysis shows Thompson Sampling can work with greedy approximations in combinatorial bandits.
problem Thompson Sampling's theoretical limits with greedy approximations in combinatorial semi-bandits.
method Study with greedy oracle, providing lower and upper bounds on regret.
result First theoretical results showing TS can work with greedy approximations, breaking misconceptions.
MAMBA learns policies competitive with multiple conflicting oracles.
problem Learning policies from multiple conflicting oracles in reinforcement learning.
method MAMBA uses a gradient estimator in the style of GAE to optimize policies, leveraging demonstrations from multiple weak oracles.
result MAMBA outperforms the state-of-the-art in learning policies competitive with multiple conflicting oracles.
We study the problem of interactively learning a binary classifier using noisy labeling and pairwise comparison oracles, where the comparison oracle answers which one in the given two instances is more likely to be positive. Learning from such oracles has multiple applications where obtaining direct labels is harder bu…
Quadratic memory is essential for optimal convex optimization queries.
problem Optimal query complexity for convex optimization and feasibility problems.
method Lower bounds on query complexity for convex optimization and feasibility problems.
result Center-of-mass algorithms are Pareto-optimal for both convex optimization and feasibility problems.
New oracle uses uncertainty for active classification with noisy feedback.
problem Improving query complexity in interactive binary classifier learning.
method Proposes a new pairwise comparison oracle that considers uncertainty and an adaptive labeling algorithm.
result Demonstrates improved performance and efficiency compared to existing methods.
Quantum oracles help identify counterfactuals better than classical ones.
problem Identifying unknown causal parameters in causal models.
method Using quantum oracles to query and identify all causal parameters and counterfactuals.
result Quantum oracles enable identification of all two-way joint counterfactuals and tighter bounds on higher-order counterfactuals.
SoQal reduces oracle label requests in active learning by up to 35%.
problem Exploiting unlabelled data in healthcare requires costly oracle labeling.
method Dynamic questioning strategy to minimize oracle label requests.
result SoQal reduces oracle label requests by up to 35%.
Paper addresses online alignment of large language models under uncertain preference feedback.
problem Online alignment of large language models with misspecified preference feedback.
method Formulates an oracle-robust objective as a worst-case optimization problem for log-linear policies, and develops projected stochastic composite updates.
result Shows that the robust objective admits an exact closed-form decomposition and achieves O ~ ( ε − 2 ) \widetilde{O}(\varepsilon^{-2}) O ( ε − 2 ) oracle complexity. We consider the problem of minimizing the sum of submodular set functions assuming minimization oracles of each summand function. Most existing approaches reformulate the problem as the convex minimization of the sum of the corresponding Lovász extensions and the squared Euclidean norm, leading to algorithms requiring …
New algorithm optimizes stochastic optimization with circular dependency.
problem Circular dependency between decision variable and importance sampling.
method Single-loop stochastic approximation algorithm based on Nesterov's dual averaging.
result Achieves minimal asymptotic variance and resolves circular optimization challenge.
New oracles improve stochastic optimization with noisy or biased measurements.
problem Optimizing functions with noisy or biased measurements.
method Introduced biased gradient oracles for stochastic optimization, analyzed RSG and SGD algorithms with these oracles.
result Derived non-asymptotic bounds for convergence rates of algorithms with biased gradient oracles.
Algorithm solves online binary classification and infinite games using ERM oracle.
problem Online learning and solving infinite games with computationally inefficient oracles.
method Proposes an algorithm relying solely on ERM oracle calls for online binary classification and nonparametric games.
result Achieves finite and sublinearly growing regret in various settings.
New algorithm learns efficiently with a simple 'yes/no' oracle.
problem Can efficient learning be achieved with a simpler oracle than ERM?
method Developed an oracle that returns 'yes' or 'no' for realizable datasets.
result Learnability is possible with a polynomial price in VC dimension.
Average Oracle outperforms DCC+NLS in portfolio optimization.
problem Optimizing portfolio performance in volatile markets.
method Comparing the Average Oracle to various DCC+NLS variants.
result The Average Oracle consistently yields higher Sharpe ratios.
New lower bounds for bilevel optimization with first-order oracles.
problem Complexity of bilevel optimization with first-order oracles.
method Development of hard instances and proof of lower bounds.
result Nontrivial lower bounds for first-order zero-respecting algorithms.
Study on tradeoffs between mistakes and ERM oracle calls in online and transductive learning.
problem Analyzing online and transductive learning with limited ERM and weak consistency oracle access.
method Proves lower bounds and upper bounds on mistakes and oracle calls, considering realizable and agnostic cases.
result Achieves optimal mistake bounds with weak consistency queries for certain concept classes.
In this paper, we propose to combine imitation and reinforcement learning via the idea of reward shaping using an oracle. We study the effectiveness of the near-optimal cost-to-go oracle on the planning horizon and demonstrate that the cost-to-go oracle shortens the learner's planning horizon as function of its accurac…
Panoptic trades options without oracles on Ethereum.
problem Trading options without relying on oracles.
method Perpetual, trustless, instant-settlement protocol on Ethereum.
result Trustless, permissionless trading of options on Uniswap v3.