LEAQI learns to query an expert less often for noisy guidance.
problem Active imitation learning with noisy guidance to reduce expert query complexity.
method Developed LEAQI, a difference classifier that predicts expert disagreement with noisy heuristic.
result Significantly fewer queries to the expert with comparable or better accuracy.
In this paper, we study Reinforcement Learning from Demonstrations (RLfD) that improves the exploration efficiency of Reinforcement Learning (RL) by providing expert demonstrations. Most of existing RLfD methods require demonstrations to be perfect and sufficient, which yet is unrealistic to meet in practice. To work o…
Regression problems assume every instance is annotated (labeled) with a real value, a form of annotation we call \emph{strong guidance}. In order for these annotations to be accurate, they must be the result of a precise experiment or measurement. However, in some cases additional \emph{weak guidance} might be given by…
Deep reinforcement learning has achieved great successes in recent years, however, one main challenge is the sample inefficiency. In this paper, we focus on how to use action guidance by means of a non-expert demonstrator to improve sample efficiency in a domain with sparse, delayed, and possibly deceptive rewards: the…
GH-PID uses guided harmonic paths for efficient SOT with interpretable diagnostics.
problem Efficiently solving Stochastic Optimal Transport with hard terminal distributions and soft costs.
method Guided Harmonic Path-Integral Diffusion (GH-PID) framework with low-dimensional guidance.
result GH-PID generates geometry-aware, cost-reducing trajectories that match terminal distributions.
End-to-end CAD system for thyroid nodule classification using multimodal data and expert guidance.
problem Improving accuracy in thyroid nodule classification for clinicians.
method Knowledge-driven DenseNet framework using multimodal ultrasound data and expert cues.
result The proposed system achieves relevant performances in thyroid nodule classification.
System guides freehand obstetric ultrasound probe movements.
problem Reducing operator expertise for standard plane acquisition.
method Artificial neural network trained on real scans.
result 88.8% accuracy for goal prediction, 90.9% for action prediction.
We propose a framework for ensuring safe behavior of a reinforcement learning agent when the reward function may be difficult to specify. In order to do this, we rely on the existence of demonstrations from expert policies, and we provide a theoretical framework for the agent to optimize in the space of rewards consist…
We study how to effectively leverage expert feedback to learn sequential decision-making policies. We focus on problems with sparse rewards and long time horizons, which typically pose significant challenges in reinforcement learning. We propose an algorithmic framework, called hierarchical guidance, that leverages the…
Survey of methods for learning from observation without requiring expert actions.
problem Lack of access to expert actions in imitation learning.
method Survey and classification of state-only imitation learning (SOIL) methods.
result Identification of open problems and future research directions.
The goal of task transfer in reinforcement learning is migrating the action policy of an agent to the target task from the source task. Given their successes on robotic action planning, current methods mostly rely on two requirements: exactly-relevant expert demonstrations or the explicitly-coded cost function on targe…
Learning to drive faithfully in highly stochastic urban settings remains an open problem. To that end, we propose a Multi-task Learning from Demonstration (MT-LfD) framework which uses supervised auxiliary task prediction to guide the main task of predicting the driving commands. Our framework involves an end-to-end tr…
EQD model improves domain-specific QA by 0.6% to 10.5%.
problem Challenges in domain-specific quantitative reasoning for LLMs.
method Two-step fine-tuning framework guided by a reward function.
result EQD outperforms state-of-the-art models and prompting strategies.
Unified scoring model improves efficiency and performance across multiple tasks.
problem Efficient and resource-efficient automated scoring for diverse tasks.
method Knowledge-distilled multi-task Mixture-of-Experts (MoE) approach.
result Comparable performance to task-specific models with significantly less storage and training resources.
FinSphere improves stock analysis quality with AI and expert-curated data.
problem Lack of objective evaluation metrics and depth in stock analysis by FinLLMs.
method Developed AnalyScore, curated Stocksis dataset, and FinSphere AI agent.
result FinSphere outperforms general and domain-specific LLMs in generating high-quality stock analysis reports.
This paper studies Learning from Observations (LfO) for imitation learning with access to state-only demonstrations. In contrast to Learning from Demonstration (LfD) that involves both action and state supervision, LfO is more practical in leveraging previously inapplicable resources (e.g. videos), yet more challenging…
Neural A* uses machine learning to improve path planning efficiency.
problem Challenges in applying machine learning to search-based path planning.
method Reformulated A* search as a differentiable network coupled with a convolutional encoder.
result Neural A* outperforms state-of-the-art planners in optimality and efficiency.
This work formalizes guidance in diffusion models and introduces a stochastic control framework.
problem Lack of a solid theoretical foundation for guidance scheduling in diffusion models.
method Introduces a stochastic optimal control framework to cast guidance scheduling as an adaptive optimization problem.
result Establishes a principled foundation for more effective guidance in diffusion models.
Framework for training-free guidance in discrete diffusion models for molecular generation.
problem No equivalent training-free guidance methods for discrete diffusion models.
method Framework using guidance functions for discrete data.
result Demonstrated utility on molecular graph generation tasks.
Exact guidance for discrete data improves posterior sampling efficiency.
problem Inefficient guidance for discrete data in posterior sampling.
method Derive exact transition rate for desired distribution given learned discrete flow matching model.
result Significantly improved efficiency with single forward pass per sampling step.
Diffusion models improve sample quality with guidance, proving it works for general data distributions.
problem Theoretical understanding of guidance effect in diffusion models for general data distributions.
method Analyzing diffusion guidance under general data distributions, proving improvement in sample quality.
result Guidance improves the average reciprocal of the classifier probability, aligning with its motivation.
URGE improves diffusion model quality without gradients or Hessian.
problem Improving sample quality in diffusion models without gradient evaluations.
method Path-wise importance reweighting via Girsanov change of measure.
result URGE achieves better generation quality than existing methods.
Research shows guidance in diffusion models does not sample from intended distribution, affecting boundary sampling.
problem Clarifying the misconception that guidance modifies the data distribution in diffusion models.
method Rigorous proof and fine-grained analysis of guidance dynamics in two cases: mixtures of compactly supported distributions and mixtures of Gaussians.
result Guidance leads to sampling more heavily from the boundary of the support of the conditional distribution as the parameter increases.
Improved image quality in diffusion models by limiting guidance to a specific noise level range.
problem Improving image quality in diffusion models with guidance.
method Restricting guidance to a specific noise level range.
result Significantly improved FID score from 1.81 to 1.40 in ImageNet-512.
Unified approach to guided generation techniques.
problem Controlling the generative process of flow/diffusion models.
method Unified posterior and end-to-end guidance techniques.
result Unified posterior guidance as a greedy strategy of end-to-end guidance.
Jeffrey guidance extends diffusion-model control to more complex applications.
problem Controlling diffusion models beyond simple cases like conditional sampling.
method Leveraging Jeffrey's rule of conditioning to update marginal distributions towards a target distribution.
result Significant reductions in FID on CIFAR-10 and FFHQ with Inception embeddings as the target.
The paper analyzes how guidance affects diffusion models using Gaussian mixture models.
problem Understanding how guidance influences diffusion models in specific contexts.
method Theoretical study using Gaussian mixture models and comparison inequalities for differential equations.
result Guidance boosts classification confidence but reduces distribution diversity, leading to lower differential entropy.
New study on guidance in masked diffusion models, showing how it shapes sampling dynamics.
problem Understanding how guidance influences the sampling behavior of masked diffusion models.
method Derived explicit solution to guided reverse dynamics, analyzing effects in 1D and 2D.
result Guidance amplifies class-specific regions and suppresses shared regions, affecting covariance structures.
The paper analyzes CFG in masked diffusion models and proposes a new method to improve sample quality.
problem Improving sample quality in conditional generation with CFG in masked diffusion models.
method Analyzing CFG's effect in a low-dimensional masked diffusion model, revealing early-stage guidance harms quality and late-stage improves it. Proposed a novel method to smooth transitions between distributions.
result The proposed method improves sample quality via a simple one-line code change.
SDG uses optimal control to improve classifier guidance in low-density regions.
problem Inefficient guidance in low-density regions of posterior distributions.
method Integrates stochastic optimal control with Stein variational inference to compute the steepest descent direction.
result SDG improves guidance in low-density regions, outperforming standard methods.
New method controls classifier guidance in diffusion models.
problem Improving classifier guidance in diffusion models.
method Cross-entropy control of classifier gradients.
result Effective guidance vectors with mean squared error O(dε). New method guides pretrained diffusion models without additional training.
problem Guidance methods for diffusion models often require extra training or are task-specific.
method Variational Control using Diffusion Trajectory Matching (DTM)
result Achieves state-of-the-art results on various problems.
Gradient guidance improves diffusion models for optimizing specific objectives.
problem Improving diffusion models for specific optimization tasks.
method Established a mathematical framework for gradient-guided diffusion, linking it to optimization theory. Developed a modified gradient guidance method and iteratively fine-tuned version.
result Gradient-guided diffusion models are essentially solutions to regularized optimization problems, preserving latent structure.
This paper combines LLMs with RL for better trading strategies.
problem Myopic behavior and opaque policies in RL for trading.
method LLMs generate strategic trading advice to guide RL agents.
result LLM-guided RL agents outperform unguided RL in return and risk metrics.
CliMB-DC combines human guidance and data-centric tools to improve ML for non-technical experts.
problem Lack of data-centric handling in LLM co-pilots for non-technical users.
method Human-guided, data-centric framework combining advanced tools and LLM reasoning.
result Significantly outperforms existing co-pilot baselines for data-centric challenges.
Self-guidance controls image generation by extracting properties from diffusion model representations.
problem Generating images from text descriptions is challenging due to the complexity of visual details.
method Self-guidance uses internal representations of diffusion models to control image generation.
result Properties like object shape, location, and appearance can be extracted and used to steer image generation.
SFG improves on-manifold sampling without labels or additional training.
problem Guiding score-based models on manifolds without labeled data or extra training.
method Developed saddle-free guidance (SFG) that uses curvature of log density estimates.
result SFG achieves state-of-the-art metrics in image generation without labeled data or additional training.
New method aligns diffusion models for inference-time properties without retraining.
problem Aligning pre-trained diffusion models for desired inference-time properties.
method Variationally stable Doob's matching for provable guidance estimation.
result Consistent estimator of guidance with non-asymptotic convergence guarantees.
LLMs can identify tax strategies, potentially revolutionizing tax enforcement.
problem Detecting and analyzing U.S. tax-minimization strategies.
method Evaluated advanced LLMs on interpreting, verifying, and generating tax strategies.
result Identified a novel tax strategy, showing LLMs' potential in tax enforcement.
Proposes a thermodynamic work minimization framework for guiding generative models.
problem Guiding generative models in sparse-data regimes with limited target samples or constraints.
method Regularization framework inspired by thermodynamic work, introducing Path Guidance and Observable Guidance.
result Improves sample efficiency and reduces bias in molecular simulations.
Certified guidance ensures generative models always meet planning objectives.
problem Lack of guarantees for generated outputs satisfying planning objectives.
method Certified guidance modifies generative models without retraining, ensuring outputs meet STL specifications.
result Certified guidance produces models that are always correct, unlike non-certified methods.
Improving a semi-supervised image segmentation task has the option of adding more unlabelled images, labelling the unlabelled images or combining both, as neither image acquisition nor expert labelling can be considered trivial in most clinical applications. With a laparoscopic liver image segmentation application, we …
New method improves training-free guidance for diffusion models, achieving state-of-the-art results.
problem Accurate, training-free guidance for conditional generation in diffusion models.
method Sequential Monte Carlo (SMC) framework with Multi-Level Monte Carlo (MLMC) variance reduction.
result Achieves state-of-the-art results on CIFAR-10 and ImageNet datasets with significant cost reduction.
A new method for offline RL using diffusion models and self-weighted guidance.
problem Challenges in computing scores for offline RL using weight functions.
method Constructing a diffusion over actions and weights, using the diffusion model for guidance.
result Performs on par with state-of-the-art methods on challenging environments.
This paper proposes a new algorithm for learning guidance rewards in RL.
problem Long-term temporal credit assignment in sparse or delayed reward environments.
method Surrogate RL objective with trajectory-space smoothing to learn guidance rewards.
result Guidance rewards can be learned without additional neural networks and have intuitive interpretation.
A new method learns continuous guidance weights to improve diffusion model quality and distributional alignment.
problem Improving perceptual quality and distributional alignment of samples from conditional diffusion models.
method Learned continuous guidance weights ωc,(s,t) are used to minimize distributional mismatch and reward guided sampling. result Improvements in Fréchet inception distance (FID) for image generation and better image-prompt alignment in text-to-image applications.
Boost-and-Skip generates minority samples without guidance, faster and more effectively.
problem Generating minority samples in low-density regions of a data manifold.
method Boost-and-Skip approach with variance-boosted initialization and timestep skipping.
result Boost-and-Skip effectively promotes the emergence of underrepresented minority features.
PriorGuide adapts diffusion models to new priors at test time.
problem Limited applicability of prior distributions in diffusion-based inference.
method PriorGuide uses a guidance approximation to adapt diffusion models to new priors at test time.
result Enhances the versatility of pre-trained inference models by allowing flexible adaptation to new priors.