The paper analyzes CFG in masked diffusion models and proposes a new method to improve sample quality.
problem Improving sample quality in conditional generation with CFG in masked diffusion models.
method Analyzing CFG's effect in a low-dimensional masked diffusion model, revealing early-stage guidance harms quality and late-stage improves it. Proposed a novel method to smooth transitions between distributions.
result The proposed method improves sample quality via a simple one-line code change.
Improved neural networks by averaging late-stage weights.
problem Improving the performance of neural networks.
method Ensemble late-stage weights and average them.
result Augmenting standard models with late-phase weights improves generalization.
SGD converges to critical points of normalized margin in late-stage training for homogeneous neural networks.
problem Analyzing the implicit bias of SGD on homogeneous neural networks.
method Interpreting SGD dynamics as an Euler-like discretization of a conservative field flow associated with the normalized classification margin.
result Normalized SGD iterates converge to the set of critical points of the normalized margin at late-stage training.
New framework optimizes deep learning training by deferring large batch sizes to late stages.
problem Optimizing batch size scheduling for deep learning training efficiency.
method Introduced the functional scaling law (FSL) framework to analyze and optimize batch size scheduling.
result Large batch sizes can be deferred to late training stages without sacrificing performance.
Machine learning can improve 2SLS first stage predictions, but nonlinear methods often introduce bias.
problem Improving the first stage of 2SLS using machine learning.
method Decomposed bias into three components, investigated through simulation.
result Nonlinear machine learning methods can introduce substantial bias in second-stage estimates.
ABS dynamically adjusts batch size based on policy stability, improving RL performance.
problem Diminishing returns with large batch sizes in RL due to non-stationary data.
method Adaptive Batch Scaling (ABS) with Behavioral Divergence metric.
result Larger batch sizes can improve RL performance, contrary to conventional wisdom.
A new multi-view clustering method that is fast, scalable, and easy to use.
problem High computational complexity, one-stage fusion, and dataset-specific hyperparameter tuning in multi-view clustering.
method Random view groups, hybrid early-late fusion, diversified base clusterings, and unified bipartite graph.
result Almost linear time and space complexity, no dataset-specific tuning required.
Enhances the Bishop-Gromov theorem for curved spaces, especially at late times.
problem The Bishop-Gromov theorem's volume growth upperbound is often too loose, especially at late times.
method Identified and quantified the effect of shear, using higher curvature invariants to improve the upperbound.
result Tighter upper bounds on late-time growth rates of geodesic balls in homogeneous spaces with non-positive sectional curvature.
Hyperfitting improves LLM generation quality by enhancing diversity, contrary to simple temperature scaling.
problem Improving open-ended generation quality of LLMs with minimal fine-tuning effort.
method Demonstrates that hyperfitting, a phenomenon where LLMs are fine-tuned to near-zero training loss, enhances generation quality and mitigates repetition.
result Hyperfitting is distinct from temperature scaling and involves a dynamic, context-dependent rank reordering mechanism in the final transformer block.
A new method reduces variance in training early-stage rankers for large-scale search systems.
problem Training early-stage rankers for large-scale search systems is challenging due to exploding variance in policy gradient methods.
method Proposes credit-assigned policy gradient (CA-PG) to mitigate variance in training early-stage rankers.
result CA-PG significantly reduces variance in training early-stage rankers compared to vanilla policy gradient.
Estimates linear model from noisy covariates and instruments using spectral regularization.
problem Estimating a linear model from many noisy covariates and instruments.
method Two-stage least squares with spectral regularization of canonical correlations.
result Upper and lower bounds on estimation error, proving optimality of the method with noisy data.
We present a reinforcement learning approach for detecting objects within an image. Our approach performs a step-wise deformation of a bounding box with the goal of tightly framing the object. It uses a hierarchical tree-like representation of predefined region candidates, which the agent can zoom in on. This reduces t…
Dynamic Influence Tracker measures changing sample importance during model training.
problem Static influence measurements during training overlook how sample importance varies over time.
method Dynamic Influence Tracker (DIT) captures time-varying sample influence across arbitrary time windows.
result DIT reveals distinct learning phases with shifting priorities and detects corrupted samples more efficiently.
Sleep apnea is a common respiratory disorder characterized by breathing pauses during the night. Consequences of untreated sleep apnea can be severe. Still, many people remain undiagnosed due to shortages of hospital beds and trained sleep technicians. To assist in the diagnosis process, automated detection methods are…
New method finds precise late-time behavior of wave equations.
problem Analyzing late-time behavior of wave equations with inverse-square potentials.
method Physical-space-based method for deriving late-time asymptotics.
result Sharp, uniform decay estimates in time for asymptotic late-time tails.
Study assesses drought and late-frost risks in Bavaria using vine copulas.
problem Assessing risks of late-frost and drought in Bavaria due to climate change.
method Used vine copula models for non-Gaussian and asymmetric dependencies, with univariate and bivariate regression analyses.
result Identified 'at-risk' regions for forest adaptation.
A family of recent successful approaches to few-shot learning relies on learning an embedding space in which predictions are made by computing similarities between examples. This corresponds to combining information between support and query examples at a very late stage of the prediction pipeline. Inspired by this obs…
This work formalizes guidance in diffusion models and introduces a stochastic control framework.
problem Lack of a solid theoretical foundation for guidance scheduling in diffusion models.
method Introduces a stochastic optimal control framework to cast guidance scheduling as an adaptive optimization problem.
result Establishes a principled foundation for more effective guidance in diffusion models.
Framework for training-free guidance in discrete diffusion models for molecular generation.
problem No equivalent training-free guidance methods for discrete diffusion models.
method Framework using guidance functions for discrete data.
result Demonstrated utility on molecular graph generation tasks.
Second part of series studying charged scalar fields on Reissner--Nordström spacetimes.
problem Analyzing late-time behavior and stability of charged scalar fields on black hole backgrounds.
method Purely physical-space based methods, energy estimates, inverse-power laws.
result First pointwise decay estimates for charged scalar fields on black hole backgrounds.
Exact guidance for discrete data improves posterior sampling efficiency.
problem Inefficient guidance for discrete data in posterior sampling.
method Derive exact transition rate for desired distribution given learned discrete flow matching model.
result Significantly improved efficiency with single forward pass per sampling step.
LITE models improve query-document relevance with learnable late interactions.
problem Improving query-document relevance with lower latency and storage.
method Proposes learnable late-interaction models (LITE) that use factorized query and document embeddings followed by a learnable scorer.
result Empirically, LITE outperforms previous late-interaction models in re-ranking tasks.
Diffusion models improve sample quality with guidance, proving it works for general data distributions.
problem Theoretical understanding of guidance effect in diffusion models for general data distributions.
method Analyzing diffusion guidance under general data distributions, proving improvement in sample quality.
result Guidance improves the average reciprocal of the classifier probability, aligning with its motivation.
A new framework integrates credit scoring into profit scoring for better P2P lending investments.
problem Maximizing profit while minimizing risk in P2P lending investments.
method Two-stage framework using Light Gradient Boosting Machine (lightGBM) to integrate credit scoring into profit scoring.
result The proposed framework identifies more profitable loans and provides better investment guidance.
Research shows guidance in diffusion models does not sample from intended distribution, affecting boundary sampling.
problem Clarifying the misconception that guidance modifies the data distribution in diffusion models.
method Rigorous proof and fine-grained analysis of guidance dynamics in two cases: mixtures of compactly supported distributions and mixtures of Gaussians.
result Guidance leads to sampling more heavily from the boundary of the support of the conditional distribution as the parameter increases.
In this paper, we propose a novel unsupervised learning method to learn the brain dynamics using a deep learning architecture named residual D-net. As it is often the case in medical research, in contrast to typical deep learning tasks, the size of the resting-state functional Magnetic Resonance Image (rs-fMRI) dataset…
Regression problems assume every instance is annotated (labeled) with a real value, a form of annotation we call \emph{strong guidance}. In order for these annotations to be accurate, they must be the result of a precise experiment or measurement. However, in some cases additional \emph{weak guidance} might be given by…
Improved image quality in diffusion models by limiting guidance to a specific noise level range.
problem Improving image quality in diffusion models with guidance.
method Restricting guidance to a specific noise level range.
result Significantly improved FID score from 1.81 to 1.40 in ImageNet-512.
Unified approach to guided generation techniques.
problem Controlling the generative process of flow/diffusion models.
method Unified posterior and end-to-end guidance techniques.
result Unified posterior guidance as a greedy strategy of end-to-end guidance.
New insights into why Transformers are hard to train, leading to a new method to stabilize them.
problem Understanding and stabilizing the training of Transformers.
method Analyzing the amplification effect in Transformer models and proposing Admin to stabilize training.
result Admin stabilizes early stage training and improves performance.
In recent years, there has been significant progress in applying deep reinforcement learning (RL) for solving challenging problems across a wide variety of domains. Nevertheless, convergence of various methods has been shown to suffer from inconsistencies, due to algorithmic instability and variance, as well as stochas…
SAM selects flatter minima late in training, improving generalization.
problem Improving neural network generalization under various settings.
method Sharpness-Aware Minimization (SAM) applied late in training.
result SAM efficiently selects flatter minima late in training, improving generalization.
A lot of attention has been devoted to multimedia indexing over the past few years. In the literature, we often consider two kinds of fusion schemes: The early fusion and the late fusion. In this paper we focus on late classifier fusion, where one combines the scores of each modality at the decision level. To tackle th…
Jeffrey guidance extends diffusion-model control to more complex applications.
problem Controlling diffusion models beyond simple cases like conditional sampling.
method Leveraging Jeffrey's rule of conditioning to update marginal distributions towards a target distribution.
result Significant reductions in FID on CIFAR-10 and FFHQ with Inception embeddings as the target.
Late fusion of clinical notes and physiological data improves ICU mortality prediction.
problem Improving ICU mortality prediction using multimodal data.
method Late fusion of clinical notes and physiological time series data with a deep learning architecture.
result Late fusion approach provides statistically significant improvement in mortality prediction performance.
The paper analyzes how guidance affects diffusion models using Gaussian mixture models.
problem Understanding how guidance influences diffusion models in specific contexts.
method Theoretical study using Gaussian mixture models and comparison inequalities for differential equations.
result Guidance boosts classification confidence but reduces distribution diversity, leading to lower differential entropy.
New study on guidance in masked diffusion models, showing how it shapes sampling dynamics.
problem Understanding how guidance influences the sampling behavior of masked diffusion models.
method Derived explicit solution to guided reverse dynamics, analyzing effects in 1D and 2D.
result Guidance amplifies class-specific regions and suppresses shared regions, affecting covariance structures.
Estimates LATE using combined datasets, overcoming data limitations.
problem Estimating LATE when compliance is incomplete and data is split.
method Combines separately observed datasets to estimate LATE using direct and weighted least squares methods.
result Proposes a stable and practical estimator for LATE.
SDG uses optimal control to improve classifier guidance in low-density regions.
problem Inefficient guidance in low-density regions of posterior distributions.
method Integrates stochastic optimal control with Stein variational inference to compute the steepest descent direction.
result SDG improves guidance in low-density regions, outperforming standard methods.
New method controls classifier guidance in diffusion models.
problem Improving classifier guidance in diffusion models.
method Cross-entropy control of classifier gradients.
result Effective guidance vectors with mean squared error O(dε). New method guides pretrained diffusion models without additional training.
problem Guidance methods for diffusion models often require extra training or are task-specific.
method Variational Control using Diffusion Trajectory Matching (DTM)
result Achieves state-of-the-art results on various problems.
Gradient guidance improves diffusion models for optimizing specific objectives.
problem Improving diffusion models for specific optimization tasks.
method Established a mathematical framework for gradient-guided diffusion, linking it to optimization theory. Developed a modified gradient guidance method and iteratively fine-tuned version.
result Gradient-guided diffusion models are essentially solutions to regularized optimization problems, preserving latent structure.
In the past few years, a lot of attention has been devoted to multimedia indexing by fusing multimodal informations. Two kinds of fusion schemes are generally considered: The early fusion and the late fusion. We focus on late classifier fusion, where one combines the scores of each modality at the decision level. To ta…
Self-guidance controls image generation by extracting properties from diffusion model representations.
problem Generating images from text descriptions is challenging due to the complexity of visual details.
method Self-guidance uses internal representations of diffusion models to control image generation.
result Properties like object shape, location, and appearance can be extracted and used to steer image generation.
SFG improves on-manifold sampling without labels or additional training.
problem Guiding score-based models on manifolds without labeled data or extra training.
method Developed saddle-free guidance (SFG) that uses curvature of log density estimates.
result SFG achieves state-of-the-art metrics in image generation without labeled data or additional training.
Theoretical analysis of t-SNE for visualizing clustered data.
problem Understanding t-SNE for visualizing high-dimensional clustered data.
method Gradient descent approach and power iterations based on graph Laplacian.
result Asymptotic equivalence and limiting behavior of t-SNE's early exaggeration stage.
Pension benefits in rural China lead to cognitive decline among the elderly.
problem Cognitive decline in late adulthood among rural Chinese elderly.
method Examined the effects of a new pension scheme on cognitive performance.
result Pension benefits negatively impact cognitive functioning, particularly delayed recall.
New method aligns diffusion models for inference-time properties without retraining.
problem Aligning pre-trained diffusion models for desired inference-time properties.
method Variationally stable Doob's matching for provable guidance estimation.
result Consistent estimator of guidance with non-asymptotic convergence guarantees.