LLM safety alignment explained as divergence estimation.
problem Aligning large language models to avoid harmful outputs.
method Presented a theoretical framework showing alignment methods as divergence estimators.
result KLDO method improves safety alignment using compliance-refusal datasets.
New approach speeds up DNA sequence alignment.
problem Efficiently estimating alignment scores for large sets of reads.
method Rank-one crowdsourcing models and multi-armed bandit algorithm.
result Adaptive algorithm identifies pairs with large alignment scores.
A new method reduces preference distortion in LLM alignment.
problem Vulnerability of traditional LLM alignment methods to human preference heterogeneity.
method Sign Estimator: A simple, provably consistent, and efficient estimator using binary classification loss.
result Substantially reduces preference distortion over a panel of simulated personas.
Bayesian framework proves thresholds for multi-graph alignment feasibility.
problem Determining when multi-graph alignment is statistically possible.
method Developed a Bayesian estimation framework over metric spaces.
result Identified thresholds for Gaussian and sparse Erdős-Rényi models.
New model optimizes feature alignment, improving statistical and computational efficiency.
problem Statistical and computational challenges in feature alignment.
method Covariance alignment model, Gromov-Wasserstein algorithm.
result Gromov-Wasserstein algorithm is minimax optimal and practical for covariance alignment.
Robustly aligns datasets with partial GW distance to handle contamination.
problem Aligning contaminated datasets using Gromov-Wasserstein distances.
method Proposes a partial GW distance estimator to minimize distortion from outliers.
result The partial GW distance estimator is minimax optimal and near-optimal in finite samples.
Extends Popularity Bias Memorization theorem to new conditions.
problem Estimating alignment with top-k singular hyperspace.
method Extending theorem to arbitrary degree distributions and proving upper and lower bounds.
result Upper and lower estimates for alignment with top-k singular hyperspace.
Extends reinforcement learning alignment to scalar rewards, improving math reasoning.
problem Designing reinforcement learning algorithms for general LLM alignment.
method Introduces f-GRPO and f-HAL, estimating f-divergences between reward-aligned and unaligned distributions.
result Improves math reasoning RLVR tasks and mitigates reward hacking.
The infinite Viterbi alignment is the limiting maximum a-posteriori estimate of the unobserved path in a hidden Markov model as the length of the time horizon grows. For models on state-space Rd satisfying a new ``decay-convexity'' condition, we develop an approach to existence of the infinite Viterbi ali…
Hölder-DPO aligns models robustly with noisy human feedback.
problem No existing alignment methods can handle severe label noise.
method Proposes Hölder-DPO, a principled alignment loss with provable redescending property.
result Hölder-DPO enables scalable human feedback valuation and improves model alignment.
We consider the problem of aligning a pair of databases with jointly Gaussian features. We consider two algorithms, complete database alignment via MAP estimation among all possible database alignments, and partial alignment via a thresholding approach of log likelihood ratios. We derive conditions on mutual informatio…
New estimator corrects bias in CKA for sparsely sampled neurons.
problem Bias in CKA for sparsely sampled neurons.
method Novel estimator that corrects for input and feature sampling.
result Reliable model-to-brain alignment with sparsely sampled neurons.
PIPA aligns preferences without reinforcement learning, improving language model performance.
problem Aligning preferences in language models efficiently and without reinforcement learning.
method Formulates preference alignment as a Maximum Likelihood Estimation problem with prior constraints.
result PIPA algorithms achieve up to 10% performance improvement on benchmarks.
New method aligns diffusion models for inference-time properties without retraining.
problem Aligning pre-trained diffusion models for desired inference-time properties.
method Variationally stable Doob's matching for provable guidance estimation.
result Consistent estimator of guidance with non-asymptotic convergence guarantees.
Paper resolves bias in ALFT training using generalized alignment games.
problem Systematic bias in estimating logarithmic rewards from small batches.
method Generalized Distributional Alignment Games, U-statistics, minimax polynomial estimators, Variance-Optimal Augmented Polynomial Optimization Program (AQP) Estimator.
result Proves optimal bias and accelerated convergence in ALFT training.
A new method improves density ratio estimation with fewer function evaluations.
problem Stable and accurate estimation of density ratios with high variance issues.
method Diffusion Secant Alignment for Score-Based Density Ratio Estimation (ISA-DRE)
result ISA-DRE achieves comparable or superior results with fewer function evaluations.
Bayesian HMM for protein alignment state estimation.
problem Estimating the MAP state sequence for a Bayesian HMM.
method Training data with Dirichlet priors, iterative algorithms for MAP path estimation.
result Bayesian setup outperforms frequentist estimation in protein alignment.
A new method for aligning multiple distributions efficiently.
problem Aligning multiple distributions in a shared latent space.
method Iterative alignment of variational approximations of distribution divergences using invertible alignment maps.
result Our method achieves competitive distribution alignment at low computational cost.
Clustering and community detection with multiple graphs have typically focused on aligned graphs, where there is a mapping between nodes across the graphs (e.g., multi-view, multi-layer, temporal graphs). However, there are numerous application areas with multiple graphs that are only partially aligned, or even unalign…
Alignment of neural network representations is influenced by SNR and sample size.
problem Understanding how neural network representations align across different conditions.
method Controlled training of neural networks on perturbed datasets, analyzing alignment and generalization.
result Alignment varies monotonically with SNR but non-monotonically with sample size, with minimal alignment near the interpolation threshold.
Decision-alignment evaluates uncertainty quantification for decision-relevant UQ
problem Evaluation of uncertainty quantification metrics
method Introduce decision-alignment
result Proper scoring rules align with decision utility
Estimates multiple networks using graphons for non-aligned graphs.
problem Estimating topology of multiple networks from nodal observations.
method Combining maximum likelihood penalty with graphon estimation schemes.
result Validated performance against competing methods in synthetic and real-world datasets.
LLMs can fail to maximize aligned values even after training, due to irrational reasoning.
problem Value misalignment in LLMs' reasoning despite training alignment.
method Formalized rational value risk and decomposed estimation error.
result Rational value risk is widespread and cannot be fully eliminated.
Paper presents a method to align unpaired samples across different modalities.
problem Challenges in collecting paired samples for multimodal representation learning.
method Uses propensity score alignment based on Rubin's framework to estimate a common space for unpaired samples.
result Optimal transport matching significantly improves alignment in real-world data.
Sharp threshold found for aligning Gaussian-weighted graphs.
problem Reconstructing planted permutations in Gaussian-weighted graphs.
method Analysis of MAP estimator and second moment method.
result Sharp information-theoretic threshold for exact recovery.
Unified framework for distribution shift estimation, explanation, and improvement.
problem Estimating, explaining, and improving model performance on target domains with distribution shift.
method Entropic Projection Alignment (EPA) aligns source and target distributions by matching moments and minimizing KL divergence.
result EPA consistently outperforms state-of-the-art baselines while offering computational efficiency.
Proposes LAMA for principled SVDD hyperparameter estimation.
problem Estimating SVDD hyperparameters is difficult and heuristic methods are unreliable.
method LAMA (Local Active Min-Max Alignment) using active learning and kernel alignment.
result LAMA provides evidence-based estimates for SVDD hyperparameters with quality scores.
Direct Density Ratio Optimization aligns LLMs with human preferences without assuming specific models.
problem Statistical inconsistency in aligning LLMs with human preferences.
method Direct Density Ratio Optimization (DDRO) estimates density ratio directly.
result DDRO is statistically consistent, converging to true human preferences as data grows.
Proposes a method to align Hawkes processes across different event spaces.
problem Aligning Hawkes processes in multiple event spaces.
method Fused Gromov-Wasserstein alignment for Hawkes processes.
result The method effectively aligns Hawkes processes and their event types.
Aligns attention distributions for improved accuracy and robustness.
problem Improving the accuracy and robustness of neural networks using attention mechanisms.
method Alignment attention that encourages key and query distributions to match within each head.
result Alignment attention leads to better accuracy, uncertainty estimation, and robustness across various tasks.
We present a probabilistic model for unsupervised alignment of high-dimensional time-warped sequences based on the Dirichlet Process Mixture Model (DPMM). We follow the approach introduced in (Kazlauskaite, 2018) of simultaneously representing each data sequence as a composition of a true underlying function and a time…
ALIEN improves uncertainty estimation of language models by refining entropy-based methods.
problem Overconfidence in uncertainty estimation for language models, especially for difficult inputs.
method ALIEN refines entropy-based uncertainty by aligning it with prediction reliability, using a lightweight uncertainty head.
result ALIEN consistently outperforms strong baselines in detecting incorrect predictions and achieving the lowest calibration error.
Foot-mounted inertial positioning (FMIP) can face problems of inertial drifts and unknown initial states in real applications, which renders the estimated trajectories inaccurate and not obtained in a well defined coordinate system for matching trajectories of different users. In this paper, an approach adopting receiv…
RCPO uses ranked choice modeling for better LLM alignment.
problem Pairwise preference optimization limits LLM alignment.
method Unified framework combining preference optimization and ranked choice modeling.
result RCPO outperforms competitive baselines in LLM alignment.
Develops methods to correct bias in AI feedback for more accurate alignment.
problem Systematic bias in AI feedback compared to human labels.
method Two debiased alignment methods: DDPO and DIPO.
result Methods improve alignment efficiency and performance close to human-labeled data.
A new method estimates protein evolutionary fields and couplings from alignments.
problem Estimating evolutionary fields and couplings from protein sequence alignments.
method Boltzmann machine with parallel, persistent Markov chain Monte Carlo method.
result Improved precision in predicting contact residue pairs.
Activists align with large fund preferences for success.
problem Aligning with large fund preferences increases activist success.
method Analyzed previous proxy voting behavior to estimate preferences and correlated them with activist success.
result Campaigns with higher alignment receive more votes and are more successful.
Introduces MFVDM for high-dimensional data analysis.
problem Non-linear dimensionality reduction of high-dimensional datasets.
method Combines multiple unitary irreducible representations for nonlinear embeddings.
result Achieves better nearest neighbor search and alignment estimation on noisy data.
One-step Bellman alignment improves online RL by reducing task mismatch.
problem Online RL struggles with task similarity defined by rewards or transitions.
method One-step Bellman alignment and re-weighted targeting (RWT) to correct task mismatch.
result Regret bounds show task shift complexity, not target MDP, affects performance.
DTS improves inference-time alignment of diffusion models with less compute.
problem Inference-time alignment of diffusion models suffers from inaccurate value estimation and inefficient reuse of past computations.
method Diffusion Tree Sampling (DTS) uses a tree-based approach to propagate terminal rewards and iteratively refine value estimates.
result DTS produces asymptotically exact samples and matches the FID of best-performing baselines with up to 10x less compute.
New method removes pseudo-label bias for unsupervised domain adaptation.
problem Class imbalance and distribution shift between domains.
method Implicit class-conditioned domain alignment without explicit pseudo-label optimization.
result Effective in handling within-domain class imbalance and between-domain class distribution shift.
A new method for unsupervised disentanglement using axis-aligned cliffs.
problem Unsupervised disentanglement of latent factors under nonlinear maps.
method Encouraging axis-aligned discontinuities (cliffs) in the estimated density of factors.
result Cliff method outperforms baselines on disentanglement benchmarks.
Study rigidity on CR Yamabe equation on Sasakian manifolds.
problem Proving rigidity on CR Yamabe equation on Sasakian manifolds.
method Using Jerison-Lee's differential identity and integral estimates.
result Prove that the manifold is CR isometric to Heisenberg group \(\mathbb{H}^n\).
Statistical-computational gap found in aligning multiple Gaussian graphs.
problem Aligning multiple Gaussian graphs with unknown signals.
method Generalized informational threshold and computational barrier analysis.
result Existence of a statistical-computational gap in multiple Gaussian graph alignment.
Novel alignment framework for text-to-image generation using diffusion models and flow matching.
problem Improving text-to-image generation with minimal computational resources.
method Proposes a novel alignment framework that decomposes the score function into pre-trained score plus a conditional expectation of the reward.
result Achieves comparable performance to finetuning-based models with reduced computational cost.
Study on estimating signals from shifted and noisy copies in high dimensions, revealing a phase transition.
problem Estimating a signal in high-dimensional space from its circularly-shifted and noisy copies.
method Analysis of sample complexity in the high-dimensional regime, focusing on the parameter α.
result A phase transition phenomenon governed by α, with different sample complexities based on α values.
Novel algorithm estimates local permutations in unlabeled multi-view sensing.
problem Estimating local permutations in unlabeled multi-view sensing.
method Graph alignment and Gromov-Wasserstein alignment exploiting multiple views.
result The proposed algorithm is scalable and applicable to challenging SNR regimes.
GALA adapts learning rates online by aligning gradients, improving deep learning model performance.
problem Fine-tuning learning rates for deep learning models requires extensive grid search.
method GALA dynamically adjusts learning rates by tracking gradient alignment and local curvature.
result GALA produces a flexible, adaptive learning rate schedule that increases when gradients align.