Method embeds numeric tabular datasets into a shared vector space for similarity and retrieval.
problem Lack of meaningful representation for numeric tabular datasets in large language models.
method Structured exploratory data analysis descriptors, sentence transformer embedding, CCA for cross-dataset alignment.
result Total P@1 score of 0.9 across 15 datasets, robust nearest-neighbor retrieval and cluster structure.
Study improves machine learning models for GI tract disease detection using comprehensive evaluations and cross-dataset testing.
problem Incomplete or incorrect evaluation of machine learning models for GI tract diseases.
method Comprehensive evaluations of five machine learning models using Global Features and Deep Neural Networks, introducing performance hexagons and cross-dataset testing.
result Demonstrates the need for more sophisticated performance metrics and evaluation methods to build generalizable models.
Unsupervised method detects earthquakes from raw waveforms, generalizing across datasets.
problem Lack of labeled data for earthquake detection.
method Uses deep autoencoders with cross-covariance triggering at bottleneck.
result Performance comparable to supervised methods, with strong cross-dataset generalization.
Parametric models, and particularly neural networks, require weight initialization as a starting point for gradient-based optimization. Recent work shows that a specific initial parameter set can be learned from a population of supervised learning tasks. Using this initial parameter set enables a fast convergence for u…
MusPy is a toolkit for symbolic music generation, providing tools for dataset management and analysis.
problem Facilitating the creation and analysis of symbolic music datasets.
method Development of an open-source Python library (MusPy) with features for dataset management, data I/O, preprocessing, and model evaluation. Demonstrated through statistical analysis and cross-dataset generalizability experiments.
result MusPy's dataset analysis reveals varying degrees of cross-genre representation across different music datasets.
Study categorizes time series anomaly detection metrics based on evaluation challenges.
problem Challenges in evaluating time series anomaly detection due to diverse application objectives and metric assumptions.
method Problem-oriented framework categorizing metrics into six dimensions based on evaluation challenges.
result Quantifies each metric's discriminative ability and reveals limitations of widely used metrics.
Study reveals how spectral bias affects learnability on real-world data.
problem Understanding how well complex datasets can be learned using kernel methods.
method Use eigenvalues and eigenfunctions from idealized data to reveal spectral bias on real-world data.
result Bound learnability on real-world data using symmetries of realistic kernels.
Study shows different trajectory prediction models generalize better under OoD conditions.
problem Comparing trajectory prediction models' robustness across different datasets.
method Training models on Argoverse 2 and testing on Waymo Open Motion, and vice versa, with various augmentation strategies.
result Smallest model with highest inductive bias performs best in OoD generalization.
This paper surveys scalable automated alignment methods for LLMs.
problem Scalability issues in traditional human-annotated alignment methods for LLMs.
method Categorizes and discusses various automated alignment methods.
result Emerging automated alignment methods are effective and scalable.
Automatic speech emotion recognition provides computers with critical context to enable user understanding. While methods trained and tested within the same dataset have been shown successful, they often fail when applied to unseen datasets. To address this, recent work has focused on adversarial methods to find more g…
Inference-aware meta-alignment of LLMs reduces computational cost.
problem Aligning LLMs to diverse human preferences is challenging due to conflicting criteria.
method IAMA trains a base model to be aligned to multiple tasks via different inference-time alignment algorithms, using non-linear GRPO for optimization.
result IAMA enables effective alignment of LLMs to multiple criteria with limited computational budget.
Conformal Alignment ensures trustworthy outputs from foundation models.
problem Ensuring outputs from foundation models align with human values in high-stakes tasks.
method A framework that trains an alignment predictor using reference data to select trustworthy outputs.
result Conformal Alignment accurately identifies trustworthy outputs via lightweight training over moderate reference data.
LPL optimizes embeddings to align local neighborhoods, improving cross-lingual word alignment.
problem Aligning embeddings across different datasets and languages.
method Locality Preserving Loss (LPL) optimizes model to project embeddings while maintaining local neighborhoods and aligning them.
result LPL-based alignment leads to better and consistent accuracy, especially in small training set settings.
Extends reinforcement learning alignment to scalar rewards, improving math reasoning.
problem Designing reinforcement learning algorithms for general LLM alignment.
method Introduces f-GRPO and f-HAL, estimating f-divergences between reward-aligned and unaligned distributions.
result Improves math reasoning RLVR tasks and mitigates reward hacking.
The paper proposes a learning-theoretic perspective on representation alignment.
problem Understanding how AI models' representations become aligned as they scale.
method Reviewing and connecting different notions of alignment, focusing on stitching.
result Relating properties of stitching to kernel alignment of representations.
Survey of alignment techniques for large language models.
problem Ensuring large language models align with human values.
method Analysis of diverse alignment methods and training paradigms.
result Preference-based methods offer more flexibility for nuanced alignment.
The paper addresses rigid alignment of noisy patches, providing a polynomial time algorithm and convergence conditions.
problem Finding a rigid alignment of overlapping local views (patches) that minimizes alignment error in a noisy setting.
method Characterizes non-degeneracy based on kernel and positivity of a matrix, provides polynomial time algorithm for testing non-degeneracy, and uses Riemannian gradient descent for alignment.
result The algorithm converges locally linearly to a non-degenerate perfect alignment under certain conditions.
Study connects database alignment and planted matching using Gaussian features.
problem Identify matching between correlated user features in anonymized databases.
method Derived results for database alignment and planted matching, showing connections and thresholds.
result Performance thresholds for database alignment converge to planted matching when feature dimensionality is sufficiently high.
Bayesian framework proves thresholds for multi-graph alignment feasibility.
problem Determining when multi-graph alignment is statistically possible.
method Developed a Bayesian estimation framework over metric spaces.
result Identified thresholds for Gaussian and sparse Erdős-Rényi models.
SMAI framework tests and integrates single-cell data alignability.
problem Lack of a rigorous statistical test for alignability and distortion during alignment.
method Spectral manifold alignment and inference (SMAI) framework.
result SMAI outperforms existing methods in alignability testing and integration.
New method aligns brain data across individuals for better brain decoding.
problem Inter-individual variability in brain response patterns limits decoder generalization.
method SpectralOT method that embeds cortical geometry into Laplace-Beltrami eigenmodes.
result SpectralOT strikes balance between aligning functional features and preserving anatomical structure.
Method aligns multilingual news for better stock return prediction.
problem Losing nuances in multilingual news translations.
method Optimal transport for sentence alignment.
result Aligned sentences correlate better with stock returns.
This paper studies aligning knowledge graphs from different sources or languages. Most existing methods train supervised methods for the alignment, which usually require a large number of aligned knowledge triplets. However, such a large number of aligned knowledge triplets may not be available or are expensive to obta…
Hölder-DPO aligns models robustly with noisy human feedback.
problem No existing alignment methods can handle severe label noise.
method Proposes Hölder-DPO, a principled alignment loss with provable redescending property.
result Hölder-DPO enables scalable human feedback valuation and improves model alignment.
This paper improves cross-domain learning using random forests for manifold alignment.
problem Improving cross-domain learning and feature integration.
method Semi-supervised manifold alignment using random forest proximities.
result Random forest proximities enhance downstream classification accuracy.
Detecting correlated trees helps align sparse graphs.
problem Detecting correlation between trees for sparse random graphs.
method MPAlign message-passing algorithm for graph alignment.
result MPAlign succeeds in polynomial time for partial alignment.
We study alignment in linear neural networks and its relation to gradient descent.
problem Understanding alignment in linear neural networks and its impact on training.
method Defined alignment for fully connected networks, analyzed alignment under gradient descent, and compared gradient descent to projected gradient descent for layer-constrained networks.
result Gradient descent can converge linearly to a global minimum when alignment is invariant, and alignment is impossible with large datasets in layer-constrained networks.
Proposes ACCA for better alignment of multiple data perspectives.
problem Unclear alignment between multiple data perspectives.
method Iteratively solves alignment and multi-view embedding.
result Improved alignment and embedding of multiple data perspectives.
Unified framework for aligning and composing diffusion models to satisfy multiple constraints.
problem Improving quality and compliance of generated samples from diffusion models.
method Constrained optimization framework that unifies alignment and composition of diffusion models.
result Proposed framework effectively satisfies multiple constraints in image generation.
Study improves understanding and performance of FA learning rules in neural networks.
problem Lack of theoretical understanding and limited applications of Feedback Alignment (FA) methods.
method Introduces a unified framework linking synaptic weight changes to implicit regularization, providing convergence conditions and empirical evidence.
result Better alignment can enhance FA performance on complex multi-class tasks.
DFA trains deep networks by aligning weights then memorizing data.
problem Understanding why DFA works for some networks but not others.
method Two-step learning process: alignment followed by memorization.
result DFA aligns weights to maximize gradient alignment, breaking degeneracy.
The study explains how neural networks align their kernels to target functions during training.
problem Understanding how neural networks align their kernels to target functions during training.
method Theoretical analysis of kernel evolution in toy models and deep networks.
result Kernel alignment naturally emerges during training to accelerate convergence and improve generalization.
We consider the problem of aligning a pair of databases with jointly Gaussian features. We consider two algorithms, complete database alignment via MAP estimation among all possible database alignments, and partial alignment via a thresholding approach of log likelihood ratios. We derive conditions on mutual informatio…
We present Manifold Alignment Determination (MAD), an algorithm for learning alignments between data points from multiple views or modalities. The approach is capable of learning correspondences between views as well as correspondences between individual data-points. The proposed method requires only a few aligned exam…
LLM safety alignment explained as divergence estimation.
problem Aligning large language models to avoid harmful outputs.
method Presented a theoretical framework showing alignment methods as divergence estimators.
result KLDO method improves safety alignment using compliance-refusal datasets.
This study presents the results of a series of simulation experiments that evaluate and compare four different manifold alignment methods under the influence of noise. The data was created by simulating the dynamics of two slightly different double pendulums in three-dimensional space. The method of semi-supervised fea…
Align-RUDDER improves reinforcement learning with few demonstrations by redistributing rewards.
problem Learning complex tasks with sparse and delayed rewards using few demonstrations.
method Align-RUDDER uses a profile model for reward redistribution based on multiple sequence alignment of demonstrations.
result Align-RUDDER outperforms competitors on complex tasks with few demonstrations.
New model optimizes feature alignment, improving statistical and computational efficiency.
problem Statistical and computational challenges in feature alignment.
method Covariance alignment model, Gromov-Wasserstein algorithm.
result Gromov-Wasserstein algorithm is minimax optimal and practical for covariance alignment.
Alignment of neural network representations is influenced by SNR and sample size.
problem Understanding how neural network representations align across different conditions.
method Controlled training of neural networks on perturbed datasets, analyzing alignment and generalization.
result Alignment varies monotonically with SNR but non-monotonically with sample size, with minimal alignment near the interpolation threshold.
MKA incorporates manifold geometry into kernel alignment for more robust representation comparison.
problem Inadequate accounting for manifold geometry in kernel alignment metrics.
method Derives a theoretical framework for Manifold Approximated Kernel Alignment (MKA).
result MKA provides a more robust foundation for measuring representations.
DTA aligns multimodal data with prior correspondence knowledge.
problem Challenges in aligning data from different domains.
method Semi-supervised manifold alignment method exploiting prior correspondence knowledge.
result DTA outperforms other methods in aligning multimodal data.
The paper examines how spike strengths and alignments affect overfitting in linear regression models.
problem The impact of spike strengths and alignments on overfitting in linear regression models.
method Characterization of generalization error through exact expressions and analysis of spike strengths, aspect ratio, and target alignment.
result Increasing spike strength can lead to catastrophic overfitting before benign overfitting, especially in well-specified aligned problems.
We study the question of how to imitate tasks across domains with discrepancies such as embodiment, viewpoint, and dynamics mismatch. Many prior works require paired, aligned demonstrations and an additional RL step that requires environment interactions. However, paired, aligned demonstrations are seldom obtainable an…
SAIL improves online alignment of large language models with minimal feedback.
problem Offline RLHF methods often lead to sub-optimal performance due to fixed preference datasets.
method SAIL uses bilevel optimization and a single-level first-order method to iteratively refine model alignment.
result SAIL significantly improves alignment performance on open-sourced datasets with minimal computational overhead.
New approach speeds up DNA sequence alignment.
problem Efficiently estimating alignment scores for large sets of reads.
method Rank-one crowdsourcing models and multi-armed bandit algorithm.
result Adaptive algorithm identifies pairs with large alignment scores.
New method for robust distribution alignment using log-likelihood ratio and normalizing flows.
problem Distribution alignment challenges in deep learning.
method Log-likelihood ratio statistic and normalizing flows.
result Minimizing the proposed objective yields robust domain alignment.
CARDS improves decoding efficiency and alignment quality for LLMs.
problem Efficiency bottlenecks in decoding-time alignment for LLMs.
method Cascade Reward Sampling (CARDS) with segment-level rejection sampling and uncertainty-based segmentation.
result Significant improvement in decoding efficiency and alignment quality.
MALI aligns distinct domains using labeled data.
problem Aligning multi-domain data for machine learning.
method MALI learns manifold structure via diffusion and uses labeled data to guide alignment.
result MALI outperforms state-of-the-art methods across multiple datasets.