Algorithm learns fair representations without sacrificing accuracy across groups.
problem Mitigating disparity among different demographic subgroups in classification.
method Balanced error rate and conditional alignment of representations.
result Improves utility-fairness trade-off on balanced datasets.
Adversarial approach for optimizing multivariate performance measures in structured predictions.
problem Inconsistency between learner's objective and desired application performance.
method Adversarial training of structured predictions to optimize exact F-score or AER while matching training data properties.
result Improvement in multivariate performance measures for word alignment and named entity recognition.
New method improves reliability of selecting individuals based on predicted treatment effects.
problem Reliability of selecting individuals based on predicted conditional average treatment effects (CATE) is unreliable.
method Denoised Conformal Alignment, combining proxy errors, variance estimation, and Benjamini-Hochberg selection.
result Significantly improved power in selecting individuals while maintaining false discovery rate control.
New welfare-based fairness notions align with existing error rate balance and predictive parity.
problem Aligning fairness notions with welfare-based criteria.
method Discussing and establishing conditions for envy freeness and prejudice freeness.
result Envy freeness and prejudice freeness are equivalent to error rate balance and predictive parity.
Method estimates LLM error rates using Pareto optimization.
problem Quantifying error rates in text-generating models.
method Pareto optimization for generating risk scores.
result Risk scores correlate well with true error rates.
GALA adapts learning rates online by aligning gradients, improving deep learning model performance.
problem Fine-tuning learning rates for deep learning models requires extensive grid search.
method GALA dynamically adjusts learning rates by tracking gradient alignment and local curvature.
result GALA produces a flexible, adaptive learning rate schedule that increases when gradients align.
This work explains early-stage alignment of logistic regression parameters with max-margin direction.
problem Early-stage alignment of logistic regression parameters with max-margin direction.
method Analyzes gradient descent dynamics and uses dataset geometry to track radial and tangential flows.
result Parameter vector weakly aligns with max-margin direction within O(exp(exp(-δ))) iterations, demonstrating faster weak alignment.
TeLeS improves ASR confidence estimation by considering temporal alignment and lexical errors.
problem Inaccurate confidence scores from E2E ASR models, especially for overconfident predictions.
method Proposes TeLeS, a novel confidence score that considers temporal alignment and lexical errors, and uses shrinkage loss to handle data imbalance.
result TeLeS generalizes well across different languages and ASR models, leading to significant WER reduction.
The paper examines how spike strengths and alignments affect overfitting in linear regression models.
problem The impact of spike strengths and alignments on overfitting in linear regression models.
method Characterization of generalization error through exact expressions and analysis of spike strengths, aspect ratio, and target alignment.
result Increasing spike strength can lead to catastrophic overfitting before benign overfitting, especially in well-specified aligned problems.
Improved deep learning for one-shot and open-set classification using alignment-based matching.
problem Limited data for one-shot classification and open-set recognition.
method Aligns images to reference images for classification, learns alignment mechanism.
result Significantly improved classification accuracy (e.g., 1.4% error rate in Omniglot, 46.5% in MiniImageNet).
The paper addresses rigid alignment of noisy patches, providing a polynomial time algorithm and convergence conditions.
problem Finding a rigid alignment of overlapping local views (patches) that minimizes alignment error in a noisy setting.
method Characterizes non-degeneracy based on kernel and positivity of a matrix, provides polynomial time algorithm for testing non-degeneracy, and uses Riemannian gradient descent for alignment.
result The algorithm converges locally linearly to a non-degenerate perfect alignment under certain conditions.
Study optimizes funding rates for cryptocurrency perpetual futures to maintain price alignment.
problem Maintaining alignment between perpetual future prices and target values in cryptocurrency markets.
method Developed replicating portfolios and path-dependent funding rates using path-dependent infinite-horizon BSDEs and arbitrage pricing theory.
result Appropriate funding rate design can keep perpetual future prices aligned with target values.
LLMs struggle with zero-shot annotation tasks due to model-internalized priors.
problem Impact of model-internalized priors on LLM performance in zero-shot annotation tasks.
method Investigated three dimensions: familiarity, decision stickiness, and susceptibility to misaligned task definitions.
result Nearly two-thirds of zero-shot errors are resistant to correction, with a rescue rate of 34.8%. Definition-Specific Familiarity (DSF) shows a positive association with model performance.
TKRR improves KRR performance by aligning target functions with kernels.
problem Improving kernel ridge regression performance through target alignment.
method Focuses on truncated kernel ridge regression (TKRR) with an additional spectral truncation parameter.
result TKRR can achieve faster rates than full KRR, reaching parametric rates.
Identifies bilinear systems from a single trajectory with optimal sample complexity.
problem Learning bilinear systems from a single trajectory of states and inputs.
method Uses a mild marginal mean-square stability assumption and martingale small-ball condition.
result Sample complexity and statistical error rates are optimal.
DRO-REBEL improves LLM alignment by robustly updating models online.
problem Overfitting and drifting of LLMs during RLHF.
method DRO-REBEL uses type-p Wasserstein, KL, and χ2 ambiguity sets for robust online updates. result DRO-REBEL achieves faster convergence and better performance than prior methods.
The paper addresses poor calibration in fine-tuned LLMs after preference alignment.
problem Poor calibration in fine-tuned Large Language Models (LLMs) after preference alignment.
method Proposes a calibration-aware fine-tuning approach to restore calibration without compromising model performance.
result Demonstrates the effectiveness of the proposed methods through extensive experiments.
New method trains deep neural networks without backpropagation.
problem Training deep neural networks efficiently and without backpropagation.
method Direct feedback alignment using random weights for error propagation.
result Achieves zero training error in convolutional and deep networks.
Faster WIND accelerates iterative BOND for LLM alignment.
problem Iterative BOND is inefficient in practice due to sample and computation inefficiency.
method Unified game-theoretic connection to self-play alignment, WIND framework with efficient algorithms.
result WIND variant achieves superior sample efficiency and faster computation.
The study analyzes implicit biases in neural networks using backward error analysis.
problem Analyzing implicit biases in multitask and continual learning settings.
method Backward error analysis to compute implicit training biases, deriving modified losses with three terms.
result The conflict term, measuring gradient alignment, is a new quantity in continual learning.
Study evaluates manifold alignment methods for noisy double pendulum dynamics.
problem Aligning manifolds of double pendulum dynamics under noise.
method Compared four manifold alignment methods: semi-supervised feature-level global and local.
result Local alignment methods were more robust to noise and faster.
New approach improves domain adaptation by relaxing distribution alignment constraints.
problem Improving domain adaptation when target distribution differs from source distribution.
method Asymmetrically-relaxed distribution alignment to minimize target error under varying conditions.
result Empirical and theoretical benefits demonstrated on synthetic and real datasets.
A new method reduces preference distortion in LLM alignment.
problem Vulnerability of traditional LLM alignment methods to human preference heterogeneity.
method Sign Estimator: A simple, provably consistent, and efficient estimator using binary classification loss.
result Substantially reduces preference distortion over a panel of simulated personas.
nGPT learns to transfer learning rates across model dimensions and token horizons.
problem nGPT does not transfer learning rates across model size and token horizon.
method Combining numerical experiments with alignment exponents, a novel nGPT parameterization νGPT is developed.
result νGPT exhibits learning rate transfer across width, depth, and token horizon.
Proposes a regularization method for unsupervised domain adaptation that aligns predictions with target data's top singular vectors.
problem Domain adaptation challenges in high joint error scenarios.
method Regularizes classifier to align with unsupervised target data guided by label alignment property (LAP).
result The method improves performance in MNIST-USPS domain adaptation and cross-lingual sentiment analysis.
LLMs can fail to maximize aligned values even after training, due to irrational reasoning.
problem Value misalignment in LLMs' reasoning despite training alignment.
method Formalized rational value risk and decomposed estimation error.
result Rational value risk is widespread and cannot be fully eliminated.
Direct feedback alignment reduces data movement in neural networks.
problem Efficiency and energy-efficiency in training large neural networks.
method Sparse feedback matrix for local learning, reducing data movement and compute.
result Orders of magnitude improvement in data movement and 2x improvement in multiply-and-accumulate operations.
We replace the Hidden Markov Model (HMM) which is traditionally used in in continuous speech recognition with a bi-directional recurrent neural network encoder coupled to a recurrent neural network decoder that directly emits a stream of phonemes. The alignment between the input and output sequences is established usin…
WiFi helps align and calibrate foot-mounted IMU trajectories.
problem Inertial drift and unknown initial states in FMIP.
method Graph-based SLAM with RSS measurements for WiFi APs.
result Aligns and calibrates trajectories accurately.
Proposes JK-EGW for multimodal alignment using shared latent space.
problem Aligning data from multiple modalities into a shared representation space.
method Joint kernel entropic Gromov--Wasserstein Optimal Transport (JK-EGW).
result Improved multimodal retrieval performance compared to baselines.
New algorithm shows neural networks can learn without full backpropagation.
problem Stochastic gradient descent with backpropagation is non-biologically plausible.
method Random and fixed backpropagation weights in a feedback alignment algorithm.
result Error converges to zero exponentially fast in overparameterized networks.
This research explores various sampling methods and probability distributions for hard alignment in sequence-to-sequence TTS synthesis.
problem Improving alignment accuracy in sequence-to-sequence text-to-speech synthesis.
method Investigated various sampling methods (greedy, beam, random) and probability distributions (Bernoulli, Concrete) for hard alignment.
result Deterministic search is more preferable than stochastic search for natural alignment transition.
Training-free source selection for LLM families with shared vocabularies
problem Source selection for LLM families with shared vocabularies
method Fisher alignment at vocabulary scale
result Fisher alignment is a cosine between kernel mean embeddings in the joint activation-error space
We study alignment in linear neural networks and its relation to gradient descent.
problem Understanding alignment in linear neural networks and its impact on training.
method Defined alignment for fully connected networks, analyzed alignment under gradient descent, and compared gradient descent to projected gradient descent for layer-constrained networks.
result Gradient descent can converge linearly to a global minimum when alignment is invariant, and alignment is impossible with large datasets in layer-constrained networks.
Decision-alignment evaluates uncertainty quantification for decision-relevant UQ
problem Evaluation of uncertainty quantification metrics
method Introduce decision-alignment
result Proper scoring rules align with decision utility
ADS explains object differences by quantifying and removing underlying properties.
problem Explaining differences between two object images.
method Align-Deform-Subtract (ADS) framework that uses semantic alignments and iterative quantification/removal of differences.
result ADS provides disentangled error measures explaining object differences in terms of underlying properties.
Study improves GMM learning performance through multi-task and transfer learning.
problem Improving GMM learning performance through similar task structures.
method Proposes a multi-task GMM learning procedure based on EM algorithm, robust to outliers.
result Achieves minimax optimal rate of convergence for parameter estimation and mis-clustering.
Alignment of neural network representations is influenced by SNR and sample size.
problem Understanding how neural network representations align across different conditions.
method Controlled training of neural networks on perturbed datasets, analyzing alignment and generalization.
result Alignment varies monotonically with SNR but non-monotonically with sample size, with minimal alignment near the interpolation threshold.
Transformer models align words through attention weights, closely approximating Optimal Transport.
problem Understanding the internal mechanism of transformer models in language processing.
method Empirical evidence and theoretical analysis of attention weights and their relation to Optimal Transport.
result Transformer models can simulate gradient descent on the dual of entropy-regularized OT problem, providing a theoretical foundation for token alignment.
Framework quantifies financial NLP robustness under regime shifts.
problem Semantic and causal drift in financial news narratives.
method Four metrics: FCAS, PCS, TSV, NLICS.
result Transformer models are more affected by semantic drift.
Develops techniques to align word embeddings from different sources.
problem Aligning word embeddings from diverse datasets or methods.
method Simple closed-form techniques for optimal rotation, translation, and scaling.
result Maximizes cosine similarity and minimizes root mean squared errors.
The paper explains how data augmentation improves semi-supervised learning efficiency.
problem Improving accuracy from a small fraction of labeled data.
method Data augmentation induces a similarity graph, which is graph-Laplacian-regularized for downstream learning.
result A fast transductive rate of O(1/nL) is achieved, reducing the number of labels needed. New approach finds optimal hidden paths in large models, scaling to high dimensions.
problem Finding optimal hidden paths in large, high-dimensional models.
method Developed a new approach to existence of the infinite Viterbi alignment for models satisfying a decay-convexity condition.
result Quantitative bounds on the distance to the infinite Viterbi alignment, demonstrating scalability to high-dimensional problems.
Local update methods' performance depends on learning rates, affecting convergence rates and alignment with true loss.
problem The performance of local update methods in federated learning and meta-learning is sensitive to learning rates.
method Proved that local update methods perform SGD on a surrogate loss function, characterized the surrogate loss, and derived convergence rates.
result Proper learning rate tuning is crucial for near-optimal behavior in communication-limited settings.
New method measures patient similarity over time, improving disease risk prediction.
problem Chronic diseases' varying progression rates and heterogeneous clinical presentations make patient comparison difficult.
method Subsequence alignment to account for pathophysiological misalignment and varying patient presentation times.
result Subsequence alignment outperforms global alignment in predicting disease progression.
This study improves knowledge distillation for RNN-T models with noisy labels.
problem Challenges in distilling knowledge from RNN-T models with variable quality teachers.
method Full-sum distillation and sequence-level knowledge distillation.
result Full-sum distillation outperforms other methods for RNN-T models, especially for bad teachers.
Paper proposes new neural network learning algorithms inspired by predictive coding.
problem Finding biologically plausible alternatives to back-propagation of errors.
method Error-driven Local Representation Alignment (LRA-E) and Difference Target Propagation.
result Both proposed algorithms yield stable performance and strong generalization in training deeper, highly nonlinear networks.
Paper characterizes optimal language model alignment methods.
problem Aligning language models to maximize reward while keeping them close to the original model.
method KL-constrained reinforcement learning and best-of-N methods.
result Optimal KL-constrained RL solution has a large deviation principle rate function.