New framework assesses neural sensitivity to small perturbations.
problem Comparing neural representations' sensitivity to small changes.
method Local decodable information, Fisher information, and projected pullback/Fisher metric.
result Reveals differences in neural sensitivity not captured by activation alignment.
Develops an ordinal-similarity framework for scalable and interpretable representation alignment.
problem Measuring representation similarity in large datasets.
method Triplet and Quadruplet Similarity Indices.
result Demonstrates inherent interpretability, robustness to outliers, and computational efficiency.
LPL optimizes embeddings to align local neighborhoods, improving cross-lingual word alignment.
problem Aligning embeddings across different datasets and languages.
method Locality Preserving Loss (LPL) optimizes model to project embeddings while maintaining local neighborhoods and aligning them.
result LPL-based alignment leads to better and consistent accuracy, especially in small training set settings.
New method learns domain-invariant local feature patterns for unsupervised domain adaptation.
problem Performance degradation due to domain-shift in unsupervised domain adaptation.
method Jointly learns domain-invariant local feature patterns and holistic feature distributions.
result Superior performance on benchmark datasets compared to state-of-the-art methods.
Paper proposes new neural network learning algorithms inspired by predictive coding.
problem Finding biologically plausible alternatives to back-propagation of errors.
method Error-driven Local Representation Alignment (LRA-E) and Difference Target Propagation.
result Both proposed algorithms yield stable performance and strong generalization in training deeper, highly nonlinear networks.
VAE learns interpretable representations by chance, aligning with PCA.
problem Understanding why VAEs learn interpretable representations.
method Analyzed the diagonal approximation and stochasticity in VAEs.
result Variance autoencoders learn interpretable representations due to local orthogonality.
The paper proposes a learning-theoretic perspective on representation alignment.
problem Understanding how AI models' representations become aligned as they scale.
method Reviewing and connecting different notions of alignment, focusing on stitching.
result Relating properties of stitching to kernel alignment of representations.
LLM trading agents show risk feedback can improve alignment without fine-tuning.
problem Aligning LLM trading agents with financial risk.
method TradeArena testbed, risk reports, execution simulation, memory replay.
result Risk feedback can improve alignment without fine-tuning, but not universally.
Using back-propagation and its variants to train deep networks is often problematic for new users. Issues such as exploding gradients, vanishing gradients, and high sensitivity to weight initialization strategies often make networks difficult to train, especially when users are experimenting with new architectures. Her…
TMDA aligns subdomain data distribution discrepancies across domains using manifold representations.
problem Transfer learning challenges due to domain divergence.
method TMDA uses low-dimensional manifolds to represent subdomains and aligns local data distribution discrepancies across domains using M3D.
result TMDA is a promising method for various transfer learning tasks.
Two semi-supervised manifold alignment methods improve cross-domain classification.
problem Aligning data from multiple sources for better analysis.
method SPUD and MASH methods using graph integration and diffusion.
result SPUD and MASH methods outperform existing methods in cross-domain classification.
MKA incorporates manifold geometry into kernel alignment for more robust representation comparison.
problem Inadequate accounting for manifold geometry in kernel alignment metrics.
method Derives a theoretical framework for Manifold Approximated Kernel Alignment (MKA).
result MKA provides a more robust foundation for measuring representations.
Neural network iteratively refines image registration, achieving compactness and speed.
problem Non-compact representation of deformations in image registration.
method Recurrent registration neural network that computes local deformations iteratively.
result Our method achieves similar accuracy but is more compact and faster.
This paper improves cross-domain learning using random forests for manifold alignment.
problem Improving cross-domain learning and feature integration.
method Semi-supervised manifold alignment using random forest proximities.
result Random forest proximities enhance downstream classification accuracy.
New approach to deep learning for domain adaptation.
problem Learning a model on a target domain using a similar source domain.
method Introducing a search framework for correct alignment of high-level representations.
result Conceptual domain adaptation improves deep learning performance.
L2G2G improves graph autoencoder accuracy without sacrificing scalability.
problem Scalability issues in graph autoencoders for large networks.
method Dynamic synchronization of latent node representations in each epoch.
result L2G2G achieves higher accuracy than standard Local2Global approach on large and dense networks.
Study evaluates manifold alignment methods for noisy double pendulum dynamics.
problem Aligning manifolds of double pendulum dynamics under noise.
method Compared four manifold alignment methods: semi-supervised feature-level global and local.
result Local alignment methods were more robust to noise and faster.
Better data representations can simplify learning tasks by aligning model distributions with true data distributions.
problem Learning complexity influenced by the alignment of model distributions with true data distributions.
method Analyzed the effect of data representations on learning complexity using a task complexity score and information coding length.
result Better representations can simplify learning tasks by aligning model distributions with true data distributions, improving learning outcomes.
Foundation models fail to preserve continuous geometry, identified as the Geometric Alignment Tax.
problem Continuous geometry is lost in foundation models due to discrete categorical bottlenecks.
method Controlled ablations on synthetic systems and evaluation of 14 biological models using rate-distortion theory and MINE.
result Replacing cross-entropy with a continuous head reduces geometric distortion by up to 8.5x.
CurvSSL improves SSL by aligning local manifold curvature.
problem Improving self-supervised learning by capturing local manifold geometry.
method CurvSSL augments Barlow Twins with a curvature-based regularizer to align and decorrelate embeddings across augmentations.
result Curvature-regularized SSL yields competitive or improved linear evaluation performance.
AI models aligned with human vision perform well on few data tasks.
problem Few-shot learning performance with limited data.
method Information-theoretic analysis and empirical testing of 491 models.
result Highly aligned models show better robustness to attacks and domain shifts.
Alignment of neural network representations is influenced by SNR and sample size.
problem Understanding how neural network representations align across different conditions.
method Controlled training of neural networks on perturbed datasets, analyzing alignment and generalization.
result Alignment varies monotonically with SNR but non-monotonically with sample size, with minimal alignment near the interpolation threshold.
Proposes a method for weakly-supervised object localization to improve few-shot learning.
problem Challenges of few-shot learning, especially with fine-grained categories.
method Introduces a Self-Attention Based Complementary Module (SAC Module) for weakly-supervised object localization.
result Significantly outperforms state-of-the-art methods on benchmark datasets, especially for fine-grained few-shot tasks.
A framework for federated learning with heterogeneous data.
problem Federated learning with data from clients using different data representations.
method FLIC framework that maps client data into a common feature space via local embedding functions, learned federally using Wasserstein barycenters and trained locally via distribution alignment.
result FLIC outperforms FL benchmarks with heterogeneous input feature spaces.
Analyzes learning dynamics of RNNs under locality constraints.
problem Understanding learning dynamics in RNNs with locality constraints.
method Dynamical systems theory applied to data-aligned linear RNNs.
result RFLO solutions are restricted to low-rank perturbations of initial parameters.
New method visualizes brain activity changes over time.
problem Understanding representational dynamics in neural responses.
method Procrustes-aligned Multidimensional Scaling (pMDS) on RDM movies.
result Multidimensional scaling alignment captures representational dynamics.
This work optimizes alignment and uniformity of features on a hypersphere for better downstream performance.
problem Improving the performance of contrastive representation learning.
method Identifying and optimizing alignment and uniformity of features on a hypersphere.
result Directly optimizing alignment and uniformity leads to comparable or better performance than contrastive learning.
Recent work on learning multilingual word representations usually relies on the use of word-level alignements (e.g. infered with the help of GIZA++) between translated sentences, in order to align the word embeddings in different languages. In this workshop paper, we investigate an autoencoder model for learning multil…
Unified deep architecture for domain-invariant network alignment.
problem Eliminate domain representation bias in network alignment.
method DANA (Domain Adversarial Network Alignment) using graph convolutional networks and semi-supervised learning.
result Achieves state-of-the-art alignment results on real-world social networks.
Gradient-free deep learning for large datasets.
problem Training deep neural networks on large-scale datasets is resource-intensive and requires specialized techniques.
method Recursive Local Representation Alignment (RLRA) for gradient-free training.
result RLRA achieves comparable performance to backprop while converging faster and being parallelizable.
CKA with Gaussian RBF kernels converges linearly as bandwidth increases.
problem Understanding the behavior of CKA with large bandwidth Gaussian kernels.
method Analyzing the convergence of CKA based on Gaussian RBF kernels in the large-bandwidth limit.
result CKA based on Gaussian RBF kernels converges linearly as bandwidth increases.
Geometric framework detects concept frustration between human concepts and machine representations.
problem Aligning human concepts with machine learning representations.
method Geometric framework and similarity measures for detecting concept frustration.
result Concept frustration affects machine learning model performance and reorganizes learned concept representations.
New estimator corrects bias in CKA for sparsely sampled neurons.
problem Bias in CKA for sparsely sampled neurons.
method Novel estimator that corrects for input and feature sampling.
result Reliable model-to-brain alignment with sparsely sampled neurons.
A novel OT-based method for aligning hyperbolic representations.
problem Aligning different hyperbolic representations of hierarchical data.
method Optimal transport (OT) on the Poincaré model of hyperbolic spaces, using gyrobarycenter mapping.
result Both Euclidean and hyperbolic OT-based methods perform similarly in retrieval tasks.
A new method for drone-based geo-localization using style and spatial alignment.
problem Geo-localization of drone-view images with satellite-view images using pre-annotated GPS tags.
method Orientation-based method to align patterns, new branch to extract aligned partial features, style alignment strategy.
result The proposed method outperforms state-of-the-art alternatives in geo-localization accuracy.
Early alignment in neural networks leads to sparse representations but hinders convergence.
problem The implicit bias of gradient descent during early training phases.
method Quantitative description of early alignment phase in small initialisation, one hidden layer networks.
result Early alignment induces a sparse representation but also hinders convergence to global minima.
The paper addresses rigid alignment of noisy patches, providing a polynomial time algorithm and convergence conditions.
problem Finding a rigid alignment of overlapping local views (patches) that minimizes alignment error in a noisy setting.
method Characterizes non-degeneracy based on kernel and positivity of a matrix, provides polynomial time algorithm for testing non-degeneracy, and uses Riemannian gradient descent for alignment.
result The algorithm converges locally linearly to a non-degenerate perfect alignment under certain conditions.
Word embedding models offer continuous vector representations that can capture rich contextual semantics based on their word co-occurrence patterns. While these word vectors can provide very effective features used in many NLP tasks such as clustering similar words and inferring learning relationships, many challenges …
Proposes COALA method for learning audio representations aligned with tags.
problem Lack of annotated data for high-performance audio representation learning.
method Aligns latent representations of audio and tags using a contrastive loss.
result Audio embedding model captures both acoustic and semantic characteristics.
High-dimensional models pose both safety benefits and risks.
problem Emergent problems in safety alignment due to high-dimensional representations.
method Detailed visualizations and lower-dimensional subspace projections.
result Dimensional reduction preserves safety alignment while avoiding linear jailbreaking.
Proposes a new method for manifold alignment using geometry-regularized twin autoencoders.
problem Traditional MA methods lack out-of-sample extension and real-world applicability.
method Guided representation learning with geometry-regularized twin autoencoders.
result Improves cross-domain generalization and robustness while maintaining alignment fidelity.
The application of machine learning to bioinformatics problems is well established. Less well understood is the application of bioinformatics techniques to machine learning and, in particular, the representation of non-biological data as biosequences. The aim of this paper is to explore the effects of giving amino acid…
Deep Divergence Graph Kernels learn graph representations without supervision.
problem Learning graph representations without feature engineering or labeled graphs.
method Unsupervised method using cross-graph attention networks and divergence scores.
result Learned representations achieve competitive results on graph classification tasks.
A new method for unsupervised disentanglement in GANs.
problem Learning disentangled representations in generative models.
method Regularizing GANs by aligning Jacobian vectors with coordinate axes.
result Unsupervised disentanglement achieved in GANs through spectral regularization.
A new deep generative model captures global dependencies without supervision.
problem Global modeling in deep generative models.
method Non-i.i.d. variational autoencoders with mixture model and global Gaussian latent variable.
result Captures interpretable disentangled representations and domain alignment.
Cross-language learning allows us to use training data from one language to build models for a different language. Many approaches to bilingual learning require that we have word-level alignment of sentences from parallel corpora. In this work we explore the use of autoencoder-based methods for cross-language learning …
Paper proposes using LSTM for LSH-based sequence alignment.
problem Sequence alignment using deep learning models.
method Deep bidirectional LSTM for feature learning and LSH-based sequence alignment.
result Higher accuracy achieved with LSTM-based model.
Paper presents a method to align unpaired samples across different modalities.
problem Challenges in collecting paired samples for multimodal representation learning.
method Uses propensity score alignment based on Rubin's framework to estimate a common space for unpaired samples.
result Optimal transport matching significantly improves alignment in real-world data.