Children learn concepts without explicit teaching by aligning internal systems.
problem Learning concepts from noisy perceptual experience without explicit teaching.
method Using information in the environment to build and align conceptual systems.
result The more concepts and systems there are, the easier learning becomes.
New approach to deep learning for domain adaptation.
problem Learning a model on a target domain using a similar source domain.
method Introducing a search framework for correct alignment of high-level representations.
result Conceptual domain adaptation improves deep learning performance.
Geometric framework detects concept frustration between human concepts and machine representations.
problem Aligning human concepts with machine learning representations.
method Geometric framework and similarity measures for detecting concept frustration.
result Concept frustration affects machine learning model performance and reorganizes learned concept representations.
SAIL improves online alignment of large language models with minimal feedback.
problem Offline RLHF methods often lead to sub-optimal performance due to fixed preference datasets.
method SAIL uses bilevel optimization and a single-level first-order method to iteratively refine model alignment.
result SAIL significantly improves alignment performance on open-sourced datasets with minimal computational overhead.
Unified framework for efficient online training of RNNs.
problem Efficient and biologically plausible online training of recurrent neural networks.
method Organizes algorithms based on criteria like past vs. future facing, tensor structure, stochastic vs. deterministic, and closed form vs. numerical.
result Algorithms cluster according to criteria, revealing conceptual connections.
New method shortens and straightens curves, proving convergence and well-posedness.
problem Shortening and straightening of curves.
method Conceptual shift in curve shortening to tangent aligning, variational study of geometric flows.
result Proves convergence to a straight line and global well-posedness for various geometric flows.
Mitigates biases in reward models using variational inference.
problem Spurious correlations in reward models that align large language models with human preferences.
method Formulates data-generating process, identifies non-spurious latent variables, and uses variational inference to recover them.
result Effective mitigation of spurious correlation issues, yielding more robust reward models.
A new method aligns source and target distributions by tuning their weights.
problem Domain adaptation on unlabeled target datasets using labeled source datasets.
method Weighted Joint Distribution Optimal Transport (WJDOT) method that finds alignment between source and target distributions and re-weighting of source distributions.
result Achieves state-of-the-art performance on simulated and real-life datasets.
New methods identify concepts in trained embeddings reliably without human labels.
problem Identifying interpretable concepts in trained embedding spaces without human labels.
method Explicitly connecting concept discovery to PCA and ICA, proposing novel approaches for dependent concepts.
result Proven methods outperform competitors on a variety of experiments, achieving up to 29% better alignment with ground truth.
New algorithm improves continual learning by balancing transfer and interference.
problem Challenges in scaling neural network learning to non-stationary data.
method Proposes a new conceptualization of continual learning as a trade-off between transfer and interference, and develops Meta-Experience Replay (MER) to optimize this.
result Demonstrates consistent improvement over recent baselines in continual learning benchmarks.
SEMASIA provides a large dataset of latent representations for model comparison.
problem Difficulty in comparing semantic structures across different neural network models.
method Collection of latent representations from 1700 pretrained models across various benchmarks.
result Consistent semantic organization across models and datasets.
Optimal transport aligns source and target distributions for linear regression in 2D.
problem Domain adaptation for linear regression in 2D with limited target data.
method Combining K-means and optimal transport for estimating geometric transformations.
result Optimal transport recovers geometric transformations like rotations, translations, and homotheties.
Deep learning model for AI co-creativity boosts design creativity.
problem Design creativity enhancement through AI collaboration.
method Novelty metric applied to deep learning-generated vector representations for conceptual shifts in a co-creative design system.
result Increasing novelty in AI contributions correlates with higher creative outcomes in design.
New model explains how concepts grow based on experience.
problem Existing models assume fixed representation; new model allows for growth.
method Geometric framework with MDL criterion for basis extension.
result Conceptual growth is selective and conservative, exposing or amplifying residual error.
This work proves win rate is key to understanding preference learning.
problem Understanding preference learning from generative models.
method Analyzing preference learning methods as win rate optimization or non-WRO.
result Proves win rate is the only evaluation respecting preferences and prevalences.
This study creates a new conceptual framework for news aggregation.
problem Aggregating and presenting diverse news sources in a unified and accessible way.
method Developed a mobile app that analyzes unstructured data patterns to create a conceptual framework for news.
result Users can easily find and navigate through updated news using a new conceptual multilevel structure.
The cognitive framework of conceptual spaces bridges the gap between symbolic and subsymbolic AI by proposing an intermediate conceptual layer where knowledge is represented geometrically. There are two main approaches for obtaining the dimensions of this conceptual similarity space: using similarity ratings from psych…
We introduce a simple recurrent variational auto-encoder architecture that significantly improves image modeling. The system represents the state-of-the-art in latent variable models for both the ImageNet and Omniglot datasets. We show that it naturally separates global conceptual information from lower level details, …
Paper classifies institutions based on credit, debit, and funding adjustment paradigms.
problem Classifying institutions based on credit, debit, and funding adjustment paradigms.
method Mathematical framework based on the principle of invariance.
result Improved solution of principle of invariance equations for accurate metrics calculation.
Following the programme set out in Part I of this work, we develop a conceptual higher order differential calculus. The '' local linear algebra '' defined in Part I is generalized by '' higher order local linear algebra ''. The underlying combinatorial object of such higher algebra is the natural n-dimensional hyper-cu…
Proposes a thermodynamic work minimization framework for guiding generative models.
problem Guiding generative models in sparse-data regimes with limited target samples or constraints.
method Regularization framework inspired by thermodynamic work, introducing Path Guidance and Observable Guidance.
result Improves sample efficiency and reduces bias in molecular simulations.
Unsupervised method constructs knowledge graph from text and code.
problem Lack of structured knowledge in scientific literature and code.
method Word embedding, clustering, and dimensionality reduction techniques.
result Enhanced understanding of scientific literature and code.
Advocates for Marr's levels of analysis to unify machine learning debates.
problem Challenges in aligning perspectives among machine learning researchers.
method Introduces Marr's levels of analysis from cognitive science and neuroscience.
result Marr's levels facilitate understanding and dissection of machine learning methods.
We present a system for identifying conceptual shifts between visual categories, which will form the basis for a co-creative drawing system to help users draw more creative sketches. The system recognizes human sketches and matches them to structurally similar sketches from categories to which they do not belong. This …
The paper explores fundamental limits of learning non-hallucinating generative models.
problem Hallucinations in generative models producing invalid outputs.
method Developed a theoretical framework to analyze learnability from a learning-theoretic perspective, incorporating inductive biases.
result Non-hallucinating learning is statistically impossible without additional inductive biases.
'Ergodicity economics' is criticized as pseudoscience.
problem Flawed conceptual basis of mainstream economic theory.
method Claims 'ergodicity economics' is more parsimonious and clearer.
result Peters' approach has not produced falsifiable implications.
This paper surveys scalable automated alignment methods for LLMs.
problem Scalability issues in traditional human-annotated alignment methods for LLMs.
method Categorizes and discusses various automated alignment methods.
result Emerging automated alignment methods are effective and scalable.
Frolicher and Nijenhuis recognized well in the middle of the previous century that the Lie bracket and its Jacobi identity could and should exist beyond Lie algebras. Nevertheless the conceptual meaning of their discovery has been obscured by the messy techniques they exploited. The principal objective in this paper is…
Inference-aware meta-alignment of LLMs reduces computational cost.
problem Aligning LLMs to diverse human preferences is challenging due to conflicting criteria.
method IAMA trains a base model to be aligned to multiple tasks via different inference-time alignment algorithms, using non-linear GRPO for optimization.
result IAMA enables effective alignment of LLMs to multiple criteria with limited computational budget.
VALC provides concept-level interpretations of FLMs, overcoming word-level limitations.
problem Lack of higher-level structure interpretation in FLMs' attention weights.
method Formal definition of conceptual interpretation, variational Bayesian framework (VALC).
result VALC finds optimal language concepts for FLM predictions, providing concept-level interpretations.
Conformal Alignment ensures trustworthy outputs from foundation models.
problem Ensuring outputs from foundation models align with human values in high-stakes tasks.
method A framework that trains an alignment predictor using reference data to select trustworthy outputs.
result Conformal Alignment accurately identifies trustworthy outputs via lightweight training over moderate reference data.
PACE explains ViTs by modeling patch-level concept distributions, surpassing existing methods.
problem Lack of trustworthy post-hoc explanations for Vision Transformers (ViTs)
method Variational Bayesian explanation framework (PACE)
result PACE surpasses state-of-the-art methods in meeting desiderata for ViT explanations.
Study evaluates manifold alignment methods for noisy double pendulum dynamics.
problem Aligning manifolds of double pendulum dynamics under noise.
method Compared four manifold alignment methods: semi-supervised feature-level global and local.
result Local alignment methods were more robust to noise and faster.
LPL optimizes embeddings to align local neighborhoods, improving cross-lingual word alignment.
problem Aligning embeddings across different datasets and languages.
method Locality Preserving Loss (LPL) optimizes model to project embeddings while maintaining local neighborhoods and aligning them.
result LPL-based alignment leads to better and consistent accuracy, especially in small training set settings.
Extends reinforcement learning alignment to scalar rewards, improving math reasoning.
problem Designing reinforcement learning algorithms for general LLM alignment.
method Introduces f-GRPO and f-HAL, estimating f-divergences between reward-aligned and unaligned distributions.
result Improves math reasoning RLVR tasks and mitigates reward hacking.
The paper proposes a learning-theoretic perspective on representation alignment.
problem Understanding how AI models' representations become aligned as they scale.
method Reviewing and connecting different notions of alignment, focusing on stitching.
result Relating properties of stitching to kernel alignment of representations.
An isometric action of a Lie group on a Riemannian manifold is of cohomogeneity one if the corresponding orbit space is one-dimensional. In this article we develop a conceptual approach to the classification of cohomogeneity one actions on Riemannian symmetric spaces of noncompact type in terms of orbit equivalence. As…
In this paper we develop a general conceptual approach to the problem of existence of action-angle variables for dynamical systems, which establishes and uses the fundamental conservation property of associated torus actions: anything which is preserved by the system is also preserved by the associated torus actions. T…
Aligns databases with Gaussian features using MAP estimation and thresholding.
problem Aligning databases with jointly Gaussian features.
method Two algorithms: MAP estimation and thresholding of log likelihood ratios.
result Conditions on mutual information for reliable performance of algorithms.
Paper proposes unsupervised knowledge graph alignment with adversarial learning.
problem Aligning knowledge graphs from different sources or languages without large amounts of aligned triplets.
method Adversarial learning framework to align entity and relation embeddings, with mutual information regularization.
result Framework effectively aligns knowledge graphs in unsupervised and weakly-supervised settings.
Survey of alignment techniques for large language models.
problem Ensuring large language models align with human values.
method Analysis of diverse alignment methods and training paradigms.
result Preference-based methods offer more flexibility for nuanced alignment.
The paper addresses rigid alignment of noisy patches, providing a polynomial time algorithm and convergence conditions.
problem Finding a rigid alignment of overlapping local views (patches) that minimizes alignment error in a noisy setting.
method Characterizes non-degeneracy based on kernel and positivity of a matrix, provides polynomial time algorithm for testing non-degeneracy, and uses Riemannian gradient descent for alignment.
result The algorithm converges locally linearly to a non-degenerate perfect alignment under certain conditions.
Study connects database alignment and planted matching using Gaussian features.
problem Identify matching between correlated user features in anonymized databases.
method Derived results for database alignment and planted matching, showing connections and thresholds.
result Performance thresholds for database alignment converge to planted matching when feature dimensionality is sufficiently high.
SMAI framework tests and integrates single-cell data alignability.
problem Lack of a rigorous statistical test for alignability and distortion during alignment.
method Spectral manifold alignment and inference (SMAI) framework.
result SMAI outperforms existing methods in alignability testing and integration.
Bayesian framework proves thresholds for multi-graph alignment feasibility.
problem Determining when multi-graph alignment is statistically possible.
method Developed a Bayesian estimation framework over metric spaces.
result Identified thresholds for Gaussian and sparse Erdős-Rényi models.
Bayesian evidence helps compare models but can overfit.
problem Comparing hypotheses consistent with observations.
method Marginal likelihood, Occam's razor, PAC-Bayes bounds.
result Marginal likelihood can negatively correlate with generalization.
New method aligns brain data across individuals for better brain decoding.
problem Inter-individual variability in brain response patterns limits decoder generalization.
method SpectralOT method that embeds cortical geometry into Laplace-Beltrami eigenmodes.
result SpectralOT strikes balance between aligning functional features and preserving anatomical structure.
Method aligns multilingual news for better stock return prediction.
problem Losing nuances in multilingual news translations.
method Optimal transport for sentence alignment.
result Aligned sentences correlate better with stock returns.