TCT learns multimodal sequence representations by translating from related sequences.
problem Challenges in learning semantic representations from multimodalities.
method Transformer based Cross-modal Translator (TCT) combined with Multimodal Transformer Network (MTN).
result Proposed method achieves new state-of-the-art performance on video-grounded dialogue.
Framework learns portable representations for diverse tasks.
problem Creating task-independent abstract representations for diverse environments.
method Autonomously learns portable representations in egocentric space.
result Portable representations enable task-independent planning and transfer.
Our work proves CSF can recover ground-truth features in RL, improving understanding of feature learning.
problem Understanding the role of representation and mutual information in reinforcement learning.
method Investigates Contrastive Successor Features (CSF) method for identifiable representation learning in reinforcement learning.
result Proves CSF can recover ground-truth features up to a linear transformation.
Unsupervised learning models can be indistinguishable without identifiability, leading to unreliable representations.
problem Unsupervised learning models may be indistinguishable without identifiability, making it impossible to recover a ground truth generative model.
method Construction based on nonlinear independent component analysis theory to illustrate potential failure cases.
result Counterexamples show that identifiability is crucial for reliable unsupervised representation learning.
New approach decouples skill learning and language grounding for autonomous agents.
problem Autonomous acquisition of skills without external instructions and feedback.
method Language-Goal-Behavior (LGB) architecture with semantic representation.
result Decouples skill learning and language grounding, enabling diversity and strategy switching.
Ground-A-Video edits videos without training, preserving intended changes.
problem Complex multi-attribute video editing with omitted or wrong changes.
method Grounding-guided video-to-video translation with Cross-Frame Gated Attention.
result Zero-shot multi-attribute video editing with improved accuracy and frame consistency.
Recent efforts on training visual navigation agents conditioned on language using deep reinforcement learning have been successful in learning policies for different multimodal tasks, such as semantic goal navigation and embodied question answering. In this paper, we propose a multitask model capable of jointly learnin…
New method learns high-quality Laplacian representations for reinforcement learning.
problem Lack of accurate Laplacian representations in large or continuous state spaces.
method Reformulated spectral graph drawing objective to have eigenvectors as unique global minimizer.
result Learned Laplacian representations more faithfully approximate the ground truth.
New pseudo-Hermitian models from non-semisimple TQFTs.
problem Constructing exactly solvable pseudo-Hermitian spin Hamiltonians.
method Identifying ground states on surfaces using non-semisimple TQFTs.
result Ground states depend only on spatial topology and can be assigned by non-semisimple TQFTs.
RDLI integrates domain logic and context grounding to detect crypto anomalies under scarce labels.
problem Extreme label scarcity and evasion strategies in crypto networks.
method Relational Domain Logic Integration (RDLI) with Retrieval Grounded Context (RGC).
result RDLI outperforms GNN baselines by 28.9% in F1 score under 0.01% label scarcity.
New framework to test neural network representation similarity measures.
problem Disagreements among dissimilarity measures in neural networks.
method Statistical testing framework to evaluate measures based on functional behavior.
result Current metrics have different weaknesses; a classical baseline performs surprisingly well.
New framework for AI to learn causal models through experience.
problem Lack of guidance for variable choice and interventions in causal models for AI.
method Defines actions as state space transformations, introduces causal variables, and identifies interventions.
result Clarifies the concept of interventions and makes causal representation learning clearer.
New method identifies causal variables from partially observed data.
problem Learning from unpaired observations with instance-dependent partial observability.
method Proposes two methods enforcing sparsity in the inferred representation.
result Establishes two identifiability results for linear and piecewise linear mixing functions.
DBLE improves confidence calibration of DNNs by learning distances in representation space.
problem Poor confidence calibration of deep neural networks (DNNs).
method DBLE trains a confidence model jointly with the classification model, using distances in the representation space.
result DBLE outperforms alternative single-model confidence calibration approaches and ensemble methods.
This paper proposes a new method to connect language and physical actions in reinforcement learning.
problem Connecting linguistic representations to the physical world in embodied agents.
method Language-conditioned goal generators to decouple sensorimotor learning from language acquisition.
result Agents can demonstrate a diversity of behaviors for any given instruction.
Paper introduces untangling number to quantify 3-periodic tangle complexity.
problem Quantifying the complexity of 3-periodic tangles in biological, chemical, and physical systems.
method Introduces untangling number, a measure of minimum distance to ground state through diagrammatic operations.
result For infinite open curves, generic ground states are crystallographic rod packings.
PoGDiff improves diffusion models on imbalanced datasets.
problem Diffusion models struggle with imbalanced text-image pairs.
method PoGDiff replaces ground-truth with a Product of Gaussians.
result PoGDiff enhances generation accuracy and quality on imbalanced datasets.
Shapelet transform improves time series classification for earthquake, wind, and wave events.
problem Autonomous detection of specific events from large time series datasets in civil engineering.
method Shapelet transform for local similarity in time series subsequences, combined with machine learning.
result Shapelet transform yields a new feature representation for time series signals in civil engineering.
New method improves domain generalization by matching object representations.
problem Existing domain generalization methods fail to generalize to unseen domains.
method Proposes matching-based algorithms to match object representations across domains.
result MatchDG algorithm matches ground-truth object representations and improves out-of-domain accuracy.
New method extracts brain age from MRI sequences over time.
problem Lack of ground-truth labels in longitudinal neuroimaging data.
method Combines factor disentanglement with self-supervised learning.
result Extracts brain age information from MRI sequences.
Tactile information is important for gripping, stable grasp, and in-hand manipulation, yet the complexity of tactile data prevents widespread use of such sensors. We make use of an unsupervised learning algorithm that transforms the complex tactile data into a compact, latent representation without the need to record g…
Method learns state representations without supervision for Atari games.
problem Learning state representations without supervision.
method Maximizes mutual information across features of neural encoder.
result New benchmark for evaluating representation learning models.
New approach uses prior knowledge to improve neural network representations.
problem Difficulty in learning robust internal representations without a large training set.
method Incorporates prior information into a pre-trained reasoning module.
result Improves representation learning in diverse self-supervised learning settings.
Paper proposes using unlabeled data for fair decision-making.
problem Bias in decision-making algorithms due to biased labels and selective labeling.
method Variational autoencoder for learning unbiased data representations from both labeled and unlabeled data.
result Method learns fair and stable decision policies with high utility.
Improved VAE learns disentangled representations with less supervision.
problem Learning disentangled representations is challenging.
method Semi-supervised disentanglement learning with label replacement.
result Significant improvement in disentanglement with minimal supervision.
Abstract: New methods for finding optimal controls in geometric problems on Lie groups.
problem Finding optimal controls for geodesics on Lie groups.
method Pontryagin maximum principle, (co)adjoint representation, geodesic vector field.
result Developed methods to find normal geodesics and locally optimal controls.
New theory for partial disentanglement from sparse graphs.
problem Disentangling latent factors from sparse causal graphs.
method Generalization of disentanglement theory to any graph, using consistency equivalence.
result Partial disentanglement captures expected factor entanglement based on graph structure.
The paper validates a method for recovering over-parameterized matrices and images from noisy measurements.
problem Recovering a low-rank matrix from noisy measurements when the rank is unknown.
method Using gradient descent with small random initialization on a nonconvex objective function built from a rank-overspecified factored representation of the matrix variable.
result Gradient descent iterations converge to the ground-truth matrix under certain conditions and can be stopped efficiently to detect a nearly optimal estimator.
TemperatureGAN generates hourly atmospheric temperature data with high fidelity.
problem Generating accurate hourly atmospheric temperature data for climate risk assessment.
method Generative Adversarial Network (GAN) conditioned on months, locations, and time periods.
result TemperatureGAN produces high-fidelity hourly atmospheric temperature data with good spatial and temporal consistency.
New method tests conditional independence using spectral representations.
problem Untestable conditional independence in many settings.
method Spectral representations of partial covariance operators, bi-level contrastive learning.
result Asymptotic validity and power guarantees for CI testing.
VCML learns concepts and metaconcepts from images and questions.
problem Learning concepts and metaconcepts from visual data.
method Bidirectional connection between visual concepts and metaconcepts.
result VCML can generalize from limited data and noisy inputs.
Neural model predicts object states and physical parameters from visual observations.
problem Computational models struggle with physical reasoning and adapting to new environments.
method Visual prior predicts particle-based system from visual observations; inference module refines estimates subject to dynamics constraints.
result Model can infer physical properties within a few observations and adapt to unseen scenarios.
We build deep RL agents that execute declarative programs expressed in formal language. The agents learn to ground the terms in this language in their environment, and can generalize their behavior at test time to execute new programs that refer to objects that were not referenced during training. The agents develop di…
MSA compares neural representations' intrinsic geometry for better understanding.
problem Existing similarity measures fail to capture subtle distinctions between neural network solutions.
method Metric similarity analysis (MSA) using Riemannian geometry.
result MSA can disentangle features of neural computations and compare nonlinear dynamics.
A new dynamical approach connects resolution cohomology to group representations.
problem Establishing a bijection between resolution cohomology and group representations.
method Gauge-theoretic Morse flow on the minimal resolution.
result The flow converges to specific irreducible representations of the group.
Self-Distilled Disentanglement improves counterfactual predictions by separating variables.
problem Improving counterfactual predictions in the presence of confounders and unobserved variables.
method Self-Distilled Disentanglement framework based on information theory.
result Effective counterfactual inference in synthetic and real-world datasets.
The paper defines metrics for evaluating disentangled representations in learning models.
problem Evaluating disentangled representations in learning models.
method Defining semantics and metrics for disentanglement learning.
result Proposed metrics correctly characterize representations learned by different methods.
A network removes irrelevant structures from chest radiographs for better analysis.
problem Clutter in chest radiographs hinders visual inspection and analysis.
method Fully Convolutional Network to suppress undesired visual structure.
result Improved classifier performance with limited training data.
New metric for disentangling multivariate representations, accounting for more complex entanglements.
problem Current disentanglement metrics fail to detect entanglements involving more than two variables.
method Partial Information Decomposition framework to analyze information sharing and propose a new disentanglement metric.
result The proposed metric correctly identifies entanglements in high-dimensional spaces.
Cosmos models scenes using neural encodings and symbolic attributes for compositional generalization.
problem Modeling scenes with high performance on unseen input scenes composed of known visual elements.
method Neurosymbolic grounding with neurosymbolic scene encodings and attention mechanisms.
result Establishes a new state-of-the-art for compositional generalization in world modeling.
ALONE uses neural networks to accelerate cine MRI without needing ground truth.
problem Accelerated MRI reconstruction without using ground truth.
method Unsupervised learning of shallow CNNs to approximate MRI patches.
result ALONE outperforms TV and DIC methods in cine MRI reconstruction.
New method learns quantum states using neural networks, revealing hidden dynamics.
problem High-precision ground state estimation of quantum many-body problems.
method Stochastic reconfiguration method with neural network Ansatz states.
result Learning landscape modes with least entanglement have largest eigenvalues, suggesting correlations are encoded in large flat valleys.
Method learns relational features for Gaifman models from knowledge bases.
problem Structure learning for Gaifman models.
method Relational tree distances to learn relational features.
result Empirical evaluation shows superiority over classical rule-learning.
CPPO learns policies from partial offline data in MDPs with structural assumptions.
problem Offline Reinforcement Learning with partial coverage assumption.
method Constrained Pessimistic Policy Optimization (CPPO) using a function class and model class constraint.
result CPPO achieves PAC guarantee with partial coverage, learning competitive policies.
This research shows how to learn shared representations from unpaired data.
problem Learning shared representations from unpaired data.
method Spectral embeddings of random walk matrices from each unimodal representation.
result Shared representations can be learned almost exclusively from unpaired data.
SAMI learns disentangled representations from data.
problem Learning disentangled representations from data.
method Combines diffusion models and VAEs to learn disentangled representations.
result SAMI learns disentangled representations that are interpretable and useful.
KINet learns object interactions without supervision for robotic pushing.
problem Lack of supervised data for object-centric forward prediction.
method End-to-end unsupervised framework using keypoint representation and contrastive estimation.
result Automatically generalizes to unseen scenarios and accurately predicts future states.
New metric measures dynamical richness without relying on accuracy.
problem Lack of a reliable metric for measuring dynamical richness.
method Developed a computationally efficient, performance-independent metric based on low-rank bias.
result Metric recovers neural collapse as a special case and captures known transitions without accuracy.