We present an approach for building an active agent that learns to segment its visual observations into individual objects by interacting with its environment in a completely self-supervised manner. The agent uses its current segmentation model to infer pixels that constitute objects and refines the segmentation model …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “object interactions”
Modeling interacting objects with latent Gaussian process ODEs.
problem Time uncertainty-aware modeling of continuous-time dynamics of interacting objects.
method A new model using latent Gaussian process ordinary differential equations to infer independent dynamics and interactions.
result Our model improves long-term predictions and successfully encapsulates independent dynamics and interactions.
R-SQAIR adds relational bias to sequential object attention models for better object interactions.
problem Traditional sequential multi-object attention models struggle with relational inferences.
method Proposes R-SQAIR, a relational extension of SQAIR with a parallel pairwise interaction module.
result Demonstrates gains in object relations and combinatorial generalization over sequential mechanisms.
STOVE predicts convincing physical behavior in videos.
problem Learning models from videos with objects and interactions.
method Compositional state-space model combining image and dynamics models.
result Predicts videos with convincing physical behavior over hundreds of timesteps.
Object-based factorizations provide a useful level of abstraction for interacting with the world. Building explicit object representations, however, often requires supervisory signals that are difficult to obtain in practice. We present a paradigm for learning object-centric representations for physical scene understan…
SPARTAN learns sparse interaction graphs between objects in scenes.
problem Capturing structured interactions between entities in complex settings remains challenging.
method SPARTAN is a Transformer-based world model that learns sparse, context-dependent interaction graphs through sparsity regularization on attention patterns.
result SPARTAN achieves significantly improved few-shot adaptation to dynamics changes and robustness against distractors.
The majority of contemporary object-tracking approaches do not model interactions between objects. This contrasts with the fact that objects' paths are not independent: a cyclist might abruptly deviate from a previously planned trajectory in order to avoid colliding with a car. Building upon HART, a neural class-agnost…
KINet learns object interactions without supervision for robotic pushing.
problem Lack of supervised data for object-centric forward prediction.
method End-to-end unsupervised framework using keypoint representation and contrastive estimation.
result Automatically generalizes to unseen scenarios and accurately predicts future states.
The paper introduces a new method to measure the shape relations between biological objects using r-parallel sets.
problem The influence of neighboring objects on the shape and function of biological objects.
method The authors develop a theory based on spatial point processes to measure the geometrical interaction between objects.
result The proposed measures provide detailed information about the shape of individual objects and their interactions.
Generative Adversarial Networks (GANs) can produce images of remarkable complexity and realism but are generally structured to sample from a single latent source ignoring the explicit spatial interaction between multiple entities that could be present in a scene. Capturing such complex interactions between different ob…
AXIOM learns games quickly with simple object models.
problem Data inefficiency in reinforcement learning.
method Expanding object-centric models with active inference.
result AXIOM learns games in 10,000 steps with minimal parameters.
A Braid Boxmath.HO
Interactive art piece shows braid groups and plane motions.
problem Understanding braid groups and their actions.
method Interactive art piece illustrating two braid group definitions and their action on free group.
result Demonstrates all plane motions in a single subspace.
Simplified interactive image segmentation using kNN graphs.
problem Interactive image segmentation with user-provided labels.
method Undirected kNN graphs for label propagation.
result Effective interactive segmentation with significant accuracy.
A new method learns object hierarchies from images to reason about physical interactions.
problem Learning about the interactions of complex objects and their dynamics.
method Unsupervised learning of object hierarchies from raw visual images.
result Improves over a strong baseline at modeling synthetic and real-world videos.
This work discovers latent field effects governing interacting dynamical systems.
problem Discovering field effects governing interacting dynamical systems.
method Proposes neural fields to learn latent force fields from observed dynamics, disentangling local object interactions and global field effects.
result Accurately discovers latent field effects in various dynamical systems.
We use deep learning to model interactions across two or more sets of objects, such as user-movie ratings, protein-drug bindings, or ternary user-item-tag interactions. The canonical representation of such interactions is a matrix (or a higher-dimensional tensor) with an exchangeability property: the encoding's meaning…
In important applications involving multi-task networks with multiple objectives, agents in the network need to decide between these multiple objectives and reach an agreement about which single objective to follow for the network. In this work we propose a distributed decision-making algorithm. The agents are assumed …
CAZSL learns to generalize physical interactions from context.
problem Designing models that can generalize over unknown objects during manipulation.
method Context-aware zero-shot learning using Siamese network architecture, embedding space masking, and regularization.
result CAZSL models can generalize to different parameters or features of interacting objects.
A novel clustering method uses torque balance to group objects.
problem Grouping similar objects in various scientific fields.
method Inspired by gravitational interactions, a parameter-free clustering algorithm based on mass and distance.
result The algorithm effectively clusters objects regardless of their shape, size, or density.
The paper introduces metrics for robust unsupervised learning of vehicle interactions.
problem Robust representation learning of temporal dynamic interactions in robotics.
method Geometric approach using Procrustes distance and optimal transport for comparing interaction distributions.
result Metrics for assessing stability and comparing interaction learning algorithms.
A game-theoretic framework identifies influential hyperparameters for neural networks.
problem Understanding which hyperparameters are most important for neural network performance.
method Employing Shapley Effects for global sensitivity analysis and Pareto front sets for identifying effective configurations.
result Reveals which hyperparameters are most influential for different objectives in neural networks.
Introduces R-SSM for modeling multi-object dynamics with GNNs and normalizing flows.
problem Complex interactions and evolutions in multi-object systems are hard to model.
method Relational state-space model (R-SSM) using graph neural networks (GNNs) and normalizing flows.
result Empirically validated on synthetic and real datasets.
Dreamer 4 learns Minecraft tasks from videos alone.
problem Accurately predicting object interactions in complex environments.
method Reinforcement learning inside a fast, accurate world model.
result Dreamer 4 outperforms previous models in Minecraft, learning from only offline data.
In this paper we study a new reinforcement learning setting where the environment is non-rewarding, contains several possibly related objects of various controllability, and where an apt agent Bob acts independently, with non-observable intentions. We argue that this setting defines a realistic scenario and we present …
Proposes Deep Scenes for interaction-aware scene understanding in reinforcement learning for autonomous driving.
problem Leveraging deep reinforcement learning for high-level decision making in autonomous driving requires handling variable-length sequences of different object types and interactions.
method Introduces Deep Scenes architecture, an extension of Deep Sets or Graph Convolutional Networks, to learn complex interaction-aware scene representations.
result Graph-Q and DeepScene-Q algorithms outperform state-of-the-art methods in evaluations with SUMO.
New method provides fine-grained feedback on interactive student programs.
problem Time-consuming manual grading of interactive student programs.
method Meta-exploration approach using reinforcement learning.
result 94.3% accuracy in providing fine-grained feedback.
Complex environments and tasks pose a difficult problem for holistic end-to-end learning approaches. Decomposition of an environment into interacting controllable and non-controllable objects allows supervised learning for non-controllable objects and universal value function approximator learning for controllable obje…
OP3 models entities for better task generalization in reinforcement learning.
problem Generalizing to unseen physical tasks with combinatorial complexity.
method Object-centric perception, prediction, and planning (OP3) framework.
result OP3 outperforms oracle models and state-of-the-art video prediction models.
Self-Predictive Representations improves data-efficient reinforcement learning from limited interaction.
problem Efficient reinforcement learning from limited data.
method Train agents to predict future latent state representations using self-supervised objectives.
result Achieves a median human-normalized score of 0.415 on Atari with 100k steps of interaction, 55% improvement over previous state-of-the-art.
Learning robot objective functions from human input has become increasingly important, but state-of-the-art techniques assume that the human's desired objective lies within the robot's hypothesis space. When this is not true, even methods that keep track of uncertainty over the objective fail because they reason about …
Develops scalable differentiable physics for complex object interactions.
problem Limited scalability of existing differentiable physics solvers.
method Adopting meshes for arbitrary geometry, localized collision handling, and accelerated implicit differentiation.
result Significantly reduces memory and computation requirements compared to particle-based methods.
Reinforcement Learning AI commonly uses reward/penalty signals that are objective and explicit in an environment -- e.g. game score, completion time, etc. -- in order to learn the optimal strategy for task performance. However, Human-AI interaction for such AI agents should include additional reinforcement that is impl…
Bayesian method helps decision-makers find preferred solutions in multi-objective optimization.
problem Identifying preferred solutions from the Pareto set in multi-objective optimization problems.
method Bayesian model to estimate decision-maker's utility function based on pairwise comparisons, guided by a principled elicitation strategy.
result Superior performance in finding high-utility solutions with a small number of queries.
When encountering novel objects, humans are able to infer a wide range of physical properties such as mass, friction and deformability by interacting with them in a goal driven way. This process of active interaction is in the same spirit as a scientist performing experiments to discover hidden facts. Recent advances i…
Observations consisting of measurements on relationships for pairs of objects arise in many settings, such as protein interaction and gene regulatory networks, collections of author-recipient email, and social networks. Analyzing such data with probabilisic models can be delicate because the simple exchangeability assu…
Bayesian optimization with preference learning identifies preferred solutions in multi-objective problems.
problem Optimizing multiple criteria with decision maker preferences in expensive functions.
method Bayesian optimization with interactive preference learning and active acquisition function.
result Identifies the most preferred solution with reduced interaction cost.
Proposes local coordinate frames for improving model performance in complex dynamical systems.
problem Improving model performance in complex, non-linear, and time-dependent dynamical systems.
method Introduces roto-translation invariant local coordinate frames for geometric graphs.
result The approach outperforms state-of-the-art models in various complex scenarios.
Infants are experts at playing, with an amazing ability to generate novel structured behaviors in unstructured environments that lack clear extrinsic reward signals. We seek to replicate some of these abilities with a neural network that implements curiosity-driven intrinsic motivation. Using a simple but ecologically …
We present a novel approach to weakly supervised object detection. Instead of annotated images, our method only requires two short videos to learn to detect a new object: 1) a video of a moving object and 2) one or more "negative" videos of the scene without the object. The key idea of our algorithm is to train the obj…
Paper studies competitive networks where teams aim to minimize their own objectives, adapting to each other's strategies.
problem Competitive networks where teams have conflicting objectives.
method Proposes diffusion learning algorithms for two classes of network games: zero-sum and non-zero-sum.
result Stability performance of proposed algorithms analyzed and demonstrated through experiments.
On Agents and Equilibriaq-fin.GN
This essay discusses the advantages of a probabilistic agent-based approach to questions in theoretical economics, from the nature of economic agents, to the nature of the equilibria supported by their interactions. One idea we propose is that "agents" are meta-individual, hierarchically structured objects, that includ…
A Human-in-the-Loop Bayesian Optimization Framework for Constraint-Aware Bioprocess Developmentcs.LG
A Human-in-the-Loop Bayesian Optimization framework for constraint-aware bioprocess development.
problem Bioprocess development
method Pareto Front Guided Sampling (PFGS) with Bayesian Optimization (BO)
result Systematic identification of high-performing, feasibility-compliant, and perturbation-resilient operating conditions.
Deep Sets improve reinforcement learning agent's object-centered navigation and generalization.
problem Improving reinforcement learning agents' ability to generalize to unseen objects and goals.
method Combining object-wise permutation invariant networks (Deep Sets) and gated-attention mechanisms.
result Agent demonstrates strong generalization to out-of-distribution goals in a procedurally-generated 2D world.
Loss Data Analyticsq-fin.RM
Loss Data Analytics is an interactive, online, freely available text. The idea behind the name Loss Data Analytics is to integrate classical loss data models from applied probability with modern analytic tools. In particular, we seek to recognize that big data (including social media and usage based insurance) are here…
Ranking items to be recommended to users is one of the main problems in large scale social media applications. This problem can be set up as a multi-objective optimization problem to allow for trading off multiple, potentially conflicting objectives (that are driven by those items) against each other. Most previous app…
Infants are experts at playing, with an amazing ability to generate novel structured behaviors in unstructured environments that lack clear extrinsic reward signals. We seek to mathematically formalize these abilities using a neural network that implements curiosity-driven intrinsic motivation. Using a simple but ecolo…
Material recognition enables robots to incorporate knowledge of material properties into their interactions with everyday objects. For example, material recognition opens up opportunities for clearer communication with a robot, such as "bring me the metal coffee mug", and recognizing plastic versus metal is crucial whe…
Neural production systems learn visual dynamics by applying rule templates to entities.
problem Modeling interactions among entities in structured visual environments.
method Inspired by production systems, the paper uses rule templates to bind placeholder variables to specific entities, scoring and applying the best fitting rules to update entity properties.
result The architecture achieves robust future-state prediction and extrapolation from simple to complex environments, outperforming GNNs.