Generative model learns motion to language and vice versa using deep RNNs.
problem Linking human motion and natural language for semantic representations and robot behaviors.
method Bidirectional mapping between motion and language using deep recurrent neural networks (RNNs) and sequence-to-sequence learning.
result Model generates realistic motions from natural language descriptions and vice versa.
Proposes VTSFE for action recognition and prediction.
problem Action recognition and prediction for whole-body movements.
method Variational autoencoders with improved noise and transition models.
result Improves performance over VAE and VAE-DMP.
Programmatic Motion Concepts learn human actions from paired videos.
problem Learning motion concepts from paired video and action sequences.
method Semi-supervised learning architecture for hierarchical motion representation.
result Outperforms established baselines, especially in small data settings.
Introduces Motion Programs for better video analysis of human motion.
problem Current video analysis focuses on raw pixels or keypoints, missing higher-level motion primitives.
method Introduces Motion Programs as a neuro-symbolic representation of motions as a composition of high-level primitives.
result Motion Programs accurately describe diverse human motions and improve downstream tasks.
Deep RL trains a robust humanoid push-recovery policy.
problem Training robust humanoid push-recovery policies.
method Model-free Deep Reinforcement Learning.
result Policy learns robust behaviors across the entire body.
New framework predicts diverse, contextually plausible 3D human motions.
problem Predicting multiple plausible future 3D poses given observed poses.
method Developed a new variational framework that conditions latent variable on past observation to encourage relevant information.
result Our approach generates motions of higher quality and preserves contextual information.
MT-VAE learns motion transitions for generating diverse future motions.
problem Learning long-term human motion sequences with transitions.
method Jointly learns motion mode embeddings and transitions using Variational Auto-Encoders.
result Generates multiple plausible future motion sequences from input.
RFC enhances humanoid control to imitate complex human motions.
problem Dynamics mismatch between humanoid models and real humans.
method Residual Force Control (RFC) augments control policies with external forces.
result RFC outperforms state-of-the-art methods in convergence speed and motion quality.
Paper learns versatile balancing and recovery motions for humanoid robots.
problem Training humanoid robots to handle unexpected perturbations.
method Hierarchical Deep Reinforcement Learning in a physics simulator.
result Learned skills comparable to preprogrammed controllers but more adaptable.
Proposes using Dynamic Mode Decomposition with delays for short-term human motion anticipation.
problem Lack of interpretability and explainability in neural network-based motion anticipation methods.
method Dynamic Mode Decomposition with delays for motion representation and prediction.
result Anticipation errors comparable or better than recurrent neural networks for very short times.
DMGNN predicts 3D human motions using adaptive multiscale graphs.
problem Predicting 3D skeleton-based human motions accurately.
method Dynamic multiscale graph neural networks (DMGNN) with adaptive multiscale graphs and MGCU.
result DMGNN outperforms state-of-the-art methods in short and long-term predictions.
Paper introduces a new method for generating diverse human motion predictions.
problem Stochastic human motion prediction with limited flexibility.
method Stochastically combines root variations with previous pose information in a recurrent network.
result Model generates more diverse motion sequences than existing techniques.
Unified framework for human motion generation on Riemannian manifolds.
problem Learning valid human motion in Euclidean spaces.
method Riemannian Motion Generation (RMG) on product manifolds, Riemannian flow matching.
result Achieves state-of-the-art FID (0.043) on HumanML3D and surpasses strong baselines on MotionMillion.
Robot learns to imitate human interactions through deep learning.
problem Teaching robots to coordinate actions with human partners.
method Deep learning framework for motion embedding, prediction, and trajectory generation.
result Importance of predictive and adaptive components for successful imitation.
Generative adversarial networks model and generate physical therapy exercises.
problem Mathematical modeling of human movements in physical therapy.
method Generative adversarial network structure with discriminative and generative models trained concurrently.
result Ability to classify and generate motion examples that resemble recorded sequences.
Deep learning predicts human survival from cardiac MRI motion data.
problem Predicting human survival from cardiac MRI motion data.
method Fully convolutional network for dense motion modeling, autoencoder for latent code learning, Cox partial likelihood loss for right-censored data.
result Predictive accuracy (C-index) significantly higher (p < .0001) for deep learning model (C=0.73) than human benchmark (C=0.59).
Selective relevance method improves motion explainability in 3D activity recognition models.
problem Models do not appropriately factor motion information into their decisions.
method Selective relevance method to adapt 2D explanation techniques for 3D inputs.
result Improves selectivity of motion explanations, revealing model's spatial bias.
Quantum genetic algorithm optimizes SVM for efficient human action recognition.
problem Efficiently extracting motion features for human skeleton dynamics.
method Quantum genetic algorithm optimization of SVM with joint angles and variance.
result Proposed approach outperforms conventional SVM by 2.3% accuracy.
End-to-end learnable network for safer self-driving with interpretable intermediate representations.
problem Safe motion planning for self-driving vehicles.
method Differentiable semantic occupancy representation for cost calculation in motion planning.
result Significantly outperforms state-of-the-art planners in imitating human behaviors and producing safer trajectories.
Paper presents a method for recognizing human actions using GLAC features from motion and static images.
problem Action recognition in 3D depth videos.
method 3D Motion Trail Model (3DMTM) for MHIs and SHIs, GLAC features extraction, l2-regularized Collaborative Representation Classifier (l2-CRC) for classification.
result The method outperforms other approaches in recognizing human actions.
A CNN-based method improves DTI of the human heart, compensating for motion.
problem Signal loss due to heart motion in DTI.
method Invertible Wavelet Scattering using CNN.
result Effective motion compensation and improved fiber structures.
Improves deep pose estimation for extreme motions using pre-trained models.
problem Accuracy and training time issues for extreme/wild motions.
method Post-data augmentation to improve pre-trained models.
result Enhanced accuracy in pose estimation for extreme motions.
Paper uses motion cues to learn features for object detection.
problem Learning effective visual representations for object detection.
method Unsupervised motion-based segmentation of video frames to train a convolutional network.
result The learned representation significantly outperforms previous unsupervised approaches in object detection, especially in limited training data scenarios.
PerceptionNet uses deep CNN for late sensor fusion in HAR, improving accuracy.
problem Improving human activity recognition using motion sensor fusion.
method Late 2D convolution on multimodal time-series data.
result PerceptionNet surpasses state-of-the-art methods by 3% average accuracy.
We present a robust multiple manifolds structure learning (RMMSL) scheme to robustly estimate data structures under the multiple low intrinsic dimensional manifolds assumption. In the local learning stage, RMMSL efficiently estimates local tangent space by weighted low-rank matrix factorization. In the global learning …
The paper presents a method to reduce arm motion complexity for prosthetics and robotics.
problem Reducing the complexity of human arm motions for robotic and prosthetic control.
method Data-driven techniques including DTW, DBA, Ward's distance, batch-DTW, and fPCA.
result Representative motion clusters and averages for different arm DOF levels.
Paper generates synthetic radar signatures for motion classification.
problem Lack of large training datasets for radar-based human activity recognition.
method Adversarial learning for synthetic data generation, kinematic sifting for consistency.
result 93% overall accuracy achieved on diverse aspect angles.
Generative model improves safety in self-driving simulators and human motion generation.
problem Improving generative models for constrained domains like safety-critical applications.
method Developed Gen-neG, a denoising diffusion model that uses oracle-assisted guidance.
result Empirically validated Gen-neG for collision avoidance and safety-guarded human motion generation.
Model predicts multi-modal sequences using N-curves.
problem Capturing multi-modal data in sequential data.
method Neural network model based on Mixture Density Networks with Bézier curves.
result Smooth multi-mode predictions without Monte Carlo simulation.
Paper introduces new motion synthesis model using normalizing flows.
problem Data-driven motion synthesis with probabilistic and controllable models.
method Probabilistic, generative, autoregressive model using normalizing flows and LSTMs.
result Randomly sampled motion from the model outperforms task-agnostic baselines.
CNNs classify human activities from IMU data.
problem Automatic identification of physical activities using motion sensors.
method Used Convolutional Neural Networks (CNNs) with raw IMU data.
result CNNs perform well in classifying 16 lower-limb activities.
EchoFusion tracks and reconstructs fetal images without external trackers.
problem Limited capture range and view-dependent artefacts in fetal ultrasound imaging.
method Combining deep learning and SLAM for image-based tracking and volume reconstruction.
result Demonstrated robust tracking and accurate volume reconstruction in fetal ultrasound.
A novel method predicts shape development using Riemannian shape spaces.
problem Predicting future shape development from a single observation.
method Proposes a novel prediction method that encodes shapes in a Riemannian shape space and learns hierarchical statistical models.
result Outperforms deep learning-supported variants and state-of-the-art methods in predicting shape development.
Deep learning uses WiFi CSI for reliable human presence detection.
problem Reliable human presence detection using ambient RF signals.
method Pre-processing of estimated CSI followed by deep learning.
result Near perfect presence detection during multiple extended periods.
mm-Pose detects human skeletons in real-time using mmWave radar and CNNs.
problem Real-time human skeletal posture estimation in various scenarios.
method mmWave radar, radar-to-image representation, forked CNN architecture.
result Accurate predictions for human skeletal joints in 3D space.
Framework learns stochastic dynamics from endpoint and intermediate distributions using soft energy constraints.
problem Learning stochastic dynamics from endpoint and intermediate distributional observations.
method Formulates generation as a McKean-Vlasov control problem with soft energy constraints, solving it through FBSDE.
result Model learns coherent stochastic trajectories matching prescribed marginal laws.
SVM with local features improves human action recognition.
problem Improving human action recognition in videos.
method Local appearance and motion features extracted using CNNs, concatenated, and used with SVM for classification.
result SVM with local features outperforms previous methods on benchmark datasets.
High dimensional time series are endemic in applications of machine learning such as robotics (sensor data), computational biology (gene expression data), vision (video sequences) and graphics (motion capture data). Practical nonlinear probabilistic approaches to this data are required. In this paper we introduce the v…
GCRL learns causal factors for motion forecasting, improving out-of-distribution prediction.
problem Sensitivity to out-of-distribution data in conventional supervised learning methods.
method Generative Causal Representation Learning (GCRL) leveraging causality for knowledge transfer.
result Significantly outperforms prior models on out-of-distribution prediction.
AirDraw uses motion sensors to write text in the air on smart watches.
problem Limited text input on smart watch screens.
method Motion sensor data and machine learning for gesture recognition.
result 71% accuracy in text entry using AirDraw system.
Geometric approach improves motion alignment accuracy and efficiency.
problem Temporal alignment of human motion data for various applications.
method Geometric point of view, principal fiber bundle, reparameterization invariant projection, dynamic programming, keyframe correspondences.
result Temporal alignment procedures are more accurate and computationally efficient.
Real-time personalization for HAR models learns from new users without prior data.
problem Poor performance of HAR models on new users without labeled data.
method Incremental online domain adaptation using batch normalization.
result Personalized HAR models adapt to new users in real-time.
Deep neural networks generate dance steps from music with minimal labeled data.
problem Generating dance steps from music with little labeled data and maintaining timing accuracy.
method Weakly supervised deep recurrent neural network with convolutional and LSTM layers.
result Model generates dance steps with low cross entropy and maintains timing accuracy.
We propose a Bayesian nonparametric approach to the problem of jointly modeling multiple related time series. Our model discovers a latent set of dynamical behaviors shared among the sequences, and segments each time series into regions defined by a subset of these behaviors. Using a beta process prior, the size of the…
A new model for simulating cloth manipulation in robots, accurate to within 1cm.
problem Accurately simulating cloth manipulation in robots, especially in moderate stress environments.
method A continuous, isometric strain model for textiles, treating them as inextensible surfaces with only isometric motions. Aerodynamic effects are incorporated through virtual uncoupling of mass.
result Simulations are accurate to within 1cm compared to real-world manipulation, even with coarse meshes.
MultiPath predicts multi-modal future trajectories for better motion planning.
problem Predicting human behavior in uncertain real-world domains like autonomous driving.
method Leverages fixed future state-sequence anchors and regresses offsets with uncertainties.
result Achieves more accurate predictions with an order of magnitude fewer trajectories.
In this paper we propose the time-dependent generalization of an `ordinary' autonomous human biomechanics, in which total mechanical + biochemical energy is not conserved. We introduce a general framework for time-dependent biomechanics in terms of jet manifolds associated to the extended musculo-skeletal configuration…
Proposes a flexible method for learning latent causal representations.
problem Limited applicability of existing causal representation learning methods.
method Imposes constraints on function classes and relaxes identifiability conditions.
result Establishes partial identifiability results under weaker conditions.