Deep learning animates objects from input images and videos.
problem Animating arbitrary objects from input images and videos.
method A deep learning framework with three modules: Keypoint Detector, Dense Motion prediction network, and Motion Transfer Network.
result Our method outperforms state-of-the-art image animation and video generation methods.
A new framework uses pixel-based images for realistic cloth animations.
problem Creating virtual cloth deformations that closely match real clothing.
method Reinterpreting cloth deformation as a 2D pattern space and using CNNs.
result Our approach achieves realistic cloth animations without accurate body shapes.
Style generators can invertibly generate images, useful for image enhancement and animation.
problem Generating all possible images and maintaining invertibility are challenging for GANs.
method Proposed style generators are invertible and can generate all possible images.
result Style generators outperform other GANs and Deep Image Prior as image priors.
We propose Progressive Structure-conditional Generative Adversarial Networks (PSGAN), a new framework that can generate full-body and high-resolution character images based on structural information. Recent progress in generative adversarial networks with progressive training has made it possible to generate high-resol…
DIFNET tackles the suspended animation problem in deep graph neural networks.
problem Deep graph neural networks suffer from the suspended animation problem.
method DIFNET uses neural gates and graph residual learning for node hidden state modeling, and includes an attention mechanism for node neighborhood information diffusion.
result DIFNET effectively addresses the suspended animation problem and improves learning performance.
Deep learning-based style transfer between images has recently become a popular area of research. A common way of encoding "style" is through a feature representation based on the Gram matrix of features extracted by some pre-trained neural network or some other form of feature statistics. Such a definition is based on…
NeuroPaint infers missing brain area dynamics from multi-animal datasets.
problem Leveraging multi-animal datasets to understand interactions between brain areas.
method Masked autoencoding approach trained across animals with partial observations.
result Models can successfully reconstruct dynamics of unrecorded brain areas.
AHA model mimics animal episodic learning without labels.
problem Machine learning's slow, statistical learning vs. animals' fast episodic learning.
method Biologically-plausible computational model of the Hippocampus trained without labels.
result AHA model performs image classification comparably to deep ANNs.
The paper learns pose variations within shape populations using constrained mixtures of factor analyzers.
problem Learning pose variations within a shape population with articulated parts and relative rotations.
method Formulated as mixtures of factor analyzers, segmentation by component posterior probabilities, and constraints on factor loading matrices for rotation matrices.
result Automatic learning of pose variations from shape populations, resulting in smooth and realistic animations.
AugurOne trains single image generators without GANs using image warps.
problem Training end-to-end single image generators without GANs.
method Non-affine augmentations of single input images for training an upscaling neural network.
result End-to-end training yields state-of-the-art performance on conditional generation tasks.
Study improves animal audio classification using data augmentation.
problem Improving automated animal audio classification accuracy.
method Exploits different data augmentation techniques for training CNNs.
result Best recognition rates on animal audio classification datasets.
In this paper, we confront the problem of deep learning's big labeled data requirements, offer a rule based strategy for extreme augmentation of small data sets and apply that strategy with the image to image translation model by Isola et al. (2016) to automate cel style cartoon coloring with very limited training data…
New framework learns interaction rules from animal trajectories.
problem Challenges in extracting interaction rules from animal movement data.
method Augmented behavioral models with neural networks and theory-guided regularization.
result Improved performance over baselines and novel biological insights.
Learning the activities of animals is important for the purpose of monitoring their welfare vis a vis their behaviour with respect to their environment and conspecifics. While previous works have largely focused on activity recognition in a single animal, little or no work has been done in learning the collective behav…
Shape analysis methods have in the past few years become very popular, both for theoretical exploration as well as from an application point of view. Originally developed for planar curves, these methods have been expanded to higher dimensional curves, surfaces, activities, character motions and many other objects. In …
New framework uses social media images to estimate wildlife populations.
problem Lack of basic data for wildlife species due to inadequate traditional methods.
method Developed a new computer vision tool to account for social media bias.
result Showed that wildlife population size estimates are learnable from social media.
Improved person detection in occluded conditions with AOS images.
problem Inaccurate classification of partially occluded people, animals, or objects.
method Combining multi-perspective images captured by Airborne Optical Sectioning (AOS) to achieve better precision and recall.
result Precision/recall of 96/93% for automated person detection.
Improved animated faces using audiovisual and modality dropout.
problem Creating realistic animated faces using speech and visual cues.
method Training a deep learning model with modality dropout to balance audio and visual inputs.
result Modality dropout improves viewer preference for audiovisual-driven animation.
In this article, we will formulate a mathematical framework that allows us to treat character animations as points on infinite dimensional Hilbert manifolds. Constructing geodesic paths between animations on those manifolds allows us to derive a distance function to measure similarities of different motions. This appro…
Emergenet predicts animal influenza strain emergence, outperforming current methods.
problem Limited ability to quantitatively assess animal influenza strain emergence.
method Infer digital twin of sequence evolution using 220,151 HA sequences.
result Emergenet predictions outperform WHO seasonal vaccine recommendations and CDC IRAT scores.
We present an instance segmentation algorithm trained and applied to a CCTV recording of beef cattle during a winter finishing period. A fully convolutional network was transformed into an instance segmentation network that learns to label each instance of an animal separately. We introduce a conceptually simple framew…
Polynomial fusion layer improves speech-driven facial animation.
problem Recent facial synthesis relies on low-dimensional representations and concatenation, ignoring higher-order interactions.
method Proposes a polynomial fusion layer to model higher-order interactions of facial encodings.
result Demonstrates improved video quality, audiovisual synchronisation, and blink generation.
GResNet tackles suspended animation in deep GNNs by adding extensive connections.
problem Performance degradation of deep GNNs, especially spectral-based models.
method Introducing GResNet framework with extensively connected highways.
result Avoids dramatic changes to node representations between layers, enhancing learnability.
This study improves lung tumor segmentation in mice MRI scans with nnU-Net, reducing annotation needs.
problem Accurate lung tumor segmentation in mice MRI scans for drug discovery.
method Optimized nnU-Net 3D model for lung tumor segmentation with minimal annotations.
result nnU-Net 3D models outperform 2D models in MRI mice scans, requiring fewer annotations.
Deep learning classifies animal behavior from wearable accelerometers.
problem Classifying animal behavior from accelerometer data.
method End-to-end deep neural network with IIR and FIR filters.
result Outperforms state-of-the-art algorithms in real-time classification.
Classical topological concepts are applied to understand high performance computing simulations of molecules writhing in three dimensional space. These simulations produce peta-bytes of floating point data, to describe 3 dimensional changes in molecular structure. A zero-th order analysis is achieved by viewing a compu…
New method ranks competitors from multiple types of comparisons.
problem Ranking individuals or teams from multiple conflicting comparison types.
method Combination of expectation-maximization algorithm and modified Bradley-Terry model.
result A ranking can be computed from multiple conflicting comparison types.
LISBET automates social behavior analysis using machine learning.
problem Manual annotation of social behaviors is time-consuming, biased, and misses subtle interactions.
method Self-supervised learning on body tracking data.
result Automated detection and segmentation of social interactions.
We introduce a new graphical model for tracking radio-tagged animals and learning their movement patterns. The model provides a principled way to combine radio telemetry data with an arbitrary set of userdefined, spatial features. We describe an efficient stochastic gradient algorithm for fitting model parameters to da…
We provide a rigorous definition of the visual cause of a behavior that is broadly applicable to the visually driven behavior in humans, animals, neurons, robots and other perceiving systems. Our framework generalizes standard accounts of causal learning to settings in which the causal variables need to be constructed …
Online Multi-Object Tracking (MOT) from videos is a challenging computer vision task which has been extensively studied for decades. Most of the existing MOT algorithms are based on the Tracking-by-Detection (TBD) paradigm combined with popular machine learning approaches which largely reduce the human effort to tune a…
StylEx trains a GAN to explain classifier decisions in StyleSpace.
problem Creating meaningful image-specific explanations for classifier decisions.
method Training a StyleGAN to learn a classifier-specific StyleSpace, incorporating the classifier model.
result StylEx finds attributes that align with semantic ones and generates human-interpretable explanations.
FEAFA dataset annotates facial expressions with high detail.
problem Lack of detailed facial expression annotations in existing datasets.
method Manual annotation of 122 participants' facial expressions.
result FEAFA dataset provides detailed annotations for facial expressions.
Diffuse optical tomography (DOT) has been investigated as an alternative imaging modality for breast cancer detection thanks to its excellent contrast to hemoglobin oxidization level. However, due to the complicated non-linear photon scattering physics and ill-posedness, the conventional reconstruction algorithms are s…
Paper uses RL for high-level character control in 3D environments.
problem Creating intelligent characters with generalizable behavior.
method Combines traditional animations, heuristics, and reinforcement learning.
result Demonstrates learning of complex behaviors in a 3D environment.
Item cold-start is a classical issue in recommender systems that affects anime and manga recommendations as well. This problem can be framed as follows: how to predict whether a user will like a manga that received few ratings from the community? Content-based techniques can alleviate this issue but require extra infor…
New method reconstructs 3D shapes from 2D images using Kendall's shape space.
problem Reconstruct 3D shapes from 2D images, especially for rare specimens.
method Kendall's shape space approach with prior information.
result More robust and plausible shapes compared to previous methods.
Paper tackles in-bed pressure-based pose estimation, improving accuracy.
problem Pose estimation models fail to generalize with in-bed pressure data.
method End-to-end framework with a deep neural network pre-processing pressure data.
result Model accurately reconstructs unclear body parts for improved pose estimation.
This paper develops a method for estimating animal motion submanifolds using distribution-free learning.
problem Estimating submanifold models of animal motion supported on a configuration manifold.
method Formulates a novel method based on distribution-free learning over the manifold of measurements.
result Derives approximations of the unknown mapping that converge with rates dependent on sample size and dimensionality.
Generative AI reduces herd behavior in trading, but can also lead to optimal herding.
problem Impact of generative AI on financial stability and herd behavior.
method Laboratory experiments with large language models replicating human trading behavior.
result AI agents make more rational decisions than humans, reducing herd behavior but also potentially leading to optimal herding.
Bayesian method infers local rules for collective animal movement.
problem Learn local rules governing long-term group behaviors.
method Bayesian Inverse Reinforcement Learning with Linearly-Solvable Markov Decision Process.
result Recover true costs and find value of collective movement.
Improved image learning using elliptically contoured tensor-variate distributions.
problem Inadequate statistical analysis for tensor-valued data, especially with heavier or lighter tails.
method Developed a family of elliptically contoured tensor-variate distributions and derived their properties and procedures for estimation.
result Tensor-variate classification rules and tensor-on-tensor regression better predict and characterize data than TVN-based methods.
NURD improves model performance by distilling representations independent of nuisance variables.
problem Models trained under spurious correlations may fail on data with different nuisance-label relationships.
method Developed Nuisance-Randomized Distillation (NURD) to find representations independent of nuisance variables.
result NURD finds representations that perform better regardless of nuisance-label relationships.
Paper introduces method to estimate animal motion on unknown submanifolds using Koopman operator.
problem Estimating animal motion on unknown submanifolds in high-dimensional space.
method Data-dependent approximation of Koopman operator in RKHS over ambient space.
result Strong rates of convergence derived for estimates in terms of fill distance.
This paper uses ML to identify prey handling in seals.
problem Automatically classify prey handling activity in seals for monitoring.
method Developed and compared three ML algorithms: Input Delay Neural Networks, Support Vector Machines, and Echo State Networks.
result Echo State Networks outperformed other algorithms in terms of accuracy and F1score.
Manually authoring transition animations for a complete locomotion system can be a tedious and time-consuming task, especially for large games that allow complex and constrained locomotion movements, where the number of transitions grows exponentially with the number of states. In this paper, we present a novel approac…
Deep learning system speeds up wildlife species identification from camera trap images.
problem Manual review of camera trap images is slow and resource-intensive.
method Combines machine and human intelligence for active learning.
result Matches state-of-the-art accuracy with minimal manual labels.
Quantifying behavior is crucial for many applications in neuroscience. Videography provides easy methods for the observation and recording of animal behavior in diverse settings, yet extracting particular aspects of a behavior for further analysis can be highly time consuming. In motor control studies, humans or other …